Charlie Harvey

Bash Tip: Remove a line from middle of file

This is a thing that I’ve wanted to do for ages, and it came up for me during this week'd episode of Hak5, which I am getting to be a big fan of by the way. The material under discussion was what to do when one of your known hosts changes. Darren was saying he was a bit slack and just did an rm of his whole known_hosts file.

As you may know, when a host key has changed, you'll get a warning like this telling you so in no uncertain terms $ ssh -o stricthostkeychecking=ask ssh.example.com @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that the RSA host key has just been changed. The fingerprint for the RSA key sent by the remote host is 4a:ad:fe:96:de:02:95:f1:76:34:23:12:e4:78:12:bb. Please contact your system administrator. Add correct host key in /home/xahria/.ssh/known_hosts to get rid of this message. Offending key in /home/xahria/.ssh/known_hosts:8 RSA host key for localhost has changed and you have requested strict checking. Host key verification failed. As you can see the offending key is on line 8 of our known hosts file. So I’d like to be able to remove that line.

Now there is probably a tool for this or something, but I thought it was a nice quick one liner to do in bash, a-la Commandline Fu. My first shot used nested backticks and piping the output of wc -l to bc. Fugly. I think this version is a little nicer.

My test data is a file t with contents1 2 3 4 5 Exciting stuff. My goal is to rewrite the file so that it loses line 3.

Here’s how I do it. (x=3; head -n$(($x-1)) t; tail -n$((`wc -l t | cut -d ' ' -f 1`-$x)) t) > t2 && cat t2 > t And lets just check it works: $ cat t 1 2 4 5

There is quite a lot going on so let's step through it.

In the real world, you would use the "Offending key" line number from the SSH warning as your x and ~/.ssh/known_hosts as your t. I suppose you could make a proper script of this if you wanted. Being a perl geek, I would perl -nle 'print if(++$i!=3)' known_hostsBut playing Bash is fun sometimes, eh?


Comments

  • You can use strong, em, code, pre, q and a HTML tags.
  • Comment and email are the only required fields.
  • Be respectful. You may want to read the comment guidelines before posting.

Privacy note: This form will forward your IP address, user agent and referrer to the Akismet, StopForumSpam and Botscout spam filtering services. I don’t log these details. Those services will. I do log everything you type into the form. Full privacy statement.






  • #192

    Acesabe

    Nice, but isn't sed or ssh-keygen -R slightly less finger/brain intensive??! ;-P



    http://www.commandlinefu.com/commands/view/6045/remove-a-line-in-a-text-file.-useful-to-fix-ssh-host-key-change-warnings

  • #193

    Unixboy

    You could use sed

    Using sed would let you do the same in fewer characters



    sed '5d' known_hosts > b && mv b known_hosts

  • #194

    Charlie Harvey

    Thanks folks

    @Acesabe Brill I had no idea that ssh-keygen -R even existed. That would definitely save some typing which is "a good thing" for a lazy person like me :-)



    @Unixboy Wow! sed is amazing. I wonder if someone will go proper old skool and come up with an awk solution?!

  • #195

    Lillee

    awk

    You would get rid of the third line of known hosts thusly:

    awk '{if (NR!=3) {print}}' known_hosts

  • #198

    Charlie

    Awk

    @Lillee Nice! I knew someone would come up with an awk solution ;-)

  • #199

    theSnake

    Python

    For the sake of completeness, there are Python solutions at:



    http://bytes.com/topic/python/answers/595470-how-remove-number-lines-text-file