Linux/Unix Tips & Tricks: The sed command

Sed (Stream EDitor) is a UNIX command that reads input, line by line from a text file or stdin (standard input) and applying the user specified operation and outputs the result to stdout (standard output). It's very good for searching for specific text patterns using regular expression and then perform some manipulation on it. A typical sed usage looks like this "sed -e 's/searchForOldStuff/replaceWithNewStuff/g' inputFileName > outputFileName". Where s stands for substitute, g for global, meaning replace all matching occurrences in a line, searchForOldStuff is the regular expression to search for and replaceWithNewStuff is what to replace it with.

Here are some handy tricks with sed.


Replace first occurrence of cat in every line with dog:

sed -e 's/cat/dog/'

Replace ALL occurrence of cat with dog:

sed -e 's/cat/dog/g'

Replace ALL occurrence of cat with dog ignoring case:

sed -e 's/cat/dog/gi'

Delete all empty (with only a carriage return) lines:

sed -e '/^$/d' or sed -e 's/^$//'

Delete all lines with only white spaces (spaces and tabs):

sed -e '/^[ \t]*$/d' or sed -e 's/^[ \t]*$//'

Delete all empty lines or lines with only whit spaces:

sed -e '/^$/d' -e '/^[ \t]*$/d'

Delete leading white space from all lines:

sed -e '/^$/d' -e 's/^[ \t]*//'

Delete trailing white spaces from all lines:

sed -e 's/[ \t]*$//'

Delete leading and trailing white spaces from all lines:

sed -e 's/^[ \t]*//' -e 's/[ \t]*$//' or sed -e 's/^[ \t]*;[ \t]*$//'

Delete lines from the first to and including the first not blank line:

sed '1,/^$/d'

Insert something at the beginning of each line:

sed -e 's/^/thiswillbeinserted/'

Insert something at the end of each line:

sed -e 's/$/thiswillbeinserted/'

Number each line of a file:

sed = yourfile.txt | sed -e 'N;s/n//'

Number each line of a file (row number and a tab):

sed = yourfile.txt | sed -e 'N;s/n/t/'

Number each line of a file (row number and a space):

sed = yourfile.txt | sed -e 'N;s/n/ /'

Reverse order of the lines:

sed -e '1!G;h;$!d' or sed -n '1!G;h;$p'

Reverse each character on the line:

sed -e '/n/!G;s/(.)(.*n)/&21/;//D;s/.//'

Convert a file with Unix newlines (CR) to DOS/Windows newlines (CRLF):

sed 's/$/r/' or sed 's/$'"/`echo \r`/"

Convert a file with DOS newlines to Unix (to make the special character '^M' press Ctrl+v and Ctrl+m):

sed 's/^M$//'

Insert a empty line after every line:

sed -e G

Insert only a empty line where not already a empty line:

sed -e '/^$/d;G'

Count number of lines:

sed -n '$='

Align all text to the right at column 79:

sed -e :a -e 's/^.{1,78}$/ &/;ta'

Center all text in the middle of 79-column width. Leading and trailing spaces are discarding.

sed -e :a -e 's/^.{1,77}$/ &/;ta' -e 's/( *)1/1/'

Print the first line:

sed q

Print the first 10 lines:

sed 10q

Print the last line:

sed '$!d'

Print the last 10 lines:

sed -e :a -e '$q;N;11,$D;ba'

Print the paragraph if it contains MYREGEX (blank lines separate paragraphs):

sed -e '/./{H;$!d;}' -e 'x;/MYREGEX/!d;'

Join pairs of lines (side-by-side) from two files:

sed '$!N;s/n/ /'

Remove html tags from a file:

sed -n '/^$/!{s/<[^>]*>//g;p;}'

Make a backup of the file using specified extension (.bak):

sed -i.bak 's/something/example/'

If you know some other handy sed scripts please post them here.

Howto kill a process in linux

If you have a program that using a lot of your computers resources or not responding it can be necessary to kill it the hard way, here are some ways to do that.

• With the kill command.
  If you know the PID (every process in your system has a uniq number, that's the PID, shortcut for Process IDentifier ) of the process, you can send it a signal with the kill command. There are several signals you can send and depending on the process it will be terminated. Some of the most common signals to end a process are SIGHUP, SIGINT, SIGTERM and SIGKILL. SIGTERM is the signal you should try first and if that don't work use SIGKILL. To send a signal to the process with the kill command type this in a console: "kill -SIGNAL PID" where you replacing SIGNAL with a signal and PID with the processes process identifier number. To find out the PID of a process you can use one of the "pidof", "pgrep", "top", or "ps" command in a console.
  
  
• With the pkill command.
  pkill is a command similar to kill in that it will send signals to a process to terminate it, the difference to kill is that you don't need know it's PID, instead you terminate a process based on its name. If you type "pkill firefox" in a console pkill will terminate all processes with firefox in it's name. pkill will by default send a SIGTERM signal but that can be changed. WARNING! be careful as pkill will terminate ALL your processes that have what you specified in its name.
  
  
• With the xkill command.
  In X-Windows you can kill processes that have a window with the xkill command. Running xkill will get you a mouse cursor of a death skull and if you click with it on a window it will be terminated. In KDE pressing the Ctrl+Alt+Esc keyboard shortcut will run the xkill command.
  
  
• With KDE.
  In a default KDE setup you can press Ctrl+Esc keyboard shortcut to open a window with all processes listed and from where you can right click on one and choose a signal to send to it.
  

• With SysRq.
  With the SysRq keyboard key you can make the kernel do some really powerful stuff with you system. Type "cat /proc/sys/kernel/sysrq" in a console to see if SysRq it is enabled, if the result is "1" it's enabled or else you can activate it by running one of this command as root "echo 1 > /proc/sys/kernel/sysrq" or with the sysctl command like this "sysctl -w kernel.sysrq=1". By pressing any of the keyboard shortcuts below you will tell the kernel to do what's described in the parentheses.
  
  Alt+SysRq+r (The r stands for put keyboard in raw mode)
  Alt+SysRq+s (The s for sync the disk)
  Alt+SysRq+e (The e for terminate all processes)
  Alt+SysRq+i (The i for kill all processes)
  Alt+SysRq+u (The u for remount all file systems read only)
  Alt+SysRq+b (The b for reboot the system)

Beware that using some of these SysRq commands it is very dangerous as it will kill all processes in your system and you will loose all data that are not saved. Use it at your own risk and only as a last option.

Good luck killing your system.