Linux/Unix Tips & Tricks: The sed command

Sed (Stream EDitor) is a UNIX command that reads input, line by line from a text file or stdin (standard input) and applying the user specified operation and outputs the result to stdout (standard output). It's very good for searching for specific text patterns using regular expression and then perform some manipulation on it. A typical sed usage looks like this "sed -e 's/searchForOldStuff/replaceWithNewStuff/g' inputFileName > outputFileName". Where s stands for substitute, g for global, meaning replace all matching occurrences in a line, searchForOldStuff is the regular expression to search for and replaceWithNewStuff is what to replace it with.

Here are some handy tricks with sed.


Replace first occurrence of cat in every line with dog:

sed -e 's/cat/dog/'

Replace ALL occurrence of cat with dog:

sed -e 's/cat/dog/g'

Replace ALL occurrence of cat with dog ignoring case:

sed -e 's/cat/dog/gi'

Delete all empty (with only a carriage return) lines:

sed -e '/^$/d' or sed -e 's/^$//'

Delete all lines with only white spaces (spaces and tabs):

sed -e '/^[ \t]*$/d' or sed -e 's/^[ \t]*$//'

Delete all empty lines or lines with only whit spaces:

sed -e '/^$/d' -e '/^[ \t]*$/d'

Delete leading white space from all lines:

sed -e '/^$/d' -e 's/^[ \t]*//'

Delete trailing white spaces from all lines:

sed -e 's/[ \t]*$//'

Delete leading and trailing white spaces from all lines:

sed -e 's/^[ \t]*//' -e 's/[ \t]*$//' or sed -e 's/^[ \t]*;[ \t]*$//'

Delete lines from the first to and including the first not blank line:

sed '1,/^$/d'

Insert something at the beginning of each line:

sed -e 's/^/thiswillbeinserted/'

Insert something at the end of each line:

sed -e 's/$/thiswillbeinserted/'

Number each line of a file:

sed = yourfile.txt | sed -e 'N;s/n//'

Number each line of a file (row number and a tab):

sed = yourfile.txt | sed -e 'N;s/n/t/'

Number each line of a file (row number and a space):

sed = yourfile.txt | sed -e 'N;s/n/ /'

Reverse order of the lines:

sed -e '1!G;h;$!d' or sed -n '1!G;h;$p'

Reverse each character on the line:

sed -e '/n/!G;s/(.)(.*n)/&21/;//D;s/.//'

Convert a file with Unix newlines (CR) to DOS/Windows newlines (CRLF):

sed 's/$/r/' or sed 's/$'"/`echo \r`/"

Convert a file with DOS newlines to Unix (to make the special character '^M' press Ctrl+v and Ctrl+m):

sed 's/^M$//'

Insert a empty line after every line:

sed -e G

Insert only a empty line where not already a empty line:

sed -e '/^$/d;G'

Count number of lines:

sed -n '$='

Align all text to the right at column 79:

sed -e :a -e 's/^.{1,78}$/ &/;ta'

Center all text in the middle of 79-column width. Leading and trailing spaces are discarding.

sed -e :a -e 's/^.{1,77}$/ &/;ta' -e 's/( *)1/1/'

Print the first line:

sed q

Print the first 10 lines:

sed 10q

Print the last line:

sed '$!d'

Print the last 10 lines:

sed -e :a -e '$q;N;11,$D;ba'

Print the paragraph if it contains MYREGEX (blank lines separate paragraphs):

sed -e '/./{H;$!d;}' -e 'x;/MYREGEX/!d;'

Join pairs of lines (side-by-side) from two files:

sed '$!N;s/n/ /'

Remove html tags from a file:

sed -n '/^$/!{s/<[^>]*>//g;p;}'

Make a backup of the file using specified extension (.bak):

sed -i.bak 's/something/example/'

If you know some other handy sed scripts please post them here.