1.1 sed
A stream editor.
1) The sed command includes many features for selecting lines to be modified and making changes only to the selected lines.
2) save sed output. e.g. sed ‘some-sed-commands’ input-file > outfile
3) Regular Expressions refer Regular Expressions.
Option
| -e |
Only print line number. |
| -n |
Not print line. Default print all lines (include edited and not edited) |
| -p |
Only print edited line. |
| -f |
Read from file rather than from command line. Use sed script. e.g. sed -f myscript.sed input_file |
Pattern
| n |
n is line number, search line n. |
| n,m |
From line n to m, e.g. 2,5 represent from line 2 to line 5 |
| /pattern/ |
Search pattern |
| /pattern1/pattern2/augu. |
Search both 2 patterns & - remember the match pattern. E.g. sed –n ‘s/nurse/”hello” &/p’ quote.txt. # origin: … nurse…, result: … hello nurse … augu. g – Global replacement, e.g. sed ‘s/\$//g’ quote.txt |
| /pattern/,n |
Search pattern in line n |
| n,/pattern/ |
As above. |
| n,m! |
Not search line n and m. |
| p |
Print match line. |
| = |
Display line number |
| d |
Remove line |
| c\ |
Replace text. |
| s |
Replace mode. |
| r |
Read text from another file. E.g. sed ‘/company./r file.txt’ quote.txt |
| w |
Write file |
| q |
Exit pattern if pattern matched |
| n |
Concatenate. |
| $ |
Last line. |
e.g.
sed "s/happy/enchanted/g" chap1 >chap1.new
Replaces each occurrence of the word happy found in the file chap1 with the word enchanted. Without the g character, replaces only the first occurrence of the word happy on a line.
sed –n ‘/\$/’p quote.txt # \$ stands for $
sed –n ‘1,$’p quote.txt # print full file
sed 's/.txt/.id/' quote.txt # replace .txt by .id
grep xxx /etc/passwd | sed -e ‘s/Taylor/Tailor/g;s/:/ /g’
cat xxx | head -15 | sed ‘/eiyo/,/rpm/d’
sed 's;INSERT INTO;TERMINATE INTO;'
1.2 awk
More powerful than sed, a real text programming kit – awk.
Finds lines in files that match a pattern and performs specified actions on those lines.
nawk – support more which written by original author;
gawk – support more which created by free software foundation GNU (may not exist in some system).
Option
| -f |
Read from file rather than from command line. |
| -Fc |
c as separator between fields (default space) |
Pattern
| awk ‘{command}’ |
Command format. |
| $n |
Field n. ($0 is the entire line) |
| “xxx” |
The constant. |
| NF |
how many fields are on a line (for the blank field cases) |
| $NF |
always the value of the last field on the line (for the blank field cases) |
| NR |
Record count. |
| END |
When pointer at the end of records. |
Command
| |
Display all lines. |
Symbol
| = =+ -= *= /= %= ^= |
Assignment. |
| ?: |
C conditional expression (nawk and gawk). |
| || |
Logical OR. |
| && |
Logical AND. |
| ~ !~ |
Match regular expression and negation. |
| < <= > >= != == |
Relational operators. |
| (blank) |
Concatenation. |
| + - |
Addition, subtraction. |
| * / % |
Multiplication, division, and modulus. |
| + - ! |
Unary plus and minus, and logical negation. |
| ^ |
Exponentiation. |
| ++ -- |
Increment and decrement, either prefix or postfix. |
| $ |
Field reference. |
String Function (awk)
| Option or argument |
Function |
| gsub(r,s,t) |
For nawk. Globally substitutes s for each match of the regular expression r in the string t. Returns the number of substitutions. If t is not supplied, defaults to $0. |
| index(s,t) |
Returns position of substring t in string s or zero if not present. (offset start from 1) |
| length(s) |
Returns length of string s or length of $0 if no string is supplied. |
| match(s,r) |
For nawk. Returns either the position in s where the regular expression r begins, or 0 if no occurrences are found. Sets the values of RSTART and RLENGTH. |
| split(s,a,sep) |
Parses string s into elements of array a using field separator sep; returns number of elements. If sep is not supplied, FS is used. Array splitting works the same way as field splitting. (array start from [1]) |
| sprintf("fmt",expr) |
Uses printf format specification for expr. e.g. sprintf(“%c”,i) |
| sub(r,s,t) |
For nawk. Substitutes s for first match of the regular expression r in the string t. Returns 1 if successful; 0 otherwise. If t is not supplied, defaults to $0. |
| substr(s,p) substr(s,p,n) |
Returns substring of string s at beginning position p up to a maximum length of n. If n is not supplied, the rest of the string from p is used. |
| tolower(s) |
For gawk. Translates all uppercase characters in string s to lowercase and returns the new string. |
| toupper(s) |
For gawk. Translates all lowercase characters in string s to uppercase and returns the new string. |