Text processing in the shell three swordsmen grep, sed, awk

grep command

1.grep format

   grep root passwad	          在passwd中过滤root
    
   grep ^root passwd	          过滤以root开头的信息
    
   grep root$ passwd	          过滤以root结尾的信息
    
   grep -i root passwd	          忽略大小写
    
   grep -E "\<root" passwd	      root字符之前不能有字符
    
   grep -E "root\>" passwd        root字符之后不能有字符
     
   grep -3 lebron passwd	      显示关键字与其前后三行
    
   grep -n lebron passwd 	      显示匹配行所在行号
    
   grep -A3 lebron passwd 	      显示匹配行以及下面3行
    
   grep -B3 lebron passwd	      显示匹配行以及上面3行
    
   grep -v	                      反向过滤

grep root passwd filter root in passwd

grep -i root passwd filter root in passwd (ignore case)

grep -i ^root passwd Filter lines starting with root in passwd (ignoring case)

grep -i root$ passwd filter lines ending with root in passwd (ignore case)

grep -iE “^root|root$” passwd starts with root or ends with root

grep -iE "^root|root$" passwd -v remove the beginning or end of root

grep -iE "<root" passwd no characters before root

grep -iE "root>" passwd no characters after root

grep -iE "<root>" passwd root cannot have characters before and after

grep -2 ROOT passwd displays the line of the keyword and its two lines before and after

grep -B2 ROOT passwd displays the line of the keyword and the two lines before it

grep -A2 ROOT passwd displays the line where the keyword is located and the next two lines

grep -n ROOT passwd displays the line of the keyword and its line number

2. Grep character number matching rules

    w...s	        w开头，s结尾中间任意3个字符
     
    ....s	        s结尾前面任意4个字符
     
    "ws*"	        w、s出现任意次
     
    "ws?"	        w、s出现0-1次
     
    "ws+"	        w、s出现1-任意次
      
    "ws{5}"	        出现5次
     
    "ws{3,5}"	出现3-5次
     
    "ws{,5}"	出现0-5次
     
    "ws{0,5}"	出现0-5次
     
    "ws{2,}"	出现2-任意次
     
    "(ws){2}"	“ws”这个字符作为整体出现2次

grep -E "w ... s" westos

grep -E “… s” westos

grep -E "in ..." westos

grep -E “ws *” westos

grep -E “ws?” westos

grep -E “ws +” westos

grep -E “ws {4}” westos

grep -E “ws {2,4}” westos

grep -E “ws {, 4}” westos

grep -E “ws {3,}” westos

grep -E “(gb){2}” westos gb appears twice as a whole

grep -E "(gb){,2}" westos gb appears as a whole at most twice

grep -E "w(gb){1,2}s" westos between w and s gb appears as a whole at most twice

sed command

    p	        打印（需加 -n）
    d	        删除
    a	        追加
    i	        插入
    c	        更改/替换
    w	        写入
    r	        整合文件
    's/xx/xx/g'	字符替换

1. p print

sed -n '3p' westos displays the third line of westos

sed -n '3,7p' westos display westos 3 to 7 lines

sed -n '3p;7p' westos displays the contents of the 3rd and 7th lines of westos

sed -n'$p' westos displays the last line of westos

sed -n'/^daddy/p' passwd display lines starting with daddy

sed -n'/bash$/p' passwd displays lines ending in bash

2. d delete

sed '1,2d' westos deletes rows 1 and 2 in westos and displays

sed '1d;4d' westos delete rows 1 and 4 in westos and display

sed'/^p/d' westos delete the line starting with p in westos and display

sed'/g$/d' westos delete the line ending with g in westos and display

sed'/g$/!d' westos delete and display the lines other than g ending in westos (the reverse operation of the previous step)

3. a append

sed'aoligay' westos append oligay after each line of westos

sed'$aoligay' westos append oligay to the end of westos

sed '1aoligay' westos append oligay after the first line of westos

sed'/gg/aoligay' westos /gg/ means to append after this line

sed'/pig/aoligay\nbro' westos \n is a newline

4. i insert

i与a的不同在于，a在指定行后追加，i在指定行前插入

sed '3i hhhhhh' westos insert hhhhh before the third row of westos

sed'$i hhhhhh' westos insert hhhhh before the end of westos

sed'/pig/i hhhhhh' westos insert hhhhh before the line /pig/ in westos

sed'/pig/i hhhhhh\ngg' westos insert hhhhh gg before the /pig/ line in westos

5. c change

sed'$c dsb' passwd replace the last line with dsb

sed'/bash$/c dsb' passwd will replace the line ending in bash with dsb

sed'/^lbj/c dsb' passwd will replace the line starting with lbj with dsb

6. w write

sed ‘/hey/w file’ westos

7. r integration

sed '2r file' westos merge the contents of the file file into the second line of westos

sed'$r file' westos merge the contents of the file file to the end of westos

8. replace

sed '1,3s/bash/dunk/' passwd Replace bash on lines 1 and 3 in passwd with dunk

sed's/lbj/kobe/g;s/bash/gkd/g' passwd replace all lbj in passwd with kobe; replace all bash with gkd

sed'/westos/,/zxn/s/bash/hhh/g' passwd replace the bash between westos and zxn in passwd with hhh

sed's///@/g' passwd replace all / in passwd with @

sed's@/@:@g' passwd has the same effect as the previous command, except that the delimiter is changed to @

awk command

    格式：
     
    awk  -F  分隔符  BEGIN{}{}END{}  filename
     
    NR	        行数
    NF	        每行的列数
    FILENAME	文件名称本身
    “westos”	字符串
    $1	        指定列数

awk -F:'{print NR}' passwd show all lines of the file

awk -F:'{print NF}' passwd shows the number of columns in each line of the file

awk -F:'{print $1}' passwd displays the contents of the first column of the file

awk -F:'{print FILENAME}' passwd display the file name (the original file will be displayed several times if there are several lines)

awk -F:'{print “ohou”}' passwd display the input string (the original file has a few lines and it will be displayed several times)

/bash$/	                  条件：以bash结尾
/条件1/ | | /条件2/	  条件1或条件2
/条件1/ && /条件2/	  条件1、2同时成立

awk -F : ‘BEGIN{print “name”}/bash$/{print $1}END{print “over”}’ passwd

awk -F : ‘BEGIN{print “name”}/bash$/||/nologin$/{print $1}END{print “over”}’ passwd

awk -F : ‘BEGIN{print “name”}/bash$/||/^root/{print $1}END{print “over”}’ passwd

awk -F : ‘$1~/root/{print $1}’ passwd

awk -F : ‘$7/nologin$/{print $1}’ passwd

Homework

Exercise 1:

List all user names that can be used in the system

Exercise 2:

Make a script apache.sh, so that after running this script, enter a number to modify the httpd port to enter a number

Exercise 3:

Count the number of users who can be used by users in the system and whose home directory is not in /home