The basic knowledge of regular expression has been discussed in the previous section, and the three key commands of regular expression are grep, sed and awk.
1) grep is used to filter row information
Function: Find and print the line that matches the keyword
Parameters: -n output matching lines and line numbers together; -v print lines that do not meet the requirements (inverted); -i are case-insensitive; -w print lines of complete words; -An print lines that meet the requirements and Lower n lines; -Bn prints the required lines and the upper n lines; -Cn prints the required lines and the upper and lower n lines.
[centos7 the root @ ~] # grep -A2 / the passwd 'halt' / etc # halt the line comprising the following two lines and outputs
[the root centos7 @ ~] # grep -B2 'halt' / etc / the passwd # comprises the The halt line and the two lines above it are output
[root@centos7 ~]# grep -C2'halt ' /etc/passwd
#Output the line containing halt and the two lines above and below [root@centos7 ~]# grep -n 'root' /etc/passwd
#Filter out lines and line numbers with the root keyword [root@centos7 ~]# grep -v'nologin ' /etc/passwd
#Filter lines without nologin [root@centos7 ~ ]# grep'[0-9]' /etc/inittab
#Filter out all lines that contain numbers [root@centos7 ~]# grep -v'[0-9]' /etc/inittab #Filter out all lines that do not contain numbers The line
[root@centos7 ~]# grep -v'^#'/etc/inittab
#Filter out lines starting with non# [root@centos7 ~]# grep'game $' /etc/inittab
#Filter out lines ending with game [root@centos7 ~]# grep -v'^#' /etc/crontab |grep -v'^$' #Remove all blank lines and lines beginning with #
2) sed is used to filter and replace line text
Sed is an online editor that processes text content one line at a time. When processing, store the currently processed line in a temporary buffer, then use the sed command to process the contents of the buffer, and after the processing is completed, send the contents of the buffer to the screen. Then process the next line, and repeat this way until the end of the file.
Role: filter query and modify the content of the replacement text
Parameters: -n Only the line (or action) that has undergone special processing by sed will be listed; -e directly edit sed actions in the command line mode; -f directly write sed actions in a file;- The action of r sed supports the syntax of extended regular notation; -i directly modifies the content of the read file without outputting to the terminal. a is added, after a can be followed by strings, and these strings will appear on a new line (the current next line); c replaced, after c can be followed by strings, these strings can replace between n1 and n2 The line of d is deleted, because it is deleted, so there is usually no content after d; i is inserted and modified, and strings can be connected after i, and these strings will be a new line; p is printed, and a certain selection Data output of s; s replace and replace, directly replace; g select all content.
Delete: d command
[root@centos7 ~]# sed '2d' app.txt #Delete the second line of the file
[root@centos7 ~]# sed '2,$d' app.txt #Delete the second line of the file to All lines at the end
[root@centos7 ~]# sed'$d' app.txt #Delete the last line of the file
[root@centos7 ~]# sed'/test/'d app.txt #Delete all lines that contain test and
replace : s command
[centos7 the root @ ~] # Sed 'S / test / Prod / g' app.txt # file replace all test prod, then no replacement g each row a
[centos7 the root @ ~] # Sed - n's/^test/prod/p' app.txt # The line beginning with test, replace test with prod and output
[root@centos7 ~]# sed -n's/\(love\)able/\1rs /p' app.txt #All loveables will be replaced with lovers
[root@centos7 ~]# sed 's#you#your#g' app.txt # "#" is the separator here, instead of the default "/"
Selected line range: comma
[root@centos7 ~]# sed -n'/test/,/chart/p' app.txt # output all lines in the range of test and cheart
[root@centos7 ~]# sed -n '6,/^test/p' app.txt #Output all lines from line 6 to the line starting with test
[root@centos7 ~]# sed'/test/,/chart/s/$/test /'app.txt #The line between test and west, add test at the end
Multi-point editing: e command
[root@centos7 ~]# sed -e '1,5d' -e's/test/dev/' app.txt #Delete lines 1 to 5 and replace test with dev
Read from the file: r command
[root@centos7 ~]# sed'/test/r 1.txt' app.txt #Match the test line, insert 1.txt content in this line and output
Write file: w command
[root@centos7 ~]# sed -n'/test/w 1.txt' app.txt #All lines containing test in app.txt are written to 1.txt
Append command: a command
[root@centos7 ~]# sed'/^test/a\\--->this is app.txt' app.txt< #The content is appended to the end of the line starting with test
Insert modification: i command
[root@centos7 ~]# sed '2i love you' app.txt #Insert love you in line 2
[root@centos7 ~]# sed -i's/\$/\!/g' app.txt #Replace the $ at the end of the line with!
3) awk filter edit text
Awk is a streaming editor like sed. It also operates on lines in a document and processes text line by line. Awk is more powerful than sed, it can do what sed can do, and it can also do more complex regularities.
1 awk command form
awk [-F|-f|-v]'BEGIN{} //{command1; command2} END{}' file
[-F|-f|-v] Large parameter, -F specifies the separator, -f calls the script, -v defines the variable var=value
'' # quote code block
BEGIN #initialize the code block, mainly refer to global variables, set the FS separator
// # matching code block, it can be a string or regular expression
{ } # Command code block, containing one or more commands
; # Multiple commands are separated by semicolons
END # Ending code block, mainly for final calculation or output summary information
2 Special variable character
$0 # Represents the entire current line
$1 #The first field of each line
NF #Field quantity variable
NR #The record number of each line, multi-file record increment
FNR # Similar to NR, multi-file record does not increment, each file All start from 1
\t # Tab character
\n # Newline character
FS # Define the separator when BEGIN
RS # Enter the record separator, the default is the newline character
~ # Match, not an exact comparison compared with ==
! ~ # Does not match , Inexact comparison
== # Equal, must all be equal, exact comparison
! = # Not equal, exact comparison
&& # Logic AND
|| # Logic OR
+ # When matched, it means 1 or more
/[0-9][ 0-9]+/ # Two or more digits
/[0-9][0-9]*/ # One or more digits
OFS # Output field separator, default is also space
ORS # Output record separator, Default is newline
-F'[:#/]' # Define three separators
3 awk example
= Etc. It is worth noting that when comparing numbers with double quotes, awk will not consider it as a number, but as a character. Without double quotes, it will be considered as a number. [root@centos7 ~]#
awk -F':''$3=="0"' /etc/passwd
#Match the line with the character 0 in the third field [root@centos7 ~]# awk -F':''$3>=500' / etc/passwd
#Match the line where the third field is greater than the number 500 [root@centos7 ~]# awk -F':''$7!="/sbin/nologin"' /etc/passwd #Match the seventh field is not nologin The line
[root@centos7 ~]# awk -F':''$3<$4' /etc/passwd
#Logical comparison between two segments [root@centos7 ~]# awk -F':''$3>" 5" && $3<"7"' /etc/passwd
#Lines that satisfy two matching conditions at the same time [root@centos7 ~]# awk -F':''$3>"5" || $7=="/bin/ bash"'/etc/passwd #The line that meets one of the conditions
commonly used variable
NF: how many paragraphs are separated by a separator, how many paragraphs NF is, and $NF is the value of the last paragraph
NR: line number line number
[root@centos7 ~ ]# head -n3 /etc/passwd | awk -F':''{print NF}'
#Output the total number of paragraphs [root@centos7 ~]# head -n3 /etc/passwd | awk -F':''{print $NF}'
#Output the value of the last paragraph [root@centos7 ~]# head -n3 /etc /passwd | awk -F':''{print NR}'
#Output line number [root@centos7 ~]# awk'NR >20' /etc/passwd
#Use line number 20 as the judgment condition [root@centos7 ~] # awk -F':''NR>20 && $1 ~ /ssh/' /etc/passwd #Output lines with NR greater than 20 and ssh in the first paragraph
modify the calculated field value
[root@centos7 ~]# head -n3 / etc/passwd |awk -F':''$1="root"'
#The first paragraph of the first three lines is modified to root output [root@centos7 ~]# head -n2 /etc/passwd |awk -F':' ' {$7=$3+$4; print $7}'
#Calculate fields and assign values [root@centos7 ~]# awk -F':''{(t=t+$3)}; END {print t}' /etc/passwd #$3 field count and
[root@centos7 ~]# awk -F':''{if ($1=="root") print $0}' /etc/passwd #output the line where $1 is root
The content in this section is basic. The command examples above can satisfy daily use. If you want more complex and powerful functions, you can go to the official website of the command tool. Especially awk, there are professional books introducing this command tool. Familiarity with commands is the prerequisite for writing scripts. Before writing a script, you must first know the execution results of each command, cooperate with the judgment loop statement, and realize the complex processing function of batches.