9.6 awk (on)
Awk is actually a programming language called a style scanning and processing language. Simply put, awk reads the file line by line, slices each line with a space as the default delimiter, and performs various analysis and processing on the cut part.
Command format: awk [options] ' mode = ({built-in variable} + expression + {action}) ' filename
The expression can be a regular expression: the search keywords in it are enclosed in / / signs, such as /root/. Where "pattern" means what awk looks for in the data, regular expressions and actions. Curly braces ({ }) do not need to appear all the time in a program, but they are used to group a sequence of instructions according to a specific pattern.
The most basic function of awk language is to browse and extract information based on specified rules in files or strings. After awk extracts information, other text operations can be performed. Complete awk scripts are often used to format information in text files.
Usually, awk is a line processing unit of the file. awk receives each line of the file, and then executes the corresponding command to process the text.
A commonly used awk function is to specify a delimiter, segment the lines of a file, and then process them.
awk -F ':' '{print $1}' 1.txt : Specify several paragraphs separated by :, and print the first paragraph.
-F Options to specify delimiters: $1 for the first segment, $0 for all segments.
[root@lgs-01 awk]# awk -F ':' '{print $1}' passwdtest
root
bin
daemon
adm
lp
sync
shutdown
halt
mail
operator
games
ftp
nobody
systemd-network
dbus
polkitd
postfix
sshd
chrony
lgs
abc
[root@lgs-01 awk]# awk -F ':' '{print $0}' passwdtest
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
polkitd:x:999:997:User for polkitd:/:/sbin/nologin
postfix:x:89:89::/var/spool/postfix:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
chrony:x:998:996::/var/lib/chrony:/sbin/nologin
lgs:x:1008:1000::/home/lgs:/bin/bash
abc:x:1001:1000::/home/abc:/bin/bash
Do not specify the delimiter with -F, the default is "space" or "whitespace" as the delimiter.
awk -F ':' '{print $2,$6,$7}': print multiple segments, separated by commas, or separated by #
[root@lgs-01 awk]# awk -F ':' '{print $2,$6,$7}' passwdtest
x /root /bin/bash
x /bin /sbin/nologin
x /sbin /sbin/nologin
x /var/adm /sbin/nologin
x /var/spool/lpd /sbin/nologin
x /sbin /bin/sync
x /sbin /sbin/shutdown
x /sbin /sbin/halt
x /var/spool/mail /sbin/nologin
x /root /sbin/nologin
x /usr/games /sbin/nologin
x /var/ftp /sbin/nologin
x / /sbin/nologin
x / /sbin/nologin
x / /sbin/nologin
x / /sbin/nologin
x /var/spool/postfix /sbin/nologin
x /var/empty/sshd /sbin/nologin
x /var/lib/chrony /sbin/nologin
x /home/lgs /bin/bash
x /home/abc /bin/bash
awk -F ':' '$1 ~ /oo/' 1.txt : The line containing oo in the first paragraph. The ~ sign indicates a match that contains oo in the specified paragraph
[root@lgs-01 awk]# awk -F ':' '$1 ~ /oo/' passwdtest
root:x:0:0:root:/root:/bin/bash
awk -F ':' '$7~ /binn+/' 1.txt: supports regular expressions, no need to add escape characters.
[root@lgs-01 awk]# awk -F ':' '$7~ /binn+/' passwdtest
1:2:3:4:5:6:binnn
Supported operators:
awk -F ':' '$3>=200 {print $0}' 1.txt : If the third segment is greater than or equal to 0, print all segments of the matching line.
[root@lgs-01 awk]# awk -F ':' '$3>=200 {print $0}' passwdtest
polkitd:x:999:997:User for polkitd:/:/sbin/nologin
chrony:x:998:996::/var/lib/chrony:/sbin/nologin
lgs:x:1008:1000::/home/lgs:/bin/bash
abc:x:1001:1000::/home/abc:/bin/bash
The operation string should be enclosed in " " signs:
awk -F ':' '$7!="/sbin/nologin" {print $0}' 1.txt: If the 7th segment is not equal to sbin, print all segments of the matching line.
[root@lgs-01 awk]# awk -F ':' '$7!="/sbin/nologin" {print $0}' passwdtest
root:x:0:0:root:/root:/bin/bash
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
lgs:x:1008:1000::/home/lgs:/bin/bash
abc:x:1001:1000::/home/abc:/bin/bash
awk -F ':' '$3>$4' test.txt : Comparison of paragraph 3 and paragraph 4.
[root@lgs-01 awk]# awk -F ':' '$3>$4 {print $0}' passwdtest
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
operator:x:11:0:operator:/root:/sbin/nologin
polkitd:x:999:997:User for polkitd:/:/sbin/nologin
chrony:x:998:996::/var/lib/chrony:/sbin/nologin
lgs:x:1008:1000::/home/lgs:/bin/bash
abc:x:1001:1000::/home/abc:/bin/bash
12:22:55
Multiple conditions match:
awk -F ':' '$3>5 && $4<7' 1.txt: paragraph 3 is greater than 5, and paragraph 4 is less than 7
[root@lgs-01 ~]# awk -F ':' '$3>5 && $4<7' passwdtest
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
operator:x:11:0:operator:/root:/sbin/nologin
12:22:55
awk -F ':' '{OFS="#"} $3>1000 || $4<5 {print $1,$2,$3,$4} ' 1.txt: OFS variable, specify the separator # to print the specified segment
[root@lgs-01 ~]# awk -F ':' '{OFS="#"} $3>1000 || $4<5 {print $1,$2,$3,$4}' passwdtest
root#x#0#0
bin#x#1#1
daemon#x#2#2
adm#x#3#4
sync#x#5#0
shutdown#x#6#0
halt#x#7#0
operator#x#11#0
lgs#x#1008#1000
abc#x#1001#1000
12#22#55#
587#1##
12#21##
12#200##
1###
9###
51###
59###
abc#0##
abc#y##
9.7 awk (below)
awk -F ':' '{print NR ": " $0}' 1.txt: NR variable, display line number.
[root@lgs-01 ~]# awk -F ':' '{print NR ": " $0}' passwdtest
1: root:x:0:0:root:/root:/bin/bash
2: bin:x:1:1:bin:/bin:/sbin/nologin
3: daemon:x:2:2:daemon:/sbin:/sbin/nologin
4: adm:x:3:4:adm:/var/adm:/sbin/nologin
5: lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
6: sync:x:5:0:sync:/sbin:/bin/sync
7: shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
8: halt:x:7:0:halt:/sbin:/sbin/halt
9: mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
10: operator:x:11:0:operator:/root:/sbin/nologin
11: games:x:12:100:games:/usr/games:/sbin/nologin
12: ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
13: nobody:x:99:99:Nobody:/:/sbin/nologin
14: systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
15: dbus:x:81:81:System message bus:/:/sbin/nologin
16: polkitd:x:999:997:User for polkitd:/:/sbin/nologin
17: postfix:x:89:89::/var/spool/postfix:/sbin/nologin
18: sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
19: chrony:x:998:996::/var/lib/chrony:/sbin/nologin
20: lgs:x:1008:1000::/home/lgs:/bin/bash
21: abc:x:1001:1000::/home/abc:/bin/bash
awk -F ':' 'NF==7 && $1 ~/root|sync/' 1.txt: NF variable, the number of segments per line. When NF is equal to 7, and the first paragraph contains root or a, the output prints the matching line.
[root@lgs-01 ~]# awk -F ':' 'NF==7 && $1 ~/root|a/' passwdtest
root:x:0:0:root:/root:/bin/bash
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
abc:x:1001:1000::/home/abc:/bin/bash
awk -F ':' '{print $NR ":" $NF}' 1.txt: Print the NR-th paragraph of each line: the character pair of the NF-th paragraph. NR, NF is a numeric variable, which is the line number and segment number of each line.
When the $ sign is not added before NR and NF: NR : NF, the output is the line number of each line: the number of segments
[root@lgs-01 ~]# awk -F ':' '{print $NR ": " $NF } ' passwdtest
root: /bin/bash
x: /sbin/nologin
2: /sbin/nologin
4: /sbin/nologin
lp: /sbin/nologin
/sbin: /bin/sync
/sbin/shutdown: /sbin/shutdown
: /sbin/halt
: /sbin/nologin
: /sbin/nologin
: /sbin/nologin
: /sbin/nologin
: /sbin/nologin
: /sbin/nologin
: /sbin/nologin
: /sbin/nologin
: /sbin/nologin
: /sbin/nologin
: /sbin/nologin
: /bin/bash
: /bin/bash
[root@lgs-01 ~]# awk -F ':' '{print NR ": " NF } ' passwdtest
1: 7
2: 7
3: 7
4: 7
5: 7
6: 7
7: 7
8: 7
9: 7
10: 7
11: 7
12: 7
13: 7
14: 7
15: 7
16: 7
17: 7
18: 7
19: 7
20: 7
21: 7
awk -F ':' '$1="root"' 1.txt: = sign, meaning assignment. : Assign the first paragraph of all lines as root. In the output result after assignment, the separator will disappear, which can be defined with OFS=":".
[root@lgs-01 ~]# awk -F ':' '$1="root"' passwdtest
root x 0 0 root /root /bin/bash
root x 1 1 bin /bin /sbin/nologin
root x 2 2 daemon /sbin /sbin/nologin
root x 3 4 adm /var/adm /sbin/nologin
root x 4 7 lp /var/spool/lpd /sbin/nologin
root x 5 0 sync /sbin /bin/sync
root x 6 0 shutdown /sbin /sbin/shutdown
root x 7 0 halt /sbin /sbin/halt
root x 8 12 mail /var/spool/mail /sbin/nologin
root x 11 0 operator /root /sbin/nologin
root x 12 100 games /usr/games /sbin/nologin
root x 14 50 FTP User /var/ftp /sbin/nologin
root x 99 99 Nobody / /sbin/nologin
root x 192 192 systemd Network Management / /sbin/nologin
root x 81 81 System message bus / /sbin/nologin
root x 999 997 User for polkitd / /sbin/nologin
root x 89 89 /var/spool/postfix /sbin/nologin
root x 74 74 Privilege-separated SSH /var/empty/sshd /sbin/nologin
root x 998 996 /var/lib/chrony /sbin/nologin
root x 1008 1000 /home/lgs /bin/bash
root x 1001 1000 /home/abc /bin/bash
[root@lgs-01 ~]# awk -F ':' '{OFS="#" } $1="root"' passwdtest
root#x#0#0#root#/root#/bin/bash
root#x#1#1#bin#/bin#/sbin/nologin
root#x#2#2#daemon#/sbin#/sbin/nologin
root#x#3#4#adm#/var/adm#/sbin/nologin
root#x#4#7#lp#/var/spool/lpd#/sbin/nologin
root#x#5#0#sync#/sbin#/bin/sync
root#x#6#0#shutdown#/sbin#/sbin/shutdown
root#x#7#0#halt#/sbin#/sbin/halt
root#x#8#12#mail#/var/spool/mail#/sbin/nologin
root#x#11#0#operator#/root#/sbin/nologin
root#x#12#100#games#/usr/games#/sbin/nologin
root#x#14#50#FTP User#/var/ftp#/sbin/nologin
root#x#99#99#Nobody#/#/sbin/nologin
root#x#192#192#systemd Network Management#/#/sbin/nologin
root#x#81#81#System message bus#/#/sbin/nologin
root#x#999#997#User for polkitd#/#/sbin/nologin
root#x#89#89##/var/spool/postfix#/sbin/nologin
root#x#74#74#Privilege-separated SSH#/var/empty/sshd#/sbin/nologin
root#x#998#996##/var/lib/chrony#/sbin/nologin
root#x#1008#1000##/home/lgs#/bin/bash
root#x#1001#1000##/home/abc#/bin/bash
awk -F ':' '{(tot=tot+$3); END {print tot}}' 1.txt: sum a column. tot starts from 0, accumulates the contents of the third paragraph of each line until the end of the last line, and then outputs the value of toto. Where there are variables, use { } curly braces.
[root@lgs-01 ~]# awk -F ':' '{(tot=tot+$3)}; END {print tot}' passwdtest
4669