Shell scripts Three Musketeers (awk, sort, uniq) contains a number of small Demo

awk tool

In Linux / UNIX systems, awk is a powerful tool for editing, the input text is read line by line, and to find the matching according to the specified pattern to meet the requirements of the output format or content filtering processing, can no longer interact the case of implementing fairly complex text manipulation, are widely used in Shell script to complete a variety of automated configuration tasks.

1.awk common usage:
Syntax awk normally used as shown below, wherein the single quotation marks plus braces "{}" is provided for processing the data of the operation. awk can deal directly with the target file, the target file can be processed by the "-f" read the script.

Demo1：

Find out / etc / passwd user names and other column, execute the following command

[root@localhost ~]# awk -F ':' '{print $1,$3,$4}' /etc/passwd
root 0 0
bin 1 1
daemon 2 2
//以下省略多行

awk contains several special built-in variables (which can be directly used) as follows:

FS：指定每行文本的字段分隔符，默认为空格或制表位。
NF：当前处理的行的字段个数。
NR：当前处理的行的行号（序数）。
$0：当前处理的行的整行内容。
$n：当前处理行的第 n 个字段（第 n 列）。
FILENAME：被处理的文件名。
RS：数据记录分隔，默认为\n，即每行为一条记录。

Row output text:

[root@localhost ~]# awk 'NR==1,NR==3{print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin

[root@localhost ~]# awk -F ":" 'NR==1,NR==3{print $1,$3}' /etc/passwd
root 0
bin 1
daemon 2

[root@localhost ~]# awk -F ":" '(NR>=1)&&(NR<=3){print $1,$3}' /etc/passwd
root 0
bin 1
daemon 2

The contents of all parity output lines:

奇数行：
[root@localhost ~]# awk -F ":" 'NR%2==1{print $1,$3}' /etc/passwd
root 0
daemon 2
lp 4
shutdown 6
mail 8
games 12
nobody 99
dbus 81
abrt 173
rpc 32
saslauth 996
pulse 171
rpcuser 29
ntp 38
usbmuxd 113
qemu 107
setroubleshoot 993
gdm 42
sshd 74
postfix 89
zhou 1000
named 25
偶数行：
[root@localhost ~]# awk -F ":" 'NR%2==0{print $1,$3}' /etc/passwd
bin 1
adm 3
sync 5
halt 7
operator 11
ftp 14
systemd-network 192
polkitd 999
libstoragemgmt 998
colord 997
rtkit 172
chrony 995
nfsnobody 65534
tss 59
geoclue 994
radvd 75
sssd 992
gnome-initial-setup 991
avahi 70
tcpdump 72
dhcpd 177

Output lines beginning with root:

[root@localhost ~]# awk '/^root/{print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash

Count the number of lines to / bin / bash at the end of:

[root@localhost ~]# awk 'BEGIN {x=0} ; /\/bin\/bash$/{x++};END {print x}' /etc/passwd
2

Statistics paragraphs of text separated by blank line:

[root@localhost opt]# vim name.txt
zhangsan:lisi:wangwu
zhaoliu:liuliu
heiba:heihei
[root@localhost opt]# awk 'BEGIN{RS=":"};END{print NR}' /opt/name.txt
5
//统计规则：遇到关键符号，折行

Output fields:

[root@localhost opt]# awk -F ":" '$3=="0"{print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash

[root@localhost opt]# awk -F ":" '$3=="0"{print $1,$7}' /etc/passwd
root /bin/bash

输出密码为空的用户的shadow记录：
[root@localhost opt]# awk 'BEGIN{FS=":"};$3=="0"{print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash

输出以冒号分隔且第7个字段包含bash：
[root@localhost opt]# awk -F : '$7~"bash"{print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
zhou:x:1000:1000:zhou:/home/zhou:/bin/bash

以冒号为分隔，输出第一个字段包含nfs且总字段数为7的第1，3，7字段
方法一：[root@localhost opt]# awk -F : '($1~"nfs")&&(NF==7){print $1,$3,$7}' /etc/passwd
nfsnobody 65534 /sbin/nologin
方法二：[root@localhost opt]# awk 'BEGIN{FS=":"};($1~"nfs")&&(NF==7){print $1,$3,$7}' /etc/passwd
nfsnobody 65534 /sbin/nologin

以冒号为分隔，第七个字段不包含/sbin/nologin且总字段为7的，1，3，7字段
[root@localhost opt]# awk 'BEGIN{FS=":"};($7!="/sbin/nologin")&&(NF==7){print $1,$3,$7}' /etc/passwd
root 0 /bin/bash
sync 5 /bin/sync
shutdown 6 /sbin/shutdown
halt 7 /sbin/halt
zhou 1000 /bin/bash
named 25 /bin/false

Wc -l command to count the number of calls the user's bash:

[root@localhost opt]# awk -F : '/bash$/{print | "wc -l"}' /etc/passwd
2

Call w command and used to count the number of online users:

[root@localhost opt]# awk 'BEGIN {while ("w" | getline) n++;{print n-2}}'
1

Call hostname, and the output current host name:

[root@localhost opt]# awk 'BEGIN {"hostname" | getline ; print $0}'
localhost.localdomain

sort tools

On Linux systems, commonly used file sorting tool, there are three: sort, uniq, wc, which is a sort tool in units of the contents of the file to be sorted, it can also be sorted according to different data types. Such as data and character Board is not the same. The syntax sort command is "sort [options] parameters", which are commonly used options include the following.

-f: ignore case

-b: Ignore spaces in front of each row

-M: Sort by month

-n: sort numerically

-r: reverse sequencing

-u: equivalent to uniq, showing the same data as the single line

-t: Specifies delimiter, using [Tab] key default partition

-o <output file>: The sorted results to the file specified dump

-k: Specifies the sorting area

uniq Tools

Uniq tool often used in conjunction with the sort command in Linux systems, for reporting or ignore file duplicates. Specific command syntax is: uniq [options] parameters. Which include the following common options.

-c: count the

-d: show only duplicate lines

-u: display only appears once in row