(4) awk of the Three Musketeers

(1) How awk works
#awk -F: '{print $1,$3}' /etc/passwd
1) awk uses one line as input, and assigns this line to the internal variable $0, each line can also be called a record, Ended with a newline
2) Then the line is split into fields by:, each field is stored in a numbered variable, starting at $1, up to 100 fields
3) How does awk know to separate fields with spaces, because there is a Internal variable FS to determine the field separator. Initially, FS is given a space
. 4) When awk prints a field, it will be printed using the print function in the set method. Awk adds a space between the print fields, because there is a comma between $1 and $3. The comma is special, and it maps to another An internal variable called the output field separator OFS, OFS defaults to space
5) After awk output, another line will be taken from the file and stored in $0, overwriting the original, then splitting the new string The process will continue until all lines are processed
(2) awk syntax format
1) syntax format : awk [options] 'command' filename
2) options
-F: define the input field separator, the default separator is a space or a tab (tab key)
3) command
BEGIN{} is usually used to define some variables, such as BEGIN{FS=":";OFS="----"}
BEGIN{} {} END{}
line After processing the previous processing line
# awk 'BEGIN{print 1/2}{print "ok"}END{print "-------"}'


ok
-------

4) Command format:
awk 'pattern' filename Example: awk -F:'/root/' /etc/passwd
awk '{action}' filename Example: awk -F: '/root/ {$1,$3}' /etc/passwd
awk 'pattern {action}' Example: awk -F: '/root/{print $1,$3}' /etc/passwd
awk 'BEGIN{FS=":"}/root /{print $1,$3}' /etc/passwd
command | awk 'pattern{action}' Example: df -P | grep "/" | awk '$4 > 25000 {print $0}'

(3) Internal variable
$0 : integer Example of line record: awk -F: '{print $0}' passwd
NR : Add a serial number at the end of the line Example: awk -F: '{print $0,NR}' passwd
FNR : Add a serial number at the beginning of the line Example: awk -F: '{print FNR,$0}' passwd
NF : count the number of fields per line Example: awk -F: '{print $0,NF}'passwd
$NF : last field of line Example: awk -F: '{print $NF }' passwd awk -F: '{print $(NF-1)}' passwd
FS: Input field separator Example: awk 'BEGIN{FS=":"}{print $1,$3}' /etc/passwd
　　awk -F'[ :\t]' '/root/{print $1,$3}' / etc/passwd
　　awk -F: '/root/{print $1,$3}' /etc/passwd
OFS : Output field separator Example: awk -F: '{print $1,$2,$3,$4}' /etc/passwd \\The default comma is a space
　　awk 'BEGIN{FS=":";OFS="+++"}{print $1,$2,$3,$4}' /etc/passwd
ORS : Output line separator Example: awk 'BEGIN {ORS=" "}{print $0}' passwd \\Merge files into one line, ORS defaults to output a record should be a carriage return, here is changed to output a space instead of a newline
RS : Input line separator example: # cat /etc/passwd | head -1 >passwd1
　　awk 'BEGIN{RS=":"}{print $0}'passwd1 \\ output a line of content into multiple lines according to the delimiter

Summary:
Field delimiter: FS OFS default space or tab
line delimiter: RS ORS default newline

(4) Formatted output
1) print function
date | awk '{print "Mount:"$2 "\nYear:" $NF}'
awk -F: '{print "Username:" $1 "\tUid:" $3}' /etc/passwd
awk -F: '{print "Username,Uid:" $1,$3"!"}' /etc/passwd

2 ) printf function
awk -F: '{printf "%-15s %-10s %-14s\n",$1,$2,$3}' /etc/passwd
%s: character type
%d: numeric type
%f: floating point Type
15: Indicates 15 characters
- : Indicates left alignment, the default is right alignment,
printf will not automatically wrap at the end of the line by default, add \n

(5) awk mode and action
Any awk statement is composed of mode and action, the mode part Determine when the action statement is triggered, the pattern can be regular expression, conditional statement, compound statement
1) Regular expression
Match the entire line:
#awk '/^root/' /etc/passwd
# awk '!/^root/' / etc/passwd
match field: operator (~ or ~!)
# awk -F: '$NF !~ /^bash/' /etc/passwd
# awk -F: '$1 ~ /^root/' /etc/passwd

2) Relational operators : < <= == != >=> String comparison requires double quotes
awk -F: '$3 == 0' /etc/passwd
awk -F: '$3 < 10' /etc/passwd
awk -F: '$7 == "/bin/bash"' /etc/passwd
awk -F: '$1 ~ /root/' /etc/passwd
awk -F : '$1 !~ /root/' /etc/passwd
df -P | awk '$4 > 25000'
3) Conditional expression :
awk -F: '$3>300{print $0}' /etc/passwd
awk -F: '{if($3>300)print $0}' /etc/passwd
awk -F: '{if ($3>300){print $0}}' /etc/passwd
awk -F: '{if($3>300) {print $3}else{print$1}}' /etc/passwd
4) Arithmetic : + - * / % ^
awk -F: '{if($3*10 > 140 ){print $0}}' /etc/passwd
5) Logical operators: && and || or !
awk '$1~/root/ && $3<15' /etc/passwd
awk '!($1~/root/ && $3<15)' /etc/passwd
(6) Conditional judgment of awk script programming
1) if statement Syntax format: {if(expression){statement;statement;...}}
awk -F":" '{if($3 == 0){print $1,"is administrator!"}}' /etc/passwd
awk -F":" '{if($3>0 && $3<1000){ count++;}} END{print count}' /etc/passwd \\ count system users
2) if..else statement
Syntax format: {if(expression){statement;} else{statement;}}
awk -F" :" '{if($3 == 0){count++;}else{i++;}} END{print "Number of administrators:"count;print "Number of system users:"i}' /etc/passwd
3)if ..else if..else statement {if(expression){statement} else if(expression){statement} else{statement}}
awk -F":" '{if($3 == 0){count++;} else if($3>0 && $3<1000){i++;} else{j++}} END{print "Number of administrators:"count;print "Number of system users:"i;print "Number of ordinary users:"j} ' /etc/passwd
(7) while loop in awk script programming
1) Syntax: {initial value;while(condition){statement;statement}}
awk 'BEGIN{i=1;while(i<=10){print i;i++}}' \\打印1到10
awk -F":" '{i=1;while(i<=7){print $i;i++}}' /etc/passwd
awk '{i=1;while(i<=NF){print $i;i++}}' /etc/hosts \\Print the fields of each line

(8) for loop in awk script programming
1) Syntax: {for( condition){statement}}
awk 'BEGIN{for(i=1;i<=10;i++){print i}}'
awk '{for(i=1;i<=NF;i++){print $i} }' /etc/hosts is equivalent to awk '{i=1;while(i<=NF){print $i;i++}}' /etc/hosts

(9) Array of awk script programming: use the object to be traversed as Array index
awk '{array name[traversal object]++} END{for(i in array name){print i,array name[i]}}'

Statistical shell type: awk -F":" '{shells[ $NF]++} END{for(i in shells){print i,shells[i]}}' /etc/passwd
statistics website status: netstat -ant | grep '\b192.168.1.253:80\b' | awk '{status[$NF]++} END{for(i in status){print i,status[i]}}'
ss -an | grep "\b192.168.1.2:80\b" | awk '{status[$1]++} END{for(i in status){print i,status[i]}}'
Count the number of each ip currently accessed:
netstat -ant| grep "\b192.168.1.2:80\b" | awk -F '[ :]+' '{ip_count[$6]++} END{for(i in ip_count){print i,ip_count[ i]}}'| sort -k2 -rn

(10) awk function:
statistical username is 4 characters
#awk -F: '$1 ~ /^....$/{count++;print $1} END{print " count is:"count}' /etc/passwd
#awk -F: 'length($1)==4{count++;print $1} END{print "count is:"count}' /etc/passwd
(11) awk reference External variable
Print the mounted partition using more than 10% of the capacity, where int($5) is an integer
df -h | awk -vi=10 '{if(int($5) > i){print $6":"$5}} '