Awk of the Shell Three Musketeers

---

Brief introduction of awk

In Linux/UNIX systems, awk is a powerful editing tool that reads input text line by line, uses space or tab as the delimiter by default, and executes editing commands by mode or condition.
The reading of AWK information is also to search for the specified matching pattern line by line, format the output or filter the content that meets the conditions, and can realize quite complex text operations without interaction. It is widely used in Shell scripts, complete Various automated configuration tasks.

The working process of awk

Step 1: Execute the statements in the BEGIN{action;...} statement block.

The second step: read a line from the file or standard input (stdin), and then execute the pattern { action;... } statement block, which scans the file line by line, repeating this process from the first line to the last line, until the file is all read complete.

Step 3: When reading to the end of the input stream, execute the END{action;...} statement block.

The BEGIN statement block is executed before awk starts to read lines from the input stream. This is an optional statement block. Statements such as variable initialization and printing output table headers can usually be written in the BEGIN statement block.

The END statement block is executed after awk reads all the lines from the input stream, such as printing the analysis results of all lines, and such information summary is completed in the END statement block, which is also an optional statement block.

Common commands in the pattern block are the most important part and are optional. If no pattern statement block is provided, { print } will be executed by default, that is, each read line will be printed, and this statement block will be executed for each line read by awk.

How awk works

The sed command is often used to process a whole line, while awk tends to divide a line into multiple "fields" and then process them. By default, the field separator is a space or a tab key.
The awk execution result can print and display the field data through the print function

Basic format of awk and its built-in variables

awk option 'pattern or condition {action}' file1 file2...

-F "separator" indicates the field separator used for input, the default separator is several consecutive blank characters
-v (small v) var=value variable assignment

Note that it must be a single quotation mark: 'pattern or condition {operation}'
specify the condition outside { }, and specify the operation inside { }.
Use commas to specify consecutive lines and || to specify discontinuous lines. && means "and".
Built-in variables cannot be enclosed in double quotes, otherwise the system will treat it as a string.

内置变量
$0:	当前处理的行的整行内容 
$n:	当前处理行的第n个字段（第n列）
NR:	当前处理的行的行号（序数）
NF:	当前处理的行的字段个数。$NF代表最后一个字段
FS:	列分割符。指定每行文本的字段分隔符，输入内容的分隔符，
默认为空格或制表位。与"-F"作用相同 用-F可以不加单引号 -F:，用FS必须用=""
OFS:输出内容的列分隔符
FILENAME:被处理的文件名
RS:	行分隔符。awk从文件中读取资料时，
将根据RS的定义把资料切割成许多条记录，而awk一次仅读入一条记录进行处理。预设值是"\n"

0 and 1 can be used to limit the printing content before placing { } (the default is "1"), if it is 0, the content will not be printed

getline

The working process of getline

1. When there is no redirection symbol ("<") or pipe symbol ("|") around getline, awk first reads the first line, and getline gets the content that the cursor jumps to the next line (that is, second line).

2. When there are pipe symbols or redirection symbols on the left and right of getline, getline acts as a directional input file. Since the file is just opened, it is not read into a line by awk, but only read by getline, so getline returns the first part of the file. line instead of jumping to a line of input.

Reason: After getline runs, awk will change internal variables such as NF, NR, $0, and FNR, so the line number that reads $0 is no longer 1, but 2.

Pass the output of ls to the getline function, line is a variable, output the content of ls to the variable, and then print out the result, if there is no content, do nothing

Text content matching filter printing

BEGIN print mode:
format: awk 'BEGIN{…};{…};END{…}' file
processing process:
1. Before awk processes the specified text, it is necessary to execute the command operation in the BEGIN{…} mode
2. The {…} in the middle is the command operation actually used to process the file.
3. The command operation in the END{…} mode will be executed after awk finishes processing the file. In the END { } statement block, statements such as printing results are often placed.

Print the fields

Usage of -v: variable assignment

The fs is: then use -v to assign a value to FS =:, when inputting, FS is:, when -v assigns a value to OFS to output, the variable is +, and then print the first and third columns

Conditional judgment print

Awk's ternary expression and precise screening usage

The ternary expression of awk inherits the usage of java, and the format is similar to that of Java

Format: awk '(conditional expression)?(A expression or value):(B expression or value)'

Accurate screening of awk:

$n(> < ==):  用于对比数值
$n~"字符串": 代表第n个字段包含某个字符串
$n!~"字符串": 代表第n个字段不包含某个字符串
$n=="字符串": 代表第n个字段为某个字符串
$n!="字符串": 代表第n个字段不为某个字符串
$NF: 代表最后一个字段

awk delimiter usage

RS specifies the line separator:
when awk reads data from a file, it will cut the data into many records according to the definition of RS,
and awk only reads one record at a time for processing. The default value of the built-in variable RS is "\n" which is a newline.
You can also use the BEGIN mode to change the line separator before the operation

Specifies the delimiter for the output

OFS : The column separator for the output content.

（$n=$n用于激活，否则不生效，n必须存在）

Awk combined with the use of arrays

Define array printing in awk:

The array in awk forms a traversal

interview questions

Monitor memory script (alarm when memory usage > 90%)

Mem_total=$(free -m | grep -w Mem | awk '{print $2}')
Mem_used=$(free -m | grep -w Mem | awk '{print $3}')

a=$(echo "scale=2;($Mem_used/$Mem_total)*100"|bc)
Mem=`echo $a|awk -F. '{print $1}'`
if [ $Mem -gt 90 ]
then
   echo "内存使用超过90%，请清理内存"
else
   echo "系统安全，请放心使用，内存使用情况：$Mem%"
fi

Monitor CPU usage script

top -b -n 1 | awk 'NR==3{print}'| awk '{print "目前CPU的使用情况："$2+$4}'

Monitor disk usage script (alarm when disk usage > 90%)

 a=$(df -T | awk 'NR==2{print}' | awk '{print $4}')

 b=$(df -T | awk 'NR==2{print}' | awk '{print $5}')
 c=$(echo "scale=2;($a/$b)*100"|bc)

sum=`echo $c |awk -F. '{print $1}'`
if [ $sum -gt 90 ]
then
   echo "磁盘使用超过90%，请清理磁盘"
else
   echo "系统安全，请放心使用，磁盘使用情况：$sum%"
fi