shell script text Three Musketeers AWK

1. AWK tool introduction

  • AWK is a language for manipulating text files and is a powerful text analysis tool
  • Complex text manipulation can be achieved in non-interactive mode
  • Compared with sed, which usually works on a whole line, awk is more inclined to divide a line into several fields, because awk is quite suitable for small text data

1.1 Basic format of AWK command

awk [选项] ‘模式条件{
    
    操作}’ 文件1 文件2...
awk -f|-v 脚本文件 文件1 文件.....

1.2 Working principle of AWK

  • As mentioned earlier, the sed command is often used to process a whole line, while awk tends to divide a line into multiple "fields" and then process them. By default, the field separator is a space or a tab key. The execution result of awk can print and display the field data through the print function.

  • In the process of using the awk command, you can use the logical operator "&&" to represent "and", "" to represent "or", "!" to represent "not"; you can also perform simple mathematical operations, such as +, -, * , /, %, ^ represent addition, subtraction, multiplication, division, remainder and power respectively.

  • Awk is followed by two single quotation marks and curly braces { } to set the processing operation you want to perform on the data. Awk can process subsequent files, and can also read the standard output from the previous command.

1.3 Common built-in variables

  • FS: column separator, specify the field separator of each line of text, the default is space or tab, the same as "-F"
  • NF: the number of fields in the currently processed row
  • NR: the row number (ordinal number) of the currently processed row
  • $0: the entire line content of the currently processed line
  • $n: the nth field (column n) of the currently processed row
  • FILENAME: the name of the file being processed
  • RS: line separator, when awk reads data from a file, it will cut the data into many records according to the definition of RS, and awk only reads one record at a time for processing, the default value is '\n'

2. Example

2.1 Print text content

  • Awk can automatically compress multiple spaces into one space
  • Printing strings requires double quotes

Example 1: Print disk usage

insert image description here

Example 2: print a string

insert image description here

Example 3: Print a string to determine how many lines a file has

insert image description here

2.2 Extract fields according to $n

Example 1: Extract IP address
$n, which means take the third column

insert image description here
Example 2: extract mac address

insert image description here

2.3 Specify the delimiter according to option -F

Example 1: Print all usernames in /etc/passwd

insert image description here

Example 2: Print multi-column content.
Commas can represent spaces when printing. If you use ":" or "+", you need to print special symbols with double quotes as strings

insert image description here

insert image description here
insert image description here
Example 3: Print the usage of the disk, remove the % sign

insert image description here
insert image description here

insert image description here
Example 4: Extract the IP address and time in the text

insert image description here

Example 5: Take the host in the text and put it back

insert image description here

2.4 Extract the row according to the keyword

Example 1: Extract /etc/passwd lines starting with root

insert image description here

Example 2: Extract the line where /etc/passwd root is located

insert image description here

Example 3: Extract lines ending in /etc/passwd nogoin

insert image description here

insert image description here

Example 4: Extract IP address

insert image description here

2.5 Use BEGIN to output lines containing specified characters and count how many lines there are

insert image description here

2.6FS: column segmentation extraction column

insert image description here

2.7NF:

Example 1: Number of fields in the currently processed row

insert image description here

Example 2: Print out the last field of each line

insert image description here
Example 3: Print out the second-to-last field on each line

insert image description here

insert image description here

2.8NR:

Example 1: The currently processed line number

insert image description here

insert image description here
Example 2: NR==n represents what the line number is equal to

insert image description here
insert image description here

Example 3: NR%2==0 takes even rows

insert image description here

Example 4: NR%2==1 takes odd rows
insert image description here

Example 5: NR==1, NR==4 take interval rows

insert image description here

Example 5: Take UID value range $n>1000
insert image description here

3. Examples

1. Count the number of occurrences of each file system type in the /etc/fstab file

insert image description here

2. Extract the hostname and put it in the original file

insert image description here

3... Count the number of occurrences of each word in the /etc/fstab file

insert image description here

4. Extract all the numbers in the string Yd$C@M05MB%9&Bdh7dq+YVixp3vpw

insert image description here

Guess you like

Origin blog.csdn.net/fyb012811/article/details/132378755