[Linux Command Explanation Encyclopedia] 050.awk built-in variable usage method and various operators detailed analysis

awk built-in variables (predefined variables)

Explanation: [A][N][P][G] indicates the first tool that supports variables, [A]=awk, [N]=nawk, [P]=POSIXawk, [G]=gawk

• $n The nth field of the current record, for example, if n is 1, it means the first field, and if n is 2, it means the second field.
• $0 This variable contains the text content of the current line during execution.
• [N] Number of ARGC command line arguments.
• [G] The position of the current file in the ARGIND command line (counting from 0).
• [N] ARGV Array containing command line arguments.
• [G] CONVFMT digital conversion format (default %.6g).
• [P] ENVIRON environment variable associative array.
• [N] ERRNO Description of the last system error.
• [G] FIELDWIDTHS List of field widths (space separated).
• [A] FILENAME The name of the current input file.
• [P] FNR is the same as NR, but relative to the current file.
• [A] FS field separator (default is any space).
• [G] IGNORECASE If true, match regardless of case.
• [A] NF indicates the number of fields, which corresponds to the current number of fields during execution.
• [A] NR represents the number of records, which corresponds to the current line number during execution.
• [A] Output format of OFMT figures (default is %.6g).
• [A] OFS output field separator (default is a space).
• [A] ORS output record separator (default is a newline).
• [A] RS record separator (default is a newline).
• [N] RSTART The first position of the string matched by the match function.
• [N] RLENGTH The length of the string matched by the match function.
• [N] SUBSEP array subscript separator (default value is 34).

escape sequence

• \ \ itself
• $ escape $
• \t tab character
• \b backspace character
• \r carriage return
• \n line break
• \c cancel line break

example

echo -e "line1 f2 f3\nline2 f4 f5\nline3 f6 f7" | awk '{print "Line No:"NR", No of fields:"NF, "$0="$0, "$1="$1, "$2="$2, "$3="$3}'

Output result:

Line No:1, No of fields:3 $0=line1 f2 f3 $1=line1 $2=f2 $3=f3
Line No:2, No of fields:3 $0=line2 f4 f5 $1=line2 $2=f4 $3=f5
Line No:3, No of fields:3 $0=line3 f6 f7 $1=line3 $2=f6 $3=f7

Use print $NFcan\; print\; out\; the\; last\; field\; in\; a\; line,\; use$ (NF-1) to print the penultimate field, and so on:

echo -e "line1 f2 f3\n line2 f4 f5" | awk '{print $NF}'

Output result:

f3
f5

echo -e "line1 f2 f3\n line2 f4 f5" | awk '{print $(NF-1)}'

Output result:

f2
f4

Print the second and third fields of each line:

awk '{ print $2,$3 }' filename

Count the number of lines in a file:

awk 'END{ print NR }' filename

The above command only uses the END statement block. When each line is read, awk will update NR to the corresponding line number. When the last line is reached, the value of NR is the line number of the last line, so the NR in the END statement block is The number of lines in the file.

An example of accumulating the first field value in each row:

seq 5 | awk 'BEGIN{ sum=0; print "总和：" } { print $1"+"; sum+=$1 } END{ print "等于"; print sum }'

Output result:

总和：
1+
2+
3+
4+
5+
等于
15

Pass external variable value to awk

External values ​​(not from stdin) can be passed to awk with the -v option:

VAR=10000
echo | awk -v VARIABLE=$VAR '{ print VARIABLE }'

Another way to pass external variables:

var1="aaa"
var2="bbb"
echo | awk '{ print v1,v2 }' v1=$var1 v2=$var2

Use when input is from a file:

awk '{ print v1,v2 }' v1=$var1 v2=$var2 filename

Find the process pid

netstat -antup | grep 7770 | awk '{ print $NF NR}' | awk '{ print $1}'

awk operation and judgment

As one of the characteristics that a programming language should have, awk supports a variety of operations, which are basically the same as those provided by the C language. Awk also provides a series of built-in operation functions (such as log, sqr, cos, sin, etc.) and some functions (such as length, substr, etc.) for string operations (operations). The reference of these functions has greatly improved the operation function of awk. As a part of conditional transfer instructions, relationship judgment is a function of every programming language, and awk is no exception. Awk allows multiple tests. As a style match, it also provides pattern matching expressions (matching) and _! (Mismatch). As an extension to testing, awk also supports logical operators.

arithmetic operator

operator description

• • add, subtract

• / & multiplication, division and remainder

• • ! Unary addition, subtraction and logical negation
    ^ *** Exponentiation
    ++ – increase or decrease, as prefix or suffix

example:

awk 'BEGIN{a="b";print a++,++a;}'

Output result:

0 2

Note: All operations are used as arithmetic operators, the operands are automatically converted to values, and all non-values ​​are turned into 0

assignment operator

Operator description
= += -= *= /= %= ^= **= assignment statement

example:

a+=5; 等价于：a=a+5; 其它同类

Logical Operators

Operator Description
|| Logical Or
&& Logical And

example:

awk 'BEGIN{a=1;b=2;print (a>5 && b<=2),(a>5 || b<=2);}'

Output result:

0 1

regular operator

operator description

!~ matches regular expression and does not match regular expression
^ start of line
$ end of line
. Any single character except newline

• Zero or more of the leading characters
  .* All characters
  [] Any character in the character group
  [^] Negates each character in the character group (does not match every character in the character group)
  ^[ ] Non-character group Lines beginning with characters within
  [az] lowercase letters [ AZ
  ] uppercase letters
  [aZ] lowercase and uppercase letters
  [0-9] numbers The end of the word regular needs to be surrounded by /regular/
  
  
  

example:

awk 'BEGIN{a="100testa";if(a ~ /^100*/){print "ok";}}'

Output result:

ok

relational operator

Operator Description
< <= > >= != == relational operators

example:

awk 'BEGIN{a=11;if(a >= 9){print "ok";}}'

Output result:

ok

Note: > < can be used as a string comparison or a numerical comparison. The key is that if the operand is a string, it will be converted to a string comparison. Both are numbers before converting to numerical comparison. String comparison: compare in ASCII order.

other operators

Operator Description
$ Field Reference
Space String Connector
?: C Conditional Expression
in Whether a certain key exists in the array

example:

awk 'BEGIN{a="b";print a=="b"?"ok":"err";}'

Output result:

ok

awk 'BEGIN{a="b";arr[0]="b";arr[1]="c";print (a in arr);}'

Output result:

0

awk 'BEGIN{a="b";arr[0]="b";arr["b"]="c";print (a in arr);}'

Output result:

1

Operational Priority Table

!The higher the level, the higher the priority
The higher the level, the higher the priority

Awk advanced input and output
Read the next record The
next statement in awk is used: match line by line in the loop, if next is encountered, the current line will be skipped, and the following statement will be ignored directly. Instead, the next line is matched. The next statement is generally used for multi-line merging:

cat text.txt
a
b
c
d
e

awk 'NR%2==1{next}{print NR,$0;}' text.txt

When the record line number is divided by 2 and the remainder is 1, the current line is skipped. The following print NR, $0 will not be executed either. At the beginning of the next line, the program begins to judge the value of NR%2. At this time, the record line number is: 2, and the following statement block will be executed: 'print NR,$0'

The analysis found that the line containing "web" needs to be skipped, and then the content needs to be merged into one line with the following line:

cat text.txt
web01[192.168.2.100]
httpd            ok
tomcat               ok
sendmail               ok
web02[192.168.2.101]
httpd            ok
postfix               ok
web03[192.168.2.102]
mysqld            ok
httpd               ok
0

awk '/^web/{T=$0;next;}{print T":"t,$0;}' text.txt

Simply read a record

Awk getline usage: The getline function is required for output redirection. getline takes input from standard input, a pipe, or another input file other than the one currently being processed. It is responsible for getting the content of the next line from the input and assigning values ​​to built-in variables such as NF, NR and FNR. The getline function returns 1 if a record is obtained, 0 if the end of the file is reached, and -1 if an error occurs, such as a failure to open the file.

getline syntax: getline var, the variable var contains the content of a specific line.

Awk getline as a whole, usage instructions:

• When there is no redirection character | or < around it: getline acts on the current file, and reads the first line of the current file to the following variable var or $0 (no variable). It should be noted that since awk has already processed getline A line is read, so the return result of getline is interlaced.
• When there are redirection characters | or < on the left and right: getline acts on the directional input file. Since the file is just opened, it is not read into a line by awk, but getline reads in, then getline returns the first line of the file. line, not every other line.

Example:

Execute the date command of linux, and output it to getline through the pipeline, then assign the output to the custom variable out, and print it:

awk 'BEGIN{ "date" | getline out; print out }' test

Execute the date command of the shell and output it to getline through the pipeline, then getline reads from the pipeline and assigns the input to out, the split function converts the variable out into an array mon, and then prints the second element of the array mon:

awk 'BEGIN{ "date" | getline out; split(out,mon); print mon[2] }' test

The output of the command ls is passed to geline as input, and the loop causes getline to read a line from the output of ls and print it to the screen. There is no input file here, because the BEGIN block executes before opening the input file, so the input file can be ignored.

awk 'BEGIN{ while( "ls" | getline) print }'

close file

Awk allows an input or output file to be closed in a program by using the awk close statement.

close("filename")

filename can be the file opened by getline, or stdin, a variable containing the filename, or the exact command used by getline. or an output file, which can be stdout, a variable containing the filename or the exact command using a pipe.

Output to a file
awk allows outputting results to a file as follows:

echo | awk '{printf("hello word!n") > "datafile"}'
# 或
echo | awk '{printf("hello word!n") >> "datafile"}'

Learn from scratchpython

[Learn python from zero] 92. Use Python's requests library to send HTTP requests and process responses
[Learn python from zero] 91. Use decorators and dictionaries to manage simple web applications for request paths
[Learn python from zero] 93. Use dictionary management Request path
[Learn python from zero] 89. Use WSGI to build a simple and efficient web server
[Learn python from zero] 88. Detailed explanation of WSGI interface: realize simple and efficient web development
[Learn python from zero] 87. Manually build HTTP server Python Implementation and multi-threaded concurrent processing
[Learn python from zero] 86. In-depth understanding of HTTP protocol and its role in browser and server communication
[Learn python from zero] 85. Parallel computing technology application of Python process pool
[Learn python from zero ] 84. In-depth understanding of threads and processes
[learning python from zero] 83. Python multi-process programming and the use of process pools
[learning python from zero] 82. Realization of chat programs based on multi-threading
[learning python from zero] 81. Python and more Application of thread communication and queue
[learning python from zero] 80. Thread access to global variables and thread safety issues
[learning python from zero] 79. Thread access to global variables and thread security issues
[learning python from zero] 78. File download cases
[ Learning python from zero] 77. TCP server programming and precautions
[learning python from zero] 76. Server and client: key components of network communication
[learning python from zero] 75. TCP protocol: reliable connection-oriented transmission layer communication protocol
[Learn python from zero] 74. UDP network program: detailed explanation of port problems and binding information
[Learn python from zero] 73. UDP network program - send data
[Learn python from zero] 72. In-depth understanding of Socket communication and socket creation The method of
[learning python from zero] 71. Network ports and their functions
[learning python from zero] 70. Network communication methods and their applications: from direct communication to routers to connect multiple networks
[learning python from zero] 69. Network communication and IP address classification analysis
[learning python from zero] 68. Greedy and non-greedy modes in Python regular expressions
[learning python from zero] 67. re module in Python: regular replacement and advanced matching technology
[learning python from zero] 66 .In-depth understanding of regular expressions: a powerful tool for pattern matching and text processing
[Learn python from zero] 65. Detailed explanation of Python regular expression modifiers and their applications
[Learn python from zero] 64. The re.compile method in Python regular expressions Detailed explanation
[learning python from zero] 63. Introduction to the re.Match class in regular expressions and its attributes and methods
[learning python from zero] 62. Python regular expressions: a powerful string matching tool
[learning python from zero] 61. Detailed explanation and application examples of property attributes in Python
[learning python from zero] 60. Exploration generator: a flexible tool for iteration
[learning python from zero] 59. Iterator: an efficient tool for optimizing data traversal
[learning python from zero] 58. Custom exceptions in Python and methods of raising exceptions
[Learn python from zero] 57. Use the with keyword in Python to correctly close resources
[Learn python from zero] 56. The importance and application of exception handling in programming
[Learn python from zero] 55. Serialization and processing in Python Deserialization, application of JSON and pickle modules
[learning python from zero] 54. Writing data in memory
[learning python from zero] 53. CSV files and Python CSV modules
[learning python from zero] 52. Reading and writing files - Python file operation guide
[learning python from zero] 51. Opening and closing of files and its application in Python
[learning python from zero] 49. Object-related built-in functions in Python and their usage
[learning python from zero] 48 .Detailed explanation of inheritance and multiple inheritance in Python
[learning python from zero] 47. The concept and basic use of inheritance in object-oriented programming
[learning python from zero] 46. __new__ and __init__ method analysis and singleton in Python Design patterns
[learning python from zero] 45. Class methods and static methods in Python
[learning python from zero] 44. Private attributes and methods in object-oriented programming
[learning python from zero] 43. Examples in Python object-oriented programming Attributes and class attributes
[Learn python from zero] 42. Built-in attributes and methods in Python
[Learn python from zero] 41. Python magic method (2)
[Learn python from zero] 40. Python magic method (1)
[Learn python from zero] 39. Object-oriented basic grammar and application examples
[Learn python from zero] 38. Use and import of Python packages
[Learn python from zero] 37. Use and precautions of Python custom modules
[From zero Learning python] 36. Methods and skills of using pip in Python for third-party package management
[learning python from zero] 35. Python common system modules and their usage
[learning python from zero] 34. Detailed explanation of the import and use of Python modules
[ Learn python from zero] 33. The role of decorators (2)
[Learn python from zero] 32. The role of decorators (1)
[Learn python from zero] 31. In-depth understanding of higher-order functions and closures in Python
[from Zero learning python] 30. In-depth understanding of recursive functions and anonymous functions
[learning python from zero] 29. "Detailed explanation of function parameters" - understand the different usages of Python function parameters
[learning python from zero] 28. Local variables and globals in Python Variables
[Learn python from zero] 27. Use and nested calls of Python functions
[Learn python from zero] 25. Functions: a tool to improve code writing efficiency
[Learn python from zero] 24. String operations and traversal methods in Python
[Learn python from zero] 23. How to use and common operations of sets (set) in Python
[Learn python from zero] 22. Addition, deletion, modification and query of dictionaries in Python and variables of dictionaries
[Learn python from zero] 21. In Python tuples and dictionaries for
[Learn python from zero] 20. Python list operation skills and examples
[Learn python from zero] 19. Application of looping through lists and list nesting
[Learn python from zero] 18. Detailed explanation of basic operations of Python lists (1)
[From Zero learning python] 17. Format method of Python string (2)
[Learn python from zero] 16. Format method of Python string (1)
[Learn python from zero] 15. In-depth understanding of string and character set encoding
[from Zero learning python] 14. Common operations of Python strings (2)
[Learn python from zero] 13. Common operations of Python strings (1)
[Learn python from zero] 12. Python string operations and applications
[Learn python from zero] 11.Python loop statement and control flow
[learn python from zero] 10. Detailed explanation of Python conditional statement and if nesting
[learn python from zero] 09. Conditional judgment statement in Python
[learn python from zero] 08. Python understands bit operations Operators, operator priority
[learn python from zero] 07. Detailed explanation of Python operators: assignment, comparison and logical operators
[learn python from zero] 06. Use arithmetic operators in Python for calculation and string concatenation
[learn from zero python] 05. Output and input in Python
[learn python from zero] 04. Basics of Python programming: variables, data types and identifiers
[learn python from zero] 03. Python interactive programming and detailed annotations
[Learn python from zero] 02. Introduction to development tools
[Learn python from zero] 01. Install and configure python