awk [single programming language interpreter]
1, awk introduce
full name: Aho Weinberger Kernaighan three men acronym;
in 1970 for the first time on a Unix machine, then use it in the open field;
so we use in Linux , renamed GNU awk; so, in fact, Linux is called gawk;
grep line filter
-o -i -v -E grep egrep fgrep [what to write on match]
sed line editor
-np 1, 3 and bound address , 5, / pat / e [2], the command PA \ I \ WC \ S / PAT / STR / G |. 1 | 2 .. | &
$ {var / PAT / STR $} var {// PAT / STR}
$ {test}, $ {# test}, $ {test: offset: length}, $ {test # * word}, $ {test ## * word}, $ {test% word}, $ {test %% * } Word, $ {Test / pattern / String}, $ {// Test pattern / String}, {$ Test: -word}
awk report generator
by pattern matching and language format itself, to obtain, and outputs the customer desired content;
[default, three tools are not to edit the source file]
examples: Get user id above system less than 1 500 user name and user ID of the user
$ I in for (Cut -d: - f3 / etc / the passwd); do
IF [-ge. 1 -a $ I $ I -le 500]; the then
echo grep $ I | Cut -d: -f1,3
Fi
DONE
-F awk: '{IF ($. 3> = $. 3. 1 && <= 500) {}}. 3 Print $' / etc / the passwd
# awk finally achieve this function only when the sentence can be!
Significance formatted output:
awk -F: -v the OFS = ":" 'the BEGIN {the printf "username UID \ n-======================== \ n "} {if ($ 3> = 1 && $ 3 <= 500) {printf" user name:% - 10s UID:% - 10d \ n ", $ 1, $ 3}} END {printf" ------- -------------------- \ Nend \ n-"} '/ etc / the passwd
2, the working principle awk
Sequentially matched to the line using tools to edit or awk formatted output (So, on the circulation line, in awk can not be used)
3, the use awk
awk [option] ... 'program' FILE .. .
1, Program must be used! apostrophe!
2, a plurality of program statements comprising braces together, side by side can be nested
awk '{} Print' / etc / the passwd
. 4, the common option awk
-F specified delimiter
awk -F [/:] '{ print $ 1 , $ 3} 'a.txt
wherein [] represents any of the plurality of characters in a
-v because awk is a language compiler, to define their own variables, but also has built-in variables (environment variables and the like)
manually specify variable parameters
awk -va = "a / b" ' {print a}' a.txt
to assign a print a variable
1, a custom variable is the FS = -v ":"
2, call in awk variable call without adding $ symbol
awk '{a = "a / b"; print a}' a.
Not stored in awk command, we can associate another command to save the results of awk;
awk 'A = { "A / B";} A Print' a.txt | TEE a.bak
2, the printf - the formatting output of
the printf "% s% d is the best class students learn" variable 1, variable 2
[] with only sequential relationship
specifier
% s string
% d% i value
% e% E scientific calculation value
% c ACSii code value
% f float
% u unsigned integer
%%% escape character only display their
modifiers
default is right-aligned
- left aligned Representative
% 5.4F
. 5 digits occupied
4 decimal places taken
awk '/ ^ UUID / {printf " is mounted file:% - 50s mount point:% - 10s file system formats:% - 10s \ n ", $ 1, $ 2, $ 3} '/ etc / fstab
Note: the address mentioned here delimitation
Sed / PATl /, / PAT2 /
. 3, variable (built-in variables, custom variable)
built-in variables - environmental variables (bash) (env, set -C + C)
variable awk language supported by default
FS defining input delimiter variables
variables defined OFS output separator
NF defines the number of parameters beyond the partition line (after the last one variable partition $ NF)
* variable reference time, do not add $, $ 0, $ ... $. 1 n-
awk the FS = -v ":" '/ \ / bash $ / {print $ 1, $ NF}' / etc / passwd
rows, the file NR definition file defining a plurality of files, the line number is superimposed
FNR document count only their row number
awk ' Print NR} { '/ etc / fstab / etc / the passwd
awk' {Print the FNR} '/ etc / fstab / etc / the passwd
fILENAME storage file name
awk' {print fILENAME} '/ etc / passwd // file name print N times, the number of rows N file
awk 'BEGIN {print FILENAME}'/ Etc / passwd // BEGIN {statement} only the beginning of the cycle line, performed once;
The number of stages of the entire command ARGC [Note: do not contain 'program' itself]
ARGV array for calling command, a designated section ARGV [2] [Note: the array nor the 'program']
awk 'BEGIN {print ARGC } '/ etc / the passwd / etc / fstab / etc / Shadow
awk' Print ARGV the BEGIN {[. 3]} '/ etc / the passwd / etc / fstab / etc / Shadow
the RS specified newline \ n can specify a new line breaks, wrap itself does not affect
the time specified output newline ORS, replace the default newline character specified
awk -v the RS = "" 'Print {}' / etc / the passwd
awk -v ORS = "" '{} Print' / etc / passwd // wrap may be used to cancel the
custom variable
-v variable = value
when the rear 'program', calls the custom variables can be directly used
or "variable value =" statement written directly in 'program' also;
awk -VA = "A / B" 'Print {A}' A.TXT
awk 'A = { "A / B";} A Print' a.txt
. 4, the pattern matching (address delimited)
1, a null value is not defined, it will default file all rows awk circulated into
2, 3 rows of stationary operation
Sed -n '1,3p' / etc / the passwd
awk '1,3 {print} '/ etc / passwd // awk 1-3 is not supported by default, 3, etc., the direct writing such value;
awk' NR> = && NR. 1 <=. 3} {Print '/ etc / // through the passwd NR variable specified
. 3, / pat1 /
Sed -n / pat1 / P / etc / the passwd
awk '/r..ter/{print}' / etc / the passwd
. 4, / pat1 /, / PAT2 / first match pat1 the line between the first match pat2,
practice: the user line between judgment /^r..ter/,/^user.*\>/, bash is the user, and displays the user name, and UID
========================================
shell:?
for i in `awk ' of NF} $ /^r..ter/,/^user.*\>/{print '/ etc / passwd`; do
IF [[$ I == "/ bin / the bash"]];then
echo ``
fi
DONE
the while Line; do
IF [[ "/ bin / the bash" == `awk -F: '{} of NF Print $' $ line`]]; the then
awk -F '. 1 {Print $, $. 3}' $ Line
Fi
DONE Sed -n `<< '/^r..ter/,/^user.*\>/p' / etc / passwd`
==================== ====================
awk -F: '/^r..ter/,/^user.*\>/{if($NF==/bin / the bash); Print $. 1, $. 3} '/ etc / the passwd
. 5, pattern matching may be used directly Analyzing statement
awk -F:' $ of NF == "/ bin / the bash" {Print $. 1, $. 3} '/ etc / the passwd
. 6 , BIGEIN | END statement
BIGEIN in the default statement cycle before the operation to be performed;
awk -F: 'the BEGIN {the printf "bash the shell program for the user to: \ n"} $ NF == "/ bin / bash" { print $ 1, $ 3} ' / etc / passwd
awk -F: 'BEGIN {printf " shell program for a user bash is: \ n"} $ NF == "/ bin / bash" {print $ 1, $ 3} END {printf "end \ n"}' / etc / passwd
generally when the output format, header and not print;
5, operators
arithmetic operators:
+ - * /% (modulo, modulo) ^ // (rounded)
comparison operators:
==! => <> = <=
! ~ ~
awk -F: '$ of NF ~ "/ bin / the bash" {Print $. 1, $. 3}' / etc / the passwd
awk -F:! '$ of NF ~ "/ bin / the bash" {print $ 1, $ 3} '/ etc / passwd
logical operators:
&&
||
!
assignment operator:
= = + - = / * =% = = ^ = = //
Conditional expression:
conditional statements? Founded condition statement: condition statement does not hold
awk '/ ^ title / {FN <= 2 print:? printf " through the parameter \ n"}' /boot/grub/grub.conf // a problem
6, the common action
Print the printf and the operation command is any action that ;
. 1, expressions common expression
2, control statement, for example, control statements: if while other
logic binding statement evaluates to
3, a combination of statements statements Compound
/ PAT1 / {{{};}}
. 4, iNPUT statements input statement
5, output statements output statements
* 7, a common language (IF BREAK the while do for the Delete the Continue Switch)
1, IF statement
syntax: if (conditional expression) {execute statements}
IF (conditional expression) {} else {execute the statement to execute a statement}
awk '/ ^ title / {if (NF <= 2) {print} else {print " too few parameters"}}' /boot/grub/grub.conf
awk '/ ^ title / {IF (of NF> = 2). 4 Print $}' /boot/grub/grub.conf
2, the while statement is
only when the parameters of the line will be used to traverse the loop;
syntax: while (Condition Expression formula) {} loop
exercise: filter kernel grub.conf file this line, then the number of characters of each parameter statistics, and displayed;
initial value
while judging condition; do
loop
initial control statements
DONE
awk '/ ^ [[: Space:]] * Kernel \> / = {I. 1; the while (I <= of NF) {$ I the printf "\ T"; Print length ($ I);}} I ++ 'the grub.conf
. 3, for statement
syntax: for (initial value; condition judgment; the initial value of control statements) {loop body}
awk '/ ^ [[: Space:]] * Kernel \> / {for (I =. 1; I <= of NF; i ++) {printf $ i " \ t"; print length ($ i)}} 'grub.conf
cycle is recommended for statement
in awk, for shell and can also be used in the same format:
回顾：
for i in 列表；do
循环体
done
for(i in 列表){循环体}
echo "xia liang z shi k hen n shuai da fa le " | awk '{for(i in {1..NF}) { ls=() if(length(ls[i])<=3){print $i}}}'
shell
=====================================
ls=(xia liang z shi k hen n shuai da fa le)
for i in `seq 0..${$ls[*]}`;do
if [[ ${#${ls[$i]}} <= 3 ]];then
print ls[$i]
fi
done
=====================================
4、do-while 循环
Syntax: do {loop} while (loop condition)
attention, and while the other was: while statement only when the condition is satisfied, will enter the circulation, and do while loop will be executed first (primary), condition judgment is performed;
5, out of loop statement
BREAK [n] out n cycles;
Continue out of the current cycle;
Next out of default the current cycle;
awk '{IF (== NR 2%. 1) {Next the else {}}} Print' / etc / the passwd
. 6, similar to the case Switch
syntax: Switch (expression The) {case or VALUE1 / Regexp /: Statement; VALUE2 or case / REGEXP2 /: Statement; ..., default: Statement}
Switch (expression) {case pattern matching value : execute a statement; case pattern matching value: execute a statement, ..., default: statements} file execution
shell:
Case variables in
PAT1)
executing a statement
;;
*)
Execute the statement
;;
esac
* 8, * arrays
in awk, the same array and an array of characteristics of the shell:
Note: awk Array not defined, as long as the use, there is an empty array of default; do this in the statistics when very common! ! ! ! !
Line traversing -> actually traverse the entire file
column traverse -> take an object fixed in a column in the same statistical data
Note: when an array of other variables to give an assignment by the for statement, the assignment is to index index information;
exercise: statistics about the number of times a file specified row word appears;
awk -v RS = ""} {Print a.txt | awk '{LS [$ 1] ++} END {for (i in LS ) {print i, ls [i ]}} '
exercise: statistics about the number of times each word in a file (string separated by a space) occur;
awk' {for (I =. 1; I <= of NF; I ++ ) {COUNT [$ I] ++}} the END {for (I in COUNT) {Print I, COUNT [I]}} '/ etc / fstab
awk' {for (I =. 1; I <= of NF; I ++) {ls [$ i] ++} } END {for (j in ls) {print j, ls [j]}} '/ var / log / httpd / access_log | sort -t "
9, the function
1, built-in functions
call functions: funcation (parameter)
length () string length statistics
using the number of rows on the math SiN () COS () ...
Sub (X, X, X) replace the first matching the value
awk '{print sub (o, O, $ 1)}' / etc / fstab // the first column (a match) o replaced with O file
gsub (x, x, x) Alternatively the line matched to all values
awk '{print gsub (o, O, $ 1)}' / etc / fstab // file will be replaced with O o the first column (all matches)
Split (X, X , x) designated to cut file delimiter
the netstat -tan | awk '/ ^ TCP \> / {Split ($. 5, IP, ":"); Print IP [. 1]}'
the netstat -tan | awk '/ ^ TCP \ > / {split ($ 5, ip, ":"); count [ip [1]] ++} END {for (i in count) {print i, count [i]}} '