linux advanced command awk

AWK is a language for processing text files and a powerful text analysis tool.

It is called AWK because it takes the first character of the Family Name of the three founders Alfred Aho, Peter Weinberger, and Brian Kernighan.
We use the ls -ldefault to output some file permissions, user names, file sizes, etc. But sometimes, what should we do if we require formatted output? At this time, we need to use awkcommands.

Small example

$ ls -l
total 1395200
-rw-r--r-- 1 MagicBook 197121  60646319  1月 12  2020 1.认识JVM的基本结构.mp4
-rw-r--r-- 1 MagicBook 197121 110583202  1月 12  2020 10.垃圾收集器.mp4
-rw-r--r-- 1 MagicBook 197121 137221102  1月 12  2020 11.收集器的选择.mp4
-rw-r--r-- 1 MagicBook 197121 204987941  1月 12  2020 12.GC调优.mp4
-rw-r--r-- 1 MagicBook 197121  73189377  1月 12  2020 2.虚拟机栈.mp4
-rw-r--r-- 1 MagicBook 197121  74493138  1月 12  2020 3.程序计数器.mp4
-rw-r--r-- 1 MagicBook 197121 123329001  1月 12  2020 4.方法区与堆.mp4
-rw-r--r-- 1 MagicBook 197121 207456340  1月 12  2020 5.JVM性能调优监控工具.mp4
-rw-r--r-- 1 MagicBook 197121 143361728  1月 12  2020 6.JVM最常问的面试题.mp4
-rw-r--r-- 1 MagicBook 197121  88398831  1月 12  2020 7.类加载机制.mp4
-rw-r--r-- 1 MagicBook 197121 105331804  1月 12  2020 8.GC算法和收集器.mp4
-rw-r--r-- 1 MagicBook 197121  98187492  1月 12  2020 9.垃圾回收算法.mp4

$ ls -l | awk '{print $9}'

1.认识JVM的基本结构.mp4
10.垃圾收集器.mp4
11.收集器的选择.mp4
12.GC调优.mp4
2.虚拟机栈.mp4
3.程序计数器.mp4
4.方法区与堆.mp4
5.JVM性能调优监控工具.mp4
6.JVM最常问的面试题.mp4
7.类加载机制.mp4
8.GC算法和收集器.mp4
9.垃圾回收算法.mp4

runoob —— awk

awk syntax

awk [选项参数] 'script' var=value file(s)
或
awk [选项参数] -f scriptfile var=value file(s)

Option parameter description:

-F fs or --field-separator fs
specifies the input file separator, fs is a string or a regular expression, such as -F:.
-v var=value or --asign var=value
Assign a user-defined variable.
-f scripfile or --file scriptfile
read awk commands from a script file.
-mf nnn and -mr nnn
set inherent limits on the value of nnn, the -mf option limits the maximum number of blocks allocated to nnn; the -mr option limits the maximum number of records. These two functions are the extended functions of Bell Labs version of awk and are not applicable in standard awk.
-W compact or --compat, -W traditional or --traditional
Run awk in compatibility mode. So gawk's behavior is exactly the same as standard awk, all awk extensions are ignored.
-W copyleft or --copyleft, -W copyright or --copyright
print short copyright information.
-W help or --help, -W usage or --usage
Print all awk options and a brief description of each option.
-W lint or --lint
print warnings about structures that cannot be ported to traditional unix platforms.
-W lint-old or --lint-old
print warnings about structures that cannot be ported to traditional unix platforms.
-W posix
opens compatibility mode. However, the following restrictions are not recognized: /x, function keywords, func, escape sequence, and when fs is a space, the new line is used as a field separator; the operator and = cannot replace ^ and ^=; fflush is invalid .
-W re-interval or --re-inerval
allows the use of interval regular expressions, refer to (Posix character class in grep), such as bracket expression [[:alpha:]].
-W source program-text or --source program-text
Use program-text as the source code, which can be mixed with the -f command.
-W version or --version
Print the version of the bug report information.

Basic usage

The text content of log.txt is as follows:

2 this is a test
3 Are you like awk
This's a test
10 There are orange,apple,mongo

Usage 1:

awk '{[pattern] action}' {filenames}   # 行匹配语句 awk '' 只能用单引号

Examples:

# 每行按空格或TAB分割，输出文本中的1、4项
 $ awk '{print $1,$4}' log.txt
 ---------------------------------------------
 2 a
 3 like
 This's
 10 orange,apple,mongo
 # 格式化输出
 $ awk '{printf "%-8s %-10s\n",$1,$4}' log.txt
 ---------------------------------------------
 2        a
 3        like
 This's
 10       orange,apple,mongo

Usage 2:

awk -F  #-F相当于内置变量FS, 指定分割字符

Examples:

# 使用","分割
 $  awk -F, '{print $1,$2}'   log.txt
 ---------------------------------------------
 2 this is a test
 3 Are you like awk
 This's a test
 10 There are orange apple
 # 或者使用内建变量
 $ awk 'BEGIN{FS=","} {print $1,$2}'     log.txt
 ---------------------------------------------
 2 this is a test
 3 Are you like awk
 This's a test
 10 There are orange apple
 # 使用多个分隔符.先使用空格分割，然后对分割结果再使用","分割
 $ awk -F '[ ,]'  '{print $1,$2,$5}'   log.txt
 ---------------------------------------------
 2 this test
 3 Are awk
 This's a
 10 There apple

Usage three:

awk -v  # 设置变量

Examples:

 $ awk -va=1 '{print $1,$1+a}' log.txt
 ---------------------------------------------
 2 3
 3 4
 This's 1
 10 11
 $ awk -va=1 -vb=s '{print $1,$1+a,$1b}' log.txt
 ---------------------------------------------
 2 3 2s
 3 4 3s
 This's 1 This'ss
 10 11 10s

Usage four:

awk -f {awk脚本} {文件名}

Examples:

 $ awk -f cal.awk log.txt