Shell programming basics-text processing

cut

Column string for processing text, grep is for processing rows

• -fn: extract n columns, or -fn,m extract n and m columns (the default tab is the column separator)
• -d: What separator is the column (you can customize the separator)

cut -f 2 /etc/passwd
#全部打印，问题是分隔分默认是tab键
cut -d ":" -f 2 /etc/passwd
cut -d : -f 2,4 /etc/passwd
#打印出第二列或者下面的第二列和第四列，2,4不是2-4的范围

The disadvantage of cut is that it does not support spaces as delimiters. If you need to support it, you must use awk.

awk programming

Processing text column string, processing is in accordance with the line

Format: awk'condition 1 {action 1} condition 2 {action 2} …'

Conditions (no conditions are all columns)

• BEGIN: Process before executing the action
• END: Process after the action is executed
• Greater than >, greater than or equal to >=
• Less than <, less than or equal to <=
• Equals ==
• Contains ~ (/abc/ required for awk recognition string)
• Not included! ~ (/Abc/ is required for awk recognition string)
• Regular rules//

action

• printf output and also supports print, the difference is that it will automatically wrap after print
• Flow control statement

Built-in variables

• $0, all columns
• $n, column n
• NF: the total number of columns
• NR: line number
• FS: Specify the separator

By default, awk uses spaces and tabs as column separators.
Variables need to be outside the quotation marks.

df -h | awk '{print $5}'
#支持空格分隔符输出列

cat /etc/passwd | awk '{FS=":"} {print $1}'
#问题是第一行被全部打印了，因为awk是先读取行再进行处理，第一行已经来不及识别分隔符
cat /etc/passwd | awk 'BEGIN {FS=":"} {print $1}'
#已分隔符为：读取第一列

df -h | awk '{print $5}' | cut -d % -f 1 | grep -v U
#获取第5列的磁盘容量值

df -h | awk '$1 ~ /root/ {print $5}'
#打印第一列包含root字符串，打印第5列

and

Modify the data of the text, add, delete, insert, replace

Format: sed option action

Options

• -n: Do not add -n to output the entire content, and output the specified content after adding -n
• -e: multiple actions are allowed, semicolon is the separator
• -i: modify the file, not print it out, the content of the file will not be modified without i
• -r: Support extended regular

Action (multiple lines plus \)

• p: printout
• a: Append, the line after the specified line
• -i: Insert, the previous line of the specified line
• -c: Replace line
• -d: delete the line
• -s: Replacement string, format:'line s/old data/new data/g'

The string deletion needs to be borrowed and replaced, and the new data is empty

sed -i 's/#SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
#修改关闭selinux

sed '/^#/d' /etc/selinux/config
#删除带有#号的行，正则需要加//

sed -e '/^#/d ; 1a 11111' /etc/selinux/config
#执行多条命令，先删除后添加

sort

• f ignore case
• -n Sort by integer, default is string
• -r reverse sort
• -u remove duplicate lines
• -t specifies the separator, -k specifies the range

 sort /etc/passwd
 #以第一个字符开始排序
 
sort -n -t : -k 3 /etc/passwd
#第三列以数值进行排序

wc statistics

• -i count the number of rows
• -w count the number of words
• -m counts the number of characters

[root@localhost ~]# wc -lwm /etc/passwd
  27   51 1301 /etc/passwd
#27行 51单词 1301字符数

Conditional judgment

test command or [] brackets (judge whether it is true, $? output)

Format: [Option File]

Option: judge according to file type (default is true)

• -b file: Determine whether the file exists and is a block device
• -c file: Determine whether the file exists and is a character device
• -d file: Determine whether the file exists and is a directory file
• -e file: determine whether the file exists
• -f file: Determine whether the file exists and is a normal file
• -L file: Determine whether the file exists and is a symbolic link file
• -p file: Determine whether the file exists, and it is a pipeline file

Option: Determine according to file permissions (default is true)

Unable to distinguish between owner and group and others

• -r file: determine whether the file exists, and the file read permission
• -w file: determine whether the file exists, and the file write permission
• -x file: determine whether the file exists, and the file execution permission
• -u file: Determine whether the file exists, and the file SUID permission
• -g file: determine whether the file exists, and the file SGID permission
• -k file: determine whether the file exists, and the file SBit permission

Options: Time comparison between files (default is true)

• File 1 -nt File 2: Is file 1 newer than file 2
• File 1 -ot File 2: Is file 1 older than file 2
• File 1 -ef File 2: Whether the indode numbers of file 1 and file 2 are the same

Options: Comparison between integers (default is true)

• Integer 1 -eq Integer 2: equal to
• Integer 1 -ne Integer 2: Not equal to
• Integer 1 -gt Integer 2: Greater than
• Integer 1 -lt Integer 2: Less than
• Integer 1 -ge Integer 2: greater than or equal to
• Integer 1 -le Integer 2: Less than or equal to

Options: compare between strings (default is true)

• String 1 == String 2: Equal to
• String 1! = String 2: Not equal
• -z string: empty
• -n string: non-empty

Options: multi-condition judgment (default is true)

• Judgment 1 -a Judgment 2: and
• Judgment 1 -o Judgment 2: Or
• ! Judgment: No, negate

[ 10 -eq 12 ]
[root@525 ~]# echo $?      
1
或
[ 10 -eq 12 ] && echo yes || echo no
no