Regular expressions and →shell programming three swordsmen

So what is a regular expression?

Regular expression concepts

Detailed explanation of grep command

Explain linux regular expressions in detail (basic regular expressions + extended regular expressions)

• Regular expression

It is a matching rule through symbols and strings, which affects the query results of the file content.

Regular expressions are composed of ordinary characters and metacharacters.
1. Common characters include uppercase and lowercase letters, numbers, punctuation marks and some other symbols.
2. Metacharacters refer to special characters with special meaning in regular expressions, which can be used to specify the appearance mode of their leading characters (that is, the characters in front of the metacharacter) in the target object.

Commonly used regularities : grep, sed, awk, pgrep, mkdir, touch...
touch a{1...100}.txt

[root@localhost ~]# ifconfig ens33 | grep "inet 192.168.6.20 " | awk ‘{print $5}’
broadcast

• The usage of grep combined regular expression

(1) n: Display the line number, the line number of the specified string in the file where the query is located

grep -n "ServerName" /usr/local/httpd/conf/httpd.conf 

(2) i: The display content is not case sensitive

grep -in "ServerName" /usr/local/httpd/conf/httpd.conf 

(3) v: Reverse query, display lines that do not contain the specified string

grep -vn "ServerName" /usr/local/httpd/conf/httpd.conf 

(3) []: Match any character in [], or mean

grep -in "Server[Nn][avty]me" /usr/local/httpd/conf/httpd.conf 
[root@localhost ~]# grep -n shu[twqwqn]do[abcde]n /etc/passwd
[root@localhost ~]# grep -n shu[twqwqn]do[abcdew]n /etc/passwd
7:shutdown:x :6:0:shutdown:/sbin:/sbin/shutdown

(4) ^: start with the specified character or string

grep -n "^[Ss]" /usr/local/httpd/conf/httpd.conf 

(5) $: Put in front of the string, which means to call the variable, and the string is the variable name. Put it after the string to indicate the end of the string.

Special usage: $ abc {$ means call variable}

Note: If the ending is ., you need to add an escape character. Because. Is also a metacharacter in regular expressions, it is necessary to use the escape character \ to convert characters with special meanings into ordinary characters.

grep -n "html$" /usr/local/httpd/conf/httpd.conf 
grep -n "\.$" /usr/local/httpd/conf/httpd.conf 
df | grep "/$"
cat httpd.conf | grep "\.$"
grep "com:80$" httpd.conf

(6) ^$: display blank lines

grep -n "^$" /usr/local/httpd/conf/httpd.conf 

(7)
[Starting Number-Ending Number]: Between the specified number range [1-10]
[Starting letter-Ending letter]: Between the specified letter range [au]
grep -n "[AZ] "/Usr/local/httpd/conf/httpd.conf
[AZ]: A continuous range of numbers or alphanumeric
examples:

[root@localhost ~]# grep -n ^[1..5] /etc/passwd           [*..*]这种写法是错误的
43:123:x:1000:1000:123:/home/123:/bin/bash
44:10
[root@localhost ~]# grep -n ^[1-30] /etc/passwd          正确写法
43:123:x:1000:1000:123:/home/123:/bin/bash
44:10
45:20
[root@localhost ~]# grep -n ^[1-10] /etc/passwd          特殊情况
43:123:x:1000:1000:123:/home/123:/bin/bash
44:10

---

• Basic regular expression

(1) Inverted value

Format one,

^[^] :取反值，不是以指定字母开头的字符串

查找“oo”前不是“w”的字符串
grep -n '[^w]oo' test.txt 

Format two,

[^] : 取反值，查询出包含“除指定的字母或数字范围外”的字符串
[^a-z] [^A-Z] [^0-9]
grep "[^a-z]ork" httpd.conf

(2) {}: the number of repeated the specified character, the need to add escape characters {} are:. \ \ * \ (\) \ $ \ /
Specified number of characters {1, 2} numbers: a string In, the range of the number of consecutive occurrences of the specified character "wo{2,5}rk" The
specified character {number 1}: Indicates the range of the number of consecutive occurrences of the specified character in the string "wo{2}rk"
Example:

查询包含1-3个o的字符串
grep -n 'wo\{1,3\}d' test.txt 			o\{1,3\}
查询数量为2以上连续出现的字符串
grep -n 'o\{2\}' test.txt 			o\{2\}            oooo：这也是oo的双倍

(3) Wildcard in regular:

.    	代替一个字符，可以连续使用，不能代表空值
*   	代替一组字符，可以为空
.*	    字符可有可无

grep'r...o' /etc/passwd to
query if there are 2 characters between r and o, a space is counted as one character

grep -n'ooo*' /etc/passwd.old
Query more than two o's.
In the query "oo o*", it must appear as a pair of oo, and in the query "ooo o*", it must be ooo Occurs in pairs.
If it is "oo*", the first o must exist, and the second o is zero or more, so it contains o, oo, ooo, oooo,...
If you query "oo o*", query Strings of more than two o's. The search keyword is two a. Contains oo, ooo, oooo,...
If you query "ooo o*", query a string of more than three o's. Contains ooo, oooo,...

grep -n'.*' /etc/passwd.old
all appear

• Extended regular expression

In the shell, some more simplified regular expression metacharacters have been added, and only a small number of commands support extended regular expressions.
Commands used: The
grep command only supports basic regular expressions. If you use extended regular expressions, you need to use egrep or awk commands!

1) egrep -n'woo+d' test.txt #"wo{2,}rk" can be written as "wo+rk"

+   ：重复一个或者一个以上的前一个字符

2) egrep -n'o?oo' test.txt #represents the query string that contains oo in the specified file, and the first o is optional

？  ：零个或者一个的前一个字符

3）egrep -n ‘o?’ test.txt

 查询没有或有一个以上o的字符串

4）egrep -n ‘oo?’ test.txt

查询一个或有一个以上o的字符串

5）egrep -n ‘ooo?’ test.txt

查询两个或有两个以上o的字符串

6）egrep -n ‘name|Name’ /usr/local/httpd/conf/httpd.conf

查询文件中与指定字符串相同的内容

7) egrep -n't(a|e)st' /usr/local/httpd/conf/httpd.conf (only one character can be restricted)

查询“组”字符串

8）egrep -n ‘A(xyz)+C’ /usr/local/httpd/conf/httpd.conf

#查询开头的“A”结尾是“C”，中间有一个以上的“xyz”字符串的意思。（一个以上代表：xyzxyzxyz...）
辨别多个重复的组                          

The last three above must be quoted, and the previous ones can be added or not! ! !

———————————————————————————————————————————————— ————— Three Musketeers of Shell Programming: GREP SED AWK sed command teaching awk command teaching

[1] sed is an online editor that processes one line of content at a time.

Sed main functions:
used to automatically edit one or more files;
simplify repeated operations on files;
write conversion programs, etc.
[By default, all sed commands are executed in the pattern space (that is, the source file is not modified)! Therefore, the input file will not change in any way, unless redirection is used to store the output]

Usage:
two formats
sed [options] 'operations' parameters
sed [options] -f scriptfile parameter
[Option]
Common sed mainly includes the following command options:
-n, -quiet or Silent: only the processing result displayed
- i: edit the text content
-e or -expression=: use the specified command or script to process the input text file
-f or -file=: use the specified script file to process the input text file
-h or -help: display help

'Operation'
"Operation" is used to specify the action behavior of file operations, that is, the sed command. Usually, the format of "[n1[,n2]]" operation parameter is adopted. n1 and n2 are optional, and may not exist, representing the parameters for selecting the operation.
For example, if the operation needs to be performed between lines 5-20, it is expressed as "5,20 action behavior".
Common operations include the following:
a: Add, add a line of specified content below the current line.
c: Replace, replace the selected line with the specified content
d: Delete, delete the selected line
i: Insert, insert a line of specified content above the selected line
p: Print, if you specify the line at the same time, it means to print the specified line. If you do not specify a row, it means that all content is printed. If there are non-printing characters, they will be output in ASCII code. It is usually used with the "-n" option.
s: replace, replace the specified character
y: character conversion

Parameters: refers to the target file of the operation. When there are multiple operation objects, the files are separated by a comma ","!
And scriptfile represents a script file, which needs to be specified with the option "-f". When the script file appears before the target file, it means that the input target file is processed through the specified script file.

You can edit and view the content of the file directly in the command line state

• a: Add a line of content after the specified line number

sed -i '1a#chkconfig:35 25 25' /usr/local/httpd/conf/httpd.conf Note that you cannot add data to an empty file.

sed -i'/#ServerName/a \192.168.100.100' /usr/local/httpd/conf/httpd.conf
add 192.168.100.100 in the next line of #ServerName

• p: output text content that meets the conditions

sed -n '90p' /usr/local/httpd/conf/httpd.conf
View the content of line 90
sed -n '90,100p' /usr/local/httpd/conf/httpd.conf
View the content of line 90-100
sed -n "95,+10p" /etc/httpd/conf/httpd.conf
read the content of the specified number of lines from the line number
sed -n'$=' /usr/local/httpd/conf/httpd.conf
view the main line number

• d: delete the specified number of lines in the file

cp /usr/local/httpd/conf/httpd.conf /usr/local/httpd/conf/httpd.txt
head -16 /usr/local/httpd/conf/httpd.txt

Delete the lines in the specified range:
sed -i “360,364d” /usr/local/httpd/conf/httpd.txt
sed -i “355, +10d” /usr/local/httpd/conf/httpd.txt

sed -i'/apache/d' /usr/local/httpd/conf/httpd.txt
delete the line with the specified string
sed -i'/apache/!d' /usr/local/httpd/conf/httpd. txt
delete all lines that do not contain the specified string
sed -i'/^#/d' /usr/local/httpd/conf/httpd.txt
delete all lines beginning with
#sed -i'/^$/d' /usr /local/httpd/conf/httpd.txt
delete all blank lines

• s: modify the content of the specified file (commonly used)

vim a.txt
server
SERVER
Server
aaabaaaa
dddkdda
sed -i's/server/SERVER1/' a.txt
modify the first string of each line

sed -i's/server/SERVER1/3'
The number 3 after a.txt means to modify the [number of strings with the same name] in each line

sed -i'/server/c #haha' /a.txt
replace the line where the specified string is located

Only replace the specified string, not the entire line.
sed -i's/server/SERVER/g' /a.txt followed by g means to modify the full text

[2] awk: powerful editing tool. Read the input text line by line, and search according to the specified matching mode (usually used to display the specified field)

[1] Separator, the default is a space separator. Split the first field of each line of the file
awk -F:'{print $1}' /etc/passwd

awk'{print}' /a.txt = cat /a.txt
awk'{print $0}' /a.txt = cat /a.txt awk uses $0 to represent the entire line (record)

[2] Output the contents of the first to third lines

awk 'NR==1,NR==3{
    
    print $0}' /a.txt 
awk '(NR>=1)&&(NR<=3){
    
    print}' /a.txt 

[3] Output the contents of the first and third lines

 awk 'NR==1||NR==3{
    
    print}' /a.txt 

[4]
awk'(NR%2)==0{print}' /a.txt
output all even lines

awk'(NR%2)==1(print)' /a.txt
output all odd lines

awk'/^root/{print}' /etc/passwd
outputs lines starting with root

awk'/nologin$/{print}' /etc/passwd
outputs lines ending with nologin

awk'BEGIN{X=0};//bin/bash$/{x++};END{print x}' /etc/passwd
counts the number of lines ending in bin/bash

The workflow of awk:

Extension:
awk contains several special built-in variables (can be used directly)
FS: specify the field separator of each line of text, the default is a space or tab stop
NF: the number of fields of the
currently processed line NR: the number of the currently processed line Line number (ordinal number)
$0: the entire line content of the
currently processed line $n: the nth field (nth column) of the currently processed line
FILENAME: the processed file name
RS: data record separation. The default is n, that is, one record per line
-F: Specify: as a separator, and the default is space separation.
Let us work hard together, come on! ! !

LAMP dynamic website platform