Shell regular expression grep
Regular expression overview
One, regular expression definition
- Regular expression, also known as regular expression, regular expression
在代码中经常简写为regex、regexp或RE - It uses a single string to describe and match a series of strings that conform to a certain syntax or grammatical rule.
Example: When mail servers filter spam, regular expressions are often used
Two, regular expression composition
(1) Ordinary characters
- Uppercase and lowercase letters, numbers, punctuation marks and some other symbols
(2) Metacharacters
- Special characters with special meaning in regular expressions
Three, basic regular expressions——grep、sed命令支持
(1) Examples of basic regular expressions
- Find specific characters
- -n display line number
- -i is not case sensitive
- -v reverse lookup
——Create test file （直接把test.txt的内容复制就行了）
[root@localhost ~]# cat test.txt
he was short and fat.
He was wearing a blue polo shirt with black pants.
The home of Football on BBC Sport online.
the tongue is boneless but it breaks bones.12!
google is the best tools for search keyword.
The year ahead will test our political establishment to the li
PI=3.141592653589793238462643383249901429
a wood cross!
Actions speak louder than words
#woood #
#woooooood #
AxyzxyzxyzxyzC
I bet this place is really spooky late at night!
Misfortunes never come alone/single.
I shouldn't have lett so tast.
——Find the and display the line number （之后的操作都是使用test.txt作为模板，可以更加直观一点）
[root@localhost ~]# grep -n 'the' test.txt
4:the tongue is boneless but it breaks bones.12!
5:google is the best tools for search keyword.
6:The year ahead will test our political establishment to th limit.
`直接'the'筛选the进行过滤`
——Search the is not case sensitive （每次做的时候可以对照一下模板test.txt）
[root@localhost ~]# grep -in 'the' test.txt
3:The home of Football on BBC Sport online.
4:the tongue is boneless but it breaks bones.12!
5:google is the best tools for search keyword.
6:The year ahead will test our political establishment to th limit.
`加了-i ，所以不区分大小写`
——Reverse search for lines that do not contain the
[root@localhost ~]# grep -vn 'the' test.txt
1:he was short and fat.
2:He was wearing a blue polo shirt with black pants.
3:The home of Football on BBC Sport online.
7:PI=3.141592653589793238462643383249901429
8:a wood cross!
9:Actions speak louder than words
10:
11:#woood #
12:#woooooood #
13:AxyzxyzxyzxyzC
14:I bet this place is really spooky late at night!
15:Misfortunes never come alone/single.
16:I shouldn't have lett so tast.
`加了-v ，所以变成了反向查找`
(2) Use square brackets "[] "to find collective characters
[] —— No matter how many characters there are, they only represent one character, which is equivalent to the relationship of "or"
[^] ——The ^ in the brackets means the negation
——Find the line containing shirt or short (还是利用刚才的测试文件test.txt进行操作)
[root@localhost ~]# grep -n 'sh[io]rt' test.txt
1:he was short and fat.
2:He was wearing a blue polo shirt with black pants.
`[io]，中括号写的是i和o，所以是i和o都可以，都符合条件`
-Find the line that repeats a single character "oo"
[root@localhost ~]# grep -n 'oo' test.txt
3:The home of Football on BBC Sport online.
5:google is the best tools for search keyword.
8:a wood cross!
11:#woood #
12:#woooooood #
14:I bet this place is really spooky late at night!
`'oo'，这里两个o其实和上面的the是同理的`
——Find the line that is not "w" before "oo"
[root@localhost ~]# grep -n '[^w]oo' test.txt
3:The home of Football on BBC Sport online.
5:google is the best tools for search keyword.
11:#woood #
12:#woooooood #
14:I bet this place is really spooky late at night!
`11,12行‘oo’前面不是‘w’，而是‘o’或多个‘o’，所以会显示出来`
——Find the line that is not lowercase before "oo"
[root@localhost ~]# grep -n '[^a-z]oo' test.txt
3:The home of Football on BBC Sport online.
`[^a-z],以a-z开头的小写字母`
——Find the line that is not a capital letter before "oo"
[root@localhost ~]# grep -n '[^A-Z]oo' test.txt
5:google is the best tools for search keyword.
8:a wood cross!
11:#woood #
12:#woooooood #
14:I bet this place is really spooky late at night!
`[^A-Z],同理以A-Z开头的大写字母`
-Find the line that contains a number
[root@localhost ~]# grep -n '[0-9]' test.txt
4:the tongue is boneless but it breaks bones.12!
7:PI=3.141592653589793238462643383249901429
`[0-9]，包含数字的行`
**（可以按自己的理解试试其他的筛选条件）**
(3) Find the beginning ^ and end characters $
- The decimal point "." Is a metacharacter in regular expressions, you need to use the escape character "\" to convert it into ordinary characters
-find the line ending with a decimal point "."
[root@localhost ~]# grep -n '\.$' test.txt
`查找空行`
[root@localhost ~]# grep -n '^$' test.txt
(4) Find any character ". "and repeated characters" * "
- Find a line starting with "w" and ending with "d" with 4 characters
[root@localhost ~]# grep -n 'w..d' test.txt
5:google is the best tools for search keyword.
8:a wood cross!
9:Actions speak louder than words
`w..d就是中间的两个'.'表示任意字符`
- "*" Means to repeat zero or more of the preceding single characters,
——The query contains at least two strings of o
[root@localhost ~]# grep -n 'ooo*' test.txt
3:The home of Football on BBC Sport online.
5:google is the best tools for search keyword.
8:a wood cross!
11:#woood #
12:#woooooood #
14:I bet this place is really spooky late at night!
`‘ooo*’---第1个'o'和第二个'o'必须存在，第三个'o'可以是0个或多个，所以oo，ooo等都符合规则`
`同理oo*的话就是第一个'o'必须存在，第二个'o'可以是0个或多个`
——Find the line that starts with "w", contains at least one "o" and ends with "d"
[root@localhost ~]# grep -n 'woo*d' test.txt
8:a wood cross!
11:#woood #
12:#woooooood #
`这个就是利用了oo*,第一个'o'必须有，如果是中间至少包含两个'o'的话就是wooo*d`
——Find lines that start with'w' and end with'd' with optional characters
[root@localhost ~]# grep -n 'w.*d' test.txt
1:he was short and fat.
5:google is the best tools for search keyword.
8:a wood cross!
9:Actions speak louder than words
11:#woood #
12:#woooooood #
`w.*d 这个'.'后面跟了* ，所以这个'.'可以是0个也可以是多个，说白了就是以'w'开头'd'结尾的不管多少字符都符合`
——Query any number of rows
[root@localhost ~]# grep -n '[0-9][0-9]*' test.txt
4:the tongue is boneless but it breaks bones.12!
7:PI=3.141592653589793238462643383249901429
`[0-9][0-9]*第一个[0-9]必须有，第二个可以是0个也可以是多个，所以可以看成只要有数字不管多少字符的行都符合`
(5) Find the range of consecutive characters {}
**使用 " . "和 " * "可以设置零个或无限多个重复的字符，如果要限制一个范围则使用' { } **
——View 2 O characters
[root@localhost ~]# grep -n 'o\{2\}' test.txt
3:The home of Football on BBC Sport online.
5:google is the best tools for search keyword.
8:a wood cross!
11:#woood #
12:#woooooood #
14:I bet this place is really spooky late at night!
`o\{
2\}即查找o并且查找字符范围是2两个及以上`
——Look at the beginning of w, the end of d, the middle is a string of 2-5 o
[root@localhost ~]# grep -n 'wo\{2,5\}d' test.txt
8:a wood cross!
11:#woood #
`wo\{
2,5\}d，2，5及2-5个字符，开头是w结尾是d的`
——Look at the beginning of w, the end of d, the middle is the string of 2 or more o
[root@localhost ~]# grep -n 'wo\{2,\}d' test.txt
8:a wood cross!
11:#woood #
12:#woooooood #
`wo\{
2,\}d，这里2后面跟了','号，就是说2个以上，不加就是两个，参考上面的案例`
Four, common metacharacters of basic regular expressions
\b 单词的开头或结尾,只匹配一个位置，不匹配分隔标点符号和空格
\d 一个数字，等价于 [0-9]
* 数量，它前面的内容以连续使用的任意次数以达到整个表达式匹配,可以是0次匹配
+ 和* 类似，但至少匹配1次, 匹配一个或多个
？ 和上面两个类似，重复0次或一次
. 匹配除了换行符以外任意字符
\s 匹配任意的空白符、制表符、换行符、中文全角空格等
\w 匹配字母、数字、汉字或者下划线
^ 用来查找的字符串的开头
$ 用来查找的字符串的结尾
{
n} 前面必须连续重复匹配n次，
{
n,m} 前面必须连续重复匹配n~m次，
{
n,} 前面必须连续重复匹配n~更多次，
\ 如果需要查找元字符，需要转义
[] 里面的字符可以不用转义，用来定义匹配集合
Five, the common metacharacters of extended regular expressions——egrep、awk命令支持
+ 重复一个或者一个以上的前一个字符
？ 零个或者一个的前一个字符
| 使用或者（or）的方式找出多个字符
() 查找“组”字符串
()+ 辨别多个重复的组