Function Description
Sed, short for Strem Editor, is a powerful tool for manipulating, filtering, and transforming text content. Common functions include addition, deletion, modification, query, filtering, and row fetching.
[root@oldboy ~]# sed --version #→ sed software version GNU but version 4.2.1
syntax format
sed [options] [sed-commands] [input-file] sed [options] [sed command] [input file] illustrate: 1. Note that there is at least one space between sed and the options that follow. 2. In order to avoid confusion, this article calls sed the sed software. Sed-commands (sed commands) are some command options built into the sed software. In order to distinguish them from the previous options (options), they are called sed commands. 3. sed-commands can be either a single sed command or a combination of multiple sed commands. 4. input-file (input file) is optional, sed can also get input from standard input such as pipes.
command execution flow
Summary process: Sed software reads a line from a file or pipe, processes one line, and outputs one line; reads another line, processes another line, and outputs another line...
Mode space: a temporary cache inside the sed software, used to store the read content.
Usage example
1. Unified experimental text
# Create a file with the following contents, which will be used by subsequent operations. [root@oldboy ~]# cat person.txt 101,oldboy,CEO 102, zhangyao, CTO 103,Alex,COO 104,yy,CFO 105,feixue,CIO
2. CRUD
a Append text to the specified line i inserts text before the specified line
2.1 Increase
2.1.1 Single-line addition
[root@oldboy ~]# sed '2a 106,dandan,CSO' person.txt 101,oldboy,CEO 102, zhangyao, CTO 106, dandan, CSO 103,Alex,COO 104,yy,CFO 105,feixue,CIO [root@oldboy ~]# sed '2i 106,dandan,CSO' person.txt 101,oldboy,CEO 106, dandan, CSO 102, zhangyao, CTO 103,Alex,COO 104,yy,CFO 105,feixue,CIO
2.1.2 Multi-line addition
[root@oldboy ~]# sed '2a 106,dandan,CSO\n107,bingbing,CCO' person.txt 101,oldboy,CEO 102,zhangyao,CTO 106,dandan,CSO #→第1种写法 107,bingbing,CCO 103,Alex,COO 104,yy,CFO 105,feixue,CIO [root@oldboy ~]# sed '2a 106,dandan,CSO \ > 107,bingbing,CCO' person.txt 101,oldboy,CEO 102,zhangyao,CTO 106,dandan,CSO #→第2种写法 107,bingbing,CCO 103,Alex,COO 104,yy,CFO 105,feixue,CIO #→sed命令i的使用方法是一样的，因此不再列出。
企业案例1：优化SSH配置（一键完成增加若干参数）
在我们学习系统优化时，有一个优化点：更改ssh服务远程登录的配置。主要的操作是在ssh的配置文件加入下面5行文本。(下面参数的具体含义见其他课程。)
Port 52113 PermitRootLogin no PermitEmptyPasswords no UseDNS no GSSAPIAuthentication no
我们可以使用vi命令编辑这个文本，但这样就比较麻烦，现在想一条命令增加5行文本到第13行前？
指定执行的地址范围
sed软件可以对单行或多行进行处理。如果在sed命令前面不指定地址范围，那么默认会匹配所有行。
用法：n1[,n2]{sed-commands}
地址用逗号分隔的，n1,n2可以用数字、正则表达式、或二者的组合表示。
例子：
10{sed-commands} 对第10行操作
10,20{sed-commands} 对10到20行操作,包括第10,20行
10，+20{sed-commands} 对10到30(10+20)行操作,包括第10,30行
1~2{sed-commands} 对1,3,5,7,……行操作
10, ${sed-commands} operate on 10 to the last line ($ represents the last line), including the 10th line
/oldboy/{sed-commands} operate on lines matching oldboy
/oldboy/,/Alex/{sed-commands} operate on lines matching oldboy to lines matching Alex
/oldboy/,${sed-commands} operate on the line matching oldboy to the last line
/oldboy/,10{sed-commands} Operate on lines matching oldboy to line 10. Note: If the first 10 lines do not match oldboy, the sed software will display the lines matching oldboy after 10 lines, if any.
1,/Alex/{sed-commands} operate on lines 1 to matching Alex
/oldboy/,+2{sed-commands} operate on the line matching oldboy to the next 2 lines
2.2 Delete
d delete the specified line [root@oldboy ~]# sed 'd' person.txt [root@oldboy ~]# [root@oldboy ~]# sed '2d' person.txt 101,oldboy,CEO 103,Alex,COO 104,yy,CFO 105,feixue,CIO [root@oldboy ~]# sed '2,5d' person.txt 101,oldboy,CEO [root@oldboy ~]# sed '3,$d' person.txt 101,oldboy,CEO 102, zhangyao, CTO [root@oldboy ~]# sed '1~2d' person.txt 102, zhangyao, CTO 104,yy,CFO [root@oldboy ~]# sed '1,+2d' person.txt 104,yy,CFO 105,feixue,CIO [root@oldboy ~]# sed '/zhangyao/d' person.txt 101,oldboy,CEO 103,Alex,COO 104,yy,CFO 105,feixue,CIO [root@oldboy ~]# sed '/oldboy/,/Alex/d' person.txt 104,yy,CFO 105,feixue,CIO [root@oldboy ~]# sed '/oldboy/,3d' person.txt 104,yy,CFO 105,feixue,CIO
Enterprise case 2: Print file content but no oldboy
[root@oldboy ~]# sed '/oldboy/d' person.txt #→delete the line containing "oldboy" 102, zhangyao, CTO 103,Alex,COO 104,yy,CFO 105,feixue,CIO
2.3 Change
2.3.1 Replace by line
c replace old line with new line
[root@oldboy ~]# sed '2c 106,dandan,CSO' person.txt 101,oldboy,CEO 106, dandan, CSO 103,Alex,COO 104,yy,CFO 105,feixue,CIO
2.3.2 Text replacement
s: use alone → replace the first matching string in each line ==> sed command
g: replace all of each line ==> one of the replacement flags of sed command, non-sed command
-i: modify the file Content ==> Options for sed software
The sed software replaces the model (the box ▇ is replaced with a triangle ▲)
sed -i 's/▇/▲/g' oldboy.log sed -i 's#▇#▲#g' oldboy.log
Observation characteristics
- The two sides are quotation marks, the two sides inside the quotation marks are
ssumg, and the middle is three same characters/or#as delimiters.#Being able to include in the replacement content/helps differentiate. The delimiter can be any symbol such as:or|, but when the replacement content contains the delimiter, it needs to be escaped:|. After long-term practice, it is recommended that you use it#as a delimiter. - Delimiter
/or#, between the first and second is the replaced content, and between the second and third is the replaced content. s#▇#▲#g, ▇ can use regular expressions, but ▲ cannot, it must be specific.- The default sed software operates on pattern space (data in memory), while the -i option changes the file contents on disk.
[root@oldboy ~]# sed 's#zhangyao#oldboyedu#g' person.txt 101,oldboy,CEO 102,oldboyedu,CTO 103,Alex,COO 104,yy,CFO 105,feixue,CIO [root@oldboy ~]# cat person.txt 101,oldboy,CEO 102, zhangyao, CTO 103,Alex,COO 104,yy,CFO 105,feixue,CIO [root@oldboy ~]# sed -i 's#zhangyao#BBB#g' person.txt [root@oldboy ~]# cat person.txt 101,oldboy,CEO 102, BBB, CTO 103,Alex,COO 104,yy,CFO 105,feixue,CIO [root@oldboy ~]# sed -i 's#oldboyedu#zhangyao#g' person.txt #→Restore the test file
Enterprise case 3: Specify a line to modify the configuration file
Specify the line to modify the configuration file exactly, which can prevent too many modifications.
[root@oldboy ~]# sed '3s#0#9#' person.txt 101,oldboy,CEO 102, zhangyao, CTO 193,Alex,COO 104,yy,CFO 105,feixue,CIO
2.3.3 Variable Substitution
[root@oldboy ~]# cat test.txt #→Create a new text a b a [root@oldboy ~]# x=a [root@oldboy ~]# y=b [root@oldboy ~]# echo $x $y a b [root@oldboy ~]# sed s#$x#$y#g test.txt b b b [root@oldboy ~]# sed 's#$x#$y#g' test.txt a b a [root@oldboy ~]# sed 's#'$x'#'$y'#g' test.txt b b b [root@oldboy ~]# sed "s#$x#$y#g" test.txt b b b [root@oldboy ~]# eval sed 's#$x#$y#g' test.txt b b b
2.3.4 Instructions for grouping replacement \( \)sum\1
The function of the sed software \( \)can remember part of the regular expression, among which, \1the first remembered pattern is the matching content in the first parentheses, and the \2second remembered pattern is the second parenthesis. Matching content, sed can remember up to 9.
Example: echo I am oldboy teacher.If you want to keep the word oldboy in this line, delete the rest, and use parentheses to mark the part you want to keep.
[root@oldboy ~]# echo I am oldboy teacher. |sed 's#^.*am \([a-z].*\) tea.*$#\1#g' oldboy [root@oldboy ~]# echo I am oldboy teacher. |sed -r 's#^.*am ([a-z].*) tea.*$#\1#g' oldboy [root@oldboy ~]# echo I am oldboy teacher. |sed -r 's#I (.*) (.*) teacher.#\1\2#g' amoldboy
Command description
Idea: oldboyreplace with charactersI am oldboy teacher.
The following explanation □replaces spaces with
^.*am□--> This sentence means to start with any characteram□and match theI am□string in the file;\([a-z].*\)□–> The shell of this sentence is parentheses\(\）, which[a-z]means to match any one of the 26 letters, which[a-z].*together match any number of characters. In this question, it is the matchingoldboystring. Since theoldboystring needs to be reserved, it is enclosed in parentheses. Match, then pass\1to take theoldboystring.□tea.*$–> means空格teastarting with any character and ending with any character, which is actually theoldboystring immediately after the matching string□teacher.;- The content that is replaced later is
\1to take the content in the preceding parentheses, which is theoldboystring we want. ()It is the meta character of extended regular expressions. The sed software recognizes basic regular expressions by default. If you want to use extended regular expressions, you need to use\escape, ie\(\）. sed uses-rthe option to recognize extended regular expressions,\(\）but it will make an error when using it.
Enterprise case 4: Optimization of system startup items
[root@oldboy ~]# chkconfig --list|grep "3:on"|grep -vE "sshd|crond|network|rsyslog|sysstat"|awk '{print $1}'|sed -r 's#^(.*)#chkconfig \1 off#g'|bash
[root@oldboy ~]# chkconfig --list|grep "3:on"
crond 0:off 1:off 2:on 3:on 4:on 5:on 6:off
network 0:off 1:off 2:on 3:on 4:on 5:on 6:off
rsyslog 0:off 1:off 2:on 3:on 4:on 5:on 6:off
sshd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
sysstat 0:off 1:on 2:on 3:on 4:on 5:on 6:off
2.3.5 Special symbols &represent replaced content
[root@oldboy ~]# sed '1,3s#C#--&--#g' person.txt #→here & equals C 101,oldboy,--C--EO #→Replace C in lines 1 to 3 with --C-- 102,zhangyao,--C--TO 103, yy,-C-OO 104,feixue,CFO 105,makeup,CIO
Business case 5: Batch renaming files
There are files in the current directory as follows:
[root@oldboy test]# ls stu_102999_1_finished.jpg stu_102999_2_finished.jpg stu_102999_3_finished.jpg stu_102999_4_finished.jpg stu_102999_5_finished.jpg
It is required to use the sed command to rename, the effect is stu_102999_1_finished.jpg==>stu_102999_1.jpgto delete the file name_finished
2.4 Check
p outputs the specified content, but it will output the result of 2 matches by default, so use n to cancel the default output
2.4.1 Query by row
[root@oldboy ~]# sed '2p' person.txt 101,oldboy,CEO 102, zhangyao, CTO 102, zhangyao, CTO 103,Alex,COO 104,yy,CFO 105,feixue,CIO [root@oldboy ~]# sed -n '2p' person.txt 102, zhangyao, CTO [root@oldboy ~]# sed -n '2,3p' person.txt 102, zhangyao, CTO 103,Alex,COO Description: Use sed to fetch a line, the easiest [root@oldboy ~]# sed -n '1~2p' person.txt 101,oldboy,CEO 103,Alex,COO 105,feixue,CIO [root@oldboy ~]# sed -n 'p' person.txt 101,oldboy,CEO 102, zhangyao, CTO 103,yy,COO 104,feixue,CFO 105,makeup,CIO
2.4.2 Query by String
[root@oldboy ~]# sed -n '/CTO/p' person.txt 102, zhangyao, CTO [root@oldboy ~]# sed -n '/CTO/,/CFO/p' person.txt 102, zhangyao, CTO 103,Alex,COO 104,yy,CFO
2.4.3 Hybrid query
[root@oldboy ~]# sed -n '2,/CFO/p' person.txt 102, zhangyao, CTO 103,Alex,COO 104,yy,CFO [root@oldboy ~]# sed -n '/feixue/,2p' person.txt 105,feixue,CIO #→Special case, if the first two lines do not match feixue, it will be matched backwards, and if it matches feixue, this line will be printed.
The blog post starts with Zhang Yao's blog: http://www.zyops.com/commands-sed