sed is a stream editor that processes content one line at a time. When processing, the currently processed line is stored in a temporary buffer, which is called "pattern space", and then the content in the buffer is processed with the sed command. After the processing is completed, the content of the buffer is sent to the screen. Then process the next line, and repeat until the end of the file. The file contents are not changed unless you use redirected storage output. Sed is mainly used to automatically edit one or more files; simplify repeated operations on files; write conversion programs, etc.

2) The working process of the sed stream editor

The workflow of sed mainly includes three processes of reading, executing and displaying:

Read: sed reads a line from the input stream (file, pipe, standard input) and stores it in a temporary buffer (also known as pattern space, pattern space)
Execution: By default, all sed commands are executed sequentially in the pattern space. Unless a line address is specified, the sed command will be executed sequentially on all lines
Display: Send the modified content to the output stream. After sending data, the pattern space will be cleared. Before all the file content has been processed, the above process will be repeated until all the content has been processed

Before all the file content has been processed, the above process will be repeated until all the content has been processed

Note: By default, all sed commands are executed in the pattern space, so the input file will not change unless you use "sed -i" to modify the source file, or redirect the output to a new file

3) How to solve the problem that the sed command processing capacity is too large, or the execution efficiency is slow due to too much content?

Solution one (recommended usage):

Use the split command to split the file (for example, if the file is a million lines, create a separate directory to divide the file into one hundred texts of 10,000 lines) and then use the sed command for processing. In addition to the split split, you can also use a Traversing split shell scripts for execution

Solution two:

Use cat file name|sed to process (but this solution can only be used for medium and large file text, if the text volume is too large, the processing effect will not be good)

Two, sed command format and option operator

1) sed command format

Basic operation format:

sed -e 'operation' file1 file2
sed -n -e 'operation' file1 file2
sed -f script file file1 file2
sed -i -e 'operation' file1 file2

Format for executing multiple commands:

method one:

sed -n -e '操作1' -e '操作2' 文件

Method 2:

 sed -n -e '操作1;操作2' 文件

Method 3:

sed -e 'n{  

操作1  

操作2  

......  

}' 文件1

2) Common options of the sed command

selection	effect
-e or --expression=	Indicates that the specified command is used to process the input text file. It can be omitted when there is only one operation command. It is generally used when executing multiple operation commands
-f or --file=	Indicates that the specified script file is used to process the input text file
-h or --help	show help
-n, --quiet, or --silent	Suppresses the sed editor output, but can be used with the p command to complete the output
-i	Modify the target text file directly

3) The operator of the sed command

operator	effect
s	Replace, replace the specified character
d	delete, delete the selected row
a	Add, add a line of specified content below the current line
i	Insert, insert a row of specified content above the selected row
c	Replace, replace the selected lines with the specified content
y	Character conversion, the character length before and after conversion must be the same
p	Print the line contents. If the line is specified at the same time, it means to print the specified line; if no line is specified, it means to print all the content; if there are non-printing characters, it will be output in ASCII code. It is usually used with the "-n" option
=	print line number
l (lowercase L)	Print text and non-printable ASCII characters in the data stream (such as the terminator $, tab character \t)

3. The printing function of the sed command

1) Default printing method

Default printing with no requirements

[root@localhost sed]#sed -e 'p'  english.txt 
 
 
[root@localhost sed]#sed -n 'p'  english.txt

Operations on line numbers when printing

[root@localhost sed]#sed -n '=' english.txt 
 
 
[root@localhost sed]#sed -n '=;p' english.txt

Addressed print of sed command

① Print the line interval in digital form

Method 1: Find the content according to the line number

[root@localhost sed]#sed -n '1p'  english.txt 
 
[root@localhost sed]#sed -n '4p'  english.txt 
 
[root@localhost sed]#sed -n '$p'  english.txt

Method 2: Print the range of line numbers

[root@localhost sed]#sed -n '1,3p' english.txt 
 
[root@localhost sed]#sed -n '5,$p' english.txt 
 
[root@localhost sed]#sed -n '5,+2p' english.txt 
 
[root@localhost sed]#sed -e '5q' english.txt

Method 3: Print at specified intervals

[root@localhost sed]#sed -n -e '5p' -e'$p'  english.txt 
 
 
[root@localhost sed]#sed -n -e '2p' -e'3p'  english.txt

Method 4: Printing of odd and even lines

[root@localhost sed]#sed -n 'n;p'  english.txt 
 
[root@localhost sed]#sed -n 'p;n'  english.txt

② Text mode filter line content

Method 1: Filter and print the included strings

[root@localhost sed]#sed -n '/o/p' english.txt 
 
[root@localhost sed]#sed -n '/th/p' english.txt

Method 2: Apply basic regular expressions to print

[root@localhost sed]#sed -n  '/^root/p'   /etc/passwd
 
[root@localhost sed]#sed -n  '/bash$/p'   /etc/passwd
 
[root@localhost sed]#sed -n  '4,/bash$/p'   /etc/passwd

Method 3: Print using extended regular expressions

Notice:

sed -r supports extended regular expressions. At the same time, when using {n}, {n,}, {n,m}, there is no need to add a backslash \ before the brackets {}

[root@localhost sed]#sed -r -n  '/(99:){2,}/p'   /etc/passwd
 
[root@localhost sed]#sed -r -n  '/^root|bash$/p'   /etc/passwd

Four, sed delete operation

Notice:

The following operations are all performed by default, and have the effect of testing. sed -i will actually operate on the text (it is recommended to back up the target file first, and then operate)

1) Delete by line number

[root@localhost sed]#sed -n '3d;p' english.txt 
 
[root@localhost sed]#sed -n '5,8d;p' english.txt 
 
[root@localhost sed]#sed -n '5,$d;p' english.txt

[root@localhost sed]#sed   '4,6!d'   english.txt

2) Matching string content deletion

[root@localhost sed]#sed   '/one/d'   english.txt 
 
[root@localhost sed]#sed   '/one/,/six/d'   english.txt

[root@localhost sed]#sed   '/one/,/six/!d'   english.txt 
 
[root@localhost sed]#sed   '/six/!d'   english.txt

3) String matching regular to delete

[root@localhost sed]#sed   '/^$/d'   english.txt

Three ways to delete blank lines:

grep -v "^$" file.txt //filter out non-empty lines
cat file.txt |tr -s "\n" //compress line breaks
sed '/^$/d' file.txt //delete blank lines

Five, sed command replacement

格式：

行范围 s/旧字符串/新字符串/替换标记


替换标记：

数字：表明新字符串将替换第几处匹配的地方  

g：表面新字符串将会替换所有匹配的地方

p：打印与替换命令匹配的行，与-n一起使用

w 文件：将替换的结果写入文件中 

 
sed命令的替换中：

s：替换字符串

c：整行替换

y：字符替换，替换前后的字符串长度必须相同

1) String replacement

Match a single character for replacement

[root@localhost sed]#sed -n 's/root/test/2p'  /etc/passwd
 
[root@localhost sed]#sed -n 's/root/test/gp'  /etc/passwd

[root@localhost sed]# sed -n '/^root/ s/^/#/p'   /etc/passwd

2) Perform alphabetic characters for size replacement

uppercase to lowercase

[root@localhost sed]#sed -i 's/[A-Z]/\l&/g'  english.txt

lowercase to uppercase

[root@localhost sed]#sed -i 's/[a-z]/\u&/'  english.txt 
 
[root@localhost sed]#sed -i 's/[a-z]/\U&/'  english.txt

[root@localhost sed]#sed -i 's/[a-z]/\U&/g'  english.txt

3) Knowledge expansion "//" can be replaced by other symbols

Classic delimiter conversion case

At this time, 9 replaces the role of /, adding \ is the number 9, otherwise it is a separator

4) replace the whole line

Using c to replace is to replace the entire line of content

[root@localhost sed]# sed '/ONE/c 22' english.txt 
 
[root@localhost sed]# sed '/TWO/c TEST' english.txt

[root@localhost sed]# sed 'y/TH/12/' english.txt

5) Single character replacement

#使用y，是对单个字符进行替换，每个字符需要一一对应，不是整体替换。前后字符串长度需要一致，不然会报错

[root@localhost sed]# sed 'y/TH/12/' english.txt

Six, the increase of sed command

a：在行后添加内容

i：在行前插入内容

r：在行后读入文件内容

1) sed command line insertion

[root@localhost sed]# sed '/THREE/a  123 ' english.txt 
 
[root@localhost sed]# sed '/THREE/i  123 ' english.txt

2) Read the file and insert it after the sed command line

[root@localhost sed]#sed  '$r english2.txt'  english.txt

Seven, sed command to copy and paste

#H复制、d删除、G粘贴到指定行下方

[root@localhost sed]#sed  '1,3 {H;G};$G' english.txt 
 
[root@localhost sed]#sed  '1,3 {H;d};$G' english.txt

Eight, the position exchange of strings and characters in the sed command

[root@localhost sed]#echo 123abc|sed -r 's/(123)(abc)/\2\1/'

[root@localhost sed]#echo 123abc|sed -r 's/(.)(.)(.)(.)(.)(.)/\6\5\4\3\2\1/'

sed stream editor for Linux text

1. The relevant knowledge of sed and its workflow

1) Introduction to sed editor