sed series of articles
- Common Linux commands (9): sed command (edit/replace/delete text)
- linux sed command deletes one line/multiple lines_sed deletes the first line/linux deletes a certain line of the file
- Linux sed batch modification and replacement of the contents/sed special characters in the file
Article directory
- sed series of articles
- 1. Introduction
- 2. sed syntax
- 3. Summary of sed syntax `(important)`
- 4. Actual use/actual combat
1. Introduction
sed (stream editor): Stream editor (The stream editor can edit data received from standard input such as pipes.)
sed 是基于行的，按顺序对每一行执行命令。- When sed processes data, it reads a line into its pattern buffer, executes the editing command, and then prints the pattern buffer. It then repeats these steps for each subsequent row.
- emphasize again,
sed是基于行的，逐行处理数据。
sed (stream editor): sed processes data in text files based on script commands, which are either entered from the command line or stored in a text file. The order in which this command executes data is as follows:
- Only read one line at a time;
- Match and modify data according to the provided rule commands. Note that sed will not directly modify the source file data by default, but will copy the data to the buffer, and the modification is limited to the data in the buffer;
- will output the execution result.
When a row of data is matched, it will continue to read the next row of data and repeat this process until all the data in the file is processed.
1.1. sed working process
The sed editor processes a file (or input) line by line and sends the output to the screen. The sed commands are those found in the vi and ed/ex editors. sed saves the line currently being processed in a temporary buffer, called the pattern space or temporary buffer. After sed has processed a line in the pattern space (that is, after executing the sed command on the line), it sends the new line to the screen (unless there was a previous command to delete the line or cancel the printing operation). sed terminates its operation after each time it has processed the last line of the input file.sed stores each line in a temporary buffer and edits this copy, so the source file is not modified or destroyed.
sed默认不会修改源文件，具体下面的3.1中有讲解。
2. sed syntax
2.1. sed syntax
sed [options] commands [inputfile...]
sed [-n] -f scriptfile files
# 注：
sed和grep不一样，不管是否找到指定的模式，它的退出状态都是0
只有当命令存在语法错误时，sed的退出状态才不是0
2.2. sed common options
options represents some options of the sed command. Common options are as follows:
| Options | illustrate |
|---|---|
| -n | -n or --quiet or --silent uses quiet mode. Under normal circumstances, all STDIN will be output to the screen. After adding -n, only lines specially processed by sed will be printed. |
| -It is | Multiple editing, and the order of commands will affect the results |
| -f | Specify a sed script file to execute on the command line |
| -r | Sed uses extended regular expressions |
| -i | Directly modify the content read from the document without outputting it on the screen |
| -in.back | When a file is modified, a backup file will be created to prevent manual errors. It is equivalent to backing up a .bak file first and then executing the -i command. |
2.3. sed positioning
The sed command will process all lines by default when no position is given
sed supports the following address types:
| instruction | illustrate |
|---|---|
| first~step | first refers to the starting matching line, step refers to the step size, for example: sed -n 2~5p meaning: start matching from the second line, and match every 5 lines, that is, 2,7,12… |
| $ | The $ symbol means matching the last line |
| /REGEXP/ | Indicates the line that matches the regular pattern, matched by the regular pattern between // |
| \cREGEXPc | This is the line that matches the regular pattern. It is matched by the regular pattern between \c and c. c can be any character. |
| addr1，add2 | Addressing addr1, add2 determines which lines are used for editing. The address can be in the form of a number, a regular expression, or a combination of both. If no address is specified, sed will process all lines in the input file. If the address is a number, this number represents the line number. If it is two line numbers separated by a comma, then the address that needs to be processed is the range between the two lines (inclusive). The range can be numeric, regular, or a combination of both. |
| addr1， +N | Match from the addr1 line to N lines below, matching N+1 lines in total. |
| addr1， ~N | Will match addr1 and the lines following addr1 until the next line whose line number is a multiple of N is entered. |
The following shows several examples of sed positioning
sed '1d' filename # 删除第一行
sed '$d' filename # 删除最后一行
sed '2,5d' filename # 删除第2到第5行
2.4. sed regular expression
Like grep, sed can also use regular expressions (RE) and various metacharacters when looking for patterns in files. Regular expressions are enclosed in slashes (/xxx/)
2.5. sed operation instructions
The sed operation command tells sed what to do with each input line specified by the address. If no address is specified, sed will process all lines of input.
The following operation instructions refer to the instructions in sed [options] commands [inputfile…]commands.
| Order | illustrate |
|---|---|
| a\ | Add one or more lines after the current line |
| c\ | Modify (replace) the text in the current line with the new text |
| d | Delete row |
| i\ | Insert text before current line |
| h | Copy the contents of the pattern space to the temporary buffer area |
| H | Append the contents of the pattern space to the temporary cache |
| g | Take out the content in the temporary buffer and copy it to the pattern space, overwriting the original content there. |
| G | Take out the content in the temporary buffer, copy it to the pattern space, and append it to the original content. |
| l | List non-printing characters |
| p | print line |
| n | Read in the next input line and start processing at the next command instead of the first |
| q | End or exit sed |
| r | Read input lines from file |
| ！ | Apply command to all lines that are not the selected line |
| s | Replace one string with another |
replacement flag
| g | Global replacement within a line |
| p | print line |
| In | Write lines to file |
| x | Swapping the contents of the scratch buffer and pattern space |
3. Summary of sed syntax（重要）
We have introduced a lot of content above: sed regularity, options, operation instructions, sed positioning, etc. 由于引入的操作指令太多，可能都不清楚怎么使用，这里先引入一些常用的语法便于理解. Here is a list of some commonly used syntaxes:
3.1. sed search content (use of regular expressions)
Regular expression is a pattern enclosed in slashes
(/xxx/)
sed [选项] '[address]/pattern/[flag]' filename
in:
- Address refers to the positioning, refer to the usage of sed positioning above.
- flag is generally the value in the abovesed operation command
Example:
sed -n '/root/p' aa.txt #打印包含root的行 /xxx/是正则表达式的写法
sed -n '10p' aa.txt #打印第10行
sed -n '5,10p' aa.txt #打印第5到第10行
sed -n '5,+5p' aa.txt #从第5行开始，往后打印5行,包括第5行
sed -n '8,/sbin\/nologin/p' aa.txt #打印aa.txt中第8行开始，到含有/sbin/nologin的内容的行结束内容
3.2
4. Actual use/actual combat
4.1. sed will not destroy the source file (sed and sed -i)
sed saves the line currently being processed in a temporary buffer, called the pattern space or temporary buffer. After sed has processed a line in the pattern space (that is, after executing the sed command on the line), it sends the new line to the screen (unless there was a previous command to delete the line or cancel the printing operation). sed terminates its operation after each time it has processed the last line of the input file. sed stores each line in a temporary buffer and edits this copy, so the source file is not modified or destroyed.
4.2. Replacement: s command(重点)
Command s is a replacement command. Replacement and replacement of text in the file can be achieved through s in sed. The text contained in slashes after s is a regular expression, followed by the text that needs to be replaced. Global replacement of lines can be done via the g flag. For more replacement-related commands, refer to "linux sed batch modification of content in replacement files/sed special characters"
grammar:
sed -i 's/原字符串/替换字符串/g' filename
illustrate:
-i: Directly modify the content in the file without adding-i. For the effect, please refer to 3.1- The s command is used for substitution. The g at the end of the command means global replacement within the line; that is, if multiple "males" appear in each line, all "males" will be replaced with "man". Without the g command, only the first "male" on each line is replaced with "man".
4.2.1. sed replacement content contains special characters
If the original string and replacement string in the sed s command contain special characters, they can be escaped. Special characters include: /, &, etc.
4.3. Add: a command
The a command is an append command, which appends new text to the end of the current line in the file (that is, the buffer line in read mode). Whether on the command line or in a sed script, the a command always follows a backslash.
The syntax is as follows:
匹配行后添加
sed -i "/匹配内容/a添加内容 " filename
而在书写的时候为便与区分，往往会在i和a前面加一个反加一个反斜扛 。代码就变成了：
sed -i "/2222222222/a\3333333333 " test.txt
sed -i '1a hello world' test.txt # 1a则是在第一行的后面添加一行，添加的内容就是a后面的内容，如果a的前面没有地址限定则在所有行的后面都会添加指定的字符串
4.4. Insert: i command
The i command is an insertion command, similar to the a command, but instead of adding text after the current line, it inserts new text in front of the current line, that is, the line just read into the buffer mode.
The syntax is as follows:
匹配行后添加
sed -i "/匹配内容/i添加内容 " filename
而在书写的时候为便与区分，往往会在i和a前面加一个反加一个反斜扛 。代码就变成了：
sed -i "/2222222222/i\3333333333 " test.txt
sed -i '1i hello world' test.txt #命令i表示在指定的行的前面插入一行，插入的内容为其后面的字符串，如果i的前面没有地址限定则在所有行的前面都会添加指定的字符串
4.5. Print: p command
The command p is a print command used to display the contents of the pattern buffer. By default, sed prints input lines on the screen, the option -n is used to cancel the default printing operation. When option -n and command p are present together, sed prints the selection
4.6. Delete: d command
Command d is used to delete input lines. sed first copies the input line from the file to the pattern buffer, then executes the sed command on the line, and finally displays the contents of the pattern buffer on the screen. If command d is issued, the input lines in the current mode buffer will be deleted and will not be displayed. For more usage of the d command, please refer to "linux sed command deletes one line/multiple lines_sed deletes the first line/linux deletes a certain line of the file"
sed -i '1d' filename 删除文件第一行
sed -i '2d' filename 删除第二行
sed -i 'nd' filename 删除第n行/n需要是实际数字
sed -i '$d' filename 删除最后一行
sed -i '1,3d' filename 删除1-3行
sed -i '3,$d' filename 删除第3行到最后一行
sed -i '/^$/d' filename # 删除所有空白行
| Edit command | meaning |
|---|---|
| 1 d | Delete the content of line 1 |
| 1,5d | Delete content from lines 1 to 5 |
| 2,+5d | Delete the contents of line 2 and the following 5 lines |
| /pattern1/d | Delete the row content matching pattern1 in each row |
| /pattern1/,/pattern2/d | Delete lines matching pattern1 until all lines matching pattern2 |
| /pattern1/,10d | Delete all lines matching pattern1 to line 10 |
| 10,/pattern1/d | Delete everything from line 10 until pattern1 is matched |
4.7. Specify the range of lines: comma
The range of lines starts at one address in the file and ends at another address. The address range can be a line number (e.g. 5,10), a regular expression (e.g. /Dick/ and /Joe/), or a combination of the two (e.g. /north/, $). The range is closed - including the start condition. line, the line that ends the condition, and the lines in between. If the end condition cannot be met, the operation will continue until the end of the file. If the end condition is met, continue to find the position that satisfies the start condition, and the range starts again.
1>Line number specifies the range
2>Regular expression specifies the range
4.8. Modification: c command
The c command is a modification command. sed Use this command to modify existing text into new text. Old text is overwritten.
sed -i '/Bill/c Billion' test.txt # 将包含Bill的行整行替换为Billion
sed -i '1c hello world' test.txt #命令c会替换指定的行的所有内容，替换成其后面的字符串，所有的新增，删除，替换行，这些命令前面的地址修饰都可以指定地址空间，也都可以使用正则表达式，命令会应用在选出的符合地址条件的所有行上面，例如：
sed -i '/^2/c hello world' test.txt #替换以2开头的行，其内容是c命令后面的字符串
4.9. Exit: q command
The q command indicates the exit command. This command will cause the sed program to exit without further processing.
After printing line 3, q causes the sed program to exit
When sorting out this article, I found some other articles worth learning from: https://blog.csdn.net/bandaoyu/article/details/120047612 https://blog.csdn.net/weixin_42445727/article/details/120431735 https://www .cnblogs.com/A121/p/10621152.html https://www.cnblogs.com/maxincai/p/5146338.html https://www.cnblogs.com/ginvip/p/6376049.html