Linux production environment, the most commonly used "Sed" skills

sedThe command is widely used and simple to use, and it is a tool for fast text processing. In fact, it does not have many skills. Recitation and use are the most suitable learning channels. They are hard skills. But it is complicated because there are too many advanced features. This article does not pay attention to the advanced functions of sed, only to explain some commonly used operations.

With the use, you will find that it vimshares some concepts with each other. The syntax of regular expressions is basically the same, and there is not much learning cost. From a personal perspective and work efficiency, the sed command is an important tool that programmers must master.

Those who say that they can use Google on the spot, mostly used to copy the text into excel, slowly grind the foreign workers, even more trouble when encountering large quantities of documents. It's not that one family doesn't enter the house, this article is not written for you.

A simple entry

As shown, a simple sed command comprises three main parts: 参数, 范围, 操作. The file to be operated can be directly hung at the end of the command line. In addition to the command line, sed can also specify a sed script through the -f parameter. This is an advanced usage and will not be described too much.

I will repeat some example commands many times. If you are smart enough to find the pattern, sometimes you ca n’t even explain it.

parameter

-n  This parameter means --quietor --silent. It indicates that the output of the execution process is ignored, and only our results can be output.

We used there is another argument: -i.

After using this parameter, all changes will be executed on the original file. Your output will overwrite the original file. Very dangerous , be sure to pay attention.

range

1,4  means find the contents of lines 1, 2, 3, 4 in the file.
The designation of this range is very flexible, please see the following example (please replace the range part in the figure yourself).

5  Select line 5.
2,5  Select 2 to 5 lines, a total of 4 lines.
1 ~ 2  select odd rows.
2 ~ 2  select even lines. The effect of
2, +3  and 2,5is the same, a total of 4 lines.
2, $  From the second line to the end of the file.

Range selection can also use regular matching. See the example below.

/ sys /, +3  select the line where sys appears, and the next three lines.
/ \ ^ sys /, / mem /  select the data between the line beginning with sys and the line where the word mem appears.

For intuitiveness, the following commands correspond to the above introduction one by one, and there can be spaces between the scope and the operation.

sed -n '5p' file
sed -n '2,5 p' file
sed -n '1~2 p' file
sed -n '2~2 p' file
sed -n '2,+3p' file
sed -n '2,$ p' file

sed -n '/sys/,+3 p' file
sed -n '/^sys/,/mem/p' file

operating

The most common operation is pto print. For example, the following two commands are equivalent:

cat file
sed -n 'p' file

In addition to printing, there are the following operations, which we commonly use.

p  Print the matching content.
d  Delete the matching content. At this time, it is necessary to remove the -nparameters, think about why.
w  Write the matching content elsewhere.

a, i, cAnd other operations, although basic but with less, does not describe. We still take some commands to illustrate.

sed -n '2,5 p' file
sed    '2,5 d' file
sed -n '2,5 w output.txt' file

Let's take a look at what the sed command can do, and give it a try.

Delete all lines starting with # and blank lines.

sed -e 's/#.*//' -e '/^$/ d' file

The most commonly used, such as the following.

sed -n '2p' /etc/group

Print the second line in the group file.

1、参数部分 比如 -n
2、模式部分 比如'2p'
3、文件，比如/etc/group

So what if I want to execute multiple commands at one time and don't want to write sed script files? Then you need to add the -e parameter.

The operation unit of sed is 行.

Replacement mode

The above is sedthe commonly used matching pattern of the command, but it also has a powerful replacement pattern, which means to find and replace some of the values, and output the result. Using the replacement mode rarely uses -nparameters.

There are a lot of parameters for the replacement mode, but both the first and fifth parts can be omitted. After replacement, the entire text will be output.

The first half is used to match some ranges, while the second half performs replacement actions.

range

This range is similar to the range syntax above. See the example below.

/ sys /, +3  select the line where sys appears, and the next three lines.
/ \ ^ sys /, / mem /  select the data between the line beginning with sys and the line where the word mem appears.

The specific commands are:

sed '/sys/,+3 s/a/b/g' file
sed '/^sys/,/mem/s/a/b/g' file

command

The command here refers to s. That is the meaning of substitute.

Find a match

The search section will find the string to be replaced. This part can accept pure strings or regular expressions. See the example below.

a  Find the string in the range line a.
[a, b, c]  Find the string a or b or c from the range line.

The command is similar:

sed 's/a/b/g' file
sed 's/[a,b,c]/<&>/g' file#这个命令我们下面解释

replace

It's time to replace the found string. The content in this section will replace the content found in the Find Match section.

It is a pity that regularity cannot be used in this part. Commonly used is precise replacement. For example, replace a with b.

But there are also advanced features. Python and java or similar regular api, sed replacement also has Matched Patternmeaning, can also be obtained Group, not the bottom. Commonly used substitutes are &.

&No., repeat it again. When it is used in the replacement string, it represents the original search matching data.

[&]  Indicates that the found data is surrounded by [].
"&"  Indicates that the searched data is surrounded by "".

The following command will enclose each line in the file with quotation marks.

sed 's/.*/"&"/' file

flag parameter

These parameters can be used individually or in multiples, only the most commonly used are introduced.

 By default, g only matches the first occurrence of the line, plus g, you can replace the full text. Commonly used.
p  When the -nparameter is used, ponly the content of the matching line will be output.
w is  similar to the w mode above, but it only outputs transformed lines.
The i  parameter is more important and means that case is ignored.
e  means execute one command for each line to be output. Not recommended, you can use xargs to complete this function.

Look at the syntax of the two commands:

sed -n 's/a/b/gipw output.txt' file
sed 's/^/ls -la/e' file

Fun

Due to regularity, many characters need to be escaped. You will do a lot in the script \\, \*processing and the like. You can use |^@!four characters to replace \.
For example, the following five commands are the same.

sed '/aaa/s/\/etc/\/usr/g' file
sed '/aaa/s@/etc@/usr@g' file
sed '/aaa/s^/etc^/usr^g' file
sed '/aaa/s|/etc|/usr|g' file
sed '/aaa/s!/etc!/usr!g' file

Note: The first half of the range cannot be used this way. I am used to using symbols @.

other

Regular expression

As you can see, regular expressions are everywhere on the command line. Below, only a brief explanation.

^  First line
$  end of line
.  Single character
*  0 or more matches
+  1 or more matches
?  0 or 1 matches
{m} The  preceding match is repeated m times
{m, n} The  preceding match is repeated m to The n-time
\  escape character
[0-9]  matches any character in parentheses, the role of
or |, or
\ b  matches a word. For example, \blucky\b only match the word lucky

Parameter i

The parameter i has been briefly introduced above, and its role is to allow the operation to be performed in the original file. No matter what you do, the original file will be overwritten. This is very dangerous.
By adding a parameter, you can make a backup of the original file.

sed -i.bak 's/a/b/' file

The above command will take effect on the original file and generate a file.bak file. It is strongly recommended to use the i parameter to specify the bak file at the same time.

Perform

We use two commands to take a look at the combined power of sed and other commands.

Output lines with a length of not less than 50 characters

sed -n '/^.{50}/p'

Count how many times each word appears in the file

sed 's/ /\n/g' file | sort | uniq -c

Find the py file in the directory and delete all line-level comments

find ./ -name "*.py" | xargs sed  -i.bak '/^[ ]*#/d'

View lines 5-7 and 10-13

sed -n -e '5,7p' -e '10,13p' file

Only output ip address

ip route show | sed -n '/src/p' | sed -e 's/  */ /g' | cut -d' ' -f9

-------------------------