sed:
1. Overview
(1) Pattern space: When processing, the currently processed line is stored in a temporary buffer, called "pattern space"
(2) sed is mainly used to automatically edit one or more files; Repeated operation of the file; writing conversion programs, etc.
(3) The content of the file does not change unless you use redirection to store the output
2. Addressing
The address consists of numbers, two lines separated by commas.
For example, 1, 3 means lines 1, 2, and 3, and the dollar sign ($) means the last line.
The range can be determined by data, regular expressions, or a combination of the two.
3. The delimitation is
separated by ":" "|" "@" "#", etc.
4. The command
calls the sed command in two forms
sed [-nefri] 'command' file(s)
sed [-nefri] -f script file(s)
(1) Common options
-n (important): Use silent mode. Only the line (or action) that has been specially treated by sed will be listed.
-r (important): The sed action supports the extended regular notation syntax. (The default is the base regular notation syntax)
-i (Important): Modify the contents of the read file directly instead of outputting it on the screen.
-e: Edit sed actions directly in command line mode;
-f: Write sed actions directly in a file, -f filename can execute sed actions in filename;
(2) Common commands
a: new , a can be followed by a string, and these strings will appear in a new line (the current next line)
c: replace, c can be followed by a string, and these strings can replace the line
d between n1 and n2 : delete, usually nothing after d
i: insert, i can be followed by strings, and these strings will appear on a new line (the current previous line)
p: print, that is, print a selected content come out. Usually p will work with the parameter sed -n
s: replace, you can directly perform the replacement work, usually the action of this s can be matched with regular notation, such as 1,20s/old/new/g That's it
(3) Replace in a line A certain part of the
format: sed 's/string to be replaced/new string/g' (the string to be replaced can use regular expressions)
awk:
1. Overview
Compared with sed, which often acts on a whole line Processing, awk is more inclined to divide a line into several "fields" for processing. Therefore, awk is quite suitable for small data processing
2. Working principle
awk 'BEGIN{ commands } pattern{ commands } END{ commands }
Step 1: Execute the statements in the BEGIN{ commands } block, only once;
Step 2: Read from a file or standard input (stdin) Take a line, then execute the pattern{commands} block, which scans the file line by line, repeating the process from the first line to the last line, until the file has been read in its entirety.
Step 3: When reading to the end of the input stream, execute the END{ commands } block, only once.
3. Built-in variable
$n The nth field of the current record
$0 This variable contains the text content of the current line during execution.
FNR Same as NR, but relative to the current file.
FS Field Separator (default is any space)
NF is the number of fields, corresponding to the current field number
during execution NR is the number of records, corresponding to the current line number during execution
OFS Output Field Separator (default is one space)
RS record separator (default is a newline)
ORS output record separator (default is a newline)
4.
(1) Arithmetic operators
* Multiplication, / division, & remainder, ^ *** exponentiation
are all used as arithmetic operators to operate, the operands are automatically converted to numeric values, and all non-numeric values become 0
(2) Regular operator
~~! Matching regular expressions and mismatching regular
expressions the following statement. And do the next line match. The net statement is generally used for multi-line merging (2) awk only allows redirected >, >> to output the result to a file 6. Set the field delimiter The default field delimiter is a space, you can use [-F" Delimiter"] explicitly specify a delimiter in the BEGIN statement block, you can use OFS="delimiter" to set the delimiter of the output field 7. Conditional judgment statement if (expression) if (expression) {statement body 1 } Statement 1 else if (expression) else
{statement body 2} statement 2
else
{statement body 3}
can be used after each statement body; semicolon end
8. Loop statement
(1) while loop
while (expression)
{statement}
(2) for loop
for (variable in array) for (variable; condition; expression)
{statement} {statement}
(3) do while loop
do
{statement} while (condition)
array:
1. Read the value of the array
{ for(item in array) {print array [item]}; } #The order of the output is random
{ for(i=1;i<=len;i++) {print array[i]}; } #Len is the length of the array
2, returns the length of the array
(1) string
length returns the string and the length of the array,
split is performed Splitting a string into an array will also return the length of the array obtained by splitting it.
(2) asort sorts the array and returns the length of the array
1. Overview
(1) Pattern space: When processing, the currently processed line is stored in a temporary buffer, called "pattern space"
(2) sed is mainly used to automatically edit one or more files; Repeated operation of the file; writing conversion programs, etc.
(3) The content of the file does not change unless you use redirection to store the output
2. Addressing
The address consists of numbers, two lines separated by commas.
For example, 1, 3 means lines 1, 2, and 3, and the dollar sign ($) means the last line.
The range can be determined by data, regular expressions, or a combination of the two.
3. The delimitation is
separated by ":" "|" "@" "#", etc.
4. The command
calls the sed command in two forms
sed [-nefri] 'command' file(s)
sed [-nefri] -f script file(s)
(1) Common options
-n (important): Use silent mode. Only the line (or action) that has been specially treated by sed will be listed.
-r (important): The sed action supports the extended regular notation syntax. (The default is the base regular notation syntax)
-i (Important): Modify the contents of the read file directly instead of outputting it on the screen.
-e: Edit sed actions directly in command line mode;
-f: Write sed actions directly in a file, -f filename can execute sed actions in filename;
(2) Common commands
a: new , a can be followed by a string, and these strings will appear in a new line (the current next line)
c: replace, c can be followed by a string, and these strings can replace the line
d between n1 and n2 : delete, usually nothing after d
i: insert, i can be followed by strings, and these strings will appear on a new line (the current previous line)
p: print, that is, print a selected content come out. Usually p will work with the parameter sed -n
s: replace, you can directly perform the replacement work, usually the action of this s can be matched with regular notation, such as 1,20s/old/new/g That's it
(3) Replace in a line A certain part of the
format: sed 's/string to be replaced/new string/g' (the string to be replaced can use regular expressions)
awk:
1. Overview
Compared with sed, which often acts on a whole line Processing, awk is more inclined to divide a line into several "fields" for processing. Therefore, awk is quite suitable for small data processing
2. Working principle
awk 'BEGIN{ commands } pattern{ commands } END{ commands }
Step 1: Execute the statements in the BEGIN{ commands } block, only once;
Step 2: Read from a file or standard input (stdin) Take a line, then execute the pattern{commands} block, which scans the file line by line, repeating the process from the first line to the last line, until the file has been read in its entirety.
Step 3: When reading to the end of the input stream, execute the END{ commands } block, only once.
3. Built-in variable
$n The nth field of the current record
$0 This variable contains the text content of the current line during execution.
FNR Same as NR, but relative to the current file.
FS Field Separator (default is any space)
NF is the number of fields, corresponding to the current field number
during execution NR is the number of records, corresponding to the current line number during execution
OFS Output Field Separator (default is one space)
RS record separator (default is a newline)
ORS output record separator (default is a newline)
4.
(1) Arithmetic operators
* Multiplication, / division, & remainder, ^ *** exponentiation
are all used as arithmetic operators to operate, the operands are automatically converted to numeric values, and all non-numeric values become 0
(2) Regular operator
~~! Matching regular expressions and mismatching regular
expressions the following statement. And do the next line match. The net statement is generally used for multi-line merging (2) awk only allows redirected >, >> to output the result to a file 6. Set the field delimiter The default field delimiter is a space, you can use [-F" Delimiter"] explicitly specify a delimiter in the BEGIN statement block, you can use OFS="delimiter" to set the delimiter of the output field 7. Conditional judgment statement if (expression) if (expression) {statement body 1 } Statement 1 else if (expression) else
{statement body 2} statement 2
else
{statement body 3}
can be used after each statement body; semicolon end
8. Loop statement
(1) while loop
while (expression)
{statement}
(2) for loop
for (variable in array) for (variable; condition; expression)
{statement} {statement}
(3) do while loop
do
{statement} while (condition)
array:
1. Read the value of the array
{ for(item in array) {print array [item]}; } #The order of the output is random
{ for(i=1;i<=len;i++) {print array[i]}; } #Len is the length of the array
2, returns the length of the array
(1) string
length returns the string and the length of the array,
split is performed Splitting a string into an array will also return the length of the array obtained by splitting it.
(2) asort sorts the array and returns the length of the array