ed
sed stream editor means (Stream Editor), as a filter and is common in the Makefile Shell scripts, i.e. the output of a program introduced before the input sed, after a series of editing commands output converted to another format. sed and vi are derived from the early UNIX-ed tools, so many sed command and the last line vi commands are the same.
The basic format of the command line is sed
sed option 'script' file1 file2 ...
sed option -f scriptfile file1 file2 ...
Option Meaning:
sed --version display version.
--help display this help document.
-n, - quiet, - silent silent output, by default, sed program after all the script instruction completes, will automatically print the contents of the pattern space, these options can be shielded automatically print.
-e script allows multiple script instruction is executed.
-f script-file,
--file = script-file commands to read from a script file, great for writing automated scripts for the program!
-i , - in-Place directly modify the source file, the script after the processed instruction content to be output to the source file (source file is modified) with caution!
-l N, --line-length = N This option specifies the line length l can be output instruction, l instruction for outputting the non-printing characters.
--posix disabled GNU sed extension.
-r, --regexp-extended use extended regular expressions in the script instructions
Under -s, --separate default, sed command will specify a file name as a plurality of long continuous input stream. The GNU sed allows them as separate files, such as cross-file regular expression matching is performed.
-u, input and output buffers --unbuffered minimum.
These are only a description of the program itself sed option function, the specific instructions of the script (ie the file content to do the operation) Later we will describe in detail, here is a brief introduction of several examples of sed program script instructions.
a, append additional
i, insert inserted
d, delete delete
s, substitution replacing
Such as: $ sed "2a itcast" ./testfile Add "itcast" in the second line of output testfile content.
$ sed "2,5d" testfile
Sed file may be processed by a standard input redirection obtained, may be passed as command line parameters, a command line parameters can be passed multiple files, sed will be sequentially processed. sed editing commands can be directly used as command line parameters passed, you can also write a script file with the -f parameter and specify the format for editing commands:
/pattern/action
Where pattern is a regular expression, action is editing. sed program line by line read out file to be processed, and if a line pattern matching, the corresponding action is executed, if a command is not the only action pattern, the action will be applied to each line of the file to be processed.
Common sed command
/ Pattern / p print lines that match pattern
/ Pattern / d delete lines that match pattern
/ Pattern / s / pattern1 / pattern2 / find the line in line with the pattern, the line matching string is replaced pattern1 pattern2
/ Pattern / s / pattern1 / pattern2 / g to find qualified pattern row, all rows matching the replacement string is pattern2 pattern1
Note that using p, sed is the content file to be processed together with the processing result output to the standard output, and therefore represents a p command to print out except that the contents of the file than further additional print lines that match pattern again. For example, the contents of a file testfile is
123
abc
456
Print lines which contain abc's
$ sed '/abc/p' testfile
123
abc
abc
456
To output only the results, should be added to the -n option, this usage is equivalent to grep command
$ sed -n '/abc/p' testfile
abc
-N do not need to use the d command parameters, and such drop a line abc
$ sed '/abc/d' testfile
123
456
Note , sed command does not modify the original file, delete the command line is not only that some of the printout, rather than deleted from the original file.
Alternatively when using the Find command, can be copied to pattern2 pattern1 string matching, such as:
$ sed 's/bc/-&-/' testfile
123
a-bc-
456
& pattern2 the string representing the current row with the original file matches pattern1
Another example:
$ sed 's/\([0-9]\)\([0-9]\)/-\1-~\2~/' testfile
-1-~2~3
abc
-4-~5~6
pattern2 the \ 1 represents a first pattern1 of () matches the braces, \ pattern1 2 represents the second () that matches the content of the parentheses. sed default Basic specification regular expressions, if the -r option is specified using Extended specification, then the () brackets would not have escaped. Such as:
But r 's / ([0-9]) ([0-9]) / - \ 1 \ 2 ~ /' out.sh
After the replacement, all the rows containing the first number of consecutive numbers are added before and after a "-" sign; are added before and after the second number of the "-" sign.
You can specify a plurality of different alternative command, with ";" separated:
$ sed 's/yes/no/;s/static/dhcp/' ./testfile
NOTE: instruction separated by a semicolon.
You can also use the -e option to specify a different substitution command, there are few replace the need to add a few -e command parameters:
$ sed -e 's/yes/no/' -e 's/static/dhcp/' testfile
Note: Use the -e option.
If the content is testfile
<html><head><title>Hello World</title></head>
<body>Welcome to the world of regexp!</body></html>
Now to remove all the HTML tags, the output is:
Hello World
Welcome to the world of regexp!
How to do it? If you use the following command
$ sed 's/<.*>//g' testfile
The result is two blank lines, all the characters are filtered out. This is because the number of regular expression matching qualifier string as long as possible, this is called greedy (Greedy). Processing a first row in such sed, <*> not <html> or <head> tag such matching, but
<html><head><title>Hello World</title>
Such a full line, since this is the beginning of the line <, any number of characters in the middle, at the end is>. So how to change this command is the correct one? Left students thinking exercises.
awk
sed in units of processing files, awk is stronger than sed place not only in units but also as a unit to process the file. awk default row is a newline delimiter, the default column delimiter is continuous spaces and the Tab , but the row and column delimiters separator can be customized, each row such as / etc / passwd file has several fields, fields are separated by:, can be re-defined as a column separator awk: and in this file as a unit process. awk is actually a very complex scripting languages, like C language as well as branching and looping structures, but the basic usage and similar sed, awk basic form of the command line is:
awk option 'script' file1 file2 ...
awk option -f scriptfile file1 file2 ...
And, like sed, awk file processing either from the standard input redirection get, can also be passed as command line arguments, you can directly edit the command line arguments passed when the command, you can specify a script file with the -f parameter editing commands the format is:
/pattern/{actions}
condition{actions}
And similar sed, pattern is a regular expression, actions are a series of operations . awk program line by line read out file to be processed, if a row matches the pattern, or condition satisfied condition, performing the corresponding actions, actions only if a portion awk command, the processing actions to be applied to each line of the file. For example, the contents of the file testfile represents a store inventory of:
Products 30
ProductB 76
ProductC 55
Print each line of the second column:
$ awk '{print $2;}' testfile
30
76
55
Automatic variable $ 1, $ 2, respectively, for the first column, second column, etc., similar to the position parameter Shell scripts, while $ 0 represents the entire current line. As another example, if a product inventory is less than 75 marked the end of the line need to order:
$ awk '$2<75 {printf "%s\t%s\n", $0, "REORDER";} $2>=75 {print $0;}' testfile
Products 30 REORDER
ProductB 76
ProductC 55 REORDER
Awk also be seen and is very similar to the C language printf function. awk condition command portion may also be two special condition-BEGIN and the END, for each file to be processed, the latter BEGIN actions performed once, the latter END actions performed after the entire file once processed before processing the entire file.
awk commands can use the same language like C variables (but do not need to define variables), such as the number of blank lines in a file of statistics
$ awk '/^ *$/ {x=x+1;} END {print x;}' testfile
Like Shell environment variables, some awk variable is predefined have special meaning:
awk commonly used built-in variables
FILENAME current input file name of the file, the variable is read-only
NR row number of the current line, the variable is read-only, R representative of record
NF number of columns in the current row have, the variable is read-only, F represents the field
Column separator OFS output format, space by default
Column melting points FS input symbol file, the default is contiguous spaces and Tab
ORS line separator output format, default newline
RS line delimiter input file, the default is a newline
List of user accounts such as printing system
$ awk 'BEGIN {FS=":"} {print $1;}' /etc/passwd
awk may be used as the C language as if / else, while, for the control structure. Scalable self-learning.