Table of contents
1.1.1 Match the beginning of the line
2.4 Basic elements and execution methods of Shell scripts
1. Text processing tools
1.1 grep
grep is a text search command, which will search for text lines that match the specified pattern from one or more files, and print the matching results. The general format is: grep [option] [pattern] [file name]
The pattern is a matching rule. The file name after the pattern is used to specify the search target. There can be multiple file names separated by spaces. The options before the pattern are used to supplement the pattern.
Common options
grep is used in conjunction with regular expressions to achieve the search function
1.1.1 Match the beginning of the line
The metacharacter "^" of the regular expression means to match the beginning of the line. When using grep to search for text, use the matching character at the beginning of the line to match the beginning of the line
For example: grep -c ^$ /etc/yum.conf (find how many blank lines are in the yum.conf file)
1.1.2 Set case
Use the i option to use the gerp command case-insensitive, and use the symbol [ ] command to achieve this function
Example: grep '[Tt] [Hh][Ii][Ss]' yum.conf (looks for "this" string case insensitive)
1.1.3 Escape characters
If the matching target string contains metacharacters, you need to use the escape character "\" to mask them. The common metacharacters are " . ", " - ", and you need to use "\ " to escape
write file test1
abcdef
---
123456
grep ' \ - \{ 3 \ } ' test (find '---' in the file test1)
1.2 sed
sed is a non-interactive text processing information that can edit text files and standard input. Standard input includes keyboard, file redirection, strings, variables, etc.
Usage environment: use this command when the editing command is complex, the file is too large, and text processing needs to execute multiple functions
Advantages: handle all editing tasks at one time, improve efficiency and save time
Format: sed [option] 'sed editing command' input file (sed editing command should be enclosed in single brackets)
Common options:
Edit command:
For example: sed '=' test1 (display line number output file test1)
1.2.1 Appending files
The editing command a\ of sed is used to append text. It can specify one or more lines of text to be appended to the specified position in the form of matching pattern "/pattern/" or line number. If not specified, it will be placed after each line by default.
For example: sed '/123456/a\add a new line!!!' test1 (append new text add a new line!!!)
Append text script implementation:
vi add.sh
/For rain/a\ we add a new line!!!
The above only appends the output results in the standard output, and does not add these new texts to the original file. If you want to modify the file, you need to use the -i option
sed -f add.sh -i poem
1.2.2 Delete text
To delete a line, use the editing command d
sed '/we add new line!!!/d' test1
#! been/sed -f
/For rain/a\
we add a new line!!!
There is no command interpreter specified in add.sh, and the corresponding command needs to be called to execute the script
sed -f add.sh test1
Specify the command interpreter in the add.sh script file (the first line)
#! /bin/sed -f
1.3 awk
When awk reads and analyzes data, it scans the contents of the file line by line from the beginning to the end, looks for lines matching the specified pattern, and processes the matched text. The workflow of awk is divided into two steps: pattern matching and processing, two special patterns Divided into BEGIN and END
The matching modes in awk mainly include relational expressions, regular expressions, mixed modes, BEGIN, END
Pattern matching:
1. If no matching pattern is specified, all data will be matched by default
Data processing:
1. Match and process data
2. If it does not match, do nothing
3. If no processing method is specified, output
Two special modes:
BEGIN: Placed before reading data, the flag data reading is about to start
END: Execute after reading the data, indicating that the data reading has been completed
Command basic format: awk [option] pattern {actions} file
pattern is the matching pattern, and actions is the operation to be performed. When the text conforms to the rules, both pattern and actions are optional, and one must be selected:
pattern is the matching pattern (optional) if pattern is omitted, actions will be performed on all text
actions is the action to perform (optional) omit actions will print the matching result to the terminal
Common commands
1.3.1 Relational expressions
Awk allows users to use relational expressions as matching patterns, and when the file line satisfies the relational expression, the corresponding operation will be performed
#! bin/bash
result = ` awk ' $2 > 80 {print} ' scores `
echo "$result"
The variable $2 means to refer to the value of the second column, and {print} means to print the text line that matches successfully
(variable $0: refers to the value of all columns variable $NF: refers to the value of the last column)