Advanced Linux commands: grep, sed, awk family bucket (detailed examples of text processing technology)

table of Contents

1. Foreword review

Two, grep

1. What is grep?

2. Analysis of detailed examples of grep usage

Three, sed

1. What is sed?

 2, sed usage detailed example analysis

Four, awk

1. What is awk?

2, awk usage detailed example analysis

Five, summary

---

1. Foreword review

The article I wrote before, " Linux Basic Commands and Programming Environment Experiments ", was once a hot article and became a popular article. As we all know, Linux is a powerful operating system with powerful functions and high stability. It is widely used in large companies and enterprises. It is very important to master Linux related command operations and knowledge systems. Linux is broad and profound. You can feel the wisdom of the original designers only when you learn it. Of course, the Linux system is also the crystallization of the wisdom of the big cows. It is constantly optimized and updated, and many version types have been derived. The relatively familiar and commonly used distribution is Ubuntu.

This article continues to learn and explore Linux, which is an advanced step. On the basis of the previous, we will master more advanced commands through detailed examples , mainly text processing technology, grep, sed, awk three commands , and feel the magical and powerful Linux!

Linux system learning related articles:

1. Linux basic commands and programming environment experiment [ https://blog.csdn.net/Charzous/article/details/108204520 ]
2. Linux environment programming inter-process communication mechanism, play with Linux, and understand more thoroughly! [ https://blog.csdn.net/Charzous/article/details/108287075 ]

Two, grep

1. What is grep?

The UNIX grep family consists of commands grep, egrep, and fgrep. The grep command searches for regular expressions globally in the file and prints all lines containing the expression. The egrep and fgrep commands are variants of grep. The egrep command is an extended grep that supports more re metacharacters. The fgrep command is called fixed grep.

grep command format:

grep [option] pattern filename

pattern is a regular expression, filename is the name of the target operation file, and option is an optional item, including:

-i	Ignore case of letters
-n	Print line number and matched line
-v	Print unmatched lines
-c	Only print the number of matching lines
-w	Search for a given pattern as a string
-l	Only print file names with matching lines

2. Analysis of detailed examples of grep usage

The following example learns the use of grep command, given the donors text in the current directory:

Mike Harrington:(510) 548-1278:250:100:175
Christian Dobbins:(408) 538-2358:155:90:201
Susan Dalsass:(206) 654-6279:250:60:50
Archie McNichol:(206) 548-1348:250:100:175
Jody Savage:(206) 548-1278:15:188:150
Guy Quigley:(916) 343-6410:250:100:175
Dan Savage:(406) 298-7744:450:300:275
Nancy McNeil:(206) 548-1278:250:80:75
John Goldenrod:(916) 348-4278:250:100:175
Chet Main:(510) 548-5258:50:95:135
Tom Savage:(408) 926-3456:250:168:200
Elizabeth Stachelin:(916) 440-1763:175:75:300

 topic:

1. Print all lines containing the string Tom.
2. Print all lines where the person's first name starts with J.
3. Print all lines ending in 175.
4. Print all lines that don't contain 175.
5. Print all lines where the phone number is in the 206 area code.

answer:

1. grep 'Tom' donors
2. grep '^J' donors
3. grep '175\>' donors
4. grep -v '175' donors
5. grep '(206)' donors

result:

1. 
2. 
3. 
4. 
5. 

Three, sed

1. What is sed?

The sed command is a streamlined non-interactive editor (Stream editor) . The sed program does not work interactively with the editor, but allows you to type editing commands on the command line, name the file, and then view the output of the editing commands on the screen. By default, all lines are printed to the screen. The sed editor is non-destructive, you can use shell redirection to save the output, otherwise it will not change the file content.

sed command format:

sed [option] '[address]command' filename

Commonly used option: -n, which means that only matching lines are printed

command is a regular expression in the form of /pattern/ . Here, the sed command is different from grep in that there is an additional address option, which indicates the operation to be performed on the matched text. Commonly used are:

d	Delete row
s	Replace the matched string
p	Print line
a	Append one or more lines of text to the current line

 2, sed usage detailed example analysis

 

topic:

1. Change John's name to Joanthan
2. Delete the first three lines
3. Print lines 5 through 10
4. Delete lines containing Mike
5. Delete all blank lines

 

answer:

1.sed 's/John/Joanthan/' donors
2.sed '1,3d' donors
3.sed -n '5,10p' donors
4.sed '/Mike/d' donors
5.sed '/^[ \t]*$/d' donors

result:

1. 
2. 
3. 
4. 

Four, awk

1. What is awk?

Awk is a UNIX/Linux programming language used to manipulate data and generate reports. nawk is a newer version, and nawk is the Gnu version. Data can come from standard input, one or more files, or it can be the output of a process. Awk can be used for simple operations on the command line, or it can be written into programs for large applications. Because awk can manipulate data, it is an indispensable tool for shell scripts and management of small databases.

awk command format:

1. awk 'pattern' filename
2. awk '{action}' filename
3. awk 'pattern {action}' filename

Also similar to the above two commands, awk is more powerful, you can specify the separator to take out the string according to the field.

2, awk usage detailed example analysis

topic:

1. Print all the phone numbers
2. Print Dan's phone number
3. Print all last names beginning with D
4. Print Main’s campaign contributions. Each value should be printed with a leading dollar sign; e.g., $250 $100 $175.
5. Print the name and total contribution of each person.

 

answer:

1.awk -F: '{print $2}' donors
2.awk -F: '/Dan/{print $2}' donors
3.awk -F'[ :]' '$2~/^D/{print $2}' donors
4.awk -F'[ :]' '$2~/Main/{print "$"$5,"$"$6,"$"$7}' donors
5.awk -F: '{print $1,$3+$4+$5}' donors

result:

1. 
2. (406) 298-7744

3. Dobbins
   Dalsass

4. $50 $95 $135

5.  

   

Five, summary

This article will continue to study and explore Linux, take a step forward, master more advanced commands through detailed examples, mainly text processing technology, grep, sed, awk three commands, and feel the magical and powerful Linux. From the study of detailed examples, it is also very important to master regular expressions. As we all know, Linux is a powerful operating system with powerful functions and high stability. It is widely used in large companies and enterprises. It is very important to master Linux related command operations and knowledge systems.

 

If you think it’s good, welcome to "one-click, three-link", like, bookmark, follow, comment directly if you have any questions, and exchange and learn!

Linux system learning related articles:

1. Linux basic commands and programming environment experiment [ https://blog.csdn.net/Charzous/article/details/108204520 ]
2. Linux environment programming inter-process communication mechanism, play with Linux, and understand more thoroughly! [ https://blog.csdn.net/Charzous/article/details/108287075 ]

---

My CSDN blog: https://blog.csdn.net/Charzous/article/details/110507267