Using the input and output pipes of the Day 13 linux

1. Redirection Overview

1. What is redirected
to the original data to be output to the screen, redirect to a specified file. For example: every morning scheduled backup data, backup data you want to save the results to a file. In this way the next day to know the data backed up yesterday's success or failure by viewing the contents of the file.

2. Why use redirection
1. When information is important screen output, but also want to save important information;
2. the background of the program, do not want him to interfere with the normal output screen;
routine 3. System command, such as the results of timed tasks, wishes to be kept down;
4. Some execute the command, we already know he error message may appear when you want to discard him directly;
5. correct the error log and the log output separately to different needs when files are saved; *

3. Learn prior knowledge redirection, standard input and output
often automatically open three standard files when you run a program, it is the standard input, standard output, error output *

name	File descriptor	effect
Standard input (STDIN)	0	The default keyboard can also be a file or other output commands.
Standard output (STDOUT)	1	The default output to the screen.
Error output (STDERR)	2	The default output to the screen.
File name (filename)	3+

Process to get the data from standard input, the normal print output to the screen terminal, the output of the error information is also printed to the screen terminal.

PS: the process is the use of file descriptors (file descriptors)to manage open files *

In cat command, for example, functions from the command line cat command is given to read the data, and the data sent directly to the standard output. If you use the following command:

#会把文件/etc/passwd的内容输出显示到屏幕上
[root@xuliangwei ~]# cat /etc/passwd

However, if the command does not keep up with the cat using the input file name, then the cat command will read data in the standard input command, and send it to standard output.

[root@xuliangwei ~]# cat
hello   #标准输入
hello   #标准输出
^C
#用户输入的每一行都立刻被cat命令输出到屏幕上。

The following look at the standard input and output process

#持续追踪查看文件内容
[root@xuliangwei ~]# tail -f /etc/passwd
ctrl+z 将进程转到后台

#查看运行的进程
[root@xuliangwei ~]# ps
PID TTY          TIME CMD
5848 pts/1    00:00:00 bash
6885 pts/1    00:00:00 tail
6888 pts/1    00:00:00 ps

#查看tail命令的pid，6885进程下的文件描述符
[root@xuliangwei ~]# ls -l /proc/6885/fd
total 0
lrwx------ 1 root root 64 Dec  3 06:57 0 -> /dev/pts/1
lrwx------ 1 root root 64 Dec  3 06:57 1 -> /dev/pts/1
lrwx------ 1 root root 64 Dec  3 06:56 2 -> /dev/pts/1
lr-x------ 1 root root 64 Dec  3 06:57 3 -> /etc/passwd
lr-x------ 1 root root 64 Dec  3 06:57 4 -> inotify

#Linux查看标准输入输出设备
[root@xuliangwei ~]# ls -l /dev/std*
lrwxrwxrwx 1 root root 15 Dec  2 22:30 /dev/stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root root 15 Dec  2 22:30 /dev/stdin -> /proc/self/fd/0
lrwxrwxrwx 1 root root 15 Dec  2 22:30 /dev/stdout -> /proc/self/fd/1

2. Output redirection

Output redirection, change the position of the output contents. There are several ways to redirect the output, as shown in Table

Types of	Operators	use
Standard output redirection coverage	>	The program outputs the correct result to the specified output file will overwrite the original file contents
Additional standard output redirection	>>	The program outputs the correct result in an additional way to specify the output file will not overwrite the original file
Coverage error output redirection	2>	The output error results of the program to execute the file, the file will overwrite the original content
Additional error output redirection	2>>	The program output error results in an additional way to specify the output file will not overwrite the original file
Standard input redirection	<<	The way to receive input commands for the specified file or change the default keyboard command

Case 1: The standard output redirection (overwrite the file every time)

#标准输出重定向, 先清空,后写入, 如果文件不存在则创建
[root@xuliangwei ~]# ifconfig eth0 > abc

Case 2: The standard output redirection (to the end of the file will be adding content)

#标准追加输出重定向, 向配置文件末尾追加内容
[xuliangwei@xuliangwei ~]$ echo "This is network conf" >> if 

Case 3: Error output redirection

#正确输出以及错误输出重定向至一个文件
[root@xuliangwei ~]# useradd xuliangwei
[root@xuliangwei ~]# su - xuliangwei

#将标准输出和标准错误输出重定向到不同文件
[xuliangwei@xuliangwei ~]$ find /etc -name "*.conf" 1>a 2>b

Case 4: right and wrong are entered into the same location

#将标准输出和标准错误输出重定向到同一个文件, 混合输出
[xuliangwei@xuliangwei ~]$ find /etc -name "*.conf" &>ab

#合并两个文件内容至一个文件
[xuliangwei@xuliangwei ~]$ cat a b > c

Case 5: right and wrong are entered into the same location *

#重定向到相同的位置
[root@xuliangwei ~]# ls /root /error >ab  2>&1

Case 6: redirected to the null device / dev / null

#将产生的任何数据放入黑洞设备，则视为丢弃。
[root@xuliangwei ~]# ls /root /error >ab 2>/dev/null
[root@xuliangwei ~]# ls /root /error >ab &>/dev/null

Case 7: script redirection

[root@xuliangwei ~]# vim ping.sh 
ping -c1 10.0.0.1
if [ $? -eq 0 ];then
    echo "10.0.0.1 is up." 
else
    echo "10.0.0.1 is down." 
fi
[root@xuliangwei ~]# chmod +x ping.sh 
[root@xuliangwei ~]# ./ping.sh

#改进后版
[root@xuliangwei ~]# vim ping.sh
ping -c1 10.0.0.1 &>/dev/null
if [ $? -eq 0 ];then
    echo "10.0.0.1 is up." 
else
    echo "10.0.0.1 is down." 
fi

Case 8: script redirection

[root@xuliangwei ~]# vim ping2.sh 
ping -c1 10.0.0.1 &>/dev/null 
if [ $? -eq 0 ];then
    echo "10.0.0.1 is up." >>up.txt 
else
    echo "10.0.0.1 is down." >>down.txt 
fi
[root@xuliangwei ~]# chmod +x ping2.sh 
[root@xuliangwei ~]# ./ping2.sh

3. Input redirection

Input redirection, i.e., input information originally obtained from the keyboard or the like, to redirect the output of the command as an input. <Equivalents 0 <*

Case 1: read from a file operation input *

#没有改变输入的方向，默认键盘
[root@xuliangwei ~]# mail alice 
Subject: hello
1111 
2222
3333
.   #结束
EOT

#检查是否收到邮件
[root@xuliangwei ~]# su - alice
[root@xuliangwei ~]# mail

#输入重定向，来自于文件
[root@xuliangwei ~]# mail -s "test01" alice < /etc/hosts

Case 2: indescribable case, see the actual operation *

#没有改变输入的方向，默认键盘，此时等待输入
[root@xuliangwei ~]# grep 'root' 
xxx
xxx

[root@xuliangwei ~]# grep 'root' < /etc/passwd
root:x:0:0:root:/root:/bin/bash 

Case 3: You can not describe the case, look at the actual operation *

[root@xuliangwei ~]# dd if=/dev/zero of=/file1.txt bs=1M count=20
[root@xuliangwei ~]# dd </dev/zero >/file2.txt bs=1M count=20

Case 4: mysql How to restore a backup, you can understand, without concern.

[root@xuliangwei ~]# mysql -uroot -p123 < bbs.sql

Case 5: Using file redirection establish multiple rows of data *

#手动执行 shell 命令
[root@xuliangwei ~]# echo "111" > file1.txt 
[root@xuliangwei ~]# cat file1.txt
111
[root@xuliangwei ~]# cat >file2.txt
111
222
333
^D

[root@xuliangwei ~]# cat >>file3.txt
aaa
bbb
ccc
^D

Case 6: A method of using a script to print menu.

[root@xuliangwei ~]# vim vm.sh
cat <<-EOF
+------------------- --- ---- --- ---- --- --- ---- --- --+ ||
| ====================== | 
| 虚拟机基本管理 v5.0 |
| by xuliangwei |
| ====================== | 
| 1\. 安装 KVM |
| 2\. 安装或重置 CentOS-6.9 | 
| 3\. 安装或重置 CentOS-7.4 | 
| 5\. 安装或重置 Windows-7  | 
| 6\. 删除所有虚拟机 |
| q. 退出管理程序 |
+------------------- --- ---- --- ---- --- --- ---- --- --+ 
EOF

Case 7: both commands are redirected

[root@xuliangwei ~]# ls; date &>/dev/null
[root@xuliangwei ~]# ls &>/dev/null; date &>/dev/null
[root@xuliangwei ~]# (ls; date) &>/dev/null

#后台执行
[root@xuliangwei ~]# (while :; do date; sleep 2; done) &
[1] 6378
[root@xuliangwei ~]# (while :; do date; sleep 2; done) &>date.txt &
[root@xuliangwei ~]# jobs
[1]+ 运行中 ( while :; do date; sleep 2;
done ) &>/date.txt &

Extension point: subshell can understand

[root@xuliangwei ~]# cd /boot; ls

//subshell 中执行
[root@xuliangwei ~]# (cd /boot; ls)

#如果不希望某些命令的执行对当前 shell 环境产生影响，请在subshell中执行

4. Process Pipeline Technology

1. What is the conduit
pipe operation symbol "|", primarily used to connect the left and right commands, the standard output of the command of the left, to the right of the standard command input

PS: Unable to deliver the standard error output to the latter command

2 a schematic flow duct

format: cmd1 | cmd2 [...|cmdn]

3. Use Cases pipeline

Case 1: The / etc / passwd user sorted by size UID

[root@xuliangwei ~]# sort -t":" -k3 -n /etc/passwd
[root@xuliangwei ~]# sort -t":" -k3 -n /etc/passwd -r
[root@xuliangwei ~]# sort -t":" -k3 -n /etc/passwd |head

Case 2: Statistics of the current shell type / etc / passwd users use

#思路:取出第七列(shell) | 排序(把相同归类)| 去重
[root@xuliangwei ~]# awk -F: '{print $7}' /etc/passwd
[root@xuliangwei ~]# awk -F: '{print $7}' /etc/passwd |sort
[root@xuliangwei ~]# awk -F: '{print $7}' /etc/passwd |sort |uniq
[root@xuliangwei ~]# awk -F: '{print $7}' /etc/passwd |sort |uniq -c

Case 4: Statistics website visits top 20

#思路: 打印所有访问的连接 | 过滤访问网站的连接 | 打印用户的 IP | 排序 | 去重

[root@xuliangwei ~]# yum -y install httpd
[root@xuliangwei ~]# systemctl start httpd
[root@xuliangwei ~]# systemctl stop firewalld

[root@xuliangwei ~]# ss -an |grep :80 |awk -F":" '{print $8}' |sort |uniq -c
[root@xuliangwei ~]# ss -an |grep :80 |awk -F":" '{print $8}' |sort |uniq -c |sort -k1 -rn |head -n 20

Case 5: Print all current IP

[root@xuliangwei ~]# ip addr |grep 'inet ' |awk '{print $2}' |awk -F"/" '{print $1}'
127.0.0.1
192.168.69.112

Case 6: Printing has been the root partition percentage space (print digital only)

[root@xuliangwei ~]# df |grep '/$' |awk '{print $5}' |awk -F"%" '{print $1}'

PS: Pipe command character that we can make further use of master mix between commands, to further improve the processing efficiency of the output command value.

4. The pipe tee art

#选项: -a追加
[root@xuliangwei ~]# ip addr |grep 'inet ' |tee ip.txt |awk -F"/" '{print $1}' |awk '{print $2}'
127.0.0.1
10.0.0.100

[root@xuliangwei ~]# cat ip.txt
inet 127.0.0.1/8 scope host lo
inet 10.0.0.100/24 brd 192.168.69.255 scope global ens32

Redirect the tee have them in the course of what is the difference

[root@xuliangwei ~]# date > date.txt    #直接将内容写入date.txt文件中
[root@xuliangwei ~]# date |tee date.txt #命令执行会输出至屏幕，但会同时保存一份至date.txt文件中

5.xargs parameter passing, mainly for some commands do not support the pipeline using pipe technology

# which cat|xargs ls- l
# ls |xargs rm -fv
# ls |xargs cp -rvt /tmp/ -或-> ls | xargs -I {} cp -rv {} /tmp/
# ls |xargs mv -t /tmp/   -或-> ls | xargs -I {}  mv {} /tmp