[BASH] Review and knowledge points six
This series of catalogs --> [BASH] review and knowledge points sorting (catalog)
Six. Pipeline command (pipe)
As mentioned above, the output data will appear when the bash command is executed! So if this group of data must go through several procedures before we can get the format we want, how should we set it? This involves the problem of the pipeline command (pipe), the pipeline command uses |the delimitation symbol " "! In addition, pipeline commands are different from "continuously issuing commands"! We will explain this further below. Below we give an example to illustrate a simple pipeline command.
Suppose we want to know how many files are under /etc/, then we can use ls /etc to look them up. However, because there are too many files under /etc/, the screen will be filled up in one breath~ I don’t know what the previous output is What? At this point, we can use the help of the less command to use:
ls -al /etc | less
In this way, the content output by using the ls command can be read by less, and using the function of less, we can flip the relevant information back and forth! Very convenient, right? Let's take a look at the purpose of this pipeline command " | "! In fact, this pipeline command "|" can only process the correct information transmitted by the previous command, that is, the information of standard output, and has no ability to directly process stdandard error. Then the overall pipeline command can be represented by the following figure:
the first data connected after each pipeline must be a "command"! And this instruction must be able to accept standard input data, so that such an instruction can be called a "pipeline command", such as less, more, head, tail, etc. are all pipeline commands that can accept standard input. For example, ls, cp, mv, etc. are not pipeline commands! Because ls, cp, mv will not accept data from stdin. In other words, there are two main points that need to be paid attention to in pipeline commands:
- Pipeline commands will only process standard output, and standard error output will be ignored
- The pipeline command must be able to accept the data from the previous command as standard input to continue processing.
Think about it, if you insist that standard error can be used by pipeline commands, how should you deal with it? In fact, it can be redirected through the data flow in the previous section! Let 2>&1 be added to the command ~ you can turn 2> into 1>! Got it? _No more talk, let's play with some pipeline commands! The dongdong below is very helpful for system management!
6.1 Extraction commands: cut, grep
What is the fetch command? To put it bluntly, it is to take out what we want after analyzing a piece of data. Or get the line we want by analyzing keywords! However, it should be noted that, generally speaking, the retrieval of information is usually analyzed for "line by line", not the analysis of the entire message~ Below we introduce two very commonly used information retrieval commands:
cut
Isn't cut just "cut"? That's right! This command can "cut" a certain section of a message for him~ The message processed is in units of "lines"! Let's talk about it below:
[dmtsai@study ~]$ cut -d'分隔字符' -f fields <==用于有特定分隔字符
[dmtsai@study ~]$ cut -c 字符区间 <==用于排列整齐的讯息
选项与参数：
-d ：后面接分隔字符。与 -f 一起使用；
-f ：依据 -d 的分隔字符将一段讯息分区成为数段，用 -f 取出第几段的意思；
-c ：以字符 (characters) 的单位取出固定字符区间；
# 范例一：将 PATH 变量取出，我要找出第五个路径。
[dmtsai@study ~]$ echo ${
PATH}
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/dmtsai/.local/bin:/home/dmtsai/bin
# 1 | 2 | 3 | 4 | 5 | 6 |
#取第五个
[root@node-135 ~]# echo $path|cut -d':' -f 5
/home/dmtsai/.local/bin
#取第三和五个
[root@node-135 ~]# echo $path|cut -d':' -f 3,5
/usr/local/sbin:/home/dmtsai/.local/bin
#取第三到第五个
[root@node-135 ~]# echo $path|cut -d':' -f 3-5
/usr/local/sbin:/usr/sbin:/home/dmtsai/.local/bin
#范例二：将 export 输出的讯息，取得第 12 字符以后的所有字符串
[dmtsai@study ~]$ export
declare -x HISTCONTROL="ignoredups"
declare -x HISTSIZE="1000"
declare -x HOME="/home/dmtsai"
declare -x HOSTNAME="study.centos.vbird"
.....(其他省略).....
# 注意看，每个数据都是排列整齐的输出！如果我们不想要『 declare -x 』时，就得这么做：
[dmtsai@study ~]$ export | cut -c 12-
HISTCONTROL="ignoredups"
HISTSIZE="1000"
HOME="/home/dmtsai"
HOSTNAME="study.centos.vbird"
.....(其他省略).....
# 知道怎么回事了吧？用 -c 可以处理比较具有格式的输出数据！
# 我们还可以指定某个范围的值，例如第 12-20 的字符，就是 cut -c 12-20 等等！
#范例三：用 last 将显示的登入者的信息中，仅留下用户大名
[dmtsai@study ~]$ last
root pts/1 192.168.201.101 Sat Feb 7 12:35 still logged in
root pts/1 192.168.201.101 Fri Feb 6 12:13 - 18:46 (06:33)
root pts/1 192.168.201.254 Thu Feb 5 22:37 - 23:53 (01:16)
# last 可以输出『账号/终端机/来源/日期时间』的数据，并且是排列整齐的
[dmtsai@study ~]$ last | cut -d ' ' -f 1
# 由输出的结果我们可以发现第一个空白分隔的字段代表账号，所以使用如上指令：
# 但是因为 root pts/1 之间空格有好几个，并非仅有一个，所以，如果要找出
# pts/1 其实不能以 cut -d ' ' -f 1,2 喔！输出的结果会不是我们想要的。
#last|cut -c 9-20
The main purpose of cut is to decompose the data in the same row! 』Most commonly used when analyzing some data or text data! This is because sometimes we use certain characters as partition parameters, and then cut the data to obtain the data we need. Brother Bird also uses this function very often! Especially when analyzing log files! However, cut may be a bit more difficult when dealing with data connected by multiple spaces, so at some point you may use the awk in the next chapter to replace it!
grep
The cut just now is to extract a certain part of a line of information that we want, and grep is to analyze a line of information, if there is the information we need, then take out the line~ The simple syntax is as follows:
[dmtsai@study ~]$ grep [-acinv] [--color=auto] '搜寻字符串' filename
选项与参数：
-a ：将 binary 文件以 text 文件的方式搜寻数据
-c ：计算找到 '搜寻字符串' 的行数
-i ：忽略大小写的不同，所以大小写视为相同
-n ：顺便输出行号
-v ：反向选择，亦即显示出没有 '搜寻字符串' 内容的那一行！
--color=auto ：可以将找到的关键词部分加上颜色的显示喔！
#范例一：将 last 当中，有出现 root 的那一行就取出来；
[dmtsai@study ~]$ last | grep 'root'
#范例二：与范例一相反，只要没有 root 的就取出！
[dmtsai@study ~]$ last | grep -v 'root'
#范例三：在 last 的输出讯息中，只要有 root 就取出，并且仅取第一栏
[dmtsai@study ~]$ last | grep 'root' |cut -d ' ' -f1
# 在取出 root 之后，利用上个指令 cut 的处理，就能够仅取得第一栏啰！
#范例四：取出 /etc/man_db.conf 内含 MANPATH 的那几行
[dmtsai@study ~]$ grep --color=auto 'MANPATH' /etc/man_db.conf
....(前面省略)....
MANPATH_MAP /usr/games /usr/share/man
MANPATH_MAP /opt/bin /opt/man
MANPATH_MAP /opt/sbin /opt/man
# 神奇的是，如果加上 --color=auto 的选项，找到的关键词部分会用特殊颜色显示喔！
grep is a great command! It supports too many grammars~ It can be used in regular notation, and it can handle a lot of data~ However, we will not talk about regular notation here~ I will explain it in the next chapter~ You should understand first, grep It can parse a line of text and get keywords. If there are keywords in this line, the entire line will be listed! In addition, in CentOS 7, the default grep has automatically added --color=auto in the alias!
6.2 Sorting commands: sort, wc, uniq
Many times, we will calculate the total number of data of the same type in the data once. For example, using last can check the identity of the person who has logged in to the host on the system. So can I find out the total number of logins for each user? At this time, instructions such as sorting and calculation must be assisted! Below we introduce several useful sorting and statistical commands!
sort
sort is a very interesting command, it can help us sort, and it can sort according to different data types! For example, the sorting of numbers and text is not the same. In addition, the characters to be sorted are related to the encoding of the language family. Therefore, if you need to sort, it is recommended to use LANG=C to make the language system one and the data sorting is better.
[dmtsai@study ~]$ sort [-fbMnrtuk] [file or stdin]
选项与参数：
-f ：忽略大小写的差异，例如 A 与 a 视为编码相同；
-b ：忽略最前面的空格符部分；
-M ：以月份的名字来排序，例如 JAN, DEC 等等的排序方法；
-n ：使用『纯数字』进行排序(默认是以文字型态来排序的)；
-r ：反向排序；
-u ：就是 uniq ，相同的数据中，仅出现一行代表；
-t ：分隔符，预设是用 [tab] 键来分隔；
-k ：以那个区间 (field) 来进行排序的意思
#范例一：个人账号都记录在 /etc/passwd 下，请将账号进行排序。
[dmtsai@study ~]$ cat /etc/passwd | sort
abrt:x:173:173::/etc/abrt:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
alex:x:1001:1002::/home/alex:/bin/bash
# 鸟哥省略很多的输出～由上面的数据看起来， sort 是预设『以第一个』数据来排序，
# 而且默认是以『文字』型态来排序的喔！所以由 a 开始排到最后啰！
#范例二：/etc/passwd 内容是以 : 来分隔的，我想以第三栏来排序，该如何？
[dmtsai@study ~]$ cat /etc/passwd | sort -t ':' -k 3
root:x:0:0:root:/root:/bin/bash
dmtsai:x:1000:1000:dmtsai:/home/dmtsai:/bin/bash
alex:x:1001:1002::/home/alex:/bin/bash
arod:x:1002:1003::/home/arod:/bin/bash
# 看到特殊字体的输出部分了吧？怎么会这样排列啊？呵呵！没错啦～
# 如果是以文字型态来排序的话，原本就会是这样，想要使用数字排序：
# cat /etc/passwd | sort -t ':' -k 3 -n
# 这样才行啊！用那个 -n 来告知 sort 以数字来排序啊！
#范例三：利用 last ，将输出的数据仅取账号，并加以排序
[dmtsai@study ~]$ last | cut -d ' ' -f1 | sort -u
sort is also a very commonly used command! Because we often need to compare some information! Take the second example above as an example! Suppose you have a lot of accounts today, and you want to know where the largest user ID is now! hehe! Use sort to know the answer at once! Of course, its use is more than that! If you have time, you might as well play it!
unique
If I have finished sorting and want to display only one duplicate data, how can I do it?
[dmtsai@study ~]$ uniq [-ic]
选项与参数：
-i ：忽略大小写字符的不同；
-c ：进行计数
#范例一：使用 last 将账号列出，仅取出账号栏，进行排序后仅取出一位；
[dmtsai@study ~]$ last | cut -d ' ' -f1 | sort | uniq
#范例二：承上题，如果我还想要知道每个人的登入总次数呢？
[dmtsai@study ~]$ last | cut -d ' ' -f1 | sort | uniq -c
1
6 (unknown
47 dmtsai
4 reboot
7 root
1 wtmp
# 从上面的结果可以发现 reboot 有 4 次， root 登入则有 7 次！大部分是以 dmtsai 来操作！
# wtmp 与第一行的空白都是 last 的默认字符，那两个可以忽略的！
Note: Deduplication should be used together with sorting, otherwise only the 'nearest' duplicates will be removed
[root@node-135 ~]# last | cut -d ' ' -f1 | uniq -c 3 root 1 reboot 4 root 1 reboot ...
This command is used to "delete duplicate lines and only display one". For example, if you want to know who is the user who logged in to your host this month, regardless of his login times, then use the above example, ( 1) First list all the data; (2) Separate the name of the person; (3) After sorting; (4) Only display one! Since this command is to reduce repetitive things, it is of course necessary to "cooperate with sorted files" to deal with it!
wc
If I want to know how many characters are in the file /etc/man_db.conf? how many lines How many characters can be done? In fact, you can use the command wc to achieve it! He can help us calculate the overall data of the output message!
[dmtsai@study ~]$ wc [-lwm]
选项与参数：
-l ：仅列出行；
-w ：仅列出多少字(英文单词)；
-m ：多少字符；
#范例一：那个 /etc/man_db.conf 里面到底有多少相关字、行、字符数？
[dmtsai@study ~]$ cat /etc/man_db.conf | wc
131 723 5171
# 输出的三个数字中，分别代表： 『行、字数、字符数』
#范例二：我知道使用 last 可以输出登入者，但是 last 最后两行并非账号内容，那么请问，
# 我该如何以一行指令串取得登入系统的总人次？
[dmtsai@study ~]$ last | grep [a-zA-Z] | grep -v 'wtmp' | grep -v 'reboot' | \
> grep -v 'unknown' |wc -l
# 由于 last 会输出空白行, wtmp, unknown, reboot 等无关账号登入的信息，因此，我利用
# grep 取出非空白行，以及去除上述关键词那几行，再计算行数，就能够了解啰！
Can wc also be used as an instruction? This is not the WC that goes to the bathroom! This is a very useful set of tools for calculating the content of files! For example, when you want to know how many accounts are currently in your account file, use this method: "cat /etc/passwd | wc -l"! Because the line in /etc/passwd represents a user! So if you know the number of lines, you know how many accounts are in it! And if you want to calculate how many characters are in a file, use the wc -m option!
6.3 Two-way redirection: tee
Think of something simple, we know from the previous section that > sends the entire data stream to a file or device, so we can't continue to use this data stream unless we read the file or device. What should I do if I want to save a certain piece of information during the processing of this data stream? You can use tee~ We can simply look at it like this:
tee will distribute the data stream to the file and the screen at the same time; and the output to the screen is actually stdout, so the next command can continue to process oh!
[dmtsai@study ~]$ tee [-a] file
选项与参数：
-a ：以累加 (append) 的方式，将数据加入 file 当中！
[dmtsai@study ~]$ last | tee last.list | cut -d " " -f1
# 这个范例可以让我们将 last 的输出存一份到 last.list 文件中；
[dmtsai@study ~]$ ls -l /home | tee ~/homefile | more
# 这个范例则是将 ls 的数据存一份到 ~/homefile ，同时屏幕也有输出讯息！
[dmtsai@study ~]$ ls -l / | tee -a ~/homefile | more
# 要注意！ tee 后接的文件会被覆盖，若加上 -a 这个选项则能将讯息累加。
tee can transfer a copy of the standard output to a file and send the same data to the screen for processing! In this way, in addition to allowing us to analyze and record a piece of data at the same time, it can also be used as an intermediate temporary disk record for processing a piece of data! tee this guy is easy on a lot of choice/filled certification exams!
6.4 Character conversion commands: tr, col, join, paste, expand
In the vim program editor, we mentioned the difference between DOS line break characters and Unix line break characters, and you can use dos2unix and unix2dos to complete the conversion. Ok, so think about it, are there any other commonly used character substitutions? For example, want to change uppercase to lowercase, or convert the [tab] key in the data to a space bar? Also, how do you combine two messages into one? Next, let's introduce how to use these character conversion commands in the pipeline:
Use vim to open the document, enter: set fileformat=unix or: set ff=unix, then enter: wq to save, and the document format conversion is completed
tr
tr(translate) can be used to delete text in a message, or to replace text messages
[dmtsai@study ~]$ tr [-ds] SET1 [SET2]
选项与参数：
-d ：删除讯息当中的 SET1 这个字符串；
-s ：取代掉重复的字符！
#范例一：将 last 输出的讯息中，所有的小写变成大写字符：
[dmtsai@study ~]$ last | tr '[a-z]' '[A-Z]'
# 事实上，没有加上单引号也是可以执行的，如：『 last | tr [a-z] [A-Z] 』
#范例二：将 /etc/passwd 输出的讯息中，将冒号 (:) 删除
[dmtsai@study ~]$ cat /etc/passwd | tr -d ':'
#范例三：将 /etc/passwd 转存成 dos 断行到 /root/passwd 中，再将 ^M 符号删除
[dmtsai@study ~]$ cp /etc/passwd ~/passwd && unix2dos ~/passwd
[dmtsai@study ~]$ file /etc/passwd ~/passwd
/etc/passwd: ASCII text
/home/dmtsai/passwd: ASCII text, with CRLF line terminators <==就是 DOS 断行
[dmtsai@study ~]$ cat ~/passwd | tr -d '\r' > ~/passwd.linux
# 那个 \r 指的是 DOS 的断行字符，关于更多的字符，请参考 man tr
[root@node-135 ~]# file /etc/passwd ~/passwd ~/passwd.linux
/etc/passwd: ASCII text
/root/passwd: ASCII text, with CRLF line terminators
/root/passwd.linux: ASCII text
In fact, this instruction can also be written in "regular notation"! Because he also replaces the data by regular representation! Taking the above example as an example, using [] can set a string of characters! Also often used to replace weird symbols in files! For example, in the third example above, the line break symbol ^M left in the DOS file can be removed! This thing is quite useful! I believe that one of the most troublesome things for people dealing with Linux & Windows systems is this! That is to say, under DOS, the line break symbol ^M will be automatically added at the end of each line! At this time, in addition to the dos2unix mentioned before, we can also use this tr to remove ^M! ^M can be used \rinstead!
col
The Linux col command is used to filter control characters. In many UNIX documentation, there are RLF control characters. When we use the shell special characters ">" and ">>" to output the content of the description file into a plain text file, the control characters will become garbled characters, and the col command can effectively filter out these control characters.
[dmtsai@study ~]$ col [-bfx][-l<缓冲区列数>]
选项与参数：
-x ：将 tab 键转换成对等的空格键
-b ：过滤掉所有的控制字符，包括RLF和HRLF
-f ：滤除RLF字符，但允许将HRLF字符呈现出来
-l<缓冲区列数> ：预设的内存缓冲区有128列，您可以自行指定缓冲区的大小。
#范例一：利用 cat -A 显示出所有特殊按键，最后以 col 将 [tab] 转成空白
[dmtsai@study ~]$ cat -A /etc/man_db.conf <==此时会看到很多 ^I 的符号，那就是 tab
[dmtsai@study ~]$ cat /etc/man_db.conf | col -x | cat -A | more
# 嘿嘿！如此一来， [tab] 按键会被取代成为空格键，输出就美观多了！
Although col has its special purpose, in many cases, it can be used to simply replace the [tab] key with a space bar! For example, in the above example, if cat -A is used, [tab] will be represented ^I by . But after processing col -x, [tab] will be replaced by the equivalent space bar!
RLF逆向换行符Reverse/Retrorse Line Feed: ESC-7 (escape then 7),HRLF半逆向换行符Half Reverse Line Feed: ESC-8 (escape then 8)
As the name implies, the newline character makes the cursor reach the beginning of the next line, and the reverse newline character will make the cursor return to the previous line at the beginning of the ; a half-reversed newline will cause the cursor to go back to the previous line, but not to the beginning.
join
From the literal meaning of join (join/participate), one can know that he is dealing with the data between two files, and it is mainly dealing with the 两个文件当中，有 "相同数据" 的那一行，才将他加在一起，加在一起后把“相同数据”提至行首显示meaning of " ". We use the simple example below to illustrate:
[dmtsai@study ~]$ join [-ti12] file1 file2
选项与参数：
-t ：join 默认以空格符分隔数据，并且比对『第一个字段』的数据，如果两个文件相同，则将两笔数据联成一行，且第一个字段放在第一个！
-i ：忽略大小写的差异；
-1 ：这个是数字的 1 ，代表『第一个文件要用那个字段来分析』的意思；
-2 ：代表『第二个文件要用那个字段来分析』的意思。
#范例一：用 root 的身份，将 /etc/passwd 与 /etc/shadow 相关数据整合成一栏
[root@study ~]# head -n 3 /etc/passwd /etc/shadow
==> /etc/passwd <==
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
==> /etc/shadow <==
root:$6$wtbCCce/PxMeE5wm$KE2IfSJr...:16559:0:99999:7:::
bin:*:16372:0:99999:7:::
daemon:*:16372:0:99999:7:::
# 由输出的资料可以发现这两个文件的最左边字段都是相同账号！且以 : 分隔
[root@study ~]# join -t ':' /etc/passwd /etc/shadow | head -n 3
root:x:0:0:root:/root:/bin/bash:$6$wtbCCce/PxMeE5wm$KE2IfSJr...:16559:0:99999:7:::
bin:x:1:1:bin:/bin:/sbin/nologin:*:16372:0:99999:7:::
daemon:x:2:2:daemon:/sbin:/sbin/nologin:*:16372:0:99999:7:::
# 透过上面这个动作，我们可以将两个文件第一字段相同者整合成一列！
# 第二个文件的相同字段并不会显示(因为已经在最左边的字段出现了啊！)
#范例二：我们知道 /etc/passwd 第四个字段是 GID ，那个 GID 记录在
/etc/group 当中的第三个字段，请问如何将两个文件整合？
[root@study ~]# head -n 3 /etc/passwd /etc/group
==> /etc/passwd <==
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
==> /etc/group <==
root:x:0:
bin:x:1:
daemon:x:2:
# 从上面可以看到，确实有相同的部分喔！赶紧来整合一下！
[root@study ~]# join -t ':' -1 4 /etc/passwd -2 3 /etc/group | head -n 3
0:root:x:0:root:/root:/bin/bash:root:x:
1:bin:x:1:bin:/bin:/sbin/nologin:bin:x:
2:daemon:x:2:daemon:/sbin:/sbin/nologin:daemon:x:
# 同样的，相同的字段部分被移动到最前面了！所以第二个文件的内容就没再显示。
# 请读者们配合上述显示两个文件的实际内容来比对！
This join is really helpful when dealing with two related data files! For example, in the above case, my /etc/passwd, /etc/shadow, /etc/group are all related, among which /etc/passwd, /etc/shadow is related to the account number, as for /etc/passwd , /etc/group takes the so-called GID (the numerical definition of the account) as its correlation. Based on this correlation, we can put related materials together! This can be quite helpful when working with data! But the above example is a bit difficult, I hope you can calm down and take a good look at the reason!
In addition, special attention should be paid to the fact that before using join, the files you need to process should be sorted ( sort) in advance ! Otherwise, some comparison items will be skipped! Special attention!
paste
This paste is much simpler than join! Compared with join, which must compare the data correlation of two files, paste directly "paste two lines together, and separate them with [tab] key"! Simple usage:
[dmtsai@study ~]$ paste [-d] file1 file2 ...
选项与参数：
-d ：后面可以接分隔字符。预设是以 [tab] 来分隔的！
- ：如果 file 部分写成 - ，表示来自 standard input 的资料的意思。
# 例一：用 root 身份，将 /etc/passwd 与 /etc/shadow 同一行贴在一起
[root@study ~]# paste /etc/passwd /etc/shadow
root:x:0:0:root:/root:/bin/bash root:$6$wtbCCce/PxMeE5wm$KE2IfSJr...:16559:0:99999:7:::
bin:x:1:1:bin:/bin:/sbin/nologin bin:*:16372:0:99999:7:::
daemon:x:2:2:daemon:/sbin:/sbin/nologin daemon:*:16372:0:99999:7:::
# 注意喔！同一行中间是以 [tab] 按键隔开的！
# 范例二：先将 /etc/group 读出(用 cat)，然后与范例一贴上一起！且仅取出前三行
[root@study ~]# cat /etc/group|paste /etc/passwd /etc/shadow -|head -n 3
# 这个例子的重点在那个 - 的使用！那玩意儿常常代表 stdin 喔！
expand
This thing is turning the [tab] key into a space bar~ You can play like this:
expand - convert tabs to spaces
[dmtsai@study ~]$ expand [-t] file
选项与参数：
-t ：后面可以接数字。一般来说，一个 tab 按键可以用 8 个空格键取代。我们也可以自行定义一个 [tab] 按键代表多少个字符呢！
#范例一：将 /etc/man_db.conf 内行首为 MANPATH 的字样就取出；仅取前三行；
[dmtsai@study ~]$ grep '^MANPATH' /etc/man_db.conf | head -n 3
MANPATH_MAP /bin /usr/share/man
MANPATH_MAP /usr/bin /usr/share/man
MANPATH_MAP /sbin /usr/share/man
# 行首的代表标志为 ^ ，这个我们留待下节介绍！先有概念即可！
#范例二：承上，如果我想要将所有的符号都列出来？(用 cat)
[dmtsai@study ~]$ grep '^MANPATH' /etc/man_db.conf | head -n 3 |cat -A
MANPATH_MAP^I/bin^I^I^I/usr/share/man$
MANPATH_MAP^I/usr/bin^I^I/usr/share/man$
MANPATH_MAP^I/sbin^I^I^I/usr/share/man$
# 发现差别了吗？没错～ [tab] 按键可以被 cat -A 显示成为 ^I
#范例三：承上，我将 [tab] 按键设定成 6 个字符的话？
[dmtsai@study ~]$ grep '^MANPATH' /etc/man_db.conf | head -n 3 | expand -t 6 - | cat -A
MANPATH_MAP /bin /usr/share/man$
MANPATH_MAP /usr/bin /usr/share/man$
MANPATH_MAP /sbin /usr/share/man$
123456123456123456123456123456123456123456123456...
# 仔细看一下上面的数字说明，因为我是以 6 个字符来代表一个 [tab] 的长度，所以，
# MAN... 到 /usr 之间会隔 12 (两个 [tab]) 个字符喔！如果 tab 改成 9 的话，
# 情况就又不同了！这里也不好理解～您可以多设定几个数字来查阅就晓得！
expand is also very interesting ~ it will automatically convert [tab] into a space bar ~ so, in the above example, using cat -A will not find the character of ^I, in addition, because [tab] is the largest The function is that the format is neatly arranged! After we convert it into a space bar, the space bar will also increase in size according to our own definition~ So, it is not that one ^I will be replaced by 8 blanks! Special attention should be paid to this place! In addition, you can also refer to unexpand, a command function that converts blanks into [tab]! ^_^
Note:
不是所有的Tab都会转换为默认或指定数量的空格符, expand will replace the tab character with an appropriate number of space characters based on the alignment principle. The principle of replacement is to make the following non-tab characters be at a physical tab boundary (that is, an integer multiple of the tab size)
unexpand - convert spaces to tabs
-t ： 自行定义一个 [tab] 按键代表多少个字符
-a ： 所有空格进行转换
--first-only : convert only leading sequences of blanks (overrides -a)
[root@node-135 ~]# cat 123 -A
123^I1231^I23$
123 123 123$
[root@node-135 ~]# unexpand -a -t 4 123|cat -A
123^I1231^I23$
123^I 123^I 123$
6.5 Partition command: split
If you have a problem with files that are too large for some portable devices to copy, hey! Just look for split! He can help you partition a large file according to the file size or the number of lines, and then you can partition the large file into a small file! Fast and effective! really good~
[dmtsai@study ~]$ split [-bl] file PREFIX
选项与参数：
-b ：后面可接欲分区成的文件大小，可加单位，例如 b, k, m 等；
-l ：以行数来进行分区。
PREFIX ：分割后的文件名，代表前导符的意思，可作为分区文件的前导文字。
#范例一：我的 /etc/services 有六百多 K，若想要分成 300K 一个文件时？
[dmtsai@study ~]$ cd /tmp; split -b 300k /etc/services services
[dmtsai@study tmp]$ ll -k services*
-rw-rw-r--. 1 dmtsai dmtsai 307200 Jul 9 22:52 servicesaa
-rw-rw-r--. 1 dmtsai dmtsai 307200 Jul 9 22:52 servicesab
-rw-rw-r--. 1 dmtsai dmtsai 55893 Jul 9 22:52 servicesac
# 那个档名可以随意取的啦！我们只要写上前导文字，小文件就会以
# xxxaa, xxxab, xxxac 等方式来建立小文件的！
#范例二：如何将上面的三个小文件合成一个文件，档名为 servicesback
[dmtsai@study tmp]$ cat services* >> servicesback
# 很简单吧？就用数据流重导向就好啦！简单！
#范例三：使用 ls -al / 输出的信息中，每十行记录成一个文件
[dmtsai@study tmp]$ ls -al / | split -l 10 - lsroot
[dmtsai@study tmp]$ wc -l lsroot*
10 lsrootaa
10 lsrootab
4 lsrootac
24 total
This is -very important! Generally speaking, if stdout/stdin is needed, but there is no file, and some are just -time, then that -will be regarded as stdin or stdout ~
Under the Windows operating system, what do you need to do to partition files? Worry about it! It's much easier under Linux! If you want to partition the file, then use it -b sizeto limit the size of a partitioned file. If it is the number of rows, then use it -l lineto partition! Very useful! In this way, you can easily partition your files into the maximum capacity supported by some software (such as 25MB for a single gmail message!), so that you can copy them easily!
6.6 Argument Substitution: xargs
What is xargs doing? In a literal sense, x is the multiplication sign for addition, subtraction, multiplication and division, and args means arguments (parameters), so this thing means generating the parameters of a certain command! xargs can read the data of stdin, and separate the data of stdin into arguments with a space character or a line break character as a distinction. Because it is separated by a space character, if some file names or nouns with other meanings contain a space character, xargs may misjudge it~ His usage is actually quite simple! Just take a look first!
[dmtsai@study ~]$ xargs [-0epn] command
选项与参数：
-0 ：如果输入的 stdin 含有特殊字符，例如 `, \, 空格键等等字符时，这个 -0 参数可以将他还原成一般字符。这个参数可以用于特殊状态喔！
-e ：这个是 EOF (end of file) 的意思。后面可以接一个字符串，当 xargs 分析到这个字符串时，就会停止继续工作！
-p ：在执行每个指令的 argument 时，都会询问使用者的意思；
-n ：后面接次数，每次 command 指令执行时，要使用几个参数的意思。
当 xargs 后面没有接任何的指令时，默认是以 echo 来进行输出喔！
#范例一：将 /etc/passwd 内的第一栏取出，仅取三行，使用 id 这个指令将每个账号内容秀出来
[dmtsai@study ~]$ id root
uid=0(root) gid=0(root) groups=0(root) # 这个 id 指令可以查询用户的 UID/GID 等信息
[dmtsai@study ~]$ id $(cut -d ':' -f 1 /etc/passwd | head -n 3)
# 虽然使用 $(cmd) 可以预先取得参数，但可惜的是， id 这个指令『仅』能接受一个参数而已！
# 所以上述的这个指令执行会出现错误！根本不会显示用户的 ID 啊！
[dmtsai@study ~]$ cut -d ':' -f 1 /etc/passwd | head -n 3 | id
uid=1000(dmtsai) gid=1000(dmtsai) groups=1000(dmtsai),10(wheel) # 我不是要查自己啊！
# 因为 id 并不是管线命令，因此在上面这个指令执行后，前面的东西通通不见！只会执行 id！
[dmtsai@study ~]$ cut -d ':' -f 1 /etc/passwd | head -n 3 | xargs id
# 依旧会出现错误！这是因为 xargs 一口气将全部的数据通通丢给 id 处理～但 id 就接受 1 个啊最多！
[dmtsai@study ~]$ cut -d ':' -f 1 /etc/passwd | head -n 3 | xargs -n 1 id
uid=0(root) gid=0(root) groups=0(root)
uid=1(bin) gid=1(bin) groups=1(bin)
uid=2(daemon) gid=2(daemon) groups=2(daemon)
# 透过 -n 来处理，一次给予一个参数，因此上述的结果就 OK 正常的显示啰！
#范例二：同上，但是每次执行 id 时，都要询问使用者是否动作？
[dmtsai@study ~]$ cut -d ':' -f 1 /etc/passwd | head -n 3 | xargs -p -n 1 id
id root ?...y
uid=0(root) gid=0(root) groups=0(root)
id bin ?...y
.....(底下省略).....
# 呵呵！这个 -p 的选项可以让用户的使用过程中，被询问到每个指令是否执行！
# 范例三：将所有的 /etc/passwd 内的账号都以 id 查阅，但查到 sync 就结束指令串
[dmtsai@study ~]$ cut -d ':' -f 1 /etc/passwd | xargs -e'sync' -n 1 id
# 仔细与上面的案例做比较。也同时注意，那个 -e'sync' 是连在一起的，中间没有空格键。
# 上个例子当中，第六个参数是 sync 啊，那么我们下达 -e'sync' 后，则分析到 sync 这个字符串时，
# 后面的其他 stdin 的内容就会被 xargs 舍弃掉了！
In fact, there are three or four small examples in man xargs, you can refer to the content by yourself. Also, xargs is a really handy tool! You really need to take a good look at the details! The reason for using xargs is that many instructions do not support pipeline commands, so we can use xargs to provide this instruction with standard input! For example, we use the following example to illustrate:
#范例四：找出 /usr/sbin 底下具有特殊权限的档名，并使用 ls -l 列出详细属性
[dmtsai@study ~]$ find /usr/sbin -perm /7000 | xargs ls -l
-rwx--s--x. 1 root lock 11208 Jun 10 2014 /usr/sbin/lockdev
-rwsr-xr-x. 1 root root 113400 Mar 6 12:17 /usr/sbin/mount.nfs
-rwxr-sr-x. 1 root root 11208 Mar 6 11:05 /usr/sbin/netreport
.....(底下省略).....
# 聪明的读者应该会想到使用『 ls -l $(find /usr/sbin -perm /7000) 』来处理这个范例！
# 都 OK！能解决问题的方法，就是好方法！
6.7 About the use of the minus sign -
Pipelining commands is very important in bash's sequential processing! In addition, it is also a very important part in the analysis of log file, so please pay special attention! In addition, in the pipeline command, the stdout of the previous command is often used as the stdin of this time. When some commands need to use the file name (such as tar) for processing, the stdin and stdout can use the minus sign "-" instead, for example:
[root@study ~]# mkdir /tmp/homeback
[root@study ~]# tar -cvf - /home | tar -xvf - -C /tmp/homeback
The above example is to say: "I package the files in /home for him, but the packaged data is not recorded in the file, but transmitted to stdout; after passing through the pipeline, it will be transmitted to the following tar -cvf - /home" tar -xvf - . The latter -one takes the stdout of the previous command, so we don't need to use filename! This is a very common example! Attention please!
This series of catalogs --> [BASH] review and knowledge points sorting (catalog)