Article directory
Preface
Continuing from the previous lecture, this blog, as seen in the title, explains character conversion commands, partition commands and parameter substitution. To be honest, I don’t use much of the stuff in this blog, and I only found out about it when I was reading a book. Now that you’ve seen it, let’s summarize it and remember it.
1. Character conversion command
1.1 tr
The tr command is used to delete text in a piece of information, or to replace text information.
General usage: tr [-ds] SET1 ...
options and parameters:
-d: delete the string SET1 in the message;
-s: replace repeated characters
It's so easy to read on paper, but I know I have to do it in detail.
例：Use cat /etc/passwd to check the password. There will be many :numbers in the corresponding output content. If I want to remove them, I have to use tr.
That is, delete a field in a piece of information.
instruction:cat /etc/passwd | tr -d ':'
This command is :to delete this field whenever it appears.
So sometimes it is quite absolute. For example, if I want to retain this character in some fields, tr will not work.
1.2 col
col is generally used to handle the tab key, replacing the tab key with a blank key.
General usage: col [-xb]
options and parameters:
-x: Convert the tab key to the equivalent space key
例：Use cat -A to display all special keys, and finally use col to convert [tab] to blank.
Use this command to view a file containing a lot of tab content. cat -A 文件路径There will be a lot of ^| symbols. After
using it cat 文件路径 | col -x | cat -A | more, we will find the corresponding ^| becomes blank.
1.3 join
The join command is used to process data between two files. It should be noted that join mainly processes "the rows with "the same data" in the two files, and then adds them together."
General usage:join [-ti12] file1 file2
Options and parameters:
-t: join By default, the data is separated by blank characters, and the data of the "first field" is compared.
If the two files are the same, the two data are concatenated into one line, and the first field is placed in the first field. one!
-i: Ignore the difference in case;
-1: This is the number 1, which means "the first file should be analyzed using that field";
-2: means "the second file should be analyzed using that field" the meaning of.
例如：The fourth field of /etc/passwd is GID, and that GID is recorded in the third field of /etc/group. How to integrate the two files?
Using the command: join -t ':' -1 4 /etc/passwd -2 3 /etc/group | head -n 3
Here you can see that the two fields have been merged.
1.4 paste
This paste is much simpler than join. Compared with join, which must compare the data correlation of the two files, paste simply "pastes two lines together and separate them with the [tab] key"! Simple usage:
General usage: paste [-d] file1 file2
options and parameters:
-d: can be followed by a delimiter character. The default is [tab] to separate!
-: If the file part is written as -, it means data from standard input.
1.5 expand
Expand is converting the [tab] key into a space key.
General usage: expand [-t] file
options and parameters:
-t: can be followed by numbers. Generally speaking, a tab key can be replaced by 8 blank keys.
We can also define how many characters a [tab] key represents.
2. Partition command split
If you have problems with files that are too large to copy on some portable devices, use split. It can help you partition a large file based on file size or number of lines, and then partition the large file into small files.
The problem with this file is that it is too large. When I used a USB flash drive to copy a single compressed package of several gigabytes, I found that it could not be copied. This is related to the USB flash drive format. FAT supports a maximum of 4GB. It must be formatted into NTFS to support 4GB. above. If I don't want to format, it might be a good idea to split a file into several smaller files.
General usage: split [-bl] file PREFIX
Options and parameters:
-b: The size of the file to be partitioned can be followed, and units can be added, such as b, k, m, etc.;
-l: Partitioning is based on the number of lines.
PREFIX: represents the meaning of the prefix character, which can be used as the leading text of the partition file.
The following examples are from the book
For example: My /etc/services has more than 600K. If I want to divide it into 300K files
cd /tmp; split -b 300k /etc/services services
ll -k services*
-rw-rw-r–. 1 dmtsai dmtsai 307200 Jul 9 22:52 servicesaa
-rw-rw-r–. 1 dmtsai dmtsai 307200 Jul 9 22:52 servicesab
-rw-rw-r–. 1 dmtsai dmtsai 55893 Jul 9 22:52 servicesac
#The file name can be freely chosen! As long as we write the leading text, the small file will be
created in the format of #xxxaa, xxxab, xxxac, etc.
3. Parameter substitution xargs
General usage: xargs [-0epn] command
options and parameters:
-0: If the input stdin contains special characters, such as `, , blank keys, etc., this -0 parameter
can restore it to normal characters. This parameter can be used in special states!
-e: This means EOF (end of file). A string can be followed. When xargs analyzes this string,
it will stop working!
-p: When executing the argument of each command, the user will be asked what the meaning is;
-n: followed by the number of times, each time the command command is executed, the meaning of several parameters will be used.
When no instructions are followed by xargs, the default is to use echo for output.
What does xargs do? In a literal sense, x is the multiplication sign for addition, subtraction, multiplication and division, and args means arguments (parameters), so this thing is generating the parameters of a certain instruction! xargs can read stdin data and separate stdin data into arguments using blank characters or line break characters as distinctions. Because they are separated by whitespace characters, xargs may misjudge if some file names or nouns with other meanings contain whitespace characters.
例如：Check all the accounts in /etc/passwd by id, but end the command string when sync is found.
Command: cut -d ':' -f 1 /etc/passwd | xargs -e'sync' -n 1 id
#Note that -e'sync' is connected together, and there is no blank key in the middle.
#The sixth parameter is sync, then after we issue -e'sync', when the sync string is analyzed,
#The contents of other stdin following it will be discarded by xargs
-4. The role of pipeline medium number
In pipeline commands, the stdout of the previous instruction is often used as the stdin this time. When some instructions need to use the file name (such as tar) for processing, the stdin and stdout can be replaced by the minus sign "-" ,
举例来说：
mkdir /tmp/homeback
tar -cvf - /home | tar -xvf - -C /tmp/homeback
The above example says: "I package the files in /home for him, but the packaged data is not recorded to the file, but sent to stdout; after passing through the pipeline,
tar -cvf - /home is sent to the subsequent tar - xvf-". The latter - is to get the stdout of the previous instruction, so we don't need to use filename! This is a very common example.
Summarize:
The basic commands of the pipeline are roughly the linux learning part 19 and 20. Most of these two articles are combined with my own practice. I really use some of the instructions very rarely, so I just rely on the examples in the book. Overall, it was quite rewarding.