0039-【Linux-Shell】-Linux_Shell脚本攻略-第二章-命令之乐

2.1 简介

grep awk sed find

2.2 用cat进行拼接

合并法输入标题

$cat <(head -n 1 aaa.txt) <(cat bbb.sh) > merge.txt

合并多个文件

cat file1 file2 file3

-s 去除空白行

cat -s merge.txt
# 等于
cat merge.txt |tr -s '\n'

-n 显示行号

–

2.4 文件查找与文件列表

find

-print以”\n”作为每个匹配的文件名

$find ./ -name "merge.txt" -print
./merge.txt

文件类型
-type

find ./ -type f

2.5 xargs

将标准输入数据转换为命令行参数。

command |xargs

将多行输入转为单行输入
原理：将换行符移除，再用“ ”空格进行替代

扫描二维码关注公众号，回复： 1559289 查看本文章

$cat test1.txt
1 2 3 4 5 6
7 8 9 10
11 12

$cat test1.txt |xargs
1 2 3 4 5 6 7 8 9 10 11 12

将单行输入转为多行输出
指定每行最大的参数数量n，每个参数由空格，作为定界符

$cat test1.txt |xargs -n 3
1 2 3
4 5 6
7 8 9
10 11 12

指定定界符，进行分割字符串

$echo "aaabcccbddd"|xargs -d b
aaa ccc ddd

$echo "aaabcccbddd"|xargs -d b -n 2
aaa ccc
ddd

传递格式化后参数


$cat ./test1.txt
echo $*

$echo "aaabcccbddd"|xargs -d b -n 1 ./test1.txt
aaa
ccc
ddd

统计行数

$find . -type f -name "*.txt" -print|xargs wc -l
  22 ./baidu.txt
  10 ./baidu2.txt
   9 ./aaa.txt
   5 ./merge.txt
   0 ./百度.txt
   1 ./test1.txt
  47 total

2.6 用tr进行转换

tr可以对来自标准输入的字符进行替换、删除、压缩。将一组字符变为另一组字符。

tr set1 set2
只能通过标准输入，将set1映射到set2，并将输出写入标准输出

大小写转换
A-Z
0-9

$echo "HELLO WHOLD" |tr 'A-Z' 'a-z'
hello whold

数字加密、解密

$echo 12345 |tr "0-9" "9876543210"
87654

$echo 87654 |tr "9876543210"  "0-9"
12345

删除字符
只需要set1

$echo "hello world"|tr -d 'lw'
heo ord

-c补集合，将不在补集合的部分删除
tr -c set1 set2
\n表示补集的输入

$echo "hello 1 world 2"|tr -d -c '0-9 \n'
 1  2

相加，将换行符替换为加好，然后最后加0

$echo -e "1\n2\n3\n4 \n5"
1
2
3
4
5

$echo -e "1\n2\n3\n4 \n5"|echo $[ $(tr '\n' '+') 0 ]
15

2.7 校验与核实

校验：从文件中生成校验和密钥，然后利用这个校验和密钥核实文件的完整性。

md5sum

$md5sum test1.txt
e3b54186d991a396af4705e376940adc  test1.txt

$md5sum test1.txt test2.sh test3.sh
e3b54186d991a396af4705e376940adc  test1.txt
b67852fcd35c2197564013ebbe00e3b3  test2.sh
f19dce7dcfabf5909b5ad20cda132f68  test3.sh

$md5sum test1.txt test2.sh test3.sh >file.md5
$md5sum -c file.md5
test1.txt: OK
test2.sh: OK
test3.sh: OK

排序、单一与重复

`sort`

uniq
- -d 非重复
- -c 统计数量

使用第一个字符排序

sort -nk 1,1 data.txt

# 默认为第一列
Sort a file in ascending order:
sort {{filename}}

# 逆序
Sort a file in descending order:
sort -r {{filename}}

# 按数字排序
Sort a file using numeric rather than alphabetic order:
sort -n {{filename}}

# 指定分隔符：，指定第三列 的数字大小进行排序
Sort the passwd file by the 3rd field, numerically:
sort -t: -k 3n /etc/passwd

# 唯一的
Sort a file preserving only unique lines:
sort -u {{filename}}

`uniq`

Display each line once:
sort {{file}} | uniq

# 仅输出唯一的
Display only unique lines:
sort {{file}} | uniq -u

# 仅输出重复行
Display only duplicate lines:
sort {{file}} | uniq -d

# 统计每行的数量
Display number of occurrences of each line along with that line:
sort {{file}} | uniq -c


Display number of occurrences of each line, sorted by the most frequent:
sort {{file}} | uniq -c | sort -nr

-s 2 忽略前两个字符
-w 2 指定第二个字符为键
-z 生成包含0值字节终止的输出

2.9 临时文件命令与随机数

1.10 分割文件和数据

split

Split a file, each split having 10 lines (except the last split):
split -l {{10}} {{filename}}

Split a file into 5 files. File is split such that each split has same size (except the last split):
split -n {{5}} {{filename}}

Split a file with 512 bytes in each split (except the last split; use 512k for kilobytes and 512m for megabytes):
split -b {{512}} {{filename}}

Split a file with at most 512 bytes in each split without breaking lines:
split -C {{512}} {{filename}}

2.11 根据扩展名切分文件名

原理：从变量中删除位于%右侧的通配符.*所匹配的字符串。通配符从右边向左边匹配。%属于非贪婪操作，找到最短的结果。%%为贪婪操作，匹配最长的结果

# 名称
$file_jpg="sample.jpg";name=${file_jpg%.*};echo $name
sample

# 后缀
$file_jpg="sample.jpg";ext=${file_jpg#*.};echo $ext
jpg

2.12 批量重命名和移动

rename

# rename 替换后字符 替换前字符的文件
Rename files using a Perl Common Regular Expression (substitute 'foo' with 'bar' wherever found):
rename {{'s/foo/bar/'}} {{\*}}

Dry-run - display which renames would occur without performing them:
rename -n {{'s/foo/bar/'}} {{\*}}

Force renaming even if the operation would overwrite existing files:
rename -f {{'s/foo/bar/'}} {{\*}}

Convert filenames to lower case (use -f in case-insensitive filesystems to prevent "already exists" errors):
rename 'y/A-Z/a-z/' {{\*}}

Replace whitespace with underscores:
rename 's/\s+/_/g' {{\*}}