Text processing tools
1、cut
- -d specified delimiter. For example -d: -d ''
- -f Specifies take the first few columns. For example -f1,3
- --output-delimiter specified display delimiter
tr -s -c -d Delete addition to compression
2, the percentage and the cut taken using disk tr
[root@glowing-bliss-1 data]# df -h | tr -s ' ' | cut -d' ' -f5 | tr -dc '[0-9]\n'
0
0
5
0
32
67
32
03, take the IP
[root@glowing-bliss-1 data]# ifconfig lo | head -2 | tail -1 | tr -s ' ' | cut -d' ' -f3
127.0.0.1
[root@glowing-bliss-1 data]# ifconfig lo | head -2 | tail -1 | tr -s ' ' : | cut -d: -f3
127.0.0.14, paste merge files
Merge the two files.-D specified delimiter
The following are combined and a.txt 1.txt, separator is:
qqq@qqq:~$ cat a.txt
a
b
c
d
qqq@qqq:~$ cat 1.txt
1
2
3
4
qqq@qqq:~$ paste 1.txt a.txt -d:
1:a
2:b
3:c
4:d5, wc command
Statistical line, words, bytes
Number of characters -L -m file widest row
[root@k8s-master ~]# wc /etc/passwd
20 28 874 /etc/passwd
[root@k8s-master ~]# wc -l /etc/passwd
20 /etc/passwd
[root@k8s-master ~]# wc -c /etc/passwd
874 /etc/passwd
[root@k8s-master ~]# wc -w /etc/passwd
28 /etc/passwd
[root@k8s-master ~]# wc -lwc /etc/passwd
20 28 874 /etc/passwd
[root@k8s-master ~]# wc -clw /etc/passwd
20 28 874 /etc/passwd6, wc show the longest length of the file name of the file in the current directory
[root@k8s-master ~]# ls -1 /etc/ | wc -L
247, sort command
The default is alphabetical order
-u delete duplicate rows in the output;
c -tc used as the field delimiter (that is, delimiter, and -d cut the same);
X-th column as a reference used to sort -kx
7.1 based on UID sort the passwd file
[root@k8s-master ~]# cat /etc/passwd | sort -t: -k3 -nr
qqq:x:1000:1000:qqq:/home/qqq:/bin/bash
polkitd:x:999:998:User for polkitd:/:/sbin/nologin
systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
...7.2 df disk utilization sorting
[root@glowing-bliss-1 data]# df -h | tr -s ' ' | cut -d' ' -f5 | tr -dc '[0-9]\n' | sort -n
0
0
0
0
5
32
32
678、uniq
After dropping duplicate rows from the output contact
-d display only the results of duplicate rows
-u never show only duplicate rows
[8.1] face questions statistic same list of files in two different directories
qqq@qqq:~$ ls . /tmp -1 | sort | uniq -d
10.txt
1.txt
2.txt
3.txt
4.txt
5.txt
6.txt
7.txt
8.txt
9.txt
qqq@qqq:~$ (ls /tmp -1;ls . -1) | sort | uniq -d #这样显示不显示目录
10.txt
1.txt
2.txt
3.txt
4.txt
5.txt
6.txt
7.txt
8.txt
9.txt[8.2] Statistics face questions in two different directories different list of files
qqq@qqq:~$ (ls /tmp -1;ls . -1) | sort | uniq -u #这样显示不显示目录
{1..txt}
a.txt
gems
myblogs
systemd-private-b8854b73c0e2479db1d56e43d8995bec-systemd-resolved.service-AcZjbU
systemd-private-b8854b73c0e2479db1d56e43d8995bec-systemd-timesyncd.service-sngpw9
vmware-root_643-39797085159, lastb display file contents btmb
[root@glowing-bliss-1 data]# lastb -f /var/log/btmp10, face questions [] to write a script for nginx log statistics, statistics of the top ten most visited IP
[root@glowing-bliss-1 data]# awk '{print $1}' /var/log/nginx/access.log | sort -nr | uniq -c | sort -nr | head11 different diff to compare two files
-u display a unified diff file format
qqq@qqq:~$ diff 1.txt 2.txt -u
--- 1.txt 2019-08-04 10:07:47.384154832 +0000
+++ 2.txt 2019-08-04 10:08:01.620224554 +0000
@@ -1,3 +1,4 @@
1
2
3
+413、patch
Replicated in other file changes the file (caution)
-b option to automatically back up files that have changed
Regular expressions (standard and expand)
First, the basic grammar
Standard regular expression characters need to be escaped are: (,), {,}, |, +,,.
Expand the regular expression under the first word, suffix, backslash quote these packets should not be spared
- * Means match any number of times, including zero
- Any character. * Denotes any length
- ? Means to match the preceding character zero or one time
- It matches the preceding character + means one or more times
- {N, m} indicates a match preceding character n to m times
- ^ Represents the beginning of a line
- $ Denote the end of line
- ^ $ Represent a blank line ^ [[: space:]] * $ represent a blank line
- \ <Beginning of a word \> end of the line
- \ <Hello \> hello word (to form words: alphanumeric characters and underscores, other characters are delimiters)
- (*) Denote packets, \ 1 \ 2 using the contents of the front bracket
2, show disk partition usage
[root@glowing-bliss-1 data]# df -h | grep '^/dev/sd' | tr -s ' ' | cut -d' ' -f5 | cut -d% -f1 | sort -n
32
673, replace the root rooter
:%s@\(root\)@\1er@g4, the regular expression to obtain an IP
qqq@qqq:~$ ifconfig ens33 | grep -Eo '([0-9]{,3}\.){3}([0-9]{,3})'
192.168.38.148
255.255.255.0
192.168.38.2555, find ntestat -atn output lines LISTEN followed by any whitespace
qqq@qqq:~$ netstat -atn | grep -E 'LISTEN[[:space:]]*$'
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp6 0 0 :::22 :::* LISTEN 6, view the number of connections per IP established
qqq@qqq:~$ cat ss.log | grep ESTAB | tr -s ' ' : | cut -d: -f6 | sort | uniq -c | sort -nr | head -3
44 127.0.0.1
10 113.234.28.244
8 124.64.18.1357, take the version number of CentOS7
[root@glowing-bliss-1 data]# cat /etc/centos-release | grep -Eo '[0-9]+' | head -1
78、grep
-A # 后多少行
-B # 前多少行
- # 前后各多少行
-i 忽略大小写
-e 相当于逻辑中的or，可以多次使用
-w 匹配整个单词9, misc directory
光盘会自动挂载到/misc/cd目录
autofs的作用，没有的话装一个并启动10, nmap scan LAN machine
View the current LAN IP which is used
[root@centos7 ~]# nmap -v -sn 192.168.10.1/24 | grep -B1 "Host is up" | grep report | cut -d' ' -f5
192.168.10.1
192.168.10.6
192.168.10.12
192.168.10.14
192.168.10.16
192.168.10.19
192.168.10.20
192.168.10.28
...11, basic regular expression metacharacters
Character matches
- . Matches any single character
- [] Matches any single character within a specified range. [. ?] Indicates a match. Or \ or a question mark
- Any single character [^] matches outside the range
- [: Digit:] and so on a series of
12、cat -A
You can view some see things : Tab, carriage return line feed
[root@centos7 ~]# cat -A 1.txt
a b^Ic$
dd$
$
d$
$
$13, program implementation
High-level language -> compiler -> machine code -> Implementation
14, .vimrc vim configuration file
root@qqq:~# vim ~/.vimrc
root@qqq:~# cat ~/.vimrc
set ignorecase
set cursorline
set autoindent
set ai
autocmd BufNewFile *.yaml exec ":call SetTitle()"
func SetTitle()
if expand("%:e") == 'yaml'
call setline(1,"#**************************************************************")
call setline(2,"#Author: uscwifi")
call setline(3,"#QQ: 2*******1")
call setline(4,"#Date: ".strftime("%Y-%m-%d"))
call setline(5,"#FileName: ".expand("%"))
call setline(6,"#URL: http://www.baidu.com")
call setline(7,"#Description: The test script")
call setline(8,"#Copyright (C): ".strftime("%Y")." Copyright © 站点名称 版权所有")
call setline(9,"#************************************************************")
call setline(10,"")
endif
endfunc
autocmd BufNewFile * normal G15, the directory path to the script into the PATH, execute scripts directly
Script execution mode
- 1、bash date.sh
- 2, the relative path method ./date.sh
- 3, absolute path method
- 4, PATH variable method, scripts directories to the PATH variable (or write bashrc profile, and then source)
- 5、cat date.sh|bash
16, two errors script
shell and python are all interpreted syntax, explain the side edge execution; C language, JAVA, etc. is a compiled language, you can perform after all the code is compiled
- 1, grammatical errors, it is best to use bash -n date.sh checks to reduce syntax errors, command syntax error will not be executed, but in front of the command will be executed
- 2, another error: there is no such command, command errors. Not affect the execution of subsequent code. In the beginning of the script together with set -e when you can make a script error stops immediately.
- bash -n check syntax errors, be sure to check the script to finish.
- bash -x for debugging scripts, see the process of execution of the script, execution logic.
17, shell summation of several methods:
17.1, let command, more commonly
[root@centos7 ~]# x=10
[root@centos7 ~]# y=17
[root@centos7 ~]# let z=$x+$y
[root@centos7 ~]# echo $z
2717.2, $ (()) more common
[root@centos7 ~]# x=100
[root@centos7 ~]# y=302
[root@centos7 ~]# z=$(($x+$y))
[root@centos7 ~]# echo $z
40217.3, bc sum, more commonly
[root@centos7 ~]# x=123
[root@centos7 ~]# y=986
[root@centos7 ~]# echo $x+$y | bc
110918, shell scripts color
echo -e parameters have to be before you can use color
[root@glowing-bliss-1 ~]# cat /data/docker_stats.sh
#!/bin/bash
RED="\e[31;1m"
GREEN="\e[32;1m"
YELLOW="\e[33;1m"
END_COLOR="\e[0m"
# 用法
echo -e "\n${YELLOW}################################${END_COLOR}\n"19、cat /proc/partitions
20, single and double quotation marks Backticks
- Single quotes: strong references, does not recognize the command, does not recognize variables
- Double quotes: not recognize the command, but do not variable;
- Reverse single quotes are identified
[root@glowing-bliss-1 ~]# echo '$HOSTNAME'
$HOSTNAME
[root@glowing-bliss-1 ~]# echo "$HOSTNAME"
glowing-bliss-1.localdomain
[root@glowing-bliss-1 ~]# ls `pwd`
virt-sysprep-firstboot.log
[root@glowing-bliss-1 ~]# echo `hostname`
-bash: echglowing-bliss-1.localdomain: command not found21, the program has a parent and child processes
21.1, parent-child relationship between the viewing process
pstree
21.2, view the current process ID
echo $BASHPID
echo $$ echo $$ often used in the script, the script runs in the process to see ID
21.3, view the parent process ID
echo $PPID
22、set
unset to delete variables
set View
set -C after the execution can not overwrite existing files
shell scripting foundation
[. ?] Representation. ? In one
grep -E "(bash|nologin)$" /etc/passwd grep -E "bash$|nologin$" /etc/passwd
1, the environment variable declaration
- export EDITOR = vim defined default editor to write profile
- declare -x EDITOR = vim equivalent to the above
- declare -r statement equal readonly name is read-only variables
2、set --
Clear all position variables
3, variable references to get used to braces
- ${10} ${DATE} ${PWD}