1. Thematic background
Recently, I used an automation platform (see Automated Operation and Maintenance Platform Spug Test for details ) to make weekly changes. The effect is very good. The platform has standardized and automated a large number of repetitive and tedious operations through script distribution. The core of the platform is distributed to each server. Shell script, I feel it is necessary to make a summary of shell script, so I have the idea of writing this topic. This topic will introduce the various uses of shell scripts in combination with operation and maintenance. It is estimated that about 10 articles will include system inspection, monitoring, ftp upload and download, database query, log cleanup, clock synchronization, timing tasks, etc., which will involve shell commonly used Grammar, notes, debugging, etc.
2. Preface
This article is the second of the topic.
The article mainly introduces some shell statements recently used in daily production operation and maintenance, mainly including replacement, deletion, search for specified lines and specified characters, file transfer, column and column interchange, etc.
Three, shell use cases
1. Prohibit root login directly
Requirements: To ensure safety, direct login with root account is prohibited in production
before fixing:
[root@ansible /etc/ssh]# ll|grep sshd_config
-rw-------. 1 root root 3907 4月 11 2018 sshd_config
[root@ansible /etc/ssh]# more /etc/ssh/sshd_config|grep PermitRootLogin
#PermitRootLogin yes
# the setting of "PermitRootLogin without-password".Before the modification, there is only one sshd_config file under the directory /etc/ssh and PermitRootLogin is configured as a comment state
After modification:
[root@ansible /etc/ssh]# sed -i.bak 's/#PermitRootLogin yes/PermitRootLogin no/g' /etc/ssh/sshd_config
[root@ansible /etc/ssh]# more /etc/ssh/sshd_config|grep PermitRootLogin
PermitRootLogin no
# the setting of "PermitRootLogin without-password".
[root@ansible /etc/ssh]# ll|grep sshd_config
-rw------- 1 root root 3905 1月 22 15:26 sshd_config
-rw-------. 1 root root 3907 4月 11 2018 sshd_config.bakAfter the modification, the directory /etc/ssh has more backup files and the PermitRootLogin parameter is uncommented and is in no state
2. Update the sudoer list
Requirements: some accounts require root privileges
before fixing:
[root@ansible /etc]# cd
[root@ansible ~]# cd /etc
[root@ansible /etc]# ll|grep sudoers
-r--r-----. 1 root root 4328 10月 30 2018 sudoers
drwxr-x---. 2 root root 6 10月 31 2018 sudoers.d
[root@ansible /etc]# more sudoers|grep 'ALL=(ALL)'
root ALL=(ALL) ALL
%wheel ALL=(ALL) ALL
# %wheel ALL=(ALL) NOPASSWD: ALLBefore modification, there is only sudoers file under /etc and no other accounts except root have root privileges
After modification:
[root@ansible /etc]# sed -i.bak "/root.*ALL=(ALL).*ALL/a\app ALL=(ALL) ALL" /etc/sudoers
[root@ansible /etc]# ll|grep sudoers
-r--r----- 1 root root 4355 1月 22 16:05 sudoers
-r--r----- 1 root root 4328 1月 22 15:53 sudoers.bak
drwxr-x---. 2 root root 6 10月 31 2018 sudoers.d
[root@ansible /etc]# more sudoers|grep 'ALL=(ALL)'
root ALL=(ALL) ALL
app ALL=(ALL) ALL
%wheel ALL=(ALL) ALL
# %wheel ALL=(ALL) NOPASSWD: ALLAfter the modification, there is an additional backup file sudoers.bak in /etc and the line under root has more information about the app account, so that the app has root privileges
3.scp copy multiple directories or files
Requirements: Copy multiple local files to the remote or copy multiple files from the remote to the local
Copy locally to remote directory:
[root@ansible ~]# touch a.txt b.txt c.txt
[root@ansible ~]# mkdir d
[root@ansible ~]# scp -v -r a.txt b.txt c.txt d [email protected]:/tmpCreate a new file a.txt b.txt c.txt and directory d locally and copy them to the /tmp directory of the remote host
Copy the remote directory to the local:
[root@157 ~]# touch 01.sh 02.sh 03.sh
[root@157 ~]# mkdir 04[root@ansible ~]# scp -v -r [email protected]:/root/\{01.sh,02.sh,03.sh,04\} /tmpCreate a new file 01.sh 02.sh 03.sh and directory 04 on 157 and copy them to the local /tmp directory
4. Delete existing scheduled tasks
Requirement: delete the specified timed task
[root@ansible /var/spool/cron]# crontab -l
0 0 * * * /usr/sbin/ntpdate -u ntpserver >> /tmp/ntp.log
[root@ansible /var/spool/cron]# sed -i.bak '/\/usr\/sbin\/ntpdate/d' /var/spool/cron/root
Note that "/" needs to be escaped when matching /usr/sbin/ntpdate
5. Swap between columns and columns
Requirement: swap the ip and hostname in the /etc/hosts file, the format of the host list in ansible is hostname+ip
before fixing:
[root@ansible ~]# cd /etc
[root@ansible /etc]# cp hosts hosts-ansible.txt
[root@ansible /etc]# more hosts-ansible.txt
10.17.6.137 loong576-file01
10.17.6.129 loong576-kxxl01
10.17.6.130 loong576-kxxl02
10.17.6.131 loong576-capp01
10.17.6.132 loong576-capp02
10.17.6.128 loong576-pbgw01
10.17.6.127 loong576-pbgw02
10.17.6.134 loong576-xucs01
10.17.6.133 loong576-xucs02
10.17.6.136 loong576-webc01
10.17.6.135 loong576-webc02
10.17.6.14 loong576-gwxx-1
10.17.6.15 loong576-gwxx-2
10.17.6.18 loong576-mysql1
10.17.6.17 loong576-mysql2After modification:
[root@ansible /etc]# awk '{print $2,$1 > "hosts-ansible.txt"}' hosts-ansible.txt
[root@ansible /etc]# more hosts-ansible.txt
loong576-file01 10.17.6.137
loong576-kxxl01 10.17.6.129
loong576-kxxl02 10.17.6.130
loong576-capp01 10.17.6.131
loong576-capp02 10.17.6.132
loong576-pbgw01 10.17.6.128
loong576-pbgw02 10.17.6.127
loong576-xucs01 10.17.6.134
loong576-xucs02 10.17.6.133
loong576-webc01 10.17.6.136
loong576-webc02 10.17.6.135
loong576-gwxx-1 10.17.6.14
loong576-gwxx-2 10.17.6.15
loong576-mysql1 10.17.6.18
loong576-mysql2 10.17.6.17
This script can easily realize the position exchange of ip and hostname
6. Add specified line
Requirements: Add the parameter'ansible_ssh_host=' at the same time before the ip address 10.17.6
[root@ansible /etc]# sed -i 's/10.17.6/ansible_ssh_host=&/' hosts-ansible.txt 
Through 5 and 6, the format of ip+hostname of /etc/hosts can be easily converted into the format of hostname+ansible_ssh_host=+ip to meet the format requirements of ansible for hostname
Through 5 and 6, the format of ip+hostname of /etc/hosts can be easily converted into the format of hostname+ansible_ssh_host=+ip to meet the format requirements of ansible for hostname
7.find, xargs, rm delete found files
Requirements: Use find to find files that meet the conditions and delete them
[root@ansible /]# find ./ -name *[0-9]\*.bak
./home/a001.bak
./opt/04.txt.bak
./opt/03.bak
./tmp/09.bak
./tmp/10.txt.bak
./usr/05.bak
./usr/06.txt.bak
./var/07.bak
./var/08.txt.bak
./root/02.txt.bak
./root/01.bak
[root@ansible /]# find ./ -name *[0-9]\*.bak|xargs rm -rf
[root@ansible /]# find ./ -name *[0-9]\*.bakFind all files starting with a number and ending with .bak, then delete
8.sed, find, grep delete/replace specified characters in files
Requirements: Find the string containing'loong576' in all files and replace or delete
before fixing:
[root@ansible-awx os-check]# find .|xargs grep -rl 'loong576'
./defaults/main.yaml
./files/check_linux.sh
./tasks/main.yaml
./defaults/main.yaml
./defaults/main.yaml
./files/check_linux.sh
./files/check_linux.sh
./tasks/main.yaml
./tasks/main.yaml
[root@ansible-awx os-check]# find .|xargs grep -ri 'loong576'
./defaults/main.yaml:# Created by loong576 2020.05
./files/check_linux.sh:# Created by loong576 2020.05
./tasks/main.yaml:# Created by loong576 2020.05
./defaults/main.yaml:# Created by loong576 2020.05
./defaults/main.yaml:# Created by loong576 2020.05
./files/check_linux.sh:# Created by loong576 2020.05
./files/check_linux.sh:# Created by loong576 2020.05
./tasks/main.yaml:# Created by loong576 2020.05
./tasks/main.yaml:# Created by loong576 2020.05 Find the list of files containing the word loong576 and point out the specific characters contained in the file
After modification: Change the comment of'Created by loong576 2020.05' to'Created by loong576 2021.01'
[root@ansible-awx os-check]# sed -i "s/2020.05/2021.01/g" `find .|xargs grep -rl 'loong576'`
[root@ansible-awx os-check]# find .|xargs grep -ri 'loong576'
./defaults/main.yaml:# Created by loong576 2021.01
./files/check_linux.sh:# Created by loong576 2021.01
./tasks/main.yaml:# Created by loong576 2021.01
./defaults/main.yaml:# Created by loong576 2021.01
./defaults/main.yaml:# Created by loong576 2021.01
./files/check_linux.sh:# Created by loong576 2021.01
./files/check_linux.sh:# Created by loong576 2021.01
./tasks/main.yaml:# Created by loong576 2021.01
./tasks/main.yaml:# Created by loong576 2021.01 Delete time:
[root@ansible-awx os-check]# sed -i "s/2021.01//g" `find .|xargs grep -rl 'loong576'`
[root@ansible-awx os-check]# find .|xargs grep -ri 'loong576'
./defaults/main.yaml:# Created by loong576
./files/check_linux.sh:# Created by loong576
./tasks/main.yaml:# Created by loong576
./defaults/main.yaml:# Created by loong576
./defaults/main.yaml:# Created by loong576
./files/check_linux.sh:# Created by loong576
./files/check_linux.sh:# Created by loong576
./tasks/main.yaml:# Created by loong576
./tasks/main.yaml:# Created by loong576
9. Specify a character to add characters to the front and the previous line, and add a new line to the last line
Requirement: When configuring ntp server, you need to add a new line on the specified character in the configuration file /etc/ntp.conf, comment some default configurations (add # before the specified character), and add a new line at the end of the configuration file /etc/hosts
Add a new line on the specified character:
[root@ansible ~]# sed -i '/driftfile/i server ntpserver iburst' /etc/ntp.conf Add'server ntpserver iburst' to the specified line of driftfile
Note some default configurations (add # before the specified characters):
[root@ansible ~]# sed -i '/centos.pool.ntp.org/s/^/#/' /etc/ntp.confComment'server [0..3].centos.pool.ntp.org iburst'
Before the change:
After the change:
Add in the last line:
[root@ansible ~]# sed -i '$a 172.16.7.157 ntpserver' /etc/hosts
4. Summary of this article
This article mainly introduces some commonly used shell use cases, involving daily search, replacement, file transfer, etc. The commands used mainly include find, sed, xargs, scp, etc. The shell script is to logically combine the various commands according to the unusable purpose, and mastering these commands will have a multiplier effect on the subsequent scripting.
For more please pay attention: shell topic