Automated operation and maintenance tools
Requirements: Hundreds or thousands of machines execute certain commands in batches and upgrade certain services in batches.
Shell and expect scripts can be automated, but we need to manually write scripts by ourselves, which is inefficient and costly to learn and maintain.
Mainstream automated operation and maintenance tools: Puppet, Saltstack, Ansible, etc.
Puppet
Introduction to Puppet: https://blog.51cto.com/ixdba/1149055
Puppet is an open source Ruby-based system configuration management tool, based on a C/S deployment architecture. It is a configuration management software designed to realize automatic management of data centers. It uses cross-platform language specifications to manage configuration files, users, software packages, system services, etc. By default, the client will communicate with the server every half an hour to confirm whether there is an update. Of course, you can also configure the active trigger to force the client to update. In this way, daily system management tasks are coded. The benefits of coding are that they can be shared, saved, avoid duplication of work, and can also be quickly restored and quickly deployed on a large scale.
Salt stack
Saltstack is developed in Python and is a very easy-to-use and lightweight management tool. The C/S architecture is composed of Master and Minion, and communicates through ZeroMQ. Saltstack's master listens to ports 4505 and 4506, 4505 is the salt message publishing system, and 4506 is the port for communication between the salt client and the server; the salt client program does not monitor the port. After the client is started, it will actively connect to the master to register. Then the TCP connection is maintained, and the master controls the client through this TCP connection. If the connection is disconnected, the master has nothing to do with the client. Of course, if the client detects that it is disconnected, it will continue to connect to the master periodically.
Ansible
What is ansible? The official title is: Ansible is Simple IT Automation. Chinese explanation: Simple IT automation tool.
The goals of this tool are as follows: automated deployment of APP; automated management of configuration items; automated continuous interaction; automated (AWS) cloud service management.
All these targets are essentially executing a series of commands on one or several servers. In layman's terms, it is to execute commands on remote servers in batches. Of course, the most important thing is that it is developed based on paramiko. What is this paramiko? It is a pure Python implementation of the ssh protocol library. Therefore, fabric and ansible have one thing in common is that they do not need to install client/agents on the remote host, because they are based on ssh to communicate with the remote host. A brief summary: development based on Python paramiko, distributed, no client required, lightweight, configuration syntax using YMAL and Jinja2 template language, stronger remote command execution operation.
Saltstack vs Ansible
Saltstack must install the agent (minion), so the deployment process is one more step, and Ansible does not need to install the agent.
SaltStack is a C/S architecture, with master and minion, Ansible only needs to have a central node.
Saltstack is based on ZeroMQ communication, Ansible is based on ssh, and the execution efficiency of Saltstack is faster than Ansible. If the number of machines is large (>200), it is better to use Saltstack
Ansible is easier to learn than Saltstack
For a more detailed comparison of other aspects, refer to
https://www.cnblogs.com/huang0925/p/4664608.html
Ansible
Official website www.ansible.com
Online e-book:https://getansible.com
Ansible is an open source software sponsored by RedHat. It is an automation language that can be used by the entire IT team, from system to network to development. It has integrated virtualization (Vmware, RHEV, Xen, etc.), network equipment (Cisco, F5, OpenSwitch), containers (Docker, LXC), public clouds (Amazon Cloud AWS, Microsoft Azure), DEVOPS (Gitlab, Github, etc.) Jenkins), monitoring/analysis (Splunk, InfluxDB) and many other fields.
Install Ansible
Documentation: https://docs.ansible.com/ansible/latest/index.html
Install Ansible on CentOS7:
yum install -y epel-release
yum install -y ansible
In addition, ansible also supports pip installation
yum install python-pip
pip install ansible
Verify that the login is successful, check the version
ansible --version
Because Ansible is an angel-less, it has only one control center, and other machines do not need to install any software packages. But if you want to control a remote machine, you need to configure key authentication.
1) Generate a key pair in the control center
It is best to check first, ls /root/.ssh/, to see if there are two files id_rsa and id_rsa.pub in this directory. If the following command is not executed
```
ssh-keygen
[root@jinkai01 ~]# ssh-keygen
[root @ jinkai01 ~] # ls /root/.ssh/
authorized_keys id_rsa id_rsa.pub known_hosts
```
2) Configure key authentication
In the control center, execute ssh-copy-id
ssh-copy-id [user@] remote machine ip
[root@jinkai01 ~]# ssh-copy-id 192.168.111.140
Verify passwordless remote login
[root@jinkai01 ~]# ssh 192.168.111.140
Last login: Mon Dec 7 21:51:42 2020 from jinkai01
[root@jinkai05 ~]#
3) Edit the hosts file
Ansible realizes the management of the device by adding the device list to the /etc/ansible/hosts file in a grouping manner. Therefore, before the formal management, the hosts file must be written first. In the hosts file, the part contained in [] represents the group name, and the device list supports host names and IP addresses.
By default, the device is managed by accessing port 22 (SSH). If the target host uses a non-default SSH port, you can also use a colon and port after the host name to indicate the configuration, separating the configuration by line unit. In addition, the hosts file also supports wildcards.
#The format is as follows
vim /etc/ansible/hosts
## [test]
## alpha.example.org
## beta.example.org
## 192.168.1.100
## 192.168.1.110
test
The results returned by Ansible are very friendly. Generally, three colors are used to indicate the execution results:
Red: indicates that there is an abnormality in the execution process;
Orange color: indicates that the target has a status change after the command is executed;
Green: indicates that the execution is successful and no target machine has been modified;
ansible all -m ping
[root@jinkai01 ~]# cat /etc/ansible/hosts
[test]
192.168.111.140
jinkai05
[root@jinkai01 ~]# ansible all -m ping
jinkai05 | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
}
192.168.111.140 | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
}
[root@jinkai01 ~]#
# Here all represents all the hosts in the hosts file, or you can specify a single
ansible jinkai05 -m ping
[root@jinkai01 ~]# ansible jinkai05 -m ping
jinkai05 | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
}
Execute commands remotely
command module
The command module executes commands on the remote host, and does not support shell features such as pipes and redirection. The commonly used parameters are as follows:
chdir: The directory to be entered in advance before running the command on the remote host;
creates: creates a file when the command is running, if the file already exists, the creation task will not be executed; removes: removes a file when the command is running, if the file does not exist, the removal task will not be executed; executeable: indicates to run Command shell program
[root@jinkai01 ~]# ansible test -m command -a "chdir=/ ls ./"
ansible test -m command -a 'hostname'
[root@jinkai01 ~]# ansible test -m command -a 'hostname'
jinkai05 | CHANGED | rc=0 >>
jinkai05
192.168.111.140 | CHANGED | rc=0 >>
jinkai05
#The test here is the host group name configured in the hosts configuration file, the module name is specified after -m, the command here is the module that executes the command remotely, and -a is followed by the specific command
ansible 127.0.0.1 -m shell -a 'w'
[root@jinkai01 ~]# ansible 127.0.0.1 -m shell -a 'w'
127.0.0.1 | CHANGED | rc=0 >>
22:07:31 up 2:17, 1 user, load average: 0.20, 0.13, 0.09
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
root pts/0 192.168.111.1 19:56 3.00s 0.95s 0.01s w
#In addition to the command module, you can also use the shell module to implement remote command execution. The shell also supports the execution of shell scripts on remote hosts.
Remote script execution
shell module
The shell module executes commands on the remote host, which is equivalent to calling the Shell process of the remote host, and then opens a sub-Shell to run the command under the Shell. The difference with the command module is that it supports Shell features: such as pipes, redirects, etc.
First create a shell script, /tmp/1.sh, with the following content
#!/bin/bash
echo date > /tmp/ansible_test.txt
Then distribute the script to each machine
ansible test -m copy -a "src=/tmp/1.sh dest=/tmp/test.sh mode=0755"
Finally, execute the shell script in batches
ansible test -m shell -a "/tmp/test.sh"
Shell module, also supports remote execution of commands and with pipes
ansible test -m shell -a "cat /etc/passwd|wc -l "
Copy files or directories
copy module
The copy module is used to copy the specified host file to the specified location of the remote host. Common parameters are as follows:
dest: Point out the location of the target directory for copying files, using an absolute path. If the source is a directory, the target must also be a directory. If the target file already exists, the original content will be overwritten;
src: Point out the path of the source file, you can use relative path and absolute path, support to specify the directory directly. If the source is a directory, the target must also be a directory;
mode: indicates the permission of the target file when copying, optional;
owner: indicates the owner of the target file when copying, optional;
group: indicates the group of the target file when copying, optional;
content: Point out that the content copied to the target host cannot be used with src, which is equivalent to copying
The data specified by content to the target file;
ansible jinkai05 -m copy -a "src=/etc/passwd dest=/tmp/test123 owner=root group=root mode=0755"
[root@jinkai01 ~]# ansible jinkai05 -m copy -a "src=/etc/passwd dest=/tmp/test123 owner=root group=root mode=0755"
jinkai05 | CHANGED => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": true,
"checksum": "75a4463c380b111d3bf0f2dae9787bc339b2f86e",
"dest": "/tmp/test123",
"gid": 0,
"group": "root",
"md5sum": "4b94e9c4fce7a5a255401706b959b739",
"mode": "0755",
"owner": "root",
"size": 1030,
"src": "/root/.ansible/tmp/ansible-tmp-1607350135.26-8003-211914668409808/source",
"state": "file",
"uid": 0
}
#Note: The source directory will be placed under the target directory. If the directory specified by the target does not exist, it will be created automatically. If the copy is a file, if the name specified by dest is different from the source, and it is not an existing directory, it is equivalent to renaming after copying. But on the contrary, if desc is a directory that already exists on the target machine, the file will be copied directly to the directory.
ansible test -m copy -a "src=/etc/passwd dest=/tmp/123"
#The /tmp/123 here is the same as the /etc/passwd on the source machine, but if there is already a /tmp/123 directory on the target machine, a passwd file will be created under the /tmp/123 directory
Reference link: https://www.cnblogs.com/keerya/p/7987886.html
Extension: Ansible common modules
1) View module help: ansible-doc module name such as: ansible-doc shell search: /EX View case
2) Ping module: test whether the ssh port connected to the target host can communicate normally
[root@jinkai01 ~]#ansible` test -m ping -o```
3) command module: used for remote host commands, the default module, shell module is recommended
[root@jinkai01 ~]# ansible test -mcommand-a'uptime'``
4) Shell module: execute remote commands, some simple shell commands
`[root@jinkai01 ~]# ``ansible test -m shell -a 'cat /etc/passwd |grep root'```
5) copy module: copy files to the remote host
[root@jinkai01 ~]# ansible test -mcopy -a 'src=/etc/hosts dest=/tmp/hosts '``
[root@jinkai01 ~]# ansible test -mcopy -a 'src=/etc/hosts dest=/tmp/hosts backup=yes'``
6) user module: create and delete users
创建用户：ansible test -m user -a'name=jack shell=/sbin/nologin'``
删除用户：ansible test -m user -a'name=jack state=absent'``
echo123| openssl passwd-1` -stdin```
ansible test -m user -a'name=jack password="$1$Ec9OCYTb$OJtUgRRglXNuEMmSCLfPl/"'``
7) yum module: install and uninstall software packages
安装：ansible test -m yum -a'name=httpd state=latest'``
卸载：ansible test -m yum -a'name=httpd state=absent'``
8) service module: control the running status of the service
ansible'name test -m service -a=httpd state=started enabled=yes'` \state`` is the start, restart, and shutdown of the service; enabled is the startup of the server ```
ansible test -m service -a'name=httpd state=stopped enabled=no'` ```
9) file module: create files or directories
创建文件：ansible test -m file -a 'path=/tmp/1.logmode=777` state=touch' ```
创建目录：ansible test -m file -a 'path =/tmp/dir1mode =777` state = directory'```
10) cron module: create and remove defined tasks
ansible test -m cron -a'name="test"minute="*/10"job="/bin/echo hello"`'```
ansible test -m cron -a'name="test2"minute="00"hour="03"job="/bin/echo hello"`'```
移除：ansible test -m cron -a `'name`=`"test2"` minute=`"00"` hour=`"03"` job=`"/bin/echo hello"` state=absent'