SSH connection and automated deployment tools paramiko and Fabric

paramiko

paramiko is a Python-based SSH2 remote secure connection that supports authentication and key methods. It can realize functions such as remote command execution, file transfer, and intermediate SSH proxy. Compared with Pexpect, the encapsulation level is higher.

pip install Paramiko

http://www.paramiko.org/
demo:https://github.com/paramiko/p...

If you are in a linux environment, you need to install dependencies: crypto, ecdsa, python3-devel.

Paramiko contains two core components: SSHClient class, SFTPClient class

login by key

import paramiko, base64,getpass
 
paramiko.util.log_to_file('syslogin.log') #日志记录
try:
        key=paramiko.RSAKey.from_private_key_file('pk_path')
except paramiko.PasswordRequiredException:
        password = getpass.getpass('RSA key password: ')
        key = paramiko.RSAKey.from_private_key_file('pk_path', password)    # 需要口令的私钥
#key = paramiko.RSAKey(data=base64.decodestring('AAA...'))
client = paramiko.SSHClient()
# client.get_host_keys().add('ssh.example.com', 'ssh-rsa', key)
client.load_system_host_keys()#~/.ssh/known_hosts
client.connect('ssh.example.com', 22,username='strongbad', password='thecheat',pkey=key)
stdin, stdout, stderr = client.exec_command('ls')
# stdin, stdout, stderr=ssh.exec_command('sudo su')
# stdin.write('123456')
for line in stdout:
    print('... ' + line.strip('\n'))
#使用send
# cmds=['sudo su\n', 'cd /var/log\n', 'ls\n'] #利用send函数发送cmd到SSH server，添加'\n'做回车来执行shell命令。注意不同的情况，如果执行完telnet命令后，telnet的换行符是\r\n
# ssh=s.invoke_shell() #在SSH server端创建一个交互式的shell，且可以按自己的需求配置伪终端，可以在invoke_shell()函数中添加参数配置。
# for cmd in cmds:
#         time.sleep(1)
#         ssh.send(cmd) #利用send函数发送cmd到SSH server，
#         out = ssh.recv(1024) #.recv(bufsize)通过recv函数获取回显。
#         print out
client.close()

Login with username and password

#####################################################################################
import paramiko
 
paramiko.util.log_to_file('syslogin.log') #日志记录
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect('192.168.8.248', 22, username='root', password='password', timeout=4)
stdin, stdout, stderr = client.exec_command('ls -l')
#print(stdout.read())
for line in stdout.readlines():
  print(line)
client.close()

SSHClient方法参数说明
connect(hostname, port=22, username=None, password=None, pkey=None, key_filename=None, timeout=None, allow_agent=True, look_for_keys=True, compress=False, sock=None, gss_auth=False, gss_kex=False, gss_deleg_creds=True, gss_host=None, banner_timeout=None)

pkey - private key type
key_filename - str or list(str) private key file or its list
timeout - in seconds
allow_agent - disables connecting to SSH agent
when False look_for_keys - disables searching for private keys in ~/.ssh file
exec_command(command, bufsize=-1, timeout=None, get_pty=False) command-
string

load_system_host_keys(filename=None) specifies the public key file, the default is ~/.ssh/known_hosts

set_missing_host_key_policy(policy): Sets the policy when the connected remote host does not have a localhost key. Currently supports three: RejectPolicy (the default), AutoAddPolicy, WarningPolicy

Upload and download files

#上传批量文件到远程主机
import paramiko
import os
import datetime
 
hostname = '74.63.229.*'
username = 'root'
password = 'abc123'
port = 22
local_dir = '/tmp/'
remote_dir = '/tmp/test/'
if __name__ == "__main__":
    #    try:
    t = paramiko.Transport((hostname, port))
    t.connect(username=username, password=password)
    sftp = paramiko.SFTPClient.from_transport(t)
    #        files=sftp.listdir(dir_path)
    files = os.listdir(local_dir)
    for f in files:
        '#########################################'
        'Beginning to upload file %s ' % datetime.datetime.now()
        'Uploading file:', os.path.join(local_dir, f)
        # sftp.get(os.path.join(dir_path,f),os.path.join(local_path,f))
        sftp.put(os.path.join(local_dir, f), os.path.join(remote_dir, f))
        'Upload file success %s ' % datetime.datetime.now()
    t.close()

Refer to http://www.cnblogs.com/yangsh...

Fabric

Fabric is a higher encapsulation based on paramiko, which is more convenient to operate.
Official website: http://www.fabfile.org/index....
github: https://github.com/fabric/fab...

Depends on crypto, paramiko. Note: fabric does not currently support Python3. But there is a version on github that supports py3 https://github.com/mathiasert…

Installation on Windows:
1. Install pycrypto.
There are several ways to install:
A. Install MSVC2010 under win7, and then compile and install it through pip install pycrypto.

B. Choose someone else to compile.
pycrypto-for-python-3-2 and below
pycrypto-for-python-3-4

2. Version installation that supports Python3: pip install Fabric3

fabfile.py
from fabric.api import run
def host_type():
    run('uname -s')

Executed by the fab command. -f specifies a file, -H specifies a list of hosts.

$ fab -f fabfile.py -H localhost,linuxbox host_type

fab parameter description:
-f specifies the entry 3 file
-g specifies the gateway device (transit, bastion machine) IP
-H specifies the target host, multiple splits with ","
-P asynchronously runs the multi-host task
-R specifies the role, based on the role Distinguish unit
-t device connection timeout, seconds
-T remote host command execution timeout, seconds
-w When command execution fails, issue a warning instead of terminating the task.
Of course, we can set these option values ​​in the code without specifying them on the command line. As follows: Global attribute setting The role of the
env object is to define the global setting of the fabfile, supporting multiple attributes and custom attributes.

env.hosts, define target hosts, list
env.exclude_hosts, exclude hosts, list
env.user, define username, str
env.port, define port, str
env.password, define password, str
env.passwords, dictionary, but form As follows: env.passwords={ '[email protected]:22':'123456','[email protected]:22':'1234'}
env.key_filename=None Specify the SSH key file, str or list
env .gateway specifies the IP of the gateway device (transit, bastion machine), str
env.roedefs defines the role grouping, dictionary: env.roledefs={ 'web':['192.168.1.21','192.168.1.23'],'db': ['192.168.1.22','192.168.1.24']}
env.parallel=False whether to execute tasks concurrently
env.path=' ' $PATH environment variable defined in run/sudo/local
env.command_timeout=None
env.timeout =10
env.shell=“/bin/bash -l -c”
env.ssh_config_path=“$HOME/.ssh/config”
env.sudo_password=None
env.sudo_passwords={}
env.use_ssh_config=False
env.warn_only=False, if True, when the operation encounters an error, issue a warning and continue execution instead of terminating
env.variable name custom variable

E.g:

@roles('web')
def webtask():
    run('/etc/init.d/nginx start')
@roles('db')
def dbtask():
    run('/etc/init.d/mysql start')
 
@roles('web','db')
def publicstask():
    run('uptime')
def deploy():
    execute(webtask)
    execute(dbtask)
    execute(publictask)

Then the terminal executes the command

$ fab deploy

Command line parameters:

def hello(name="world"):
    print("Hello %s!" % name)
$ fab hello:name=Jeff
Hello Jeff!
Done.

Common API

fabric.api module:

local, execute local commands, such as local('uname -s')
lcd, switch local directories, such as lcd('/home')
cd, switch remote directories
run, execute remote commands
sudo, execute remote commands
put in sudo mode, upload files Go to the remote host put('/home/aaa','/home/xby/aaa')
get, download the file from the remote host to the local get('/opt/bbb','/home/bbb')
prompt, get the user Enter
confirm to get a confirmation message, such as confirm('Continue[Y/N]?')
reboot, restart the remote host, such as reboot()
@task function decorator, the identification function is callable by fab, otherwise it will not be visible to fab
@runs_once, the identification function will only be executed once and will not be affected by multiple hosts.
@roles, indicates the host role when the function is executed
@parallel(pool_size=)
@with_settings()

fabric.contrib.console.confirm(question, default=True) User input Y/n, return True/False

Example 1: View local and remote host information:

from fabric.api import *
 
env.user='root'
env.hosts=['192.168.1.2','192.168.1.3']
env.password='123'
 
@runs_once #即使有多台主机，但它只会执行一次
def local_task():
    local('uname -a')
def remote_task():
    with cd("/data/logs"): #这个with的作用是让后面的表达式语句继承当前的状态，实现"cd /data/logs && ls -l"的效果。
        run("ls -l")
$ fab -f sample.py local_task
$ fab -f sample.py remote_task

Example 2: Dynamically get remote directory

from fabric.api import *
from fabric.contrib.console import confirm
 
env.user='root'
env.hosts=['192.168.1.2','192.168.1.3']
env.password='123'
 
@runs_once
def input_raw():
    return prompt("please input dir name:",default='/home')
def worktask(dirname):
    run("ls -l "+dirname)
@task
def go():
    dirname=input_raw()
    worktask(dirname)

Example 3: Gateway mode file upload and execution

In fact, just define the ip of env.gateway. Compared with paramiko, it is indeed simplified a lot.

from fabric.api import *
from fabric.contrib.console import confirm
from fabric.context_managers import *
 
env.user='root'
env.hosts=['192.168.1.2','192.168.1.3']
env.password='123'
env.gateway='192.168.22.2'
 
lpath='/home/install/lnmp.tar.gz'
rpath='/tmp/install'
 
@task
def put_task():
      run("mkdir -p /tmp/install")
      with settings(warn_only=True): #put出现异常时，发出警告，继续执行，不要终止。
        result=put(lpath,rpath) #上传
       if result.failed and not confirm("put failed,continue[Y/N]?"):
        abort("Aborting")
@task 
def run_task():
      with cd("/tmp/install"):
         run("tar -zxvf lnmp.tar.gz")
       with cd("lnmp"):
        run("./install.sh")
 
@task go():
  put_task()
  run_task()

colorful output

fabric.colors.blue(text, bold=False)
fabric.colors.cyan(text, bold=False)
fabric.colors.green(text, bold=False)
fabric.colors.magenta(text, bold=False)
fabric.colors.red(text, bold=False)
fabric.colors.white(text, bold=False)
fabric.colors.yellow(text, bold=False)

from fabric.colors import red, green
print(red("This sentence is red, except for " + green("these words, which are green") + "."))

Example - Fabric deploys Flask app

Example 1: It can upload the current source code to the server and install it into a pre-existing virtual environment:

 

from fabric.api import *
# 使用远程命令的用户名
env.user = 'appuser'
# 执行命令的服务器
env.hosts = ['server1.example.com', 'server2.example.com']
def pack():
    # 创建一个新的分发源，格式为 tar 压缩包
    local('python setup.py sdist --formats=gztar', capture=False)
def deploy():
    # 定义分发版本的名称和版本号
    dist = local('python setup.py --fullname', capture=True).strip()
    # 把 tar 压缩包格式的源代码上传到服务器的临时文件夹
    put('dist/%s.tar.gz' % dist, '/tmp/yourapplication.tar.gz')
    # 创建一个用于解压缩的文件夹，并进入该文件夹
    run('mkdir /tmp/yourapplication')
    with cd('/tmp/yourapplication'):
        run('tar xzf /tmp/yourapplication.tar.gz')
        # 现在使用 virtual 环境的 Python 解释器来安装包
        run('/var/www/yourapplication/env/bin/python setup.py install')
    # 安装完成，删除文件夹
    run('rm -rf /tmp/yourapplication /tmp/yourapplication.tar.gz')
    # 最后 touch .wsgi 文件，让 mod_wsgi 触发应用重载
    run('touch /var/www/yourapplication.wsgi')