Ansible automated cluster deployment K8S

Ansible automated cluster deployment K8S

Introduction 1.1 Ansible

Ansible is an IT automation tools. It can be configured systems, software, and coordinate the deployment of more advanced IT tasks, such as continuous deployment, rollover. Ansible applicable to the management of enterprise IT infrastructure, business environment thousands of examples from small-scale to have a small number of hosts. Ansible is a simple automation language, perfectly describe IT application infrastructure.

Have the following three features:

• Simple: reduce the cost of learning
  
• Powerful: coordination of application lifecycle
• Agentless: predictable, reliable and secure

Use document: https://docs.ansible.com/

安装Ansible：yum install ansible -y

• Inventory: Ansible host information management, including the IP address, SSH port, account numbers, passwords, etc.
• Modules: module task has completed, you can also customize the module, for example, a script often used.
• Plugins: the use of plug-in adds Ansible core functionality itself provides many plug-ins, you can also customize the plug-ins. E.g. plug connection for connection to the target host.
• Playbooks: "script", a modular series of tasks defined for the unified external calls. Ansible core functionality.

1.2 Host list

[webservers]
alpha.example.org
beta.example.org
192.168.1.100
www[001:006].example.com

[dbservers]
db01.intranet.mydomain.net
db02.intranet.mydomain.net
10.25.1.56
10.25.1.57
db-[99:101]-node.example.com

1.3 using the command line

ad-hoc command input, quickly perform an action, but do not want to retain records.

ad-hoc command is to understand the basics and Ansible Before learning playbooks need to know.

In general, Ansible real ability lies in the script.

1, connect to a remote host authentication

SSH password authentication:

[webservers]
192.168.1.100:22 ansible_ssh_user=root ansible_ssh_pass=’123456’
192.168.1.101:22 ansible_ssh_user=root ansible_ssh_pass=’123456’

SSH key authentication:

[webservers]
10.206.240.111:22 ansible_ssh_user=root ansible_ssh_key=/root/.ssh/id_rsa 
10.206.240.112:22 ansible_ssh_user=root

也可以ansible.cfg在配置文件中指定：
[defaults]
private_key_file = /root/.ssh/id_rsa  # 默认路径

2, commonly used options

Options	description
-C, --check	Run a check, do nothing
-e EXTRA_VARS, - the extra-= EXTRA_VARS	Key = value provided additional variables
-u REMOTE_USER, --user=REMOTE_USER	SSH connection users, the default None
-k, --ask-pass	SSH connection user password
-b, --become	Mention the right, the default root
-K, --ask-become-pass	Mention the right password

3, use the command line

ansible all -m ping ansible all -m ping ansible all -m shell -a "ls /root" -u root -k ansible webservers -m copy –a "src=/etc/hosts dest=/tmp/hosts"

1.4 Common Module

ansible-doc -l to view all modules

ansible-doc -s copy View module documentation

Module Documentation: https://docs.ansible.com/ansible/latest/modules/modules_by_category.html

1、shell

Execute shell commands on the target host.

- name: 将命令结果输出到指定文件
  shell: somescript.sh >> somelog.txt
- name: 切换目录执行命令
  shell:
    cmd: ls -l | grep log
    chdir: somedir/
- name: 编写脚本
  shell: |
      if [ 0 -eq 0 ]; then
         echo yes > /tmp/result
      else
         echo no > /tmp/result
      fi
  args:
    executable: /bin/bash

2、copy

Copy the file to the remote host.

- name: 拷贝文件
  copy:
    src: /srv/myfiles/foo.conf
    dest: /etc/foo.conf
    owner: foo
    group: foo
    mode: u=rw,g=r,o=r
    # mode: u+rw,g-wx,o-rwx
    # mode: '0644'
    backup: yes

3、file

Manage files and file attributes.

- name: 创建目录
  file:
    path: /etc/some_directory
    state: directory
    mode: '0755'
- name: 删除文件
  file:
    path: /etc/foo.txt
    state: absent
- name: 递归删除目录
  file:
    path: /etc/foo
    state: absent

present, latest: express installation

absent: representation Uninstall

4、yum

Package management.

- name: 安装最新版apache
  yum:
    name: httpd
    state: latest
- name: 安装列表中所有包
  yum:
    name:
      - nginx
      - postgresql
      - postgresql-server
    state: present
- name: 卸载apache包
  yum:
    name: httpd
    state: absent 
- name: 更新所有包
  yum:
    name: '*'
    state: latest
- name: 安装nginx来自远程repo
  yum:
    name: http://nginx.org/packages/rhel/7/x86_64/RPMS/nginx-1.14.0-1.el7_4.ngx.x86_64.rpm
    # name: /usr/local/src/nginx-release-centos-6-0.el6.ngx.noarch.rpm
    state: present

5、service/systemd

Management services.

- name: 服务管理
  service:
    name: etcd
    state: started
    #state: stopped
    #state: restarted
    #state: reloaded
- name: 设置开机启动
  service:
    name: httpd
    enabled: yes

- name: 服务管理  
  systemd: 
    name=etcd 
    state=restarted 
    enabled=yes 
    daemon_reload=yes

6、unarchive

- name: 解压
  unarchive: 
    src=test.tar.gz 
    dest=/tmp

7、debug

During execution print statement.

- debug:
    msg: System {{ inventory_hostname }} has uuid {{ ansible_product_uuid }}

- name: 显示主机已知的所有变量
  debug:
    var: hostvars[inventory_hostname]
    verbosity: 4

1.5 Playbook

Playbooks is Ansible configuration, deployment and orchestration language. They can describe what you want to do something or IT processes are described in a series of steps on the remote machine. Use easy to read format YAML file organization Playbook.

If Ansible module is your work tool, then the Playbook is your manual, assets and your hosts file is your raw material.

Compared with the adhoc task execution mode, Playbooks use ansible is a completely different way, and is especially powerful.

https://docs.ansible.com/ansible/latest/user_guide/playbooks.html

---
- hosts: webservers
  vars:
    http_port: 80
    server_name: www.ctnrs.com
  remote_user: root
  gather_facts: false
  tasks:
  - name: 安装nginx最新版
    yum: pkg=nginx state=latest
  - name: 写入nginx配置文件
    template: src=/srv/httpd.j2 dest=/etc/nginx/nginx.conf
    notify:
    - restart nginx
  - name: 确保nginx正在运行
    service: name=httpd state=started
  handlers:
    - name: restart nginx
      service: name=nginx state=reloaded

1, host and user

- hosts: webservers
  remote_user: lizhenliang
  become: yes
  become_user: root

ansible-playbook nginx.yaml -u lizhenliang -k -b -K

2, the definition of variables

Variable is a convenient way to apply to multiple hosts; the actual execution before the host, each host variables will be added, and then referenced in the execution.

• Pass the command line

  -e VAR=VALUE

• Host variables and group variables

Variables defined in the Inventory.

[webservers]
192.168.1.100 ansible_ssh_user=root hostname=web1
192.168.1.100 ansible_ssh_user=root hostname=web2

[webservers:vars]
ansible_ssh_user=root hostname=web1

• Single file storage

The preferred approach is not to Ansible variables are stored in the Inventory.

In addition to the variables stored in the Inventory file directly, and a host of variables may also be stored in a single file with respect to the Inventory file.

Set of variables:

group_vars is stored in the variable group

group_vars / all.yml active all the host devices, is equivalent to [all: vars]

grous_vars / etcd.yml group represented ETCD effective host, equivalent to [etcd: vars]

# vi /etc/ansible/group_vars/all.yml
work_dir: /data
# vi /etc/ansible/host_vars/webservers.yml
nginx_port: 80

• Defined in the Playbook

- hosts: webservers
  vars:
    http_port: 80
    server_name: www.ctnrs.com

• Register variables

- shell: /usr/bin/uptime
  register: result
- debug:
    var: result

3, the task list

Each play contains a series of tasks. These tasks are performed according to the order in the play, all the hosts can perform the same task instructions. The aim is to play the selected host mapping to the task.

  tasks:
  - name: 安装nginx最新版
    yum: pkg=nginx state=latest

4, syntax checking and debugging

Grammar checker: ansible-playbook --check /path/to/playbook.yaml

Test run, not practical: ansible-playbook -C /path/to/playbook.yaml

debug module during the execution of print statements for debugging variables or expressions, without having to stop play. And 'when:' debug command better together.

  - debug: msg={{group_names}}
  - name: 主机名
    debug:
      msg: "{{inventory_hostname}}"

5, Mission Control

If you have a great script, it is possible to run a particular section may be useful in without running the entire script.

  tasks:
  - name: 安装nginx最新版
    yum: pkg=nginx state=latest
    tags: install
  - name: 写入nginx配置文件
    template: src=/srv/httpd.j2 dest=/etc/nginx/nginx.conf
    tags: config

use:

ansible-playbook example.yml --tags "install"
ansible-playbook example.yml --tags "install,config"
ansible-playbook example.yml --skip-tags "install"

6, process control

condition:

tasks:
- name: 只在192.168.1.100运行任务
  debug: msg="{{ansible_default_ipv4.address}}"
  when: ansible_default_ipv4.address == '192.168.1.100'

cycle:

tasks:
- name： 批量创建用户
  user: name={{ item }} state=present groups=wheel
  with_items:
     - testuser1
     - testuser2

- name: 解压
  copy: src={{ item }} dest=/tmp
  with_fileglob:
    - "*.txt"

Common loop:

Statement	description
with_items	Standard cycle
with_fileglob	Traverse directory file
with_dict	Traversal Dictionary

7, Templates

 vars:
    domain: "www.ctnrs.com"
 tasks:
  - name: 写入nginx配置文件
    template: src=/srv/server.j2 dest=/etc/nginx/conf.d/server.conf

# server.j2
{% set domain_name = domain %}
server {
   listen 80;
   server_name {{ domain_name }};
   location / {
        root /usr/share/html;
   }
}

Ansible used directly in variable jinja {} {} reference. Use ansible variable assignment jinja variable references do {}.

Define the variable :

{% set local_ip = inventory_hostname %}

Conditionals and loops :

{% set list=['one', 'two', 'three'] %}
{% for i in list %}
    {% if i == 'two' %}
        -> two
    {% elif loop.index == 3 %}
        -> 3
    {% else %}
        {{i}}
    {% endif %}
{% endfor %}

For example: generating a string connected etcd

{% for host in groups['etcd'] %}
    https://{{ hostvars[host].inventory_hostname }}:2379
    {% if not loop.last %},{% endif %}
{% endfor %} 

Which can also be used in variable ansible.

1.6 Roles

Roles method is based on certain variable file, and the task handler known file structure of an automatic loading. By role to group content, suitable for building complex deployment environment.

1. Define Roles

Roles directory structure:

site.yml
webservers.yml
fooservers.yml
roles/
   common/
     tasks/
     handlers/
     files/
     templates/
     vars/
     defaults/
     meta/
   webservers/
     tasks/
     defaults/
     meta/

• tasks - The main task list contains the role to be performed.
• handlers - Contains handler, this role even anywhere outside of this role can use these handlers.
• defaults- the role of the default variable
• vars- the role of other variables
• files - Contains files can be deployed in this role.
• templates - contains templates can be deployed by this role.
• meta- the definition of some of the metadata for this role. Please see below for more details.

The usual practice is from tasks/main.ymlincludes platform-specific tasks file:

# roles/webservers/tasks/main.yml
- name: added in 2.4, previously you used 'include'
  import_tasks: redhat.yml
  when: ansible_facts['os_family']|lower == 'redhat'
- import_tasks: debian.yml
  when: ansible_facts['os_family']|lower == 'debian'

# roles/webservers/tasks/redhat.yml
- yum:
    name: "httpd"
    state: present

# roles/webservers/tasks/debian.yml
- apt:
    name: "apache2"
    state: present

2, using roles

# site.yml
- hosts: webservers
  roles:
    - common
    - webservers


定义多个：
- name: 0
  gather_facts: false
  hosts: all 
  roles:
    - common

- name: 1
  gather_facts: false
  hosts: all 
  roles:
    - webservers

3, the role of control

- name: 0.系统初始化
  gather_facts: false
  hosts: all 
  roles:
    - common
  tags: common 

1.7 Automated Deployment K8S (offline)

1, familiar with binary deployment K8S step

1. Server Planning

1. system initialization
   1. Close selinux, firewalld
   2. Close swap
   3. Time Synchronization
   4. Write hosts
2. Etcd cluster deployment
   1. Certificate generation etcd
   2. Three cluster deployment etcd
   3. View cluster status
3. Master deployment
   1. Certificate generation apiserver
   2. Deployment apiserver, controller-manager and scheduler components
   3. Start TLS Bootstrapping
4. Department Node
   1. Installation Docker
   2. Deployment kubelet and kube-proxy
   3. On the Master allowed to issue certificates to the new Node
   4. Apiserver authorized access kubelet
5. Deployment of plug-in (ready Mirror)
   1. Flannel
   2. Web UI
   3. CoreDNS
   4. Ingress Controller
6. Master availability
   1. Increased Master node (Master1 consistent with)
   2. Deploy Nginx load balancer
   3. Nginx + Keepalived availability
   4. Modify Node connection VIP

2, Roles Organization K8S each component deployment resolve

Prepare recommendations:

1. Roles carding processes and structure
2. If the configuration file does not have a fixed content, rendering the use jinja
3. Manual intervention should change the content of a document written in unity

3, download the required files

All nodes to ensure consistent system time

Download Ansible deployment file:

git clone https://github.com/lizhenliang/ansible-install-k8s
cd ansible-install-k8s

Download and unzip the package:

Cloud disk address: https://pan.baidu.com/s/1lTXolmlcCJbei9HY2BJRPQ

tar zxf binary_pkg.tar.gz

4, modify the file Ansible

Modify the hosts file, according to the plan to modify the corresponding IP and name.

vi hosts

Modify group_vars / all.yml file, modify the package directory and certificate trusted IP.

vim group_vars/all.yml
software_dir: '/root/binary_pkg'
...
cert_hosts:
  k8s:
  etcd:

5, a key deployment

Chart

Single Master Architecture

Multi-Master architecture

Deployment Commands
Single Master version:

ansible-playbook -i hosts single-master-deploy.yml -uroot -k

Multi Master version:

ansible-playbook -i hosts multi-master-deploy.yml -uroot -k

6, deployment control

If the installation fails at some stage, be targeted testing.

For example: the deployment of plug-ins to run only

ansible-playbook -i hosts single-master-deploy.yml -uroot -k --tags addons