1. Introduction to the Kubernetes series
1. Background introduction
Cloud computing develops rapidly
-
IaaS
-
PaaS
-
SaaS
-
-
Docker technology is advancing by leaps and bounds
-
Build once, run anywhere
-
Containers are fast and lightweight
-
complete ecological environment
-
-
2. What is kubernetes?
First of all, it is a new leading solution for distributed architecture based on container technology. Kubernetes (k8s) is Google's open source container cluster management system (Google internal: Borg). Based on Docker technology, it provides a series of complete functions such as deployment and operation, resource scheduling, service discovery, and dynamic scaling for containerized applications, making large-scale container cluster management more convenient.
Kubernetes is a complete distributed system support platform with complete cluster management capabilities, multi-level security protection and access mechanisms, multi-tenant application support capabilities, transparent service registration and discovery mechanisms, built-in intelligent load balancers, and powerful fault detection and self-healing capabilities, rolling service upgrades and online expansion capabilities, scalable automatic resource scheduling mechanisms, and multi-granularity resource quota management capabilities. At the same time, Kubernetes provides complete management tools, covering all aspects including development, deployment testing, operation and maintenance monitoring.
In Kubernetes, Service is the core of the distributed cluster architecture. A Service object has the following key characteristics:
Have a uniquely designated name
Have a virtual IP (Cluster IP, Service IP, or VIP) and port number
Able to embody certain remote service capabilities
is mapped to a set of container applications that provide this service capability
The service processes of Service currently provide external services based on Socket communication, such as Redis, Memcache, MySQL, Web Server, or a specific TCP Server process that implements a specific business. Although a Service usually consists of multiple related Service processes provide services. Each service process has an independent Endpoint (IP+Port) access point, but Kubernetes allows us to connect to the specified Service through the service. With the transparent load balancing and failure recovery mechanism of Kubernetes, no matter how many service processes there are on the backend, or whether a service process will be redeployed to other machines due to a failure, it will not affect our normal calls to services. More importantly, the Service itself will not change once it is created, which means that in the Kubernetes cluster, we do not have to worry about changes in the IP address of the service.
Containers provide powerful isolation functions. It is necessary to isolate the group of processes that provide services into the container. To this end, Kubernetes designed the Pod object to package each service process into the corresponding Pod, making it a container running in the Pod. In order to establish the association management between Service and Pod, Kubernetes attaches a label to each Pod. For example, a Pod running MySQL is labeled with name=mysql, a Pod running PHP is labeled with name=php, and then the corresponding Service is labeled Define the label selector Label Selector, which can cleverly solve the problem of association between Service and Pod.
In terms of cluster management, Kubernetes divides the machines in the cluster into a Master node and a group of working node Nodes. Among them, the Master node runs a set of processes related to cluster management, kube-apiserver, kube-controller-manager and kube-scheduler. These processes realize the management capabilities of the entire cluster, such as resource management, Pod scheduling, elastic scaling, security control, system monitoring and error correction, and are all completed automatically. Node serves as a working node in the cluster and runs real applications. The smallest running unit managed by Kubernetes on Node is Pod. Node runs Kubernetes' kubelet and kube-proxy service processes. These service processes are responsible for the creation, startup, monitoring, restarting, and destruction of Pods and the implementation of software-mode load balancers.
In the Kubernetes cluster, it solves the two major problems of service expansion and upgrade in traditional IT systems. You only need to create a Replication Controller (RC) for the Pod associated with the Service that needs to be expanded, and problems such as expansion and subsequent upgrades of the Service will be easily solved. Include the following 3 key information in an RC definition file.
Definition of target Pod
The number of replicas that the target Pod needs to run (Replicas)
The target Pod label to be monitored (Label)
After creating the RC, Kubernetes will filter out the corresponding Pod instances through the Label defined in the RC and monitor their status and quantity in real time. If the number of instances is less than the defined number of replicas, it will create one based on the Pod template defined in the RC. The new Pod is then scheduled to the appropriate Node to start running until the number of Pod instances reaches the predetermined target. This process is completely automated.
Kubernetes advantages:
-
Container orchestration
-
lightweight
-
Open source
-
Elastic scaling
-
load balancing
3. Core concepts of Kubernetes
3.1、Master
The management node of the k8s cluster is responsible for managing the cluster and providing access to the resource data of the cluster. Have Etcd storage service (optional), run the Api Server process, Controller Manager service process and Scheduler service process, and associate the working node Node.
-
Kubernetes API server provides the key service process of the HTTP Rest interface and is the only entry point for operations such as adding, deleting, modifying, and querying all resources in Kubernetes. It is also the entry process of cluster control;
-
Kubernetes Controller Manager is the automated control center for all Kubernetes resource objects;
-
Kubernetes Schedule is the process responsible for resource scheduling (Pod scheduling)
3.2、Node
Node is the service node (also called agent or minion) that runs Pod in the Kubernetes cluster architecture. Node is the unit of Kubernetes cluster operation. It is used to host the operation of the assigned Pod and is the host where the Pod runs. Associated with the Master management node, it has name, IP, and system resource information.
-
Run docker eninge service
-
daemon kunelet
-
Load balancer kube-proxy
Each Node node runs the following set of key processes
kubelet: Responsible for tasks such as creating, starting and stopping containers for Pods
kube-proxy: an important component that implements the communication and load balancing mechanism of Kubernetes Service
Docker Engine (Docker): Docker engine, responsible for the creation and management of native containers
Node nodes can be dynamically added to the Kubernetes cluster during operation. By default, kubelet will register itself with the master. This is also the Node management method recommended by Kubernetes. The kubelet process will regularly report its own information to the master, such as operating system, Docker version, CPU and memory, as well as which Pods are running, etc., so that the Master can know the resource usage of each Node node and implement an efficient and balanced resource scheduling strategy.
3.3、Sub
A combination of several related containers running on Node nodes. The containers contained in the Pod run on the same host, use the same network namespace, IP address and port, and can communicate through localhost. Pod is the smallest unit for creation, scheduling and management of Kurbernetes. It provides a higher level of abstraction than containers, making deployment and management more flexible. A Pod can contain one container or multiple related containers.
There are actually two types of Pods, ordinary Pods and static Pods. The latter is special. It does not exist in the etcd storage of Kubernetes, but is stored in a specific file on a specific Node, and only on this Node. start up. Once a normal Pod is created, it will be placed in the etcd storage, and will then be scheduled by the Kubernetes Master to a specific Node for binding. The Pod will then be instantiated into a set of related Dockers by the kubelet process on the corresponding Node. The container is started and started. By default, when a container in a Pod stops, Kubernetes will automatically detect this problem and restart the Pod (restart all containers in the Pod). If the Node where the Pod is located goes down, it will All Pods on this Node are rescheduled to other nodes.
3.4、Replication Controller
Replication Controller is used to manage Pod copies and ensure that there are a specified number of Pod copies in the cluster. If the number of replicas in the cluster is greater than the specified number, excess containers beyond the specified number will be stopped. Otherwise, containers less than the specified number will be started to ensure that the number remains unchanged. Replication Controller is the core for realizingelastic scaling, dynamic expansion and rolling upgrade.
3.5、Service
Service defines a logical collection of Pods and a strategy for accessing the collection, and is an abstraction of a real service. Service provides a unified service access entrance, service proxy and discovery mechanism, and associates multiple Pods with the same Label. Users do not need to know how the background Pod runs.
Problems with accessing Service from external systems
First, we need to understand the three IPs of Kubernetes.
-
Node IP: The IP address of the Node node
-
Pod IP: The IP address of the Pod
-
Cluster IP: IP address of Service
First of all, Node IP is the physical network card IP address of the node in the Kubernetes cluster. All servers belonging to this network can communicate directly through this network. This also shows that when nodes outside the Kubernetes cluster access a node or TCP/IP service within the Kubernetes cluster, they must communicate through Node IP.
Secondly, Pod IP is the IP address of each Pod. It is allocated by Docker Engine based on the IP address segment of the docker0 bridge. It is usually a virtual second-layer network.
Finally, Cluster IP is a virtual IP, but it is more like a fake IP network for the following reasons:
Cluster IP only acts on the Kubernetes Service object, and is managed and assigned IP addresses by Kubernetes
The Cluster IP cannot be pinged, it does not have an "entity network object" to respond to
Cluster IP can only be combined with Service Port to form a specific communication port. Cluster IP alone does not have the basis for communication, and they belong to a closed space such as the Kubernetes cluster.
Within the Kubernetes cluster, the communication between the Node IP network, Pod IP network and Cluster IP network uses a special programming routing rule designed by Kubernetes itself.
3.6、Label
Any API object in Kubernetes is identified by Label. The essence of Label is a series of Key/Value pairs, where the key and Value are specified by the user. Labels can be attached to various resource objects, such as Node, Pod, Service, RC, etc. A resource object can define any number of Labels, and the same Label can be added to any number of resource objects. Label is the basis for the operation of Replication Controller and Service. The two are associated with the Pod running on the Node through Label.
We can implement multi-dimensional resource grouping management functions by bundling one or more different Labels with specified resource objects, so that resource allocation, scheduling, configuration and other management tasks can be performed flexibly and conveniently.
Version tags: "release":"stable","release":"canary"......
Environment tags: "environment":"dev","environment":"qa","environment":"production"
Schema tags: "tier":"frontend","tier":"backend","tier":"middleware"
Partition label:"partition":"customerA","partition":"customerB"
Quality control tags: "track":"daily","track":"weekly"
Label is equivalent to the label we are familiar with. Defining a Label for a resource object is equivalent to giving it a label. Then you can query and filter resource objects with certain Labels through the Label Selector (label selector). Kubernetes uses this This method implements a simple and universal object query mechanism similar to SQL.
The important usage scenarios of Label Selector in Kubernetes are as follows:
The kube-Controller process filters the number of Pod copies to be monitored by defining a Label Selector on the resource object RC, thereby achieving a fully automatic control process in which the number of copies always meets the expected settings.
The kube-proxy process selects the corresponding Pod through the Label Selector of the Service, and automatically establishes a request forwarding routing table for each Service to the corresponding Pod, thereby achieving intelligent load balancing of the Service.
By defining specific Labels for certain Nodes and using the label scheduling strategy Nodeselector in the Pod definition file, the kuber-scheduler process can implement the "directed scheduling" feature of the Pod
4. Kubernetes architecture and components
- Service grouping, small cluster, multi-cluster
- Service grouping, large cluster, single cluster
4.1. Kubernetes components:
Kubernetes Master control component, schedules and manages the entire system (cluster), including the following components:
4.1.1、Kubernetes API Server
As the entrance to the Kubernetes system, it encapsulates the addition, deletion, modification and query operations of core objects, and provides it to external customers and internal components in the form of a RESTful API interface. The maintained REST objects are persisted and stored in Etcd.
4.1.2.Kubernetes Scheduler
Perform node selection (i.e. allocate machines) for the newly created Pod and be responsible for resource scheduling of the cluster. The components are detached and can be easily replaced with other schedulers.
4.1.3.Kubernetes Controller
Responsible for executing various controllers. Many controllers have been provided to ensure the normal operation of Kubernetes.
4.1.4. Replication Controller
Manage and maintain Replication Controller, associate Replication Controller and Pods, and ensure that the number of replicas defined by Replication Controller is consistent with the actual number of running Pods.
4.1.5. Node Controller
Manage and maintain Node, regularly check the health status of Node, and identify (failed | not failed) Node nodes.
4.1.6. Namespace Controller
Manage and maintain Namespace, and regularly clean up invalid Namespace, including API objects under Namespace, such as Pod, Service, etc.
4.1.7. Service Controller
Manage and maintain Service, provide loads and service agents.
4.1.8.EndPoints Controller
Manage and maintain Endpoints, associate Service and Pod, create Endpoints as the backend of Service, and update Endpoints in real time when Pod changes.
4.1.9. Service Account Controller
Manage and maintain Service Account, create a default Service Account for each Namespace, and create a Service Account Secret for the Service Account.
4.1.10. Persistent Volume Controller
Manage and maintain the Persistent Volume and Persistent Volume Claim, allocate a Persistent Volume for binding to the new Persistent Volume Claim, and perform cleanup and recycling for the released Persistent Volume.
4.1.11. Daemon Set Controller
Manage and maintain Daemon Set, responsible for creating Daemon Pod and ensuring normal running of Daemon Pod on the designated Node.
4.1.12. Deployment Controller
Manage and maintain Deployment, associate Deployment and Replication Controller, and ensure that the specified number of Pods are running. When the Deployment is updated, control the updates of the Replication Controller and Pod.
4.1.13.Job Controller
Manage and maintain jobs, create one-time task pods for Jod, and ensure that the number of tasks specified by the job is completed.
4.1.14. Pod Autoscaler Controller
Realize automatic scaling of Pods, obtain monitoring data regularly, perform policy matching, and execute Pod scaling actions when conditions are met.
4.2、Kubernetes Nodes
Kubernetes Nodes run nodes and run and manage business containers, including the following components
4.2.1、Kubelet
Responsible for managing and controlling containers, Kubelet will receive Pod creation requests from Kubernetes API Server, start and stop containers, monitor the running status of containers, and report to Kubernetes API Server.
4.2.2、Kubernetes Proxy
Responsible for creating proxy services for Pods, Kubernetes Proxy will obtain all Service information from the Kubernetes API Server, and create proxy services based on the Service information to implement request routing and forwarding from the Service to the Pod, thus realizing a Kubernetes-level virtual forwarding network.
4.3.3、Docker
Container service needs to be run on Node
2. Practical practice of building a Docker cluster environment based on Kubernetes
Kubernetes is a distributed cluster built by Google based on docker and consists of the following main components
1、etcd：
High-availability storage shared configuration and service discovery, used in conjunction with flannel on the node machine, are used to enable docker running on each node to have different IP segments. The ultimate goal is to enable docker containers running on different nodes to have different ip segments. An IP address that is different from any other container (docker container running on other nodes).
2、flannel：
Network structure support
3、to apiserver：
Whether controlled directly through kubectl or using remote api, it must go through apiServer
4、kube-controller-manager:
Loop control over replication controller, endpoints controller, namespace controller, and serviceaccounts controller, interact with kube-apiserver to ensure that these controllers work
5、be a scheduler
The function of the Kubernetes scheduler is to schedule pods to specified working nodes (nodes) according to a specific scheduling algorithm. This process is also called binding.
6、kubelet
Kubelet runs on Kubernetes Node. It is the logical successor of container agent
7、be a proxy
kube-proxy is a component in kubernetes that runs on node nodes. It plays the role of a service proxy.
3. Prepare the environment
1. Architecture topology diagram
2. Environment planning for two CentOS7 system machines:
192.168.10.130: used to install Kubernetes master
192.168.10.131: used as kubernetes node (node1)
角色 IP 组件 master 192.168.10.130 etcd、kube-apiserver、kube-controller-manager、kube-scheduler node01 192.168.10.131 kubelet、kube-proxy、docker node02 192.168.10.132 kubelet、kube-proxy、docker
3. Environmental description
Operating system: CentOS7
Kubernetes version: v1.8.3
Docker version: v17.09-ce
All use the latest stable version.
Turn off selinux.
Note: The version I use is as follows:
Check the system kernel: uname -r displays the result 3.10.0-229.el7.x86_64
Check the system version: cat /etc/centos-release
Show results CentOS Linux release 7.1.1503 (Core)
Check the system bit number: getconf LONG_BIT
Operating system: CentOS Linux release 7.1.1503 (Core)
Kubernetes version: v1.10.
Docker version: v17.05-ce
Etcd version: v3.34
Turn off selinux.
Deployment part
4. Deploy the cluster
1. Download the binary package
Kubernetes software package and ETCD
You can download it from the official download address. The address is: Click to open the link. The latest download version is V1.10, which will be blocked. You can refer to Click to open the link. In any case, I still put the client binary, server binary, and node binary of V1.0. Downloaded. ETCD can be downloaded by clicking on the link.
Software package purpose
kubernetes-server-linux-amd64.tar.gz
The version is V1.10, including KUBE-APISERVER, KUBE-CONTROLLER-MANAGER, KUBE-SCHEDULE
Download address: kubernetes-server-linux-amd64.tar.gz_Free high-speed download | Baidu Netdisk-Unlimited sharing
kubernetes-node-linux-amd64.tar.gz
The version is V1.10, which includes KUBELET, KUBE-PROXY, and KUBECTL. Docker needs to be installed first.
Download address:kubernetes-node-linux-amd64.tar.gz_Free high-speed download | Baidu Netdisk-Unlimited sharing
etcd-v3.3.4-linux-amd64.tar.gz
The version is V3.34, Kubernetes Master requires ETCD data storage
Download address:etcd-v3.3.4-linux-amd64.tar.gz_Free high-speed download | Baidu Netdisk-Unlimited sharing
Docker
Version V1.17.05
2. After the download is completed, upload it to the server
Upload kubernetes-server-linux-amd64.tar.gz to the master node.
kubernetes-node-linux-amd64.tar.gz is uploaded to the node.
Upload etcd-v3.3.4-linux-amd64.tar.gz to the node node
5. Turn off the firewall and selinux running on the system and set the time zone host name
1. If the system has a firewall enabled, follow the steps below to turn off the firewall (all machines)
# systemctl stop firewalld # systemctl disable firewalld
2. Close selinux
vim /etc/sysconfig/selinux Find and modify the following values SELINUX=disabled After modification, run the following command to make the modification take effect setenforce 0
3. Set the host name master1 node1... time zone respectively
Set the time zone, both master and node machines need to be executed, and the time is unified
timedatectl set-timezone Asia/Shanghai
Set master host name
hostnamectl set-hostname master
Set node hostname
hostnamectl set-hostname node #node执行
6. Installation and configuration of master (i.e. kubernetes-server) host machine
1. Install etcd
ETCD is a distributed and consistent KV storage system for shared configuration and service discovery. It mainly includes addition, deletion, modification, security authentication, clustering, election, transaction, distributed lock, watch mechanism, etc., and implements the RAFT protocol. Very powerful
1.1. Unzip the file
etcd-v3.3.4-linux-amd64.tar.gz
1.2 Copy the file to the specified directory or the current directory, but it needs to be in the bin directory
Copy etcd and etcdctl in the decompression directory to the /usr/local/bin directory, or create a bin directory yourself, and then place etcd and etcdctl in the created bin directory.
cd /usr/local/bin/
1.3 Configure etcd configuration file
Make sure that the listed items are configured correctly and are not commented out. This is true for the following configurations.
[telecom@master ~]$ vim /etc/etcd/etcd.conf ETCD_NAME="default" ETCD_DATA_DIR="/var/lib/kubelet" ETCD_LISTEN_PEER_URLS="http://192.168.10.130:2380" ETCD_LISTEN_CLIENT_URLS="http://127.0.0.1:2379,http://127.0.0.1:2379,http://192.168.10.130:2379,http://192.168.10.130:4001" ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.10.130:2380" # if you use different ETCD_NAME (e.g. test), set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..." ETCD_INITIAL_CLUSTER="default=http://192.168.10.130:2380" ETCD_INITIAL_CLUSTER_STATE="new" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379,http://0.0.0.0:4001" ETCD_HEARTBEAT_INTERVAL=6000 ETCD_ELECTION_TIMEOUT=30000
1.4. Configure systemctl
Create etcd.service in the /etc/systemd/system/ directory. If there is no system directory, just create it. First create the ETCD storage directory address /home/chen/etcd/data, and then create the ETCD configuration file directory. /etc/etcd/, because we use the default configuration of ETCD, so the /etc/etcd/etcd.conf file can be empty.
[telecom@master ~]$ vim /etc/systemd/system/etcd.service [Unit] Description=etcd server After=network.target After=network-online.target Wants=network-online.target [Service] #Type=simple Type=notify WorkingDirectory=/var/lib/kubelet EnvironmentFile=-/etc/etcd/etcd.conf ExecStart=/usr/local/bin/etcd [Install] WantedBy=multi-user.target
1.5. After configuration. Execute the command
systemctl daemon-reload systemctl enable etcd.service systemctl start etcd.service systemctl status etcd.service systemctl stop etcd.service
Problems encountered
spawning /usr/local/bin/etcd: Not a directory New mkdir -p /var/lib/kubelet Change /etc/systemd/system/etcd.service Modify path in WorkingDirectory [Service] WorkingDirectory=/var/lib/kubelet And in /etc/etcd/etcd.conf in ETCD_DATA_DIR="/var/lib/kubelet" Then execute the above command as needed and it will be successful
1.6. Check etcd status
1.7. Verify whether writing data to etcd is successful and execute
export ETCDCTL_API=3 etcdctl set 键 “值” etcdctl get 键 例：[telecom@master ~]$ etcdctl set li "123" 123 [telecom@master ~]$ etcdctl get li
1.8. Set environment variables
vi /etc/profile中 加入 export ETCDCTL_API=3 执行source /etc/profile
If an error is reported, use journalctl -f -t etcd and journalctl -u etcd to locate the problem.
