Introduction to k8s

1. Features of k8s

• Lightweight: consumes less resources
• Open source
• Elastic scaling
• Load balancing: IPVS

Two, architecture design

The number of high-availability cluster replicas is preferably an odd number >= 3

2.1 Borg architecture (predecessor of k8s)

2.2 k8s architecture

• etcd: It is officially positioned as a reliable distributed key-value storage service , which can store some key data for the entire distributed cluster and assist the normal operation of the distributed cluster. etcd is a key-value pair database that stores all important information (persistence) of the k8s cluster . It is recommended to use Etcd v3 version in k8s cluster, v2 version has been deprecated in k8s v1.11
• api server: a unified entrance for all service access
• ControllerManager: Maintain the expected number of copies
• Scheduler: Responsible for introducing tasks, selecting appropriate nodes to assign tasks
• kubelet: Directly interact with the container engine to realize the life cycle management of the container
• kube-proxy: Responsible for writing rules to IPTABLES, IPVS to achieve service mapping access
• coredns: can create a domain name and IP correspondence resolution for the SVC in the cluster
• dashboard: Provide a B/S structure access system for k8s cluster
• ingress controller: The official can only implement four-layer proxy, and ingress can implement seven-layer proxy
• federaction: Provides a unified management function that can cross cluster centers with multiple k8s
• prometheus: Provides the monitoring capability of k8s cluster
• ELK: Provide a unified analysis and intervention platform for k8s cluster logs

2.3 Service classification

• Stateful service: DBMS
• Stateless Service: LVS APACHE

2.4 Controller

2.4.1 ReplicationController&ReplicaSet&Deployment

The ReplicationController is used to ensure that the number of replicas of the container application always remains at the user-defined number of replicas. That is, if a container exits abnormally, a new Pod will be automatically created to replace it; and if the container exits abnormally, it will also be automatically recycled. In the new version of Kubernetes, it is recommended to use ReplicaSet to replace ReplicationController.

There is no essential difference between ReplicaSet and ReplicationController, except that the name is different, and ReplicaSet supports collective selectors.

Although ReplicaSet can be used independently, it is generally recommended to use Deployment to automatically manage ReplicaSet. In this way, there is no need to worry about incompatibility with other mechanisms (for example, ReplicaSet does not support rolling-update, but Deployment supports)

2.4.2 HPA（Horizontal Pod Autoscaling）

Horizontal Pod Autoscaling is only applicable to Deployment and ReplicaSet. In the v1 version, it only supports expansion and contraction based on the CPU utilization of the Pod. In the vlalpha version, it supports expansion and contraction based on memory and user-defined metrics.

2.4.3 StatefulSet

StatefulSet is to solve the problem of stateful services (corresponding to Deployment and ReplicaSet are designed for stateless services), and its application scenarios include:

• Stable persistent storage, that is, the same persistent data can be accessed after the Pod is rescheduled. Based on PVC
• A stable network symbol, that is, its podname and hostname remain unchanged after the Pod is rescheduled. Implementation based on Headless Service (that is, Service without Cluster IP)
• Orderly deployment, orderly expansion, that is, Pods are in order. When deploying or expanding, they must be carried out according to the defined order (that is, from 0 to N-1. Before the next Pod runs, all Pods must be Running and Ready status), based on init containers
• Orderly shrink, orderly delete (ie from N-1 to 0)

2.4.4 DaemonSet

DaemonSet ensures that all (or some) Nodes run a copy of the Pod. When a Node joins the cluster, it will also add a Pod for them. When a Node is removed from the cluster, these Pods will also be recycled. Deleting DaemonSet will delete all Pods it created

Some typical usage of DaemonSet:

• Run cluster storage daemon, for example, run glustered and ceph on each Node.
• Run the log collection daemon on each Node, such as fluentd, logstash
• Run monitoring daemon on each Node, such as Prometheus NodeExporter

3. K8S network communication mode

The Kubernetes network model assumes that all Pods are in a flat network space that can be directly connected. This is a ready-made network model in GCE (Google Compute Engine), and Kubernetes assumes that this network already exists. When building a Kubernetes cluster in a private cloud, you cannot assume that this network already exists. We need to implement this network assumption by ourselves, first open up the mutual access between Docker containers on different nodes, and then run Kubernetes

Communication mode:

• Between multiple containers in the same Pod: lo
• Communication between Pods: Overlay Network
• Communication between Pod and Service: Iptables rules of each node

Network solution: Kubernetes + Flannel

Flannel is a network planning service designed by the CoreOS team for Kubernetes. Simply put, its function is to allow Docker containers created by different node hosts in the cluster to have a unique virtual IP address in the entire cluster. And it can also establish an overlay network between these IP addresses, through this overlay network, the data packet is passed to the target container intact

The relationship between ETCD and Flannel:

• Storage management Flannel can allocate IP address segment resources
• Monitor the actual address of each Pod in ETCD, and establish and maintain a Pod node routing table in memory

Network communication method in different situations

• Internal communication of the same Pod: The same Pod shares the same network namespace and the same Linux protocol stack
• Pod1 accesses Pod2:
  (1) Pod1 and Pod2 are not on the same host, and the address of Pod is on the same network segment as docker0, but the network segment of docker0 and the host network card are two completely different IP network segments, and they are different from Node. The communication between them can only be carried out through the physical network card of the host computer. Associate the IP of the Pod with the IP of the Node where the Pod is located. Through this association, the Pod can access each other
  (2) Pod1 and Pod2 are on the same host, and the docker0 bridge directly forwards the request to Pod2 without going through Flannel
• Pod access to the Service network: currently based on performance considerations, all are maintained and forwarded by iptables
• Pod access to the external network: Pod sends a request to the external network, looks up the routing table, and forwards the data packet to the host's network card. After the host's network card completes routing, iptables executes Masquerade, changes the source IP to the host's network card IP, and then External server sends a request
• Access Pod from the Internet: Service