Kubernetes resource object

In k8s All objects are called resources, such as: pod, service, etc.

Pod resources

pod is the smallest unit in k8s in front mentioned, k8s support advanced features self-healing, flexible expansion and so on, so if you simply run business docker in k8s node is no way to support these advanced features, there must be customized container , then, is this pod officials have customized a good container supports advanced features, when starting a pod, there will be at least two containers pod容器``和业务容器, container traffic will share more than a pod container (a collection container), then Pod in a shared network namespace container,

Pod container classification

• Infrastructure Container: base container, maintenance of the entire network space Pod
• InitContainers: initialization container, before the container business started
• Containers: container business, initiated in parallel

The meaning of existence Pod: to close applications exist

• File interaction between the two applications
• Two applications require communication through 127.0.0.1 or socker
• Two applications require frequent occurrence of call

Mirror pulling strategy

imagePullPolicy
1、ifNotPresent：默认值，镜像在宿主机上不存在时才拉取
2、Always：每次创建Pod都会重新拉取一次镜像
3、Never：Pod永远不会主动拉取这个镜像 

1.pod basic operations

// 指定yaml文件创建pod
kubectl create -f [yaml文件路径]

// 查看pod基本信息
kubectl get pods

// 查看pod详细信息
kubectl describe pod [pod名]

// 更新pod（修改了yaml内容）
kubectl apply -f [yaml文件路径]

// 删除指定pod
kubectl delete pod [pod名]

// 强制删除指定pod
kubectl delete pod [pod名] --foce --grace-period=0

2.pod yaml profile

apiVersion: v1
kind: Pod
metadata:
  name: nginx01
  labels:
    app: web
spec:
  containers:
    - name: nginx01
      image: reg.betensh.com/docker_case/nginx:1.13
      ports:
        - containerPort: 80

Pod of relations with controllers

• controllers: object management and operation of container on a cluster
• By label-selectorassociated
• Pod applied by the controller to achieve operation and maintenance, such as stretching, rolling upgrade.

RC copy controller

Replication Controller copy controller, application hosting after Kubernetes, Kubernetes need to ensure that applications can continue to run, this is the work content RC, it will ensure that any time Kubernetes in both a specified number of Pod is running. On this basis, RC also provides some advanced features, such as a rolling upgrade, upgrade rollback and so on.

Recommended ReplicaSet (referred to as RS) in the new version of Kubernetes in place of ReplicationController

1. Create a rc

apiVersion: v1
kind: ReplicationController
metadata:
  name: myweb
spec:
  replicas: 2
  selector:
    app: myweb
  template:
    metadata:
      labels:
        app: myweb
    spec:
      containers:
      - name: nginx01
        image: reg.betensh.com/docker_case/nginx:1.13
        ports:
          - containerPort: 80

By default, pod名it will be rc名+随机值composed as follows:

[root@k8s-master01 rc]# kubectl get pods
NAME          READY     STATUS    RESTARTS   AGE
myweb-0lp57   1/1       Running   0          45s
myweb-zgfcf   1/1       Running   0          45s

RC to control the pod by tag selector (labels), RC name must be the same name and the tag selector

[root@k8s-master01 rc]# kubectl get rc -o wide
NAME      DESIRED   CURRENT   READY     AGE       CONTAINER(S)   IMAGE(S)                                 SELECTOR
myweb     2         2         2         12m       nginx01        reg.betensh.com/docker_case/nginx:1.13   app=myweb

2.RC rolling upgrade
We have already created a v1 http-server version of the pod in k8s environment, how to do if we want to do a version upgrade it? Is the original pod stopped, and then use the new image to pull up a new pod it, this would be clearly inappropriate.

kubectl rolling-update myweb -f nginx_rc_v2.yaml  --update-period=30s

3.RC rolling fallback
Suppose now myweb upgrade to myweb2, appeared bug, then forced to interrupt the upgrade

kubectl  rolling-update myweb -f nginx_rc_v2.yaml --update-period=50s

Then roll back to version myweb2 myweb

kubectl  rolling-update  myweb myweb2 --rollback 

deployment of resources

deployment is also a way to keep pod highly available, there is obviously why RC also introduced deployment of it?
Because the deployment solves a pain point of the RC, when the RC version upgrade container, the label will change, so svc tag or the original, so you need to manually modify configuration files svc.

Deployment of Podand ReplicaSetabove, to provide a statement defined in formula (Declarative) method, used to replace previous ReplicationControllerto facilitate the management application.
You need only Deploymentdescription you want 目标状态what it is, Deployment controllerit will help you Podand ReplicaSetthe actual state of change to your 目标状态. You can define a new Deploymentto create ReplicaSetor delete existing Deploymentand create a new one to replace. That Deploymentis 管理多个ReplicaSet, as shown below:

Although ReplicaSet can be used independently, but it is recommended to use Deployment to automatically manage ReplicaSet, so no need to worry about with other mechanisms incompatibilities

1. Create a deployment

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.13
        ports:
        - containerPort: 80

// 启动
[root@k8s-master01 deploy]# kubectl create -f nginx_deploy.yaml

View deployment startup state

deployment will first start a rs, then start pod

[root@k8s-master01 deploy]# kubectl get all
NAME                                   READY   STATUS    RESTARTS   AGE
pod/nginx-deployment-fcfcc984f-t2bk4   1/1     Running   0          33s
pod/nginx-deployment-fcfcc984f-vg7qt   1/1     Running   0          33s
pod/nginx-deployment-fcfcc984f-zhwxg   1/1     Running   0          33s

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   16h

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-deployment   3/3     3            3           33s

NAME                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/nginx-deployment-fcfcc984f   3         3         3       33s

2. associated service

kubectl expose deployment nginx-deployment --port=80 --type=NodePort

View svc

[root@k8s-master01 deploy]# kubectl  get svc
NAME               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
kubernetes         ClusterIP   10.96.0.1       <none>        443/TCP        16h
nginx-deployment   NodePort    10.96.171.141   <none>        80:31873/TCP   25s

Svc address and port access

[root@k8s-master01 deploy]# curl -I 10.0.0.33:31873
HTTP/1.1 200 OK
Server: nginx/1.13.12
Date: Thu, 14 Nov 2019 05:44:51 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Mon, 09 Apr 2018 16:01:09 GMT
Connection: keep-alive
ETag: "5acb8e45-264"
Accept-Ranges: bytes

3.deployment upgrade

// 直接编辑对应deployment，并修改镜像版本
kubectl edit deployment nginx-deployment

// 通过 set image 发布新的镜像
kubectl set image deploy nginx-deployment nginx-deployment=nginx:1.17

4.deployment rollback

// 回滚到上一级版本
kubectl  rollout undo deployment nginx-deployment

// 回滚到指定版本
kubectl rollout undo deployment nginx-deployment --to-revision=1

// 查看当前deploy历史版本
kubectl rollout history deployment nginx-deployment

The command line version to achieve release

# kubectl  run nginx --image=nginx:1.13 --replicas=3 --record 
# kubectl  rollout history deployment nginx
deployment.extensions/nginx 
# kubectl set image deployment nginx nginx=nginx:1.15                                           

Headless Service

In K8S, the way we want to access the service through a name that is Deploymentto add a layer on top of Service, so that we can Service nameaccess the service, and that which is the principle and CoreDNSrelevant, it will be Service nameparsed into Cluster IP, so we visited Cluster IPwhen on by Cluster IP for load balancing the traffic across 各个PODthe top. I think the question is CoreDNSwhether it will directly resolve POD's name, service in the Service, is not possible because there are Service Cluster IP, directly CoreDNS resolved, then how can it do to resolve POD, a large cattle raised you can use Headless Service, so we have to explore what is Headless Service.

Headless ServiceIt is a kind of Service, except that defines spec:clusterIP: None, that is, do not need Cluster IP的Service.

We first think Service的Cluster IPof the works: a Servicemay correspond to multiple EndPoint(Pod), clientaccess is Cluster IPthrough iptablesthe rules go Real Server, so as to achieve load balancing. Specific operation is as follows:

1, web-demo.yaml

#deploy
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: web-demo
  namespace: dev
spec:
  # 指定svc名称
  serviceName: web-demo-svc
  replicas: 3
  template:
    metadata:
      labels:
        app: web-demo
    spec:
      containers:
      - name: web-demo
        image: 10.0.0.33/base_images/web-demo:v1.0
        ports:
        - containerPort: 8080
        resources:
          requests:
            memory: 1024Mi
            cpu: 500m
          limits:
            memory: 2048Mi
            cpu: 2000m
        livenessProbe:
          tcpSocket:
            port: 8080
          initialDelaySeconds: 20
          periodSeconds: 10
          failureThreshold: 3
          successThreshold: 1
          timeoutSeconds: 5
        readinessProbe:
          httpGet:
            path: /hello
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 20
          periodSeconds: 10
          failureThreshold: 1
          successThreshold: 1
          timeoutSeconds: 5
---
#service
apiVersion: v1
kind: Service
metadata:
  name: web-demo-svc
  namespace: dev
spec:
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP
  clusterIP: None
  selector:
    app: web-demo

View svc, found ClusterIPasNone

$ kubectl get svc -n dev | grep "web-demo-svc"
web-demo-svc   ClusterIP   None          <none>        80/TCP         12s

pod creates order

$ kubectl get pod -n dev
NAME                         READY   STATUS    RESTARTS   AGE
web-demo-0                   1/1     Running   0          7m2s
web-demo-1                   1/1     Running   0          6m39s
web-demo-2                   1/1     Running   0          6m15s

Log on to the inside of the pod Cluster

$ kubectl exec -it web-demo-0 sh -n dev
/ # nslookup web-demo-svc

Name:      web-demo-svc
Address 1: 10.244.2.67 web-demo-0.web-demo-svc.dev.svc.cluster.local
Address 2: 10.244.3.12 web-demo-2.web-demo-svc.dev.svc.cluster.local
Address 3: 10.244.1.214 web-demo-1.web-demo-svc.dev.svc.cluster.local

Summary: dns accessed through, returns a list of backend pods

StatefulSet

First, Deploymentonly a stateless service, and there is no undifferentiated Pod order, the order between the support StatefulSet plurality Pod, for each Pod has its own number, need access to each other and to distinguish between persistent storage

Pod sequential

1, headless-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: springboot-web-svc
spec:
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP
  clusterIP: None
  selector:
    app: springboot-web

2, statefulset.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: springboot-web
spec:
  # serviceName 该字段是告诉statefulSet用那个headless server去保证每个的解析
  serviceName: springboot-web-svc
  replicas: 2
  selector:
    matchLabels:
      app: springboot-web
  template:
    metadata:
      labels:
        app: springboot-web
    spec:
      containers:
      - name: springboot-web
        image: 10.0.0.33/base_images/web-demo:v1.0
        ports:
        - containerPort: 8080
        livenessProbe:
          tcpSocket:
            port: 8080
          initialDelaySeconds: 20
          periodSeconds: 10
          failureThreshold: 3
          successThreshold: 1
          timeoutSeconds: 5

View pod

$ kubectl get pod
NAME               READY   STATUS    RESTARTS   AGE
springboot-web-0   1/1     Running   0          118s
springboot-web-1   1/1     Running   0          116s

Enter a pod, and visit another pod by pod name

$ kubectl exec -it springboot-web-0 sh
/ # ping springboot-web-1.springboot-web-svc
PING springboot-web-1.springboot-web-svc (10.244.2.68): 56 data bytes
64 bytes from 10.244.2.68: seq=0 ttl=62 time=1.114 ms
64 bytes from 10.244.2.68: seq=1 ttl=62 time=0.698 ms

Persistent storage
is automatically created based on pod pvc

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nginx-demo
spec:
  serviceName: springboot-web-svc
  replicas: 2
  selector:
    matchLabels:
      app: nginx-demo
  template:
    metadata:
      labels:
        app: springboot-web
    spec:
      containers:
      - name: springboot-web
        image:  10.0.0.33/base_images/nginx:1.13
        ports:
        - containerPort: 8080
        volumeMounts:
        - name: data
          mountPath: /
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes:
      - ReadWriteOnce
      storageClassName: glusterfs-storage-class
      resources:
        requests:
          storage: 1Gi

DaemonSet

DaemonSet in Kubernetes1.2 new version of a resource objects

DaemonSetIt allows 所有（或者一些特定）的Nodenodes 仅运行一份Pod. When a cluster node is added to the kubernetes, Pod will be (DaemonSet) scheduled to run on the node, when the node is removed from kubernetes cluster is (DaemonSet) scheduled Pod will be removed if you delete DaemonSet, with all this DaemonSet related pods will be deleted.

When to run the application using kubernetes, we need a lot of time 区域（zone）or 所有Noderunning 同一个守护进程（pod）, for example, the following scene:

• Each Node running on a distributed storage daemons, such as glusterd, ceph
• Run Log collector on each Node, e.g. fluentd, logstash
• Acquisition operation monitoring terminal in each Node, e.g. prometheus node exporter, collectd etc.

Pod DaemonSet of dispatch and RC is similar, except that the system scheduling algorithm built in each scheduling Node, or may be used NodeSelector NodeAffinity Pod in definition to specify a range satisfying the condition scheduling Node

DaemonSet resource file format

apiVersion: extensions/v1beta1
kind: DaemonSet
  metadata:

The following examples are defined as the start of each Node on a filebeatcontainer, wherein the host mounts the directory "/ var / log / messages"

$ vi k8s-log-filebeat.yaml
apiVersion: v1
kind: ConfigMap # 定义一个config文件内容
metadata:
  name: k8s-logs-filebeat-config
  namespace: kube-system
data:
  # 填写filebeat读取日志相关信息
  filebeat.yml: |-
    filebeat.prospectors:
      - type: log
        paths:
          - /messages
        fields:
          app: k8s
          type: module
        fields_under_root: true
    output.logstash:
      # specified logstash port (by default 5044)
      hosts: ['10.0.0.100:5044']
---
apiVersion: apps/v1
kind: DaemonSet  # DaemonSet 对象，保证在每个node节点运行一个副本
metadata:
  name: k8s-logs
  namespace: kube-system
spec:
  selector:
    matchLabels:
      project: k8s
      app: filebeat
  template:
    metadata:
      labels:
        project: k8s
        app: filebeat
    spec:
      containers:
      - name: filebeat
        image: docker.elastic.co/beats/filebeat:6.8.1
        args: [
          "-c", "/etc/filebeat.yml",
          "-e",
        ]
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
          limits:
            cpu: 500m
            memory: 500Mi
        securityContext:
          runAsUser: 0
        # 进行实际挂载操作
        volumeMounts:
        # 将configmap里的配置挂载到 /etc/filebeat.yml 文件中
        - name: filebeat-config
          mountPath: /etc/filebeat.yml
          subPath: filebeat.yml

        # 将宿主机  /var/log/messages 路径挂载到 /messages中
        - name: k8s-logs
          mountPath: /messages
      # 定义卷
      volumes:
      - name: k8s-logs
        hostPath:
          path: /var/log/messages
          type: File
      - name: filebeat-config
        configMap:
          name: k8s-logs-filebeat-config

The DeamonSet created using kubectl create command

$ kubectl create -f  k8s-log-filebeat.yaml
configmap/k8s-logs-filebeat-config created
daemonset.apps/k8s-logs created

Check the created DeamonSet and Pod, can be seen on each Node creates a Pod

$ kubectl get ds -n kube-system | grep "k8s-logs"
NAME                      DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                 AGE
k8s-logs                  2         2         0       2            0           <none>                        2m15s

$ kubectl get pods -n kube-system -o wide | grep "k8s-logs"  
k8s-logs-gw4bs                         0/1     Running   0          87s     <none>        k8s-node01     <none>           <none>
k8s-logs-p6r6t                         0/1     Running   0          87s     <none>        k8s-node02     <none>           <none>

In version 1.6 later in kubernetes, DaemonSet can perform a rolling upgrade, that update a template DaemonSet when the old Pod copy will be automatically deleted, while new Pod copy is automatically created, this time DaemonSet update strategy (updateStrategy) is RollingUpdate, as follows:

apiVersion: apps/v1
kind: DaemonSet 
metadata:
  name: k8s-logs
  namespace: kube-system
spec:
  updateStrategy:
    type: RollingUpdate

updateStrategy another value is OnDelete, namely only when manually delete the copy DaemonSet Pod created, a new copy will be created out of Pod, if you do not set the value updateStrategy, and then in later versions of 1.6 kubernetes will be the default setting RollingUpdate (rolling upgrade).

Service Resources

We all know that in kubernetes to Pod minimum scheduling unit, and its characteristic is uncertainty, that will be destroyed at any time and re-create uncertainty will lead each Pod will be deployed to different N by the scheduler a node node, this will cause Pod ip address will change;

For example, web scene is divided into front-end back-end, front-end needs to call back-end resources, if the backend of Pod born of uncertainty lead to a different IP address, so the front is definitely not do IP address to connect back-end automatic switching, so it is necessary to discover through Service Pod and get its IP address.

Pod relationship with the Service

• Pod prevent lost contact., Pod obtain information (via the associated label-selector)
• Pod defines a set of access policies (load balancing TCP / UDP 4 layers)
• Support ClusterIP, NodePort and LoadBalancer three types
• Server implementations underlying iptables and there are two kinds of network modes IPVS

Each application is associated with a Service

Service Type

• ClusterIP: By default, a distribution within the cluster can access the virtual IP (vip)
• NodePort: assign a port as an inlet on each external access Node
• LoadBalancer: work on a specific Cloud Provider, such as Google Cloud, AWS, OpenStack

Detailed 1.Cluster IP:

Cluster IP, also known as VIP, the main achievement of visits between different Pod

type: NodePort  
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP

Open proxy access
kubectl proxy to allow external network access K8S service of ClusterIP

kubectl proxy --address='0.0.0.0'  --accept-hosts='^*$' --port=8009

http://[k8s-master]:8009/api/v1/namespaces/[namespace-name]/services/[service-name]/proxy

Details: https: //blog.csdn.net/zwqjoy/article/details/87865283

Detailed 2.Node Port:

External users can access node implemented Node, node Node will flow forward into the interior thereof Pod

Access Process: Users -> Domain -> Load Balancer -> NodeIP: Port -> PodIP: Port

It can also be deployed in front of a load balancing Node LB directly, as shown:

type: NodePort
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30008
      protocol: TCP

Parameter Description

spec.ports.port：vip端口（cluster ip）
spec.ports.nodePort：映射到宿主机的端口，即提供外部访问的端口
spec.ports.targetPort：pod 端口
spec.selector: 标签选择器

Creating nodePort type of time will be assigned a Cluster IP, providing convenient access to between Pod

3, LoadBalancer Comments:
Access Process: Users -> Domain -> Load Balancer -> NodeIP: Port -> PodIP : Port

Conventional docker mapping scenarios:

Access -> node IP: 10.0.0.12 -> docker container: 172.16.48.2

If hung up when docker, a docker, container IP address change occurs in the restart, before doing so and docker node mapping is invalid, you need to manually modify the map, so it is very troublesome

so, add in k8s in the cluster IP, network segment 10.254.0.0/16,series will automatically create the cluster IP, also known as vip, when the pod is created automatically registered in the service, and load balancing (the default rule as rr), if a pod hung up, it will automatically be rejected

Access -> node IP: 10.0.0.13 -> cluster IP: 10.254.0.0/16 (service) -> pod IP: 172.16.48.2

Create a service

apiVersion: v1
kind: Service
metadata:
  name: myweb
spec:
  type: NodePort
  ports:
  - port: 80
    nodePort: 30000
    targetPort: 80
  selector:
    app: myweb               

// 启动
kubectl create -f nginx-svc.yaml

Service to take over the pod to see if the normal network services:

[root@k8s-master01 svc]# kubectl get endpoints
NAME         ENDPOINTS                       AGE
kubernetes   10.0.0.31:6443                  2d
myweb        172.16.12.2:80,172.16.15.4:80   2m

ipvs and iptables works

service underlying traffic forwarding and load balancing to achieve:

• iptables
• ipvs

1, a service will create a lot of iptables rules (update, non-incremental)
2, iptables rules are matched one by one from top to bottom (large delay).

Savior: IPVS (kernel mode)

LVS load balancing IPVS kernel scheduler module implemented, such as: the SLB Ali cloud, based on four LVS achieve load balancing.

iptables：

• Flexible, powerful (the package can operate at different stages of the packet)
• And updating traversal rules match, a linear delay

IPVS：

• Working in kernel mode, better performance
• Rich scheduling algorithms: rr, wrr, lc, wlc, ip hash ....