kubernetes affinity Detailed Scheduling

Article Directory

Overview:

Creating pod scheduling by default is to be scheduled according to kubernetes scheduler default scheduling rules, but there are times when some applications have special requirements such as specifying deployed to the corresponding node, a node needs to be deployed in different between multiple pod, require mutual exclusion, between the pod and the pod communicate with each other more frequently need to run on the same node requires affinity. Then you need the flexibility to configure scheduler to achieve.

A scenario: scheduling to a group of hosts having the same characteristics (label + nodeSelector)

Kubernetes example, the entire cluster, there is a good set of configuration servers, and there are some special applications require hardware configuration requirements, these need to be scheduled to the server, then go kubernetes achieved by the label + nodeSelector, in fact, in kubernetes inside label is a very flexible concept, associations between different resource objects within kubernetes is to be by way of label.
Cases
we give those nodes high profile marked with a label

Host to a high-profile marked with label Configuration = hight (rke-node2 here for example)

kubectl label node rke-node2 Configuration=hight

In the show labels you can just see us play up label

Create a pod use nodeSelector scheduled on the node of the specified label

mysql-deployment.yaml

apiVersion: apps/v1beta1
kind: Deployment
metadata:
 name: mysql-2
spec:
 selector:
   matchLabels:
     app: mysql
 template:
   metadata:
     labels:
       app: mysql
   spec:
     containers:
     - image: mysql:5.6
       name: mysql
       env:
       - name: MYSQL_ROOT_PASSWORD
         value: password
       ports:
       - containerPort: 3306
         name: mysql
     nodeSelector:
       Configuration: hight

Configuring nodeSelector scheduled Configuration: the hight of the host
to create deployment

kubectl apply -f mysql-deployment.yaml

View pod is scheduled to label as Configuration: the host hight of the

Scene 2: application deployment do not want to schedule some nodes (nodeaffinity)

For example, in a kubernetes cluster, you need to deploy applications to the specified label on the machine, but if not, then this type of machine, then it is normal scheduling scheduling rules. Then nodeSelector can not be achieved by this demand, you can go to achieve by way of nodeaffinity in kubernetes.

nodeaffinityAnd nodeSelecotrvery similar, are configured scheduling policy node level, but the difference is nodeSelect single function, only the corresponding host selected according to label, but nodeaffinitymore flexible, nodeaffinitythere are two scheduling types requiredDuringSchedulingIgnoredDuringExecution(hard requirement) and preferredDuringSchedulingIgnoredDuringExecution(soft requirements), the difference is that the two types, hard requirements must be met scheduling rules set when scheduling requirements, otherwise Throws scheduling fails, and soft requirements, but priority scheduling rules set when scheduling, rules fail when the settings, press kube -scheduler default scheduling policy for scheduling. nodeaffinityAlso supports multiple configuration rule matching condition as
In：labela value in the list of
NotIn：labelvalues is not within the list of
Gt：labelvalues is greater than the set
Lt：labelvalue is less than a set value of
Exists：label present setting
DoesNotExist：label set does not exist

Case

apiVersion: apps/v1beta1
kind: Deployment
metadata:
 name: mysql-2
spec:
 selector:
   matchLabels:
     app: mysql
 template:
   metadata:
     labels:
       app: mysql
   spec:
     containers:
     - image: mysql:5.6
       name: mysql
       env:
       - name: MYSQL_ROOT_PASSWORD
         value: password
       ports:
       - containerPort: 3306
         name: mysql
     affinity:
       nodeAffinity:
         requiredDuringSchedulingIgnoredDuringExecution:
           nodeSelectorTerms:
           - matchExpressions:
             - key: kubernetes.io/hostname
               operator: NotIn
               values:
               - rke-node3
               - rke-node4
         preferredDuringSchedulingIgnoredDuringExecution:
         - weight: 1 preference: matchExpressions: - key: disk_type operator: In values: - ssd

如上所例，要求pod不能被调度到rke-node3和rke-node4上，但如果有满足dis_type=ssd的label的节点则优先选择。
如果在 nodeAffinity 类型中指定了多个 nodeSelectorTerms，那么 pod 将会被调度到只要满足其中一个 nodeSelectorTerms 的 node 上。
如果在 nodeSelectorTerms 中指定了多个 matchExpressions，那么 pod 将会被调度到满足所有 matchExpressions 的 node 上。

场景三：部署的应用关联性很强，需要尽量在一个节点上

比如在kubernetes集群，有些pod和pod之间的交互很频繁，这时就需要将它们尽可能的调度到一台主机上，通过pod与pod之间的关系来选择调度，这时候我们就可以使用podaffinity。根nodeaffinity一样，podaffinity也有两种调度类型requiredDuringSchedulingIgnoredDuringExecution(硬要求)和preferredDuringSchedulingIgnoredDuringExecution(软要求)，多种规则匹配条件配置。
例部署一个nginx和mysql因为nginx和mysql之间的交互很频繁，所以尽量将他们部署在一个host上。
先启动一个nginx。

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  containers:
  - name: nginx
    image: nginx

启动mysql

apiVersion: v1
kind: Pod
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  containers:
  - name: mysql
    image: nginx
    env:
     - name: "MYSQL_ROOT_PASSWORD"
       value: "123456"
  affinity:
    podAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - nginx
        topologyKey: kubernetes.io/hostname

topologyKey可以设置成如下几种类型
kubernetes.io/hostname　　＃Node
failure-domain.beta.kubernetes.io/zone　＃Zone
failure-domain.beta.kubernetes.io/region #Region
可以设置node上的label的值来表示node的name,zone,region等信息，pod的规则中指定topologykey的值表示指定topology范围内的node上运行的pod满足指定规则

这里我们配置podaffinity，mysql会调度到有app:nginx这个label的pod的host上，所以你部署出来的mysql会调度到有nginx的pod上，因为默认部署出来的nginx就自带app:nginx这个label。

场景四：部署应用需要互斥，不能同时运行在一台主机上，会冲突

我们继续以我们刚刚的nginx+mysql这个应用组合为例，现在我们修改mysql的podaffinity为podAntiAffinity

apiVersion: v1
kind: Pod
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  containers:
  - name: mysql
    image: nginx
    env:
     - name: "MYSQL_ROOT_PASSWORD"
       value: "123456"
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - nginx
        topologyKey: kubernetes.io/hostname

将podaffinity改为podAntiAffinity这样调度mysql这个pod时会避开有label为app:nginx的pod的主机

高级特性Taints and tolerations

Taints其实根nodeaffinity正好相反，nodeaffinity是将设置策略的pod调度到期望的节点上，而taints正好相反，如果一个节点被标记为taints，除非pod配置了tolerations，否则是不会被允许调度过来在生产环境中我们一般会将master节点配置Taints，因为master只跑kubernetes 系统组件，如果跑了用户应用pod容易把资源耗尽，造成master节点崩溃，当然后期如果要添加额外的系统组件，这时就可以通过给对应的pod配置toleration。

设置rke-node2不能被调度

kubectl taint nodes rke-node2 key=value:NoSchedule

value可以配置多个值如
NoSchedule：不能调度，当之前调度的不管。
PreferNoSchedule：尽量不调度上去，其实就是Noschedule软策略版。
NoExecute：不能调度，但之前已经调度上去的也会自动迁移走。

NoSchedule更像执行kubectl cordon xxx
NoExecute更想执行kubectl cordon xxx+kubectl drain

取消taints

kubectl taint nodes rke-node2 key-

如何让pod调度到配置了taints主机上?
通过配置tolerations，可以让pod调度到配置了taints的机器上
如

apiVersion: apps/v1beta1
kind: Deployment
metadata:
 name: mysql-2
spec:
 selector:
   matchLabels:
     app: mysql
 template:
   metadata:
     labels:
       app: mysql
   spec:
     containers:
     - image: mysql:5.6
       name: mysql
       env:
       - name: MYSQL_ROOT_PASSWORD
         value: password
       ports:
       - containerPort: 3306
         name: mysql
     tolerations:
     - key: "key"
       operator: "Equal" value: "value" effect: "NoSchedule"