Kubernetes HPA自动扩缩容

HPA 自动更新工作负载资源（例如 Deployment 或者 StatefulSet），目的是自动扩缩工作负载以满足需求。

水平扩缩意味着对增加的负载的响应是部署更多的 Pod。这与 “垂直（Vertical）” 扩缩不同，对于 Kubernetes，垂直扩缩意味着将更多资源（例如：内存或 CPU）分配给已经为工作负载运行的 Pod。

如果负载减少，并且 Pod 的数量高于配置的最小值， HPA 会指示工作负载资源（Deployment、StatefulSet 或其他类似资源）缩减。

使用HPA前提是需要安装metrics server来获取资源指标

1.1 基于CPU资源HPA

首先启动一个 Deployment 用 hpa-example 镜像运行一个容器

[root@k8s-master01 hpa]# cat hpa-example.yamlapiVersion: apps/v1
kind: Deployment
metadata:
  name: php-apache
spec:
  selector:
    matchLabels:
      run: php-apache
  replicas: 1
  template:
    metadata:
      labels:
        run: php-apache
    spec:
      containers:
      - name: php-apache
        image: hub.kce.ksyun.com/leoxinyuan/hpa-example:latest
        ports:
        - containerPort: 80
        resources:
          limits:
            cpu: 500m
          requests:
            cpu: 200m
---
apiVersion: v1
kind: Service
metadata:
  name: php-apache
  labels:
    run: php-apache
spec:
  ports:
  - port: 80
  selector:
    run: php-apache


[root@k8s-master01 hpa]# kubectl apply -f hpa-example.yaml deployment.apps/php-apache createdservice/php-apache created
[root@k8s-master01 hpa]# kubectl get po | grep phpphp-apache-6585f85cff-vfmgr               1/1     Running            0          3m18s

1.1.1 创建HPA

使用命令创建

使用 kubectl 创建自动扩缩器。kubectl autoscale 子命令是 kubectl 的一部分。

HPA 控制器将根据设定的CPU 利用率为 20%，进行动态的增加或伸缩，最大的Pod数量位5 最小的伸缩Pod数量位1

1.创建HPA[root@k8s-master01 hpa]# kubectl autoscale deployment php-apache --cpu-percent=20 --min=1 --max=5horizontalpodautoscaler.autoscaling/php-apache autoscaled
[root@k8s-master01 hpa]# kubectl get hpaNAME         REFERENCE               TARGETS   MINPODS   MAXPODS   REPLICAS   AGEphp-apache   Deployment/php-apache   0%/20%    1         5         1          52s#请注意当前的 CPU 利用率是 0%，这是由于我们尚未发送任何请求到服务器 （`TARGET` 列显示了相应 Deployment 所控制的所有 Pod 的平均 CPU 利用率）。

1.1.2 增加CPU负载测试

扫描二维码关注公众号，回复： 14747202 查看本文章

1. 增加负载进行测试扩容启动一个不同的 Pod 作为客户端。客户端 Pod 中的容器在无限循环
[root@k8s-master01 hpa]# kubectl run -i --tty load-generator --rm --image=busybox:1.28 --restart=Never -- /bin/sh -c "while sleep 0.01; do wget -q -O- http://php-apache; done"
If you don't see a command prompt, try pressing enter.
OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!O

2.验证可以看到当前CPU使用上升了[root@k8s-master01 ~]# kubectl get hpaNAME         REFERENCE               TARGETS    MINPODS   MAXPODS   REPLICAS   AGEphp-apache   Deployment/php-apache   151%/20%   1         5         1          3m57s

3.目前已经扩容5个Pod
[root@k8s-master01 ~]# kubectl get po | grep phpphp-apache-6585f85cff-9s7nr               1/1     Running            0          72sphp-apache-6585f85cff-bglmq               1/1     Running            0          72sphp-apache-6585f85cff-j9tpn               1/1     Running            0          72sphp-apache-6585f85cff-tms9v               1/1     Running            0          57sphp-apache-6585f85cff-vfmgr               1/1     Running            0          12m

1.1.2 停止CPU负载测试

结束增加负载的容器load-generator，查看是否缩容ctrl+c
[root@k8s-master01 ~]# kubectl get hpaNAME         REFERENCE               TARGETS   MINPODS   MAXPODS   REPLICAS   AGEphp-apache   Deployment/php-apache   0%/20%    1         5         5          7m56s[root@k8s-master01 ~]# kubectl get po | grep phpphp-apache-6585f85cff-9s7nr               0/1     Terminating        0          7m35sphp-apache-6585f85cff-j9tpn               1/1     Running            0          7m35s

一旦 CPU 利用率降至 0，HPA 会自动将副本数缩减为 1。

controller-manager 默认 5 分钟过后会进行缩放，到这里就完成了基于CPU的 HPA 操作

使用配置清单创建

1.删除hpa使用yaml清单创建[root@k8s-master01 ~]# kubectl get hpaNAME         REFERENCE               TARGETS   MINPODS   MAXPODS   REPLICAS   AGEphp-apache   Deployment/php-apache   0%/20%    1         5         1          10m[root@k8s-master01 ~]# kubectl delete hpa php-apache horizontalpodautoscaler.autoscaling "php-apache" deleted[root@k8s-master01 ~]# [root@k8s-master01 ~]# vim hpa.yaml
apiVersion: autoscaling/v1kind: HorizontalPodAutoscalermetadata:  name: php-apache  namespace: defaultspec:  maxReplicas: 5  minReplicas: 1  scaleTargetRef:    apiVersion: apps/v1    kind: Deployment    name: php-apache  targetCPUUtilizationPercentage: 20


[root@k8s-master01 hpa]# kubectl apply -f hpa.yamlhorizontalpodautoscaler.autoscaling/php-apache created[root@k8s-master01 hpa]# kubectl get hpaNAME         REFERENCE               TARGETS   MINPODS   MAXPODS   REPLICAS   AGEphp-apache   Deployment/php-apache   0%/20%    1         5         1          <invalid>

1.2 基于内存资源HPA

首先启动一个 Deployment 用 hpa-mem 镜像运行一个容器

#vim hpa-mem.yamlapiVersion: apps/v1kind: Deploymentmetadata:  name: php-memspec:  selector:    matchLabels:      app: php-mem  template:    metadata:      labels:        app: php-mem    spec:      containers:      - name: php-mem        image: hub.kce.ksyun.com/leoxinyuan/hpa-example:latest        ports:        - containerPort: 80        resources:          requests:            memory: 50Mi            cpu: 50m        securityContext:          privileged: true
[root@k8s-master01 hpa]# kubectl apply -f hpa-mem.yaml deployment.apps/php-mem created

1.2.1 创建HPA

#vim hpa.yamlapiVersion: autoscaling/v2beta1kind: HorizontalPodAutoscalermetadata:  name: php-memspec:  scaleTargetRef:    apiVersion: apps/v1    kind: Deployment    name: php-mem  minReplicas: 1  maxReplicas: 3  metrics:  - type: Resource    resource:      name: memory      targetAverageUtilization: 60
[root@k8s-master01 hpa]# kubectl apply -f hpa.yamlhorizontalpodautoscaler.autoscaling/php-mem created
[root@k8s-master01 hpa]# kubectl get hpaNAME         REFERENCE               TARGETS   MINPODS   MAXPODS   REPLICAS   AGEphp-apache   Deployment/php-apache   0%/20%    1         5         1          13mphp-mem      Deployment/php-mem      17%/60%   1         3         1          112s

1.2.2 增加内存负载测试

1. 进入php-mem容器进行增加内存负载
#可以多执行几次dd[root@k8s-master01 hpa]# kubectl exec -it php-mem-5556f56d55-zrztn bashkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.root@php-mem-5556f56d55-zrztn:/var/www/html# root@php-mem-5556f56d55-zrztn:/var/www/html# root@php-mem-5556f56d55-zrztn:/var/www/html# mkdir /tmp/memory  root@php-mem-5556f56d55-zrztn:/var/www/html#  mount -t tmpfs -o size=40M tmpfs /tmp/memory root@php-mem-5556f56d55-zrztn:/var/www/html# dd if=/dev/zero of=/tmp/memory/block  dd: writing to '/tmp/memory/block': No space left on device81921+0 records in81920+0 records out41943040 bytes (42 MB) copied, 0.133548 s, 314 MB/s
2. 查看内存变化[root@k8s-master01 hpa]# kubectl get hpaNAME         REFERENCE               TARGETS   MINPODS   MAXPODS   REPLICAS   AGEphp-mem      Deployment/php-mem      99%/60%   1         3         2          2m56s
3. 查看Pod扩容[root@k8s-master01 hpa]# kubectl get po | grep memphp-mem-5556f56d55-t9j82                  1/1     Running            0          108sphp-mem-5556f56d55-zrz12                  1/1     Running            0          6m59sphp-mem-5556f56d55-zrztn                  1/1     Running            0          6m59s

1.2.3 停止内存负载测试

稍等片刻查看等内存降下来，可以看到Pod已经缩容为1

controller-manager 默认 5 分钟过后会进行缩放，到这里就完成了基于内存的 HPA 操作

[root@k8s-master01 hpa]# kubectl exec -it php-mem-5556f56d55-zrztn bashkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.root@php-mem-5556f56d55-zrztn:/var/www/html# rm -rf /tmp/memory/block

[root@k8s-master01 hpa]# kubectl get po | grep memphp-mem-5556f56d55-t9j82                  1/1     Running            0