In-depth analysis of cloud native deployment and monitoring of MySQL in K8S environment

1. Preparation

• MySQL is a relational database management system, developed by the Swedish company MySQL AB, which is a product of Oracle. MySQL is one of the most popular relational database management systems. In terms of WEB applications, MySQL is one of the best RDBMS (Relational Database Management System, relational database management system) application software.
• This article mainly analyzes how mysql is deployed on k8s. The main advantages of mysql deployed on k8s are as follows:
• • resource isolation;
• • Dynamic elastic scaling;
• • environmental consistency;
• • Easy operation and maintenance.
• For the use of MySQL, you can refer to: MySQL official documentation .
• The principle of MySQL is roughly as follows:

• The general process of deployment is as follows:

2. Add source

helm repo add bitnami https://charts.bitnami.com/bitnami
helm pull bitnami/mysql
tar -xf mysql-9.3.3.tgz

3. Modify the configuration

• Modify mysql/values.yaml:

...

image:
  registry: myharbor.com
  repository: bigdata/mysql
  tag: 8.0.30-debian-11-r15

...

architecture: replication

...

primary:
  persistence:
    enabled: true
    size: 10Gi
    storageClass: "mysql-local-storage"
    # 目录需要提前在宿主机上创建
    local:
    - name: mysql-0
      host: "local-168-182-110"
      path: "/opt/bigdata/servers/mysql/data/data1"
  service:
    type: NodePort
    nodePorts:
      mysql: "30306"

secondary:
  replicaCount: 2
  persistence:
    enabled: true
    size: 10Gi
    storageClass: "mysql-local-storage"
    # 目录需要提前在宿主机上创建
    local:
    - name: mysql-1
      host: "local-168-182-111"
      path: "/opt/bigdata/servers/mysql/data/data1"
    - name: mysql-2
      host: "local-168-182-112"
      path: "/opt/bigdata/servers/mysql/data/data1"
  service:
    type: NodePort
    nodePorts:
      mysql: "30307"

...

metrics:
  ## @param metrics.enabled Start a side-car prometheus exporter
  ##
  enabled: true
  image:
    registry: myharbor.com
    repository: bigdata/mysqld-exporter
    tag: 0.14.0-debian-11-r33

• Add mysql/templates/pv.yaml:

{
    
    {
    
    - range .Values.primary.persistence.local }}
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: {
    
    {
    
     .name }}
  labels:
    name: {
    
    {
    
     .name }}
spec:
  storageClassName: {
    
    {
    
     $.Values.primary.persistence.storageClass }}
  capacity:
    storage: {
    
    {
    
     $.Values.primary.persistence.size }}
  accessModes:
    - ReadWriteOnce
  local:
    path: {
    
    {
    
     .path }}
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
                - {
    
    {
    
     .host }}
---
{
    
    {
    
    - end }}

{
    
    {
    
    - range .Values.secondary.persistence.local }}
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: {
    
    {
    
     .name }}
  labels:
    name: {
    
    {
    
     .name }}
spec:
  storageClassName: {
    
    {
    
     $.Values.secondary.persistence.storageClass }}
  capacity:
    storage: {
    
    {
    
     $.Values.secondary.persistence.size }}
  accessModes:
    - ReadWriteOnce
  local:
    path: {
    
    {
    
     .path }}
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
                - {
    
    {
    
     .host }}
---
{
    
    {
    
    - end }}

• 添加 mysql/templates/storage-class.yaml：

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: {
    
    {
    
     .Values.primary.persistence.storageClass }}
provisioner: kubernetes.io/no-provisioner

Fourth, start the installation

# 创建持久化目录
mkdir -p /opt/bigdata/servers/mysql/data/data1

# 先准备好镜像
docker pull docker.io/bitnami/mysql:8.0.30-debian-11-r15
docker tag docker.io/bitnami/mysql:8.0.30-debian-11-r15 myharbor.com/bigdata/mysql:8.0.30-debian-11-r15
docker push myharbor.com/bigdata/mysql:8.0.30-debian-11-r15

# mysqld-exporter
docker pull docker.io/bitnami/mysqld-exporter:0.14.0-debian-11-r33
docker tag docker.io/bitnami/mysqld-exporter:0.14.0-debian-11-r33 myharbor.com/bigdata/mysqld-exporter:0.14.0-debian-11-r33
docker push myharbor.com/bigdata/mysqld-exporter:0.14.0-debian-11-r33

# 开始安装
helm install mysql ./mysql -n mysql --create-namespace

NAME: mysql
LAST DEPLOYED: Mon Sep 19 23:57:18 2022
NAMESPACE: mysql
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: mysql
CHART VERSION: 9.3.3
APP VERSION: 8.0.30

** Please be patient while the chart is being deployed **

Tip:

  Watch the deployment status using the command: kubectl get pods -w --namespace mysql

Services:

  echo Primary: mysql-primary.mysql.svc.cluster.local:3306
  echo Secondary: mysql-secondary.mysql.svc.cluster.local:3306

Execute the following to get the administrator credentials:

  echo Username: root
  MYSQL_ROOT_PASSWORD=$(kubectl get secret --namespace mysql mysql -o jsonpath="{.data.mysql-root-password}" | base64 -d)

To connect to your database:

  1. Run a pod that you can use as a client:

      kubectl run mysql-client --rm --tty -i --restart='Never' --image  myharbor.com/bigdata/mysql:8.0.30-debian-11-r15 --namespace mysql --env MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD --command -- bash

  2. To connect to primary service (read/write):

      mysql -h mysql-primary.mysql.svc.cluster.local -uroot -p"$MYSQL_ROOT_PASSWORD"

  3. To connect to secondary service (read-only):

      mysql -h mysql-secondary.mysql.svc.cluster.local -uroot -p"$MYSQL_ROOT_PASSWORD"



To access the MySQL Prometheus metrics from outside the cluster execute the following commands:

    kubectl port-forward --namespace mysql svc/mysql-metrics 9104:9104 &
    curl http://127.0.0.1:9104/metrics

• View pod status:

kubectl get pods,svc -n mysql -owide

Five, Prometheus monitoring

• Prometheus：

https://prometheus.k8s.local/targets?search=mysql

• You can view the collected data with the command:

kubectl get --raw http://10.244.0.74:9104/metrics
kubectl get --raw http://10.244.1.125:9104/metrics
kubectl get --raw http://10.244.2.178:9104/metrics

• Grafana：

https://grafana.k8s.local/

• Account: admin, password obtained through the following command:

kubectl get secret --namespace grafana grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

• Import grafana template, cluster resource monitoring: 7362, official module download address:

https://grafana.com/grafana/dashboards/

6. Uninstall

helm uninstall mysql -n mysql

kubectl delete pod -n mysql `kubectl get pod -n mysql |awk 'NR>1{
    
    print $1}'` --force
kubectl patch ns mysql -p '{
    
    "metadata":{
    
    "finalizers":null}}'
kubectl delete ns mysql --force