kubernetes- deployment environment LNMP run Discuz

Description: In this study, the chapter https://www.cnblogs.com/douyi/p/12031600.html simulation of a real online environment cluster based on the deployment environment LNMP run Discuz Forum k8s cluster

 

First, download the image [public] deploy on operations

 

1: Download mysql5.7 Mirror

 1 [root@master ~]#  docker pull mysql:5.7
 2 5.7: Pulling from library/mysql
 3 d599a449871e: Pull complete 
 4 f287049d3170: Pull complete 
 5 08947732a1b0: Pull complete 
 6 96f3056887f2: Pull complete 
 7 871f7f65f017: Pull complete 
 8 1dd50c4b99cb: Pull complete 
 9 5bcbdf508448: Pull complete 
10 02a97db830bd: Pull complete 
11 c09912a99bce: Pull complete 
12 08a981fc6a89: Pull complete 
13 818a84239152: Pull complete 
14 Digest: sha256:5779c71a4730da36f013a23a437b5831198e68e634575f487d37a0639470e3a8
15 Status: Downloaded newer image for mysql:5.7

2: Download nginx-php-fpm mirroring

 1 [root@master ~]#  docker pull richarvey/nginx-php-fpm
 2 Using default tag: latest
 3 latest: Pulling from richarvey/nginx-php-fpm
 4 9d48c3bd43c5: Pull complete 
 5 4bf02c0a37c8: Pull complete 
 6 9ce49f939c6f: Pull complete 
 7 2fa33c09831c: Pull complete 
 8 64d703475f7c: Pull complete 
 9 687e14517432: Pull complete 
10 5d8ec717fece: Pull complete 
11 9fe12d3f6add: Pull complete 
12 e860070fe546: Pull complete 
13 36baa049bdfe: Pull complete 
14 4073f665d7d8: Pull complete 
15 42a3d0ae6c56: Pull complete 
16 682e06ece949: Pull complete 
17 3c3d9faab7c5: Pull complete 
18 258d52d772e4: Pull complete 
19 41336c37b6cd: Pull complete 
20 9ef84aa1b775: Pull complete 
21 94c642e23a42: Pull complete 
22 f601338f6a53: Pull complete 
23 0c5f20947667: Pull complete 
24 50f6eb03fd0b: Pull complete 
25 75174a25ab08: Pull complete 
26 8e16258eb3ba: Pull complete 
27 447fabc9be1b: Pull complete 
28 a485db129518: Pull complete 
29 de30cb0b9f89: Pull complete 
30 5724050c9b48: Pull complete 
31 c20deb9ffa63: Pull complete 
32 52031a5fa502: Pull complete 
33 6aea67ff4e17: Pull complete 
34 Digest: sha256:f8109029deb401ee83c4fcd1e737cc496f42aca6a6e8f445f8501ea977be6784
35 Status: Downloaded newer image for richarvey/nginx-php-fpm:latest

二、上传镜像到harbor【deploy上操作】

说明：harbor安装方法参考次篇文章 https://www.cnblogs.com/douyi/p/12057218.html

1：mysql镜像push到harbor上

 1 [root@master ~]# docker tag mysql:5.7 harbor.ghs.com/lnmp/mysql:5.7
 2 [root@master ~]# docker push harbor.ghs.com/lnmp/mysql:5.7
 3 The push refers to repository [harbor.ghs.com/lnmp/mysql]
 4 94a471180601: Pushed 
 5 934de0c0816e: Pushed 
 6 fc12e028de3b: Pushed 
 7 398ef8a407f7: Pushed 
 8 7055b7f82e4c: Pushed 
 9 2a9aab74013a: Pushed 
10 414373ffccb4: Pushed 
11 6599033b2ab2: Pushed 
12 51734435c93c: Pushed 
13 5a8a245abd1c: Pushed 
14 99b5261d397c: Pushed 
15 5.7: digest: sha256:9e02c7c9a87d363588e85c87b8c6f637254c5c67b915b1666482f54121bb0926 size: 2622

2：nginx-php-fpm镜像push到harbor上

 1 [root@master ~]# docker tag richarvey/nginx-php-fpm harbor.ghs.com/lnmp/nginx-php-fpm:latest
 2 [root@master ~]# docker push harbor.ghs.com/lnmp/nginx-php-fpm:latest
 3 The push refers to repository [harbor.ghs.com/lnmp/nginx-php-fpm]
 4 2624e7202e4b: Pushed 
 5 eb30cdb90ca1: Pushed 
 6 ae93c067c849: Pushed 
 7 9801a5392a22: Pushed 
 8 7b2d66211161: Pushed 
 9 9d4627716f06: Pushed 
10 d3cf99377a74: Pushed 
11 f647f7714c77: Pushed 
12 8ed73ab2d11b: Pushed 
13 525bffddcce1: Pushed 
14 d93022710c58: Pushed 
15 28e868b9a611: Pushed 
16 ebbffb8bead8: Pushed 
17 c4ffba011c55: Pushed 
18 9ac09047b17d: Pushed 
19 3fefc401e422: Pushed 
20 3b73d52fb480: Pushed 
21 73f0a2737684: Pushed 
22 bb46af78e4a5: Pushed 
23 5d02268e182d: Pushed 
24 84bd64e5da78: Pushed 
25 14d91eeebaab: Pushed 
26 67ee9b354071: Pushed 
27 d822f3d53ee3: Pushed 
28 a0c85631af8f: Pushed 
29 7c1ad7fa7002: Pushed 
30 03ecc22184eb: Pushed 
31 1581c77670a3: Pushed 
32 14ee8729351c: Pushed 
33 03901b4a2ea8: Pushed 
34 latest: digest: sha256:f8109029deb401ee83c4fcd1e737cc496f42aca6a6e8f445f8501ea977be6784 size: 6576

三、安装NFS【master2上操作】

1：创建web和DB目录

##/data/k8s下的两个目录用于集群中运行LNMP环境存放的数据，web目录为nginx网站的根目录，DB为数据库存放的数据目录

[root@master2 ~]# mkdir -p /data/k8s/{web,db}

2：yum安装nfs

[root@master2 ~]# yum -y install nfs-utils

3：设置共享目录

[root@master2 ~]# vim /etc/exportfs

/data/k8s 192.168.1.0/24(sync,rw,no_root_squash)

4：启动服务

[root@master2 ~]# systemctl start nfs

[root@master2 ~]# systemctl enable nfs

Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service.

四、安装Mysql服务【deploy上操作】

1：创建存放mysql服务的yaml文件目录

[root@master2 ~]# mkdir -p /home/k8s/lnmp/mysql

2：创建secret，将mysql数据库的密码存放到secret中

[root@master mysql]# kubectl create secret generic mysql-pass --from-literal=password=abc123

3：创建mysql-pv

 1 [root@master mysql]# vim mysql-pv.yaml 
 2 apiVersion: v1
 3 kind: PersistentVolume
 4 metadata:
 5   name: mysql-pv
 6 spec:
 7   capacity:
 8     storage: 10Gi
 9   accessModes:
10     - ReadWriteMany
11   nfs:
12     path: /data/k8s/db              ##该参数指定你的NFS端的共享目录
13     server: 192.168.1.201           ##指定nfs端的IP
14 [root@master mysql]# kubectl create -f mysql-pv.yaml 
15 persistentvolume/mysql-pv created

4：创建mysql-pvc

 1 [root@master mysql]# vim mysql-pvc.yaml 
 2 apiVersion: v1
 3 kind: PersistentVolumeClaim
 4 metadata:
 5   name: mysql-claim
 6   labels:
 7     app: discuz
 8 spec:
 9   accessModes:
10     - ReadWriteMany
11   resources:
12     requests:
13       storage: 10Gi
14 [root@master mysql]# kubectl create -f mysql-pvc.yaml 
15 persistentvolumeclaim/mysql-claim created

5：查看pv，pvc信息

##下面可以看出状态Bound（绑定）正常，pv中的CLAIM列绑定的信息为pvc定义的name，pvc中VOLUME列的信息显示绑定了mysql-pv的name

1 [root@master mysql]# kubectl get pv,pvc
2 NAME                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                 STORAGECLASS   REASON   AGE
3 persistentvolume/mysql-pv   10Gi       RWX            Retain           Bound    default/mysql-claim                           71s
4 
5 NAME                                STATUS   VOLUME     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
6 persistentvolumeclaim/mysql-claim   Bound    mysql-pv   10Gi       RWX                           15s

6：创建mysql-deploy

 1 [root@master mysql]# vim mysql-dp.yaml 
 2 apiVersion: apps/v1
 3 kind: Deployment
 4 metadata:
 5   name: dz-mysql
 6   labels:
 7     app: discuz
 8 spec:
 9   selector:
10     matchLabels:
11       app: discuz
12       tier: mysql
13   strategy:
14     type: Recreate
15   template:
16     metadata:
17       labels:
18         app: discuz
19         tier: mysql
20     spec:
21       imagePullSecrets:
22        - name: my-secret 
23       containers:
24       - image: harbor.ghs.com/lnmp/mysql:5.7      ##此处指定你的harbor需要拉取的镜像
25         name: mysql 
26         env:
27         - name: MYSQL_ROOT_PASSWORD
28           valueFrom:
29             secretKeyRef:
30               name: mysql-pass                    ##此模块是调用secret中存放的mysql密码
31               key: password
32         ports:
33         - containerPort: 3306
34           name: dz-mysql
35         volumeMounts:
36         - name: mysql-persistent-storage
37           mountPath: /var/lib/mysql
38       volumes:
39       - name: mysql-persistent-storage
40         persistentVolumeClaim:                     ##此模块参数调用定义创建的pvc名称
41           claimName: mysql-claim
42 [root@master mysql]# kubectl create -f mysql-dp.yaml 
43 deployment.apps/dz-mysql created

7：创建mysql-service

 1 [root@master mysql]# vim mysql-svc.yaml 
 2 apiVersion: v1
 3 kind: Service
 4 metadata:
 5   name: dz-mysql
 6   labels:
 7     app: discuz
 8 spec:
 9   ports:
10     - port: 3306
11   selector:
12     app: discuz
13     tier: mysql
14 [root@master mysql]# kubectl create -f mysql-svc.yaml 
15 service/dz-mysql created

8：查看mysql服务的pod和svc信息

##下面信息可以看出，dz-mysql状态Running（运行）正常，分配的cluster-ip=10.68.235.35，该IP在集群中可以访问，外部无法访问该IP

1 [root@master mysql]# kubectl get pod,svc
2 NAME                            READY   STATUS    RESTARTS   AGE
3 pod/dz-mysql-787586b8d6-9kl5b   1/1     Running   0          5m35s
4 pod/nginx-7c45b84548-4998z      1/1     Running   2          20d
5 
6 NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
7 service/dz-mysql     ClusterIP   10.68.235.35    <none>        3306/TCP   5m13s
8 service/kubernetes   ClusterIP   10.68.0.1       <none>        443/TCP    20d
9 service/nginx        ClusterIP   10.68.241.175   <none>        80/TCP     20d

五、安装Nginx服务【deploy上操作】

1：创建nginx-pv

 1 [root@master nginx]# vim web-pv.yaml 
 2 apiVersion: v1
 3 kind: PersistentVolume
 4 metadata:
 5   name: web-pv
 6 spec:
 7   capacity:
 8     storage: 10Gi
 9   accessModes:
10     - ReadWriteMany
11   nfs:
12     path: /data/k8s/web
13     server: 192.168.1.201
14 [root@master nginx]# kubectl create -f web-pv.yaml 
15 persistentvolume/web-pv created

2：创建nginx-pvc

 1 [root@master nginx]# vim web-pvc.yaml 
 2 apiVersion: v1
 3 kind: PersistentVolumeClaim
 4 metadata:
 5   name: web-claim
 6   labels:
 7     app: discuz
 8 spec:
 9   accessModes:
10     - ReadWriteMany
11   resources:
12     requests:
13       storage: 10Gi
14 [root@master nginx]# kubectl create -f web-pvc.yaml 
15 persistentvolumeclaim/web-claim created

3：查看nginx的pv，pvc信息

1 [root@master nginx]# kubectl get pv,pvc
2 NAME                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                 STORAGECLASS   REASON   AGE
3 persistentvolume/mysql-pv   10Gi       RWX            Retain           Bound    default/mysql-claim                           34m
4 persistentvolume/web-pv     10Gi       RWX            Retain           Bound    default/web-claim                             114s
5 
6 NAME                                STATUS   VOLUME     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
7 persistentvolumeclaim/mysql-claim   Bound    mysql-pv   10Gi       RWX                           33m
8 persistentvolumeclaim/web-claim     Bound    web-pv     10Gi       RWX                           108s

4：创建nginx-deploy

 1 [root@master nginx]# vim web-dp.yaml 
 2 apiVersion: apps/v1
 3 kind: Deployment
 4 metadata:
 5   name: dz-web
 6   labels:
 7     app: discuz
 8 spec:
 9   replicas: 1
10   selector:
11     matchLabels:
12       app: discuz
13       tier: nginx-php
14   template:
15     metadata:
16       labels:
17         app: discuz
18         tier: nginx-php
19     spec:
20       imagePullSecrets:
21       - name: my-secret
22       containers:
23       - image: harbor.ghs.com/lnmp/nginx-php-fpm:latest
24         name: dz-web
25         ports:
26         - containerPort: 9000
27         - containerPort: 80
28           name: dz-web
29         volumeMounts:
30         - name: mysql-persistent-storage
31           mountPath: /var/www/html/
32       volumes:
33       - name: mysql-persistent-storage
34         persistentVolumeClaim:
35           claimName: web-claim
36 [root@master nginx]# kubectl create -f web-dp.yaml 
37 deployment.apps/dz-web created

5：创建nginx-service

##映射端口30001提供集群外部的机器访问web网站

 1 [root@master nginx]# vim web-svc.yaml 
 2 apiVersion: v1
 3 kind: Service
 4 metadata:
 5   name: dz-web
 6   labels:
 7     app: discuz
 8 spec:
 9   type: NodePort
10   ports:
11     - port: 80
12       nodePort: 30001        
13   selector:
14     app: discuz
15     tier: nginx-php
16 [root@master nginx]# kubectl create -f web-svc.yaml 
17 service/dz-web created

6：查看nginx的pod,svc信息

 1 [root@master nginx]# kubectl  get pod,svc
 2 NAME                            READY   STATUS    RESTARTS   AGE
 3 pod/dz-mysql-787586b8d6-9kl5b   1/1     Running   1          38m
 4 pod/dz-web-d4968cf8d-clppp      1/1     Running   0          3m37s
 5 pod/nginx-7c45b84548-4998z      1/1     Running   2          20d
 6 
 7 NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
 8 service/dz-mysql     ClusterIP   10.68.235.35    <none>        3306/TCP       38m
 9 service/dz-web       NodePort    10.68.53.228    <none>        80:30001/TCP   11m
10 service/kubernetes   ClusterIP   10.68.0.1       <none>        443/TCP        20d
11 service/nginx        ClusterIP   10.68.241.175   <none>        80/TCP         20d

六、安装Discuz【NFS上操作】

1：下载Discuz

1 [root@master2 ~]# cd /data/
2 [root@master2 data]# git clone https://gitee.com/ComsenzDiscuz/DiscuzX.git
3 正克隆到 'DiscuzX'...
4 remote: Counting objects: 9085, done.
5 remote: Compressing objects: 100% (5400/5400), done.
6 remote: Total 9085 (delta 3895), reused 8535 (delta 3632)
7 接收对象中: 100% (9085/9085), 12.96 MiB | 166.00 KiB/s, done.
8 处理 delta 中: 100% (3895/3895), done.

2：将Discuz项目中upload目录下所有文件，放到web中

1 [root@master2 data]# mv DiscuzX/upload/* k8s/web/
2 [root@master2 data]# cd k8s/web/
3 [root@master2 web]# chmod 100 data uc_server/data uc_client/data config

3：进入mysql，创建dz库和授权用户

 1 [root@master ~]# kubectl exec -it dz-mysql-787586b8d6-9kl5b -- mysql -u root -p
 2 Welcome to the MySQL monitor.  Commands end with ; or \g.
 3 Your MySQL connection id is 4
 4 Server version: 5.7.28 MySQL Community Server (GPL)
 5 
 6 Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.
 7 
 8 Oracle is a registered trademark of Oracle Corporation and/or its
 9 affiliates. Other names may be trademarks of their respective
10 owners.
11 
12 Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
13 
14 mysql> create database dz;
15 Query OK, 1 row affected (0.19 sec)
16 
17 mysql> grant all on dz.* to 'dz'@'%' identified by 'abc123';
18 Query OK, 0 rows affected, 1 warning (0.11 sec

4：浏览器访问web安装页面

 

 实验成功

5：通过设置Nginx代理供外部访问

##目前nginx服务是运行在kubernetes集群里，node节点以及master节点上是可以通过cluster-ip访问到，但是外部的客户端就不能访问了。所以，可以在任意一台node或者master上建一个nginx反向代理即可访问到集群内的nginx。 kubectl get svc dz-web //查看cluster-ip，我的ip是10.68.235.35

nginx代理配置文件内容如下：

 1 server {
 2           listen 80;
 3           server_name dz.yuankeedu.com;
 4 
 5           location / {
 6               proxy_pass      http://10.68.235.35;
 7               proxy_set_header Host   $host;
 8               proxy_set_header X-Real-IP      $remote_addr;
 9               proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
10       }
11 }