Kubernetes(5)-在集群测试运行Nginx+tomcat

其他 2020-04-02 09:15:02 阅读次数: 0

集群环境：

IP 主机名 角色
192.168.100.142 kube-master1,kube-master1.suosuoli.cn K8s 集群主节点 1
192.168.100.144 kube-master2,kube-master2.suosuoli.cn K8s 集群主节点 2
192.168.100.146 kube-master3,kube-master3.suosuoli.cn K8s 集群主节点 3
192.168.100.160 node1,node1.suosuoli.cn K8s 集群工作节点 1
192.168.100.162 node2,node2.suosuoli.cn K8s 集群工作节点 2
192.168.100.164 node3,node3.suosuoli.cn K8s 集群工作节点 3
192.168.100.150 ha1,ha1.suosuoli.cn K8s 主节点访问入口 1(高可用及负载均衡)
192.168.100.152 ha2,ha2.suosuoli.cn K8s 主节点访问入口 1(高可用及负载均衡)
192.168.100.154 harbor,harbor.suosuoli.cn 容器镜像仓库
192.168.100.200 无 VIP

一. 运行 nginx 和 tomcat 准备

1.1 拉取 nginx 和 tomcat 镜像

root@kube-master1:~# docker pull nginx:1.16.1
root@kube-master1:~# docker pull tomcat:8.5-jdk11-adoptopenjdk-hotspot

1.2 上传到本地 harbor 仓库

1.2.1 新建 baseimages 项目

在 karbor 新建一个项目来存放镜像：

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-yVSr4hsb-1585788726733)(http://39.105.69.237/static/blog_images/article/k8s-basic/2020-03-29-16-32-37.png)]

别忘了需要添加不安全的主机列表给 docker:

~# vim /lib/systemd/system/docker.service
...
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --insecure-registry harbor.suosuoli.cn
...
~# systemctl daemon-reload
~# systemctl restart docker

1.2.2 打标签并上传镜像到 harbor

将从 docker hub 官方 pull 下来的 nginx 和 tomcat 镜像打标签，并上传到 harbor：

打标签：

root@kube-master1:~# docker tag tomcat:8.5-jdk11-adoptopenjdk-hotspot harbor.suosuoli.cn/baseimages/tomcat:8.5-jdk11-adoptopenjdk-hotspot
root@kube-master1:~# docker tag nginx:1.16.1 harbor.suosuoli.cn/baseimages/nginx:1.16.1

上传到 harbor.suosuoli.cn:

root@kube-master1:~# docker login harbor.suosuoli.cn
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
root@kube-master1:~# docker push tomcat
tomcat                                 tomcat:8.5-jdk11-adoptopenjdk-hotspot
root@kube-master1:~# docker push harbor.suosuoli.cn/baseimages/tomcat:8.5-jdk11-adoptopenjdk-hotspot
The push refers to repository [harbor.suosuoli.cn/baseimages/tomcat]
aec076861234: Pushed
b5fcbb958d72: Pushed
3fc7e3fa7a96: Pushed
5f919249e8cf: Pushed
85fa6e80789c: Pushed
16542a8fc3be: Pushed
6597da2e2e52: Pushed
977183d4e999: Pushed
c8be1b8f4d60: Pushed
8.5-jdk11-adoptopenjdk-hotspot: digest: sha256:e05a00d18af68ace3e1d63a82318e166cb4adbf84fcd204636bf6460b5a760a1 size: 2204

root@kube-master1:~# docker push harbor.suosuoli.cn/baseimages/nginx:1.16.1
The push refers to repository [harbor.suosuoli.cn/baseimages/nginx]
bc21985928b6: Pushed
e370c53ec242: Pushed
f2cb0ecef392: Pushed
1.16.1: digest: sha256:2500df07b831096e9207522576c20132b3f645865185d519f3f911602b585c4b size: 948

1.3 测试下载 harbor 镜像

查看 harbor 仓库：

测试下载，在 kube-master2 测试，也要改 docker 的 service 文件，添加非安全仓库：

root@kube-master2:~# docker pull harbor.suosuoli.cn/baseimages/nginx:1.16.1
1.16.1: Pulling from baseimages/nginx
68ced04f60ab: Pull complete
e6edbc456071: Pull complete
9f915b082e59: Pull complete
Digest: sha256:2500df07b831096e9207522576c20132b3f645865185d519f3f911602b585c4b
Status: Downloaded newer image for harbor.suosuoli.cn/baseimages/nginx:1.16.1
harbor.suosuoli.cn/baseimages/nginx:1.16.1
root@kube-master2:~# docker images | grep nginx
harbor.suosuoli.cn/baseimages/nginx                                           1.16.1              8c5ec390a315        4 weeks ago         127MB

二. 在集群运行 nginx 和 tomcat

准备好 kubeadm 使用的 yaml 配置文件，使用 yaml 文件部署
nginx 和 tomcat。

2.1 nginx 部署

编辑部署配置文件：

root@kube-master1:~# mkdir /opt/kubeadm-deploy
root@kube-master1:~# cd /opt/kubeadm-deploy/
root@kube-master1:/opt/kubeadm-deploy# vim nginx.yaml
root@kube-master1:/opt/kubeadm-deploy# cat nginx.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: default
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:1.16.1
          ports:
            - containerPort: 80

---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: suosuoli-nginx-service-label
  name: suosuoli-nginx-service
  namespace: default
spec:
  type: NodePort
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: 80
      nodePort: 30004
  selector:
    app: nginx

部署：

~# kubectl apply -f nginx.yaml

2.2 tomcat 部署

编辑部署文件：

root@kube-master1:/opt/kubeadm-deploy# vim tomcat.yaml
root@kube-master1:/opt/kubeadm-deploy# cat tomcat.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: default
  name: tomcat-deployment
  labels:
    app: tomcat
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tomcat
  template:
    metadata:
      labels:
        app: tomcat
    spec:
      containers:
        - name: tomcat
          image: tomcat:8.5-jdk11-adoptopenjdk-hotspot
          ports:
            - containerPort: 8080

---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: suosuoli-tomcat-service-label
  name: suosuoli-tomcat-service
  namespace: default
spec:
  type: NodePort
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: 8080
      nodePort: 30005
  selector:
    app: tomcat

部署：

~# kubectl apply -f tomcat.yaml

2.3 dashboard 查看

三. 测试访问

3.1 nginx

查看 nginx 被分配到哪个主机：

root@kube-master1:~# kubectl get pod -o wide | grep nginx
nginx-deployment-5d66cc795f-g7bmr     1/1     Running   0          56m   10.20.4.7    node2   <none>           <none>

root@kube-master1:~# kubectl describe pod nginx-deployment-5d66cc795f-g7bmr -n default | grep -w Node:
Node:         node2/192.168.100.162

访问http://192.168.100.162:80测试：

3.2 tomcat

查看 tomcat 被分配到哪个主机：

root@kube-master1:~# kubectl get pod -o wide | grep tomcat
tomcat-deployment-5948d795c9-qtqlr    1/1     Running   0          59m   10.20.5.13   node3   <none>           <none>

root@kube-master1:~# kubectl describe pod tomcat-deployment-5948d795c9-qtqlr -n default | grep -w Node:
Node:         node3/192.168.100.164

访问http://192.168.100.164:30005测试：

没有默认 app：

进入 tomcat 容器，写个测试页面：

root@kube-master1:~# kubectl exec -it tomcat-deployment-5948d795c9-qtqlr -n default -- /bin/bash

root@tomcat-deployment-5948d795c9-qtqlr:/usr/local/tomcat# ls
bin  BUILDING.txt  conf  CONTRIBUTING.md  include  lib  LICENSE  logs  native-jni-lib  NOTICE  README.md  RELEASE-NOTES  RUNNING.txt  temp  webapps  webapps.dist  work
root@tomcat-deployment-5948d795c9-qtqlr:/usr/local/tomcat# cd webapps
root@tomcat-deployment-5948d795c9-qtqlr:/usr/local/tomcat/webapps# ll
total 12
drwxr-xr-x 2 root root 4096 Mar 26 18:56 ./
drwxr-xr-x 1 root root 4096 Mar 26 18:56 ../
root@tomcat-deployment-5948d795c9-qtqlr:/usr/local/tomcat/webapps# mkdir app
root@tomcat-deployment-5948d795c9-qtqlr:/usr/local/tomcat/webapps# echo "A test message..." > app/index.html

访问http://192.168.100.164:30005/app测试：

四. nginx 配置动静分离

4.1 配置 nginx

4.1.1 查看 nginx 和 tomcat 情况

root@kube-master1:~# kubectl get pod -A | grep "nginx\|tomcat"
default                nginx-deployment-5d66cc795f-g7bmr            1/1     Running   0          67m
default                tomcat-deployment-5948d795c9-qtqlr           1/1     Running   0          67m
root@kube-master1:~# kubectl get service -A | grep "nginx\|tomcat"
default                suosuoli-nginx-service      NodePort    172.20.15.35   <none>        80:30004/TCP             76m
default                suosuoli-tomcat-service     NodePort    172.20.7.237   <none>        80:30005/TCP             75m

4.1.2 进入 nginx 容器配置

生产中是不会进入容器更改的，这样做只是为了实验需要：

# 进入nginx容器
root@kube-master1:~# kubectl exec -it nginx-deployment-5d66cc795f-g7bmr -n default -- /bin/bash
root@nginx-deployment-5d66cc795f-g7bmr:/# cat /etc/issue
Debian GNU/Linux 10 \n \l

# 安装工具
root@nginx-deployment-5d66cc795f-g7bmr:/# apt update
root@nginx-deployment-5d66cc795f-g7bmr:/# apt install procps vim iputils-ping net-tools curl

root@nginx-deployment-5d66cc795f-g7bmr:/# vim /etc/hosts
...
172.20.7.237   suosuoli-tomcat-service
...

# ping 一下tomcat的service名称是否通
root@nginx-deployment-5d66cc795f-g7bmr:/# ping suosuoli-tomcat-service
PING suosuoli-tomcat-service (172.20.7.237) 56(84) bytes of data.
...
root@nginx-deployment-5d66cc795f-g7bmr:/# ping suosuoli-tomcat-service.default.svc.suosuoli.local
PING suosuoli-tomcat-service.default.svc.suosuoli.local (172.20.7.237) 56(84) bytes of data.
...

# telnet访问一下
root@nginx-deployment-5d66cc795f-g7bmr:/# telnet suosuoli-tomcat-service 80
Trying 172.20.7.237...
Connected to suosuoli-tomcat-service.
Escape character is '^]'.
Connection closed by foreign host.

root@nginx-deployment-5d66cc795f-g7bmr:/# telnet suosuoli-tomcat-service.default.svc.suosuoli.local 80
Trying 172.20.7.237...
Connected to suosuoli-tomcat-service.default.svc.suosuoli.local.
Escape character is '^]'.
Connection closed by foreign host.

# 获取index内容试试
root@nginx-deployment-5d66cc795f-g7bmr:/# curl suosuoli-tomcat-service/app/index.html
A test message...

# 编辑nginx配置文件
root@nginx-deployment-5d66cc795f-g7bmr:/# vim /etc/nginx/conf.d/default.conf
...
location = /app {
      proxy_pass http://suosuoli-tomcat-service;
    }
...

# 重新加载配置文件
root@nginx-deployment-5d66cc795f-g7bmr:/# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

root@nginx-deployment-5d66cc795f-g7bmr:/# nginx -s reload
2020/03/29 11:16:24 [notice] 3322#3322: signal process started

4.2 访问测试

访问 nginx 首页：

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-fUZnA94N-1585788726736)(http://39.105.69.237/static/blog_images/article/k8s-basic/2020-03-29-19-47-43.png)]

访问请求 tomcat：

五. 通过 HAProxy 实现高可用和反向代理

于 haproxy 和 keepalived 实现高可用的反向代理，并访问到运行在 kubernetes
集群中业务 Pod。

5.1 HAProxy 和 Keepalived 配置

5.1.1 HAProxy

ha1:

root@ha1:~# vim /etc/haproxy/haproxy.cfg
root@ha1:~# cat /etc/haproxy/haproxy.cfg
...
listen stats
 	mode http
 	bind 0.0.0.0:9999
 	stats enable
 	log global
 	stats uri     /haproxy-status
 	stats auth    haadmin:stevenux

listen k8s-adm-6443
  mode tcp
  balance roundrobin
  bind 192.168.100.200:6443
  server 192.168.100.142 192.168.100.142:6443 check inter 3s fall 3 rise 5
  server 192.168.100.144 192.168.100.144:6443 check inter 3s fall 3 rise 5
  server 192.168.100.146 192.168.100.146:6443 check inter 3s fall 3 rise 5

listen k8s-suosuoli-80
  mode tcp
  balance roundrobin
  bind 192.168.100.201:80
  server 192.168.100.162 192.168.100.162:30004 check inter 3s fall 3 rise 5
  server 192.168.100.164 192.168.100.164:30004 check inter 3s fall 3 rise 5
  server 192.168.100.166 192.168.100.166:30004 check inter 3s fall 3 rise 5

root@ha1:~# systemctl restart haproxy

ha2:

root@ha2:~# vim /etc/haproxy/haproxy.cfg
root@ha2:~# cat /etc/haproxy/haproxy.cfg
...
listen stats
 	mode http
 	bind 0.0.0.0:9999
 	stats enable
 	log global
 	stats uri     /haproxy-status
 	stats auth    haadmin:stevenux

listen k8s-adm-6443
  mode tcp
  balance roundrobin
  bind 192.168.100.200:6443
  server 192.168.100.142 192.168.100.142:6443 check inter 3s fall 3 rise 5
  server 192.168.100.144 192.168.100.144:6443 check inter 3s fall 3 rise 5
  server 192.168.100.146 192.168.100.146:6443 check inter 3s fall 3 rise 5

listen k8s-suosuoli-80
  mode tcp
  balance roundrobin
  bind 192.168.100.201:80
  server 192.168.100.162 192.168.100.162:30004 check inter 3s fall 3 rise 5
  server 192.168.100.164 192.168.100.164:30004 check inter 3s fall 3 rise 5
  server 192.168.100.166 192.168.100.166:30004 check inter 3s fall 3 rise 5

root@ha2:~# systemctl restart haproxy

5.1.2 Keepalived

ha1:

root@ha1:~# cat /etc/keepalived/keepalived.conf
global_defs {
    notification_email {
    root@localhost
    }
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id ha1.example.com
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0

vrrp_mcast_group4 224.0.0.18
vrrp_iptables
}

vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 80
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass stevenux
}
virtual_ipaddress {
    192.168.100.200 dev eth0 label eth0:0
    192.168.100.201 dev eth0 label eth0:1 # 新增的VIP
} }

ha2:

root@ha2:~# cat /etc/keepalived/keepalived.conf
global_defs {
    notification_email {
    root@localhost
    }
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id ha1.example.com
vrrp_skip_check_adv_addr
vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0

vrrp_mcast_group4 224.0.0.18
vrrp_iptables
}

vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 80
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass stevenux
}
virtual_ipaddress {
    192.168.100.200 dev eth0 label eth0:0
    192.168.100.201 dev eth0 label eth0:1
} }

5.2 访问 VIP 试试

访问 nginx 主页：

访问 tomcat 页面：

六. kubeadm 命令

kubeadm 命令参考

6.1 kubeadm init phase

kubeadm init phase preflight
kubeadm init phase kubelet-start
kubeadm init phase certs
kubeadm init phase kubeconfig
kubeadm init phase control-plane
kubeadm init phase etcd
kubeadm init phase upload-config
kubeadm init phase upload-certs
kubeadm init phase mark-control-plane
kubeadm init phase bootstrap-token
kubeadm init phase addon

6.2 kubeadm join phase

kubeadm join phase
kubeadm join phase preflight
kubeadm join phase control-plane-prepare
kubeadm join phase kubelet-start
kubeadm join phase control-plane-join

6.1 kubeadm token

kubeadm token create
kubeadm token delete
kubeadm token generate
kubeadm token list

6.2 kubeadm config

kubeadm config upload from-file
kubeadm config view
kubeadm config print init-defaults
kubeadm config print join-defaults
kubeadm config migrate
kubeadm config images list
kubeadm config images pull

6.3 kubeadm reset

该命令尽力还原由 kubeadm init 或 kubeadm join 所做的更改
kubeadm reset [flags]
选项:

--cert-dir string     默认值："/etc/kubernetes/pki"
# 存储证书的目录路径。如果已指定，则需要清空此目录。
--cri-socket string
# 要连接的 CRI 套接字的路径。如果为空，则 kubeadm 将尝试自动检测此值；仅当安装了多个CRI 或具有非标准 CRI 插槽时，才使用此选项。
-f, --force
# 在不提示确认的情况下重置节点。
-h, --help
# reset 操作的帮助命令
--ignore-preflight-errors stringSlice
# 错误将显示为警告的检查列表；例如：'IsPrivilegedUser,Swap'。取值为 'all' 时将忽略检查中的所有错误。
--kubeconfig string     默认值："/etc/kubernetes/admin.conf"
# 与集群通信时使用的 kubeconfig 文件。如果未设置该标志，则可以在一组标准位置中搜索现有的 kubeconfig 文件。
--skip-phases stringSlice
# 要跳过的阶段列表

6.4 kubeadm reset phase

kubeadm reset phase
kubeadm reset phase preflight
kubeadm reset phase update-cluster-status
kubeadm reset phase remove-etcd-member
kubeadm reset phase cleanup-node

stevenux
发布了118 篇原创文章 · 获赞 61 · 访问量 2万+
私信 关注

猜你喜欢

转载自blog.csdn.net/YouOops/article/details/105261595
今日推荐
《美国对全球网络空间安全与发展的威胁和破坏》报告发布
火速冲上 GitHub 热榜 —— 开源编程语言、框架哪有这么可爱？
北京人形机器人创新中心发布全球首个纯电驱拟人奔跑的全尺寸人形机器人“天工”
LFOSSA 源来如此公开课 | 掌握云原生未来：CNCF 认证全面攻略与备考秘籍
周排行
循环神经网络（rnn）讲解
Tigao教程四：单独的关节运动
金蝶K3WISE15.0-注册套打教程
如何在Mac上配置Kubernetes
Android应用结束自身进程的方法
SpringMVC学习 十三 拦截器栈
中国驻洛杉矶总领馆举行新春招待会
HttpClient get post 发送
11 - three.js 笔记 - 绘制三维字体模型
Mysql递归获取某个父节点下面的所有子节点和子节点上的所有父节点
每日归档
更多
2024-05-01(4)
2024-04-30(1)
2024-04-29(40)
2024-04-28(0)
2024-04-27(56)
2024-04-26(39)
2024-04-25(22)
2024-04-24(36)
2024-04-23(26)
2024-04-22(39)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022