默认的 dashboard 没啥用，我们用 kubesphere 可以打通全部的 devops 链路。
Kubesphere 集成了很多套件，集群要求较高 https://kubesphere.io/ Kuboard 也很不错，集群要求不高
官方文档：https://kuboard.cn/support/，但是我们也不用它。
一 、简介
KubeSphere 是一款面向云原生设计的开源项目，在目前主流容器调度平台 Kubernetes 之 上构建的分布式多租户容器管理平台，提供简单易用的操作界面以及向导式操作方式，在降 低用户使用容器调度平台学习成本的同时，极大降低开发、测试、运维的日常工作的复杂度。
二、安装
kubesphere官网：https://kubesphere.com.cn/
2.1 前提条件
https://kubesphere.io/docs/v2.1/zh-CN/installation/prerequisites/
2.2 安装前提环境
安装文档：https://helm.sh/zh/docs/intro/install/
## 在master节点安装 helm
curl -L https://git.io/get_helm.sh | bash
##
chmod 700 get_helm_ma.sh
2.2.1 安装 helm（master 节点执行）
Helm 是 Kubernetes 的包管理器。包管理器类似于我们在 Ubuntu 中使用的 apt、Centos 中使用的 yum 或者 Python 中的 pip 一样，能快速查找、下载和安装软件包。Helm 由客 户端组件 helm 和服务端组件 Tiller 组成, 能够将一组 K8S 资源打包统一管理, 是查找、共 享和使用为 Kubernetes 构建的软件的最佳方式。
错误代码:
SHA sum of /tmp/helm-installer-ezodFH/helm–linux-amd64.tar.gz does not match. Aborting
安装文档：https://helm.sh/zh/docs/intro/install/
二进制安装
第一步：下载 需要的版本
第二步：解压(tar -zxvf helm-v2.17.0-linux-amd64.tar.gz)
第三步：在解压目中找到helm程序，移动到需要的目录中(mv linux-amd64/helm /usr/local/bin/helm)
## 在mster节点随便找一个文件加解压这个文件
tar -zxvf helm-v2.17.0-linux-amd64.tar.gz
## 把helm 文件移动到/usr/local/bin/helm 目录下
mv linux-amd64/helm /usr/local/bin/helm
## 查看版本，会发现有两个版本，一个服务端的，一个客户端的。
helm version
kubectl get pods --all-namespaces
## 创建权限（master 执行）新建一个文件，helm-rbac.yaml 文件的内容在下方
vi helm-rbac.yaml
## 执行这个文件 新建tiller
kubectl apply -f helm-rbac.yaml
## 初始化装好 –tiller-image 指定镜像，否则会被墙，等待节点上部署的tiller完成即可。这个过程可能需要4-5分钟，请耐心等待
helm init --service-account=tiller --tiller-image=sapcc/tiller:v2.17.0 --history-max 300
helm-rbac.yaml 文件内容如下
apiVersion: v1
kind: ServiceAccount
metadata:
name: tiller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: tiller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system
过程：
第一步：下载 需要的版本
https://helm.sh/zh/docs/intro/install/
https://github.com/helm/helm/tags
第二步：解压(tar -zxvf helm-v2.16.3-linux-amd64.tar.gz)
第三步：在解压目中找到helm程序，移动到需要的目录中(mv linux-amd64/helm /usr/local/bin/helm)
验证，等待节点上部署tiller
2.2.2 存储类型
https://v2-1.docs.kubesphere.io/docs/zh-CN/appendix/install-openebs/
## 查看所有节点，找到主节点名称
kubectl get node -o wide
## 在主节点上查看有没有污点有污点，表示被人占用
kubectl describe node k8s-node1 | grep Taint
## 去除污点
kubectl taint nodes k8s-node1 node-role.kubernetes.io/master:NoSchedule-
## 创建 openebs 为存储器
kubectl apply -f https://openebs.github.io/charts/openebs-operator-1.5.0.yaml
## 查看安装情况
kubectl get pods --all-namespaces
## 安装 OpenEBS 后将自动创建 4 个 StorageClass
kubectl get sc --all-namespaces
##如下将 openebs-hostpath设置为 默认的 StorageClass
kubectl patch storageclass openebs-hostpath -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
## 将污点重新设置回来
kubectl taint nodes k8s-node1 node-role.kubernetes.io/master=:NoSchedule
## 测试存储类型是否安装成功demo-openebs-hostpath.yaml 这个文件下面有
kubectl apply -f demo-openebs-hostpath.yaml -n openebs
## 如果 PVC 的状态为 Bound并且 Pod 状态为 running，则说明已经成功挂载，证明了默认的 StorageClass（openebs-hostpath）是正常工作的
kubectl get pvc -n openebs
测试：如下创建一个 demo-openebs-hostpath.yaml，其中定义的 Deployment 与 PVC 用作测试，检验 openebs-hostpath 的 StorageClass 是否创建成功
## 注意，若您的 K8s 版本是 1.15，这里的 apiVersion 需要改为 apps/v1beta1
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: percona
labels:
name: percona
spec:
replicas: 1
selector:
matchLabels:
name: percona
template:
metadata:
labels:
name: percona
spec:
securityContext:
fsGroup: 999
tolerations:
- key: "ak"
value: "av"
operator: "Equal"
effect: "NoSchedule"
containers:
- resources:
limits:
cpu: 0.5
name: percona
image: percona
args:
- "--ignore-db-dir"
- "lost+found"
env:
- name: MYSQL_ROOT_PASSWORD
value: k8sDem0
ports:
- containerPort: 3306
name: percona
volumeMounts:
- mountPath: /var/lib/mysql
name: demo-vol1
volumes:
- name: demo-vol1
persistentVolumeClaim:
claimName: demo-vol1-claim
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: demo-vol1-claim
spec:
storageClassName: openebs-hostpath
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5G
---
apiVersion: v1
kind: Service
metadata:
name: percona-mysql
labels:
name: percona-mysql
spec:
ports:
- port: 3306
targetPort: 3306
selector:
name: percona
异常情况
https://blog.csdn.net/CSDN877425287/article/details/108267932
error: unable to recognize "demo-openebs-hostpath.yaml": no matches for kind "Deployment" in version "extensions/v1beta1"
error: unable to recognize "demo-openebs-hostpath.yaml": no matches for kind "Deployment" in version "apps/v1beta1"
error: unable to recognize "demo-openebs-hostpath.yaml": no matches for kind "Deployment" in version "apps/v1beta2"
这种异常情况是由于k8s的版本不同导致的
我这里使用的k8s版本为1.17.0
piVersion: apps/v1
kind: Deployment
从新apply即可
配置成功后的状态：
2.3 安装kubeshpere
由于完全安装需要：8核16g
所以我们选择最小安装
若集群可用 CPU > 1 Core 且可用内存 > 2 G，可以使用以下命令最小化安装 KubeSphere
安装文档：https://github.com/kubesphere/ks-installer/blob/master/README_zh.md
安装之前检查以下pod是否启动正常
2.3.1最小化快速部署
## 安装installer
kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.1.1/kubesphere-installer.yaml
## 查看ks-installer有没有安装完成
kubectl get pods --all-namespaces
## 安装kubesphere
kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.1.1/cluster-configuration.yaml
# 查看部署进度及日志
$ kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f
安装installer
查看ks-installer有没有安装完成
安装kubesphere
查看部署进度及日志（整个安装时间：20分钟左右）
2.4 访问测试
Collecting installation results ...
#####################################################
### Welcome to KubeSphere! ###
#####################################################
Console: http://10.0.2.10:30880
Account: admin
Password: P@88w0rd
NOTES：
1. After you log into the console, please check the
monitoring status of service components in
"Cluster Management". If any service is not
ready, please wait patiently until all components
are up and running.
2. Please change the default password after login.
#####################################################
https://kubesphere.io 2021-08-02 23:47:24
#####################################################
访问地址：我们的master地址是
http://192.168.56.100:30880
Account: admin
Password: P@88w0rd
把密码修改为：Admin123456
2.5 安装可插拔的功能组件
暂时我们安装两个：
1.安装内置的DevOps系统
2.安装告警通知系统
安装之前首先调整以下虚拟机的内存和cup核
为了正常安装插件，先设置两个节点的内存为12G 处理器为6核
master不用设置，因为它是负责调度的，node是负责干活的。
安装文档：https://kubesphere.com.cn/docs/pluggable-components/
我们只安装KubeSphere DevOps 和 KubeSphere 告警系统
https://kubesphere.com.cn/docs/pluggable-components/alerting/
## 安装kubesphere
kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.1.1/kubesphere-installer.yaml
## 安装组件
kubectl apply -f cluster-configuration.yaml
## 安装进度监控
kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f
安装过程
cluster-configuration.yaml
---
apiVersion: installer.kubesphere.io/v1alpha1
kind: ClusterConfiguration
metadata:
name: ks-installer
namespace: kubesphere-system
labels:
version: v3.1.1
spec:
persistence:
storageClass: "" # If there is no default StorageClass in your cluster, you need to specify an existing StorageClass here.
authentication:
jwtSecret: "" # Keep the jwtSecret consistent with the Host Cluster. Retrieve the jwtSecret by executing "kubectl -n kubesphere-system get cm kubesphere-config -o yaml | grep -v "apiVersion" | grep jwtSecret" on the Host Cluster.
local_registry: "" # Add your private registry address if it is needed.
etcd:
monitoring: false # Enable or disable etcd monitoring dashboard installation. You have to create a Secret for etcd before you enable it.
endpointIps: localhost # etcd cluster EndpointIps. It can be a bunch of IPs here.
port: 2379 # etcd port.
tlsEnable: true
common:
redis:
enabled: false
openldap:
enabled: false
minioVolumeSize: 20Gi # Minio PVC size.
openldapVolumeSize: 2Gi # openldap PVC size.
redisVolumSize: 2Gi # Redis PVC size.
monitoring:
# type: external # Whether to specify the external prometheus stack, and need to modify the endpoint at the next line.
endpoint: http://prometheus-operated.kubesphere-monitoring-system.svc:9090 # Prometheus endpoint to get metrics data.
es: # Storage backend for logging, events and auditing.
# elasticsearchMasterReplicas: 1 # The total number of master nodes. Even numbers are not allowed.
# elasticsearchDataReplicas: 1 # The total number of data nodes.
elasticsearchMasterVolumeSize: 4Gi # The volume size of Elasticsearch master nodes.
elasticsearchDataVolumeSize: 20Gi # The volume size of Elasticsearch data nodes.
logMaxAge: 7 # Log retention time in built-in Elasticsearch. It is 7 days by default.
elkPrefix: logstash # The string making up index names. The index name will be formatted as ks-<elk_prefix>-log.
basicAuth:
enabled: false
username: ""
password: ""
externalElasticsearchUrl: ""
externalElasticsearchPort: ""
console:
enableMultiLogin: true # Enable or disable simultaneous logins. It allows different users to log in with the same account at the same time.
port: 30880
alerting: # (CPU: 0.1 Core, Memory: 100 MiB) It enables users to customize alerting policies to send messages to receivers in time with different time intervals and alerting levels to choose from.
enabled: true # Enable or disable the KubeSphere Alerting System.
# thanosruler:
# replicas: 1
# resources: {}
auditing: # Provide a security-relevant chronological set of records，recording the sequence of activities happening on the platform, initiated by different tenants.
enabled: false # Enable or disable the KubeSphere Auditing Log System.
devops: # (CPU: 0.47 Core, Memory: 8.6 G) Provide an out-of-the-box CI/CD system based on Jenkins, and automated workflow tools including Source-to-Image & Binary-to-Image.
enabled: true # Enable or disable the KubeSphere DevOps System.
jenkinsMemoryLim: 2Gi # Jenkins memory limit.
jenkinsMemoryReq: 1000Mi # Jenkins memory request.
jenkinsVolumeSize: 8Gi # Jenkins volume size.
jenkinsJavaOpts_Xms: 512m # The following three fields are JVM parameters.
jenkinsJavaOpts_Xmx: 512m
jenkinsJavaOpts_MaxRAM: 2g
events: # Provide a graphical web console for Kubernetes Events exporting, filtering and alerting in multi-tenant Kubernetes clusters.
enabled: false # Enable or disable the KubeSphere Events System.
ruler:
enabled: true
replicas: 2
logging: # (CPU: 57 m, Memory: 2.76 G) Flexible logging functions are provided for log query, collection and management in a unified console. Additional log collectors can be added, such as Elasticsearch, Kafka and Fluentd.
enabled: false # Enable or disable the KubeSphere Logging System.
logsidecar:
enabled: true
replicas: 2
metrics_server: # (CPU: 56 m, Memory: 44.35 MiB) It enables HPA (Horizontal Pod Autoscaler).
enabled: false # Enable or disable metrics-server.
monitoring:
storageClass: "" # If there is an independent StorageClass you need for Prometheus, you can specify it here. The default StorageClass is used by default.
# prometheusReplicas: 1 # Prometheus replicas are responsible for monitoring different segments of data source and providing high availability.
prometheusMemoryRequest: 400Mi # Prometheus request memory.
prometheusVolumeSize: 20Gi # Prometheus PVC size.
# alertmanagerReplicas: 1 # AlertManager Replicas.
multicluster:
clusterRole: none # host | member | none # You can install a solo cluster, or specify it as the Host or Member Cluster.
network:
networkpolicy: # Network policies allow network isolation within the same cluster, which means firewalls can be set up between certain instances (Pods).
# Make sure that the CNI network plugin used by the cluster supports NetworkPolicy. There are a number of CNI network plugins that support NetworkPolicy, including Calico, Cilium, Kube-router, Romana and Weave Net.
enabled: false # Enable or disable network policies.
ippool: # Use Pod IP Pools to manage the Pod network address space. Pods to be created can be assigned IP addresses from a Pod IP Pool.
type: none # Specify "calico" for this field if Calico is used as your CNI plugin. "none" means that Pod IP Pools are disabled.
topology: # Use Service Topology to view Service-to-Service communication based on Weave Scope.
type: none # Specify "weave-scope" for this field to enable Service Topology. "none" means that Service Topology is disabled.
openpitrix: # An App Store that is accessible to all platform tenants. You can use it to manage apps across their entire lifecycle.
store:
enabled: false # Enable or disable the KubeSphere App Store.
servicemesh: # (0.3 Core, 300 MiB) Provide fine-grained traffic management, observability and tracing, and visualized traffic topology.
enabled: false # Base component (pilot). Enable or disable KubeSphere Service Mesh (Istio-based).
kubeedge: # Add edge nodes to your cluster and deploy workloads on edge nodes.
enabled: false # Enable or disable KubeEdge.
cloudCore:
nodeSelector: {
"node-role.kubernetes.io/worker": ""}
tolerations: []
cloudhubPort: "10000"
cloudhubQuicPort: "10001"
cloudhubHttpsPort: "10002"
cloudstreamPort: "10003"
tunnelPort: "10004"
cloudHub:
advertiseAddress: # At least a public IP address or an IP address which can be accessed by edge nodes must be provided.
- "" # Note that once KubeEdge is enabled, CloudCore will malfunction if the address is not provided.
nodeLimit: "100"
service:
cloudhubNodePort: "30000"
cloudhubQuicNodePort: "30001"
cloudhubHttpsNodePort: "30002"
cloudstreamNodePort: "30003"
tunnelNodePort: "30004"
edgeWatcher:
nodeSelector: {
"node-role.kubernetes.io/worker": ""}
tolerations: []
edgeWatcherAgent:
nodeSelector: {
"node-role.kubernetes.io/worker": ""}
tolerations: []
#####################################################
### Welcome to KubeSphere! ###
#####################################################
Console: http://10.0.2.10:30880
Account: admin
Password: P@88w0rd
NOTES：
1. After you log into the console, please check the
monitoring status of service components in
"Cluster Management". If any service is not
ready, please wait patiently until all components
are up and running.
2. Please change the default password after login.
#####################################################
https://kubesphere.io 2021-08-07 12:19:29
#####################################################
http://192.168.56.100:30880
三、kubesphere进阶
说明文档：https://v2-1.docs.kubesphere.io/docs/zh-CN/quick-start/wordpress-deployment/
WordPress 是使用 PHP 开发的博客平台，用户可以在支持 PHP 和 MySQL 数据库的环境中架设属于自己的网站。我们以部署wordpress（项目）和mysql（数据库）为例进行以下操作。
从上图可以看出我们需要配置4个东西，分别为：部署deployment，对外端口service，存储pvc，密钥secret
3.1 建立多租户系统和创建项目
给不同的用户分配不同的权限来管理项目
3.2 创建WordPress应用-密钥
为什么需要配置密钥，为了更安全，比如mysql的登录密码我们不能直接配置在配置文件中，而是要把它放在密钥中进行加密
1）mysql密钥
https://hub.docker.com
mysql镜像地址：https://hub.docker.com/_/mysql
添加密钥的数据：这个数据的键名需要更具docker镜像中mysql 需要的环境变量去设置
MYSQL_ROOT_PASSWORD：123456
2）wordpress密钥
wordpress镜像地址：https://hub.docker.com/_/wordpress
WORDPRESS_DB_PASSWORD 连接数据库的密码
3.3 创建WordPress应用-创建存储卷
1）创建mysql存储卷
2）创建wordpress存储卷
3）最终结果
3.4 创建WordPress应用-创建容器
我们需要部署两个容器，一个是mysql，一个是wordgress项目，mysql是有状态（数据库一般都是有状态的，项目是无状态的。）
3.4.1 部署mysql
第一步：部署mysql，填写基本信息
第二步：添加容器镜像
添加mysql镜像
mysql:5.6
第三步：挂载存储
1）mysql的挂在路径
/var/lib/mysql
需要去服务中看一下mysql
mysql-deploy-iy7c
3.4.2 部署wordpress
wordpress:4.8-apache
mysql的连接地址和密码
WORDPRESS_DB_HOST：mysql-deploy-nsd2（我这儿是个老图，按照自己配置拿到的名称写）
WORDPRESS_DB_PASSWORD
/var/www/html
3.5 创建WordPress应用-外网访问
此处我们的mysql不需要外网地址的访问，可以不操作，想操作也可以这样做
3.5.1 mysql外网访问
3.5.2 wordgress外网访问
第一步：设置基本信息
第二步：服务设置
第三步：高级设置
我再上一步的设置中 服务端口设置的不是80 而是 10086 这个端口没有实际作用。
3.5.3 测试
最后我们通过
192.168.56.100:31721
进行测试
http://192.168.56.100:31721/wp-login.php
四、DevOps 流水线
