- statefulset容器IP地址的变化不可控,所以证书必须实时生成,当前的做法是将一个自认证的证书打包进etcd镜像,起pod时自己生成,而访问etcd所需的证书也需要从容器中获取。
- 当前三台机器都只对某一块nas盘有访问权限,所以每个pod的落点都是固定的,如etcd-0对应100.68.34.8,若有偏差只能手动调整。
etcd参数只使用了最基本的参数,docker打包镜像的代码和部署文件已添加在附件
环境在
[root@ecam40931 etcd]# kubectl get pod -o wide|grep etcd
etcd-0 1/1 Running 0 16m 172.1.50.2 100.68.34.8
etcd-1 1/1 Running 0 16m 172.1.34.2 100.68.34.9
etcd-2 1/1 Running 0 16m 172.1.95.2 100.68.34.10
当前已经可使用加密方式正常访问
[root@ecam40931 etcd]# kubectl exec -it etcd-0 -- sh
/ # etcdctl --ca-file /etc/etcd/ssl/ca.pem --key-file /etc/etcd/ssl/etcd-key.pem --cert-file /etc/etcd/ssl/etcd.pem --endpoints=https://172.1.34.2:2379 cluster-health
member 1293bb6c66f7bfa1 is healthy: got healthy result from https://172.1.34.2:2379
member 5fefc8eefc1469cb is healthy: got healthy result from https://172.1.50.2:2379
member e38762190fc12c09 is healthy: got healthy result from https://172.1.95.2:2379
cluster is healthy
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv0001
spec:
capacity:
storage: 100Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
storageClassName: nas-etcd
nfs:
path: /csp_csmp_id100020_vol1004_prd
server: 100.68.21.4
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv0002
spec:
capacity:
storage: 100Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
storageClassName: nas-etcd
nfs:
path: /csp_csmp_id100020_vol1005_prd
server: 100.68.21.4
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv0003
spec:
capacity:
storage: 100Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
storageClassName: nas-etcd
nfs:
path: /csp_csmp_id100020_vol1006_prd
server: 100.68.21.4
---
apiVersion: v1
kind: Service
metadata:
name: etcd
namespace: kube-system
spec:
selector:
app: etcd
clusterIP: None
ports:
ports:
- port: 2379
targetPort: 2379
name: port2379
- port: 2380
targetPort: 2380
name: port2380
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: etcd
namespace: kube-system
spec:
serviceName: "etcd"
replicas: 3
template:
metadata:
labels:
app: etcd
spec:
terminationGracePeriodSeconds: 10
nodeSelector:
caas_cluster: storage
# host_name: ecam41060
containers:
- name: etcd
image: hub.yun.paic.com.cn/etcd:test
ports:
- containerPort: 2379
name: port2379
containerPort: 2380
name: port2380
volumeMounts:
- name: datadir
mountPath: /var/lib/etcd
volumeClaimTemplates:
- metadata:
name: datadir
namespace: etcd
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 10Gi
storageClassName: nas-etcd