For etcd api v3 data backup and recovery methods
# export ETCDCTL_API=3
# etcdctl --endpoints localhost:2379 snapshot save snapshot.db （备份）
# etcdctl snapshot restore snapshot.db --name m3 --data-dir=/home/etcd_data （还原）
File recovery need to modify the permissions for ETCD: ETCD
-name: reassign a data directory, you can not specify, the default is default.etcd
-data-dir: Specifies the data directory
is recommended but do not specify the name specified data-dir use, and the data-dir corresponding to the data-dir arranged in the service etcd
Practices
Stand-alone backup
[root@k8s-master1 ~]# etcdctl --endpoints 127.0.0.1:2379 snapshot save snashot.db
Snapshot saved at snashot.db
[root@k8s-master1 ~]# ll
-rw-r--r-- 1 root root 3756064 Apr 18 10:38 snashot.db
[root@k8s-master1 ~]#
Cluster Backup
[root@k8s-master1 ~]# etcdctl --endpoints="https://192.168.32.129:2379,https://192.168.32.130:2379,192.168.32.128:2379" --cacert=/etc/kubernetes/cert/ca.pem --key=/etc/etcd/cert/etcd-key.pem --cert=/etc/etcd/cert/etcd.pem snapshot save snashot1.db
Snapshot saved at snashot1.db
[root@k8s-master1 ~]#
[root@k8s-master1 ~]# ll
-rw-r--r-- 1 root root 3756064 Apr 18 10:53 snashot1.db
-rw-r--r-- 1 root root 3756064 Apr 18 10:38 snashot.db
Data Recovery
Do the following operation, please carefully, it may cause a cluster crash data loss. Please test in a laboratory environment.
Execute the command: systemctl STOP ETCD
ETCD service all the nodes stopped.
Execute the command: rm -rf / var / lib / etcd /
all of the nodes delete data etcd
Recovery of data v3
[root@k8s-master1 ~]# etcdctl --name=k8s-master1 --endpoints="https://192.168.32.128:2379" --cacert=/etc/kubernetes/cert/ca.pem --key=/etc/etcd/cert/etcd-key.pem --cert=/etc/etcd/cert/etcd.pem --initial-cluster-token=etcd-cluster-1 --initial-advertise-peer-urls=https://192.168.32.128:2380 --initial-cluster=k8s-master1=https://192.168.32.128:2380,k8s-master2=https://192.168.32.129:2380,k8s-master3=https://192.168.32.130:2380 --data-dir=/var/lib/etcd snapshot restore snashot1.db
2019-04-18 13:43:42.570882 I | mvcc: restore compact to 148651
2019-04-18 13:43:42.584194 I | etcdserver/membership: added member 4c99f52323a3e391 [https://192.168.32.129:2380] to cluster 2a0978507970d828
2019-04-18 13:43:42.584224 I | etcdserver/membership: added member 5a74b01f28ece933 [https://192.168.32.128:2380] to cluster 2a0978507970d828
2019-04-18 13:43:42.584234 I | etcdserver/membership: added member b29b94ace458096d [https://192.168.32.130:2380] to cluster 2a0978507970d828
[root@k8s-master2 ~]# etcdctl --name=k8s-master2 --endpoints="https://192.168.32.129:2379" --cacert=/etc/kubernetes/cert/ca.pem --key=/etc/etcd/cert/etcd-key.pem --cert=/etc/etcd/cert/etcd.pem --initial-cluster-token=etcd-cluster-1 --initial-advertise-peer-urls=https://192.168.32.129:2380 --initial-cluster=k8s-master1=https://192.168.32.128:2380,k8s-master2=https://192.168.32.129:2380,k8s-master3=https://192.168.32.130:2380 --data-dir=/var/lib/etcd snapshot restore snashot1.db
2019-04-18 13:43:56.313096 I | mvcc: restore compact to 148651
2019-04-18 13:43:56.324779 I | etcdserver/membership: added member 4c99f52323a3e391 [https://192.168.32.129:2380] to cluster 2a0978507970d828
2019-04-18 13:43:56.324806 I | etcdserver/membership: added member 5a74b01f28ece933 [https://192.168.32.128:2380] to cluster 2a0978507970d828
2019-04-18 13:43:56.324819 I | etcdserver/membership: added member b29b94ace458096d [https://192.168.32.130:2380] to cluster 2a0978507970d828
[root@k8s-master3 ~]# etcdctl --name=k8s-master3 --endpoints="https://192.168.32.130:2379" --cacert=/etc/kubernetes/cert/ca.pem --key=/etc/etcd/cert/etcd-key.pem --cert=/etc/etcd/cert/etcd.pem --initial-cluster-token=etcd-cluster-1 --initial-advertise-peer-urls=https://192.168.32.130:2380 --initial-cluster=k8s-master1=https://192.168.32.128:2380,k8s-master2=https://192.168.32.129:2380,k8s-master3=https://192.168.32.130:2380 --data-dir=/var/lib/etcd snapshot restore snashot1.db
2019-04-18 13:44:10.643115 I | mvcc: restore compact to 148651
2019-04-18 13:44:10.649920 I | etcdserver/membership: added member 4c99f52323a3e391 [https://192.168.32.129:2380] to cluster 2a0978507970d828
2019-04-18 13:44:10.649957 I | etcdserver/membership: added member 5a74b01f28ece933 [https://192.168.32.128:2380] to cluster 2a0978507970d828
2019-04-18 13:44:10.649973 I | etcdserver/membership: added member b29b94ace458096d [https://192.168.32.130:2380] to cluster 2a0978507970d828
Service to get up
[root@k8s-master1 ~]# tail -n 30 /var/log/messages
Apr 18 13:46:41 k8s-master1 systemd: Starting Etcd Server...
Apr 18 13:46:41 k8s-master1 etcd: etcd Version: 3.3.7
Apr 18 13:46:41 k8s-master1 etcd: Git SHA: 56536de55
Apr 18 13:46:41 k8s-master1 etcd: Go Version: go1.9.6
Apr 18 13:46:41 k8s-master1 etcd: Go OS/Arch: linux/amd64
Apr 18 13:46:41 k8s-master1 etcd: setting maximum number of CPUs to 1, total number of available CPUs is 1
Apr 18 13:46:41 k8s-master1 etcd: error listing data dir: /var/lib/etcd
Apr 18 13:46:41 k8s-master1 systemd: etcd.service: main process exited, code=exited, status=1/FAILURE
Apr 18 13:46:41 k8s-master1 systemd: Failed to start Etcd Server.
Apr 18 13:46:41 k8s-master1 systemd: Unit etcd.service entered failed state.
Apr 18 13:46:41 k8s-master1 systemd: etcd.service failed.
Apr 18 13:46:41 k8s-master1 flanneld: timed out
Apr 18 13:46:41 k8s-master1 flanneld: E0418 13:46:41.858283 63943 main.go:349] Couldn't fetch network config: client: etcd cluster is unavailable or misconfigured; error #0: EOF
Apr 18 13:46:41 k8s-master1 flanneld: ; error #1: EOF
Apr 18 13:46:41 k8s-master1 flanneld: ; error #2: EOF
[root@k8s-master1 ~]#
Directory permissions to modify the data, the default is the root: the root
chown -R & lt ETCD: ETCD / var / lib / ETCD
normal.
[root@k8s-master1 ~]# etcdctl member list
4c99f52323a3e391, started, k8s-master2, https://192.168.32.129:2380, https://192.168.32.129:2379
5a74b01f28ece933, started, k8s-master1, https://192.168.32.128:2380, https://192.168.32.128:2379
b29b94ace458096d, started, k8s-master3, https://192.168.32.130:2380, https://192.168.32.130:2379
[root@k8s-master1 ~]#
V3 can see the data recovery success.
Reference address
https://blog.csdn.net/liukuan73/article/details/78986652
https://blog.51cto.com/goome/2380854
https://yq.aliyun.com/articles/336781