etcd集群部署及入门使用

环境准备

单节点启动

简单读写

环境准备

环境：etcd3.4.3 + ubuntu18.04

二进制安装

从github下载etcd3.4.3, 在本地解压后可直接使用etcd, etcdctl两个可执行文件, 可以把etcd, etcdctl直接放到/usr/local/bin下

$ etcd --version
etcd Version: 3.4.3
Git SHA: 3cf2f69b5
Go Version: go1.12.12
Go OS/Arch: linux/amd64

$ etcdctl version
etcdctl version: 3.4.3
API version: 3.4

etcd官方的demo已经详细介绍了入门级操作

单节点启动

为方便测试，采用非证书模式

./etcd \
--name 'demo-01' \
--data-dir '/etcd-v3.4.3-linux-amd64/etcdwork/data' \
--listen-client-urls 'http://0.0.0.0:2379' \
--advertise-client-urls 'http://0.0.0.0:2379',http://127.0.0.1:2379 \
--listen-peer-urls 'http://0.0.0.0:2380' \
--initial-advertise-peer-urls 'http://0.0.0.0:2380,http://127.0.0.1:2380'

查看节点信息

./etcdctl endpoint status --write-out=table

简单读写

./etcdctl put key1 "Hello world"
./etcdctl get key1

租约lease

#新建lease
./etcdctl lease grant 100  #TTL为100秒

#将key绑定到这个租约
./etcdctl put --lease=7f0c7789bd33d008 foo10 bar

#查看这个租约及其中绑定的key
./etcdctl lease timetolive 7f0c7789bd33d008
./etcdctl lease timetolive --keys 7f0c7789bd33d008

#刷新租约
./etcdctl lease keep-alive 7f0c7789bd33d00e

集群安装

因为条件受限（qiong），在单机上模拟集群，通过端口来区分etcd实例（“节点”）

“节点”	Node IP	Client Port	Peer Port
node1	10.x.x.17	12379	12380
node2	10.x.x.17	22379	22380
node3	10.x.x.17	32379	32380

从配置文件启动etcd各个节点

以下注意将ip修改为本机ip

node1配置文件

name: node1
data-dir: /etcd-v3.4.3-linux-amd64/cluster-test2/data/node-01
listen-client-urls: 'http://10.×.×.17:12379'
advertise-client-urls: 'http://10.×.×.17:12379'
listen-peer-urls: 'http://10.×.×.17:12380'
initial-advertise-peer-urls: 'http://10.×.×.17:12380'
initial-cluster: node1=http://10.×.×.17:12380,node2=http://10.×.×.17:22380,node3=http://10.×.×.17:32380
initial-cluster-token: etcd-cluster-1
initial-cluster-state: new

node2配置文件

name: node2
data-dir: /etcd-v3.4.3-linux-amd64/cluster-test2/data/node-02
listen-client-urls: 'http://10.×.×.17:22379'
advertise-client-urls: 'http://10.×.×.17:22379'
listen-peer-urls: 'http://10.×.×.17:22380'
initial-advertise-peer-urls: 'http://10.×.×.17:22380'
initial-cluster: node1=http://10.×.×.17:12380,node2=http://10.×.×.17:22380,node3=http://10.×.×.17:32380
initial-cluster-token: etcd-cluster-1
initial-cluster-state: new

node3配置文件

name: node3
data-dir: /etcd-v3.4.3-linux-amd64/cluster-test2/data/node-03
listen-client-urls: 'http://10.×.×.17:32379'
advertise-client-urls: 'http://10.×.×.17:32379'
listen-peer-urls: 'http://10.×.×.17:32380'
initial-advertise-peer-urls: 'http://10.×.×.17:32380'
initial-cluster: node1=http://10.×.×.17:12380,node2=http://10.×.×.17:22380,node3=http://10.×.×.17:32380
initial-cluster-token: etcd-cluster-1
initial-cluster-state: new

启动

nohup ./etcd --config-file=./cluster-test2/node1 &
nohup ./etcd --config-file=./cluster-test2/node2 &
nohup ./etcd --config-file=./cluster-test2/node2 &

查看节点状态

./etcdctl --endpoints="http://10.×.×.17:12379,http://10.×.×.17:22379,http://10.×.×.17:32379" endpoint status --write-out=table

查看集群

./etcdctl --endpoints="http://10.×.×.17:12379,http://10.×.×.17:22379,http://10.×.×.17:32379"  member list --write-out=table
# 指定集群中某个“节点”就可以了
./etcdctl --endpoints="http://10.×.×.17:32379"  member list --write-out=table

模拟节点故障及恢复

当前node1为leader，模拟node1故障

kill node1(10.×.×.17:12380)的etcd进程

重新选举后，node3(10.×.×.17:32380)被选举为leader，集群仍可用

日志输出：

d1f88dc1b11d98da became leader at term 5
d1f88dc1b11d98da 就是对应node3

node1重启恢复后，node1作为follower加入集群

节点的删除和添加

以下是当前集群

删除node1(端口12379，id为ec9e77ae631cb150)

& ./etcdctl --endpoints="http://10.x.x.17:22379,http://10.x.x.17:32379" member remove ec9e77ae631cb150

Member ec9e77ae631cb150 removed from cluster c975e2342085d03d

删除后集群

添加node4

新节点的端口为42379和42380

# 添加一个新节点
$ ./etcdctl --endpoints="http://10.x.x.17:22379,http://10.x.x.17:32379" member add node4 --peer-urls=http://10.x.x.17:42380
# 返回添加成功，已经启动参数
Member 3909c345000fe1e7 added to cluster c975e2342085d03d

ETCD_NAME="node4"
ETCD_INITIAL_CLUSTER="node4=http://10.x.x.17:42380,node2=http://10.x.x.17:22380,node3=http://10.x.x.17:32380"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.x.x.17:42380"
ETCD_INITIAL_CLUSTER_STATE="existing"

启动新节点node4

新节点的配置文件如下

根据上面member add返回的参数进行配置

注意：initial-cluster-state: existing

initial-cluster中增加node4=http://10.x.x.17:42380，删除原来node1=http://10.x.x.17:12380

# ./cluster-test2/node4 
name: node4
data-dir: /etcd-v3.4.3-linux-amd64/cluster-test2/data/node-04
listen-client-urls: 'http://10.x.x.17:42379'
advertise-client-urls: 'http://10.x.x.17:42379'
listen-peer-urls: 'http://10.x.x.17:42380'
initial-advertise-peer-urls: 'http://10.x.x.17:42380'
initial-cluster: node2=http://10.x.x.17:22380,node3=http://10.x.x.17:32380,node4=http://10.x.x.17:42380
initial-cluster-token: etcd-cluster-1
initial-cluster-state: existing

启动

nohup ./etcd --config-file=./cluster-test2/node4 &

node4添加后的集群：

注意：

如果添加新成员到一个节点的集群，在新成员启动前集群无法继续工作，因为它需要两个成员作为 galosh 才能在一致性上达成一致。这种情况仅会发生在 etcdctl member add影响集群和新成员成功建立连接到已有成员的时间内。 -- LG教育《etcd 原理与实践》