Author: Chen Jinjian
personal blog: HTTPS: //jian1098.github.io
CSDN blog: https: //blog.csdn.net/c_jian
Jane book: https: //www.jianshu.com/u/8ba9ac5706b6
Contact: jian1098 @qq.com
About etcd
Introduction
etcdIt is an open source, highly available distributed key-value storage system developed using the Go language, which can be used to configure sharing and service registration and discovery.
Features
- Full replication: each node in the cluster can use the full archive
- High availability: Etcd can be used to avoid single points of failure of hardware or network problems
- Consistency: Every read will return the latest write across multiple hosts
- Simple: Includes a well-defined, user-oriented API (gRPC)
- Security: Implemented automated TLS with optional client certificate authentication
- Fast: benchmark speed of 10,000 writes per second
- Reliable: The Raft algorithm is used to achieve a strongly consistent and highly available service storage catalog
Cluster
As a high-availability key-value storage system, etcd is inherently designed for clustering. Since the Raft algorithm requires a majority of nodes to vote when making decisions, etcd generally recommends an odd number of nodes for cluster deployment, and the recommended number is 3, 5, or 7 nodes to form a cluster.
Service discovery
Service discovery is to solve one of the most common problems in distributed systems, that is, how can processes or services in the same distributed cluster find each other and establish a connection. Essentially, service discovery is to know whether there are processes in the cluster listening on udp or tcp ports, and you can find and connect by name. To solve the problem of service discovery, the following three pillars are needed, none of which are indispensable
-
Strongly consistent and highly available service storage catalog. Etcd based on the Raft algorithm is a highly consistent and highly available service storage directory.
-
A mechanism for registering services and monitoring the health status of services. Users can register services in etcd, and set the registered services, and
key TTLkeep the heartbeat of the service regularly to achieve the effect of monitoring the health status. -
A mechanism for finding and connecting services. Services registered under the subject specified by etcd (service catalog composed of service names) can also be found under the corresponding subject.
Core components
-
HTTP Server: Used to process API requests sent by users and requests for synchronization and heartbeat information from other etcd nodes.
-
Store: Used to process various functions supported by etcd, including data indexing, node status changes, monitoring and feedback, event processing and execution, etc. It is the specific implementation of most of the API functions provided by users by etcd.
-
Raft: The specific implementation of Raft's strong consistency algorithm is the core of etcd.
-
WAL: Write Ahead Log is the data storage method of etcd. In addition to storing the state of all data in memory and the index of the node, etcd uses WAL for persistent storage. In WAL, all data will be logged in advance before submission. Snapshot is a state snapshot taken to prevent excessive data; Entry represents the specific log content stored.
Install etcd
From the https://github.com/etcd-io/etcd/releasesget the latest version, download unpack etcdand etcdctltwo programs (linux and windows the same). Among them etcdis the binary file that runs the etcd service, which etcdctlis the official command line etcd client, which etcdctlcan be used to access the etcd service on the command line.
In order to facilitate the operation, the two files can be added to the system environment variable by soft link
ln -fs /root/eosio/2.0/bin/etcd /usr/local/bin/etcd
ln -fs /root/eosio/2.0/bin/etcdctl /usr/local/bin/etcdctl
View version
etcd --version
Start etcd
etcd
Optional parameters:
-name Node name, the default is UUID
-data-dir The directory where logs and snapshots are saved, the default is the current working directory
-addrThe published ip address and port. The default is 127.0.0.1:2379
-bind-addr The listening address used for client connections, the default is -addr configuration
-peers A comma-separated list of cluster members, for example, 127.0.0.1:2380,127.0.0.1:2381
-peer-addr The published IP address of the cluster service communication, the default is 127.0.0.1:2380.
-peer-bind-addr The listening address of the cluster service communication, the default is -peer-addr configuration
The above configuration can also set the configuration file, the default isetcd目录/etcd.conf
Key-value library operations
写（put）
etcdctl put name "hello world" //新增和更新都是put
Read (get)
etcdctl get name
Query by prefix
etcdctl get name --prefix //查找前缀为name的
Delete (del)
etcdctl del name
Transaction (txn)
Transactions in etcd are executed atomically, and only support expressions like if… then… else…
//先赋值
etcdctl put user1 bad
//开启事务
etcdctl txn --interactive
compares:
// 输入以下内容,注意=号两边要有空格，输入结束按两次回车
value("user1") = "bad"
//如果 user1 = bad，则执行 get user1
success requests (get, put, del):
get user1
//如果 user1 != bad，则执行 put user1 good
failure requests (get, put, del):
put user1 good
// 运行结果，执行 success
SUCCESS
user1
bad
Watch
// 当 stock1 的数值改变（ put 方法）的时候，watch 会收到通知，在这之前进程会阻塞
$ etcdctl watch stock1
// 新打开终端etcd 多台服务器集群
$ export ETCDCTL_API=3
$ etcdctl put stock1 1000
//在watch 终端显示
PUT
stock1
1000
$ etcdctl watch stock --prefix
$ etcdctl put stock1 10
$ etcdctl put stock2 20
Lease
Lease can set the expiration time of access
$ etcdctl lease grant 300 //创建一个300秒的租约
# lease 326963a02758b527 granted with TTL(300s)
$ etcdctl put sample value -- lease=326963a02758b527 //写入操作时将租约id为326963a02758b527的租约分配给sample
OK
$ etcdctl get sample
$ etcdctl lease keep-alive 326963a02758b520 //续约
$ etcdctl lease revoke 326963a02758b527 //手动释放租约
lease 326963a02758b527 revoked
# or after 300 seconds //自动释放租约
$ etcdctl get sample
Lease provides several functions:
- Grant: Assign a lease.
- Revoke: Release a lease.
- TimeToLive: Get the remaining TTL time.
- Leases: List all leases in etcd.
- KeepAlive: Automatically renew a lease at a fixed time.
- KeepAliveOnce: Renew a lease once.
- Close: It seems to close all leases established by the current client.
Distributed lock (lock)
Distributed lock, when one person operates, the other person can only see but not operate
# 第一终端
$ etcdctl lock mutex1
mutex1/326963a02758b52d
# 第二终端
$ etcdctl lock mutex1
# 当第一个终端结束了，第二个终端会显示
mutex1/326963a02758b53
Election
The election node is the leader, only the leader node has the write permission, and the ordinary node only has the read permission to ensure data consistency; the leader node will periodically send heartbeats to the ordinary nodes, and the new leader will be automatically elected when the ordinary nodes cannot receive the heartbeat
$ etcdctl elect one p1
one/326963a02758b539
p1
# another client with the same name blocks
$ etcdctl elect one p2
//结束第一终端，第二终端显示
one/326963a02758b53e
p2
Cluster status monitoring (endpoint)
Cluster health check
$ etcdctl --write-out=table endpoint status
$ etcdctl endpoint health
Snapshot
Used to save snapshots of etcd database
etcdctl snapshot save my.db
Snapshot saved at my.db
etcdctl --write-out=table snapshot status my.db
Cluster member management (Member)
Used to view, add, delete, update members
export ENDPOINTS=127.0.0.1:2379,127.0.0.1:2479,127.0.0.1:2579 //windows下export换成set
etcdctl member list -w table //成员列表-w可省略
etcdctl --endpoints=$ENDPOINTS member remove b9057cfdc8ff17ce //删除成员
etcdctl --endpoints=$ENDPOINTS member add cd3 --peer-urls=http://127.0.0.1:2180 //添加成员cd3为成员名
Start new node
etcd --name cd3 --listen-client-urls http://127.0.0.1:2179 --advertise-client-urls http://127.0.0.1:2179 --listen-peer-urls http://127.0.0.1:2180 --initial-advertise-peer-urls http://127.0.0.1:2180 --initial-cluster-state existing --initial-cluster cd2=http://127.0.0.1:2580,cd0=http://127.0.0.1:2380,cd3=http://127.0.0.1:2180,cd1=http://127.0.0.1:2480 --initial-cluster-token etcd-cluster-1
Go language operation etcd
connection
Download the driver package
go get github.com/coreos/etcd/clientv3
Connection service
cli, err := clientv3.New(clientv3.Config{
Endpoints: []string{
"localhost:2379"},
// Endpoints: []string{"localhost:2379", "localhost:22379", "localhost:32379"}
DialTimeout: 5 * time.Second,
})
Read and write
write
The first parameter is goroutinethe context Context, and the next two parameters are key and value.
kv := clientv3.NewKV(cli)
putResp, err := kv.Put(context.TODO(),"/test/key1", "Hello etcd!")
// PutResp: &{cluster_id:14841639068965178418 member_id:10276657743932975437 revision:3 raft_term:7 <nil>}
General query
getResp, err := kv.Get(context.TODO(), "/test/key1")
Back to structure
type RangeResponse struct {
Header *ResponseHeader `protobuf:"bytes,1,opt,name=header" json:"header,omitempty"`
// kvs is the list of key-value pairs matched by the range request.
// kvs is empty when count is requested.
Kvs []*mvccpb.KeyValue `protobuf:"bytes,2,rep,name=kvs" json:"kvs,omitempty"`
// more indicates if there are more keys to return in the requested range.
More bool `protobuf:"varint,3,opt,name=more,proto3" json:"more,omitempty"`
// count is set to the number of keys within the range when requested.
Count int64 `protobuf:"varint,4,opt,name=count,proto3" json:"count,omitempty"`
}
The Kvs field saves all the kv pairs found in this Get. Because the above example only gets a single key, you only need to judge whether len(Kvs) is equal to 1 to know whether the key exists.
Search by prefix
rangeResp, err := kv.Get(context.TODO(), "/test/", clientv3.WithPrefix())
Paging query
RangeResponse.MoreAnd Count, it will work when we use withLimit()other options to proceed Get, which is equivalent to page-turning query.
op operation
Op literally means "operation". Both Get and Put belong to Op, which are just special APIs open to simplify user development.
The parameter Op is an abstract operation, which can be Put/Get/Delete...; and OpResponse is an abstract result, which can be PutResponse/GetResponse...
Op can be created by some methods defined in Client:
- func OpDelete (key string, opts… OpOption) Op
- func OpGet(key string, opts …OpOption) Op
- func OpPut(key, val string, opts …OpOption) Op
- func OpTxn (cmps [] Cmp, thenOps [] On, elseOps [] On) On
In fact, it is no different from calling KV.Put and KV.GET directly.
cli, err := clientv3.New(clientv3.Config{
Endpoints: endpoints,
DialTimeout: dialTimeout,
})
if err != nil {
log.Fatal(err)
}
defer cli.Close()
ops := []clientv3.Op{
clientv3.OpPut("put-key", "123"),
clientv3.OpGet("put-key"),
clientv3.OpPut("put-key", "456")}
for _, op := range ops {
if _, err := cli.Do(context.TODO(), op); err != nil {
log.Fatal(err)
}
}
lease
Create a lease
grantResp, err := lease.Grant(context.TODO(), 10)
Assign lease
kv.Put(context.TODO(), "/test/vanish", "vanish in 10s", clientv3.WithLease(grantResp.ID))
If Lease has expired before Put, then this Put operation will return an error, and you need to reassign Lease at this time
Renew
keepResp, err := lease.KeepAliveOnce(context.TODO(), grantResp.ID)
If Lease has expired before execution, then Lease needs to be re-allocated. etcd does not provide an API to implement atomic Put with Lease, we need to judge err to reallocate Lease.
Affairs
txn := kv.Txn(context.TODO())
kv.Txn(context.TODO()).If(
clientv3.Compare(clientv3.Value(k1), ">", v1),
clientv3.Compare(clientv3.Version(k1), "=", 2)
).Then(
clientv3.OpPut(k2,v2), clentv3.OpPut(k3,v3)
).Else(
clientv3.OpPut(k4,v4), clientv3.OpPut(k5,v5)
).Commit()
Similar to clientv3.Value()\ used to specify key attributes, there are several methods:
- func CreateRevision(key string) Cmp: The creation version of key=xxx must meet...
- func LeaseValue(key string) Cmp: Lease ID with key=xxx must satisfy...
- func ModRevision(key string) Cmp: The last modified version of key=xxx must meet...
- func Value(key string) Cmp: The creation value of key=xxx must satisfy...
- func Version(key string) Cmp: The cumulative update times of key=xxx must meet...
monitor
Watch is used to monitor the changes of a certain key Watchand returns one after the call WatchChan. Its type is declared as follows:
type WatchChan <-chan WatchResponse
type WatchResponse struct {
Header pb.ResponseHeader
Events []*Event
CompactRevision int64
Canceled bool
Created bool
}
Reference article
https://juejin.im/post/5dba5bedf265da4d461eb8ff#heading-3
https://zhuanlan.zhihu.com/p/38300827