Huawei Cloud Yaoyun Server L instance evaluation | Installing etcd on Huawei Cloud

Huawei Cloud Yaoyun Server L instance evaluation | Installing etcd on Huawei Cloud

1. What is etcd

Official website: https://etcd.io/
github: https://github.com/etcd-io/etcd

etcd is a distributed key-value pair storage system developed by coreos. It uses the raft protocol internally as a consensus algorithm to reliably and quickly save key data and provide access. Reliable distributed collaboration is achieved through distributed locks, leader election, and write barriers.

Any application, from simple web applications to highly complex container orchestration engines such as Kubernetes, can use standard HTTP/JSON tools to read and write data in etcd.

Official hardware recommendations

Official hardware recommendation: https://etcd.io/docs/v3.5/op-guide/hardware/

etcd generally works well with limited resources for development or testing purposes; it is common to use etcd for development on a laptop or cheap cloud machine. However, some hardware guidelines are useful for proper management when running etcd clusters in production.

As shown above, for a small cluster, 2C8G is enough.

CPU:
Few etcd deployments require large amounts of CPU. A typical cluster requires two to four cores to run smoothly. Heavily loaded etcd deployments (serving thousands of clients or tens of thousands of requests per second) tend to be CPU bound because etcd can serve requests in memory. Such a heavy deployment typically requires eight to sixteen cores.

Memory:
etcd has a relatively small memory footprint, but its performance still depends on having enough memory. The etcd server will aggressively cache key-value data and spend most of the remaining memory tracking observers. Usually 8GB is enough. For heavy-duty deployments with thousands of observers and millions of keys, allocate 16GB to 64GB of memory accordingly.

Hard disk:
Fast disk is the most critical factor for the performance and stability of etcd deployment.

Slow disks increase etcd request latency and may harm cluster stability. Because etcd's consensus protocol relies on persisting metadata into a log, most etcd cluster members must write every request to disk. Additionally, etcd will incrementally checkpoint its state to disk so that it can truncate this log. If these writes take too long, the heartbeat may time out and trigger an election, destabilizing the cluster. Generally speaking, to determine whether the disk is fast enough for etcd, you can use benchmarking tools such as fio. Read an example here.

2. Huawei cloud host preparation

1. When purchasing a Huawei cloud host, the evaluation system is as follows:
   Note: In this article, we use the 2C4G environment for testing, not 2C2G~

2. Create a new security group and develop all ports for testing
   
   . Change the security group as follows, select our security group for development of all ports:
   

3. After developing all the ports, we can log in to the Huawei Cloud host via ssh~

3. etcd installation

Official installation documentation: https://etcd.io/docs/v3.5/install/
Official quick start documentation:

https://etcd.io/docs/v3.5/quickstart/

It is generally recommended to deploy etcd in a cluster mode in a production environment. A three-node cluster can achieve high availability, but there is a risk of downtime on a single node. This article is positioned as an introductory article to test and verify the installation of Huawei Cloud Yaoyun Server L instance. It mainly talks about single-node installation and basic use.

1. Install pre-built binaries

The easiest way to install etcd is to use the prebuilt binaries:

1. Download the compressed archive appropriate for your platform from Releases, select version v3.5.0 or higher.

Unzip the archive file. This will produce a directory containing the binaries.

Add the executable binary to your path. For example, rename and/or move the binary to a directory in your path (such as /usr/local/bin), or add the directory created in the previous step to your path.

mkdir -p etcd-download
cd etcd-download

ETCD_VER=v3.5.0

# choose either URL
GOOGLE_URL=https://storage.googleapis.com/etcd
GITHUB_URL=https://github.com/etcd-io/etcd/releases/download
DOWNLOAD_URL=${GOOGLE_URL}

rm -f etcd-${ETCD_VER}-linux-amd64.tar.gz

curl -L ${DOWNLOAD_URL}/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz -o etcd-${ETCD_VER}-linux-amd64.tar.gz

# 改完自己的安装目录
export INSTALL_PATH="/opt/lighthouse/server/env/"
mkdir -p $INSTALL_PATH
tar xzvf etcd-${ETCD_VER}-linux-amd64.tar.gz  -C $INSTALL_PATH/
mv $INSTALL_PATH/etcd-* $INSTALL_PATH/etcd


$INSTALL_PATH/etcd/etcd --version

2. Build from source

If you have Go version 1.16+, you can follow these steps to build etcd from source:

Download the etcd repository as a zip file and unzip it, or clone the repository using the following command.

$ git clone -b v3.5.0 https://github.com/etcd-io/etcd.git

To build main@HEAD from main, omit the -b v3.5.0 flag.

Change directory:

$ cd etcd

Run the build script:

$ ./build.sh

The binary files are located in the bin directory.

Add the full path to the directory bin to your path, for example:

$ export PATH="$PATH:`pwd`/bin"

Test what etcd has in your path:

$ etcd --version

4. etcd service registration and discovery

etcd is implemented based on Go language and is mainly used in shared configuration, service discovery, cluster monitoring, leader election, distributed locks and other scenarios.

etcd is a highly available, distributed key-value storage system that can be used to implement service registration and discovery.

In this article we focus on service registration and discovery~

Service registration and discovery refers to registering service instances to the service registration center so that other services can discover and use the service. etcd can be used as a service registration center to store the address of service instances and other related information.

When the service provider starts, it writes its own service instance information (such as IP address, port number, service name, health status, etc.) into etcd, and also specifies a TTL (Time To Live) expiration time. etcd will automatically maintain the heartbeat information of these service instances. If the heartbeat of a service instance times out, etcd will delete it from the registration center.

When a service consumer starts, it can query the instance information of the corresponding service through the API in etcd. Consumers can use this information to call service providers without the need to hardcode the service address.

1. Configure etcd

1. etcd configuration item explanation
   https://etcd.io/docs/v3.5/op-guide/configuration/This
   document explains the various configuration parameters of etcd in detail.
2. etcd production deployment recommendations
   https://etcd.io/docs/v3.5/op-guide/production/
   introduces the security, performance and other settings that need to be paid attention to when deploying etcd in a production environment.
3. etcd cluster setup example
   https://etcd.io/docs/v3.5/dev-guide/clusters/
   provides examples of etcd cluster configuration in different scenarios.

Enter the following command in the terminal to start the etcd service:

etcd

If you need custom configuration, you can use etcd command line parameters, for example:

etcd --name my-etcd --listen-client-urls http://0.0.0.0:2379 --advertise-client-urls http://0.0.0.0:2379

• –name my-etcd: Specifies the name of the etcd node, which should be unique in the cluster.
• –listen-client-urls http://0.0.0.0:2379: The listening address for client access, 0.0.0.0 represents all addresses, and 2379 is the default etcd client port.
• –advertise-client-urls http://0.0.0.0:2379: The externally announced client address, used to access the etcd node.

It should be noted that this is only the minimum configuration of a single etcd node. To build a cluster, other nodes and initial cluster information need to be configured. But the command contains the basic network and identification configuration of etcd nodes.

Of course, etcd supports the use of configuration files to manage service configuration. The command line parameter specifies the configuration file. When starting etcd, use the –config-file parameter to specify the configuration file:

etcd --config-file=/etc/etcd/etcd.conf.yml

Here is a single node and configuration demo:

# 节点名称
name: 'etcd-node1'

# 数据目录
data-dir: /opt/lighthouse/server/env/etcd/etcd-data

# 快照设置
snapshot-count: 10000

# 心跳间隔
heartbeat-interval: 100

# 通讯和客户端监听地址
listen-peer-urls: http://0.0.0.0:2380
listen-client-urls: http://0.0.0.0:2379

# 初始集群配置
initial-advertise-peer-urls: http://0.0.0.0:2380
initial-cluster: etcd-node1=http://0.0.0.0:2380
initial-cluster-token: etcd-cluster-1
initial-cluster-state: new

# 启用详细日志
log-level: debug
log-dir: /opt/lighthouse/server/logs/etcd
log-file: etcd.log
log-max-size: 100
log-backups: 3
log-format: text

# 压缩设置
auto-compaction-mode: periodic
auto-compaction-retention: "1"

# 安全传输
#client-transport-security:
#  trusted-ca-file: /path/to/ca.crt
#  cert-file: /path/to/etcd.crt
#  key-file: /path/to/etcd.key
#
#peer-transport-security:
#  trusted-ca-file: /path/to/ca.crt
#  cert-file: /path/to/etcd.crt
#  key-file: /path/to/etcd.key

2. Use systemctl to manage and start the etcd service

[Unit]
Description=etcd key-value store
Documentation=https://etcd.io/docs/
After=network.target

[Service]
CPUAccounting=true
MemoryAccounting=true
CPUQuota=200%
MemoryLimit=2G
Type=notify
ExecStart=/opt/lighthouse/server/env/etcd/etcd  --config-file=/opt/lighthouse/server/conf/etcd/etcd.conf.yml
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

Use: –config-file= to specify the configuration file for startup.

Configure as systemd service:

# etcd 数据目录 $ETCD_DIR/etcd-data
sudo mkdir -p $ETCD_DIR/etcd-data


sudo rm -rf /etc/systemd/system/etcd.service
sudo cp $SERVER_CONF_PATH/etcd.service /etc/systemd/system/etcd.service

sudo systemctl daemon-reload
sudo systemctl enable etcd
sudo systemctl restart etcd

3. Registration service

When a service provider starts, it needs to write its own service instance information into etcd. You can use the etcdctl command line tool.

etcdctl put /services/my-service/1 '{"host":"192.168.0.1","port":8080}'

The above command registers a service instance named my-service into etcd, with an IP address of 192.168.0.1 and a port number of 8080.

etcd supports client libraries in multiple programming languages, such as Go, Java, Python, etc. Service registration and discovery can be made more convenient using client libraries.

4. Discovery services

When a service consumer starts, it can query service instance information from etcd.

./etcdctl get /services/my-service --prefix

The above command will query all service instance information named my-service. You can automatically monitor changes in service instances through the –watch parameter of etcdctl.

5. Other references

etcd service registration and discovery
reference URL: https://zhuanlan.zhihu.com/p/625634172