etcd is a highly available distributed key (key-value) database. Internal etcd using raft protocol as the consensus algorithm, etcd based Go language.
etcd is a service discovery system, have the following characteristics:
Simple: simple installation configuration, but also provides HTTP API to interact is also very simple to use
Security: Support for SSL certificate validation
Quick: The benchmark official data, single-instance support 2k + read operations per second
Reliable: The raft algorithm, distributed systems of data availability and consistency
etcd scenarios
For service discovery, service discovery (ServiceDiscovery) to solve the distributed system is one of the most common problems, process or service that is distributed in the same cluster of how to find each other and establish a connection.
To solve the problem of service discovery, features three essential attributes required below.
A strong consistency, high availability storage service catalog.
Based etcd Ralf algorithm is inherently such a strong consistency, high availability storage service catalog.
A service and a mechanism for health services health registration.
Users can sign up for service in etcd, and the key TTL for service configuration register, the timings of holding services in order to achieve the effect of heart rate monitor health status.
A method of finding and mechanisms for service connections.
It can be found in the corresponding theme by registering at etcd subject specified services. To ensure connectivity, we can deploy etcd a proxy mode service on each machine, so that you can ensure access etcd cluster services are able to connect with each other.
K8s principle of etcd analysis
k8s cluster uses etcd as its back-end data, etcd is a stateless distributed data storage cluster data stored therein in the form of key-value. etcd colleagues today for the principle of operation of the cluster made a lecture, to summarize .
A. etcd organization of data
etcd The API is divided into two, respectively export ETCDCTL_API = 3 and export ETCDCTL_API = 2 to distinguish between. two different API call interface, the data organization or different at API_2, and its key value are stored in memory.
Down API_3, key stored in memory, value stored in the hard disk. Obviously, API_3 advantage, because the key is to be short compared to the value it more. Here is a more commonly used data in our discussion API_3 organization.
In etcd in, key stored as a B-tree in memory, value stored as a B + tree on the hard disk. Why should the Form B / B + tree to store it? This involves a All systems must face the question of how to spend less time
The data read out from the hard disk. As we all know, the computer's system memory, cache> >>> disk memory, that is to say for etcd, the access time to a maximum consumption data in disk access. So we must find ways to reduce access disk
Times. This time B / B + tree of superiority manifested. Analyze in detail below.
B / B + source tree model is AVL (balanced binary tree). For AVL, it each node stores only one data, thus for a very large AVL tree, the data access time complexity is log2 n. where n is the total number of data stored in the tree AVL tree. Suppose there is a total of 1023 data AVL, a data access worst case 10 requires access nodes. AVL tree node number unlike because between continuously stored in the memory element, which node 10 access operation
Is likely to contain multiple disk access, so slow down the access speed. For B / B + tree, the designers of each node size to a memory page size (typically 4kb), and a memory and tab size equivalent to the size of a disk block. Accordingly, B / B + AVL tree with respect to the advantages that it is, it reads data in the hard disk unit is a data block rather than a single data 4kb Thus, it data block read into memory and then further search, thus greatly reducing the number of disk I / O's.
About B / B + tree database system applications more detailed description of the Internet there are a lot of relevant information. Omitted. As for the difference between B-tree and B + tree, B + tree only stores data in a leaf node, the only non-leaf nodes storage search_key, B tree stored in the non-leaf nodes is the real data.
B. etcd in how to store a key-value
Understand the concept of B / B + tree, we analyze etcd how the data is stored to the hard disk. First, etcd There is a concept called revision, the revision can be understood as a global variable. Each time a user to perform an action, such as inserting a key-pair, this revision will be incremented by one, to be understood that this is a global revision ID, it indicates how many operations have been performed, each time the operation has a unique revision identified for the B-tree in memory for, search-key when it is used during lookup etcd key, stored in the node is revision information.
While the hard disk stored in B + trees is revision of search-key value which is stored in the node and key ETCD etcd value. With this organization, each of ETCD did save a history of each key.
At this point, we can sort out what etcd find keywords, for example "spe", of course, first of all etcd traverse B-tree based on "spe" to memory, to find the key corresponding to the revision, where revision is a set of numbers, including " spe "every time changes. from this group found that a revision of the largest, if a user specifies a revision, then you remove that user-specified then took revision B + tree to find the hard disk, followed by the node read into memory to find. until it reaches the leaf node, and ultimately find the desired value.