Elasticsearch Deep (b)

Elasticsearch the basis of distributed architecture

Elasticsearch transparent hidden properties of complex distributed mechanism

Elasticsearch is a distributed system, distributed in response to large amount of data.

Elasticsearch hides the complexity of distributed mechanisms:

• Fragmentation: Before we casually inserted into the document will be some of the cluster es go, we do not care about how the data is distributed, the data to which shard went.
• Cluster discovery mechanism (cluster discovery): If you start a new process es, es then the process will be found as a node and cluster es, then automatically join in.
• Load balancing shard: For example, assume there are three nodes, a total of 25 shard to assign up to three nodes, will be evenly divided ES automatically allocated to ensure that a balanced load of read requests each node
• shard duplicate
• Request Routing
• Cluster expansion
• shard redistribution

Horizontal expansion and vertical expansion Elasticsearch

Expansion Plan:

　　6 servers, each hold 1T data immediately increase the amount of data you want to 8T, this time there are two options.

(1) Vertical expansion: re-purchase of two servers, each server's capacity is 2T, instead of the old two servers, the server 6 now total capacity is 4 * 1T + 2 * 2T = 8T.

(2) the level of expansion: the new purchase two servers, each server's capacity is 1T, added directly to the cluster to go, so now the total capacity of the server is 8 * 1T = 8T

Vertical expansion: purchase a more powerful server, the cost is very high, and there will be a bottleneck, assuming that the world's most powerful server capacity is 10T, but when you reach the total number 5000T time, how many you want to buy the most powerful server ah.

Horizontal expansion: the industry is frequently used programs, more and more ordinary server procurement, performance more general, but many ordinary servers grouped together, can constitute a powerful computing and storage capacity.

When data is increased or decreased or reduced rebalance node

For example, there are now four node, three of which node has a shard, 1 Ge node has two shard, but this time if there is a new node is added in, the es automatically to one shard assigned to the newly added node up.

master node

Es will always have a cluster node is a master node:

• Es cluster management metadata: for example, index creation and deletion, maintain index metadata; add and remove nodes, clusters of maintenance data
• By default, it will automatically select a node as a master node
• master node does not carry all of the requests, it will not be a single point of bottlenecks

Node distributed architecture equality

1. Peer nodes, each node can receive all requests
2. Route Request: Any node after receiving the request, the request can be automatically routed to the relevant node up to process the request.
3. Response collection: the most primitive node in response to receiving data from another node, then returns the data to the client.

primary shard and a replica shard again combing mechanism

1. An index (index) comprising a plurality of shard
   

    

2. Each shard is a minimum unit of work, carrying part of the data, Lucene instance, full index and the ability to handle requests.
   

    

3. When changes in the node, shard will automatically load balancing nodes in.

    

4. primary shard and replica shard, each document must exist only a certain primary shard and its corresponding replica shrad in, can not exist in a plurality of primary shard.
5. replica shard is the primary shard copy, responsible for fault tolerance, load and bear a read request.
6. The number of primary shard in the creation of the index is fixed, and the number of replica shard can be modified at any time.
7. The default number of primary shard is 5, the number of replica shrad default is 1.
8. primary shard and its replica shard can not be placed on the same node (or node goes down, primary shard and a replica shard were missing, would not achieve the role of fault tolerance.), but other primary shard and a replica shard is placed on the same node.

Create a single-node environment index look like?

1. Single-node environment, create an index: There are three primary shard, 3 Ge replica shard
   

   PUT /test_index
   {
      "settings" : {
         "number_of_shards" : 3,
         "number_of_replicas" : 1
      }
   }

2. 集群状态是yellow
3. 这个时候，只会将3个primary shard分配到仅有的一个node上去，另外3个replica shard是无法分配的
4. 集群可以正常工作，但是一旦出现节点宕机，数据全部丢失，而且集群不可用，无法承担任何请求
   

两个node环境下replica shard是如何分配的？ 

 此时的情况，1个node、3个primary shard、3个replica shard

如果此时新增一个node进来，构成了一个由2个node组成的es集群，如下:

并且：

1. primary shard会自动把数据同步到对应的replica shard上去
2. 客户端的读请求可以发送到primary shard上去，也可以发送到replica shard上去

Elasticsearch横向扩容

1. primary shard 和 replica shard自动负载均衡
   目前情况：2个node， 3个primary shard，3个replica shard
   
   
   如果此时给es集群增加一个节点(node),es会自动对primary shard和replica shard进行负载均衡
   
2. 每个Node有更少的shard, IO/CPU/Memory资源给每个shard分配更多，每个shard性能更好
3. 扩容的极限，6个shard(3个primary shard,3个replica shard),最多扩容到6台机器，每个shard可以占用单台服务器的所有资源，性能最好
   
4. 超出扩容极限，动态修改replica数量，9个shard（3primary，6 replica），扩容到9台机器，比3台机器时，拥有3倍的读吞吐量
5. 3台机器下，9个shard（3 primary，6 replica），资源更少，但是容错性更好，最多容纳2台机器宕机，6个shard只能容纳0台机器宕机
6. 这里的这些知识点，你综合起来看，就是说，一方面告诉你扩容的原理，怎么扩容，怎么提升系统整体吞吐量；另一方面要考虑到系统的容错性，怎么保证提高容错性，让尽可能多的服务器宕机，保证数据不丢失

Elasticsearch容错机制

1. master选举、replica容错、数据恢复 
   目前es集群情况：3个node，9个shard(3个primary shard,6个replica shard)
   

   如果此时master node宕机：
   

   因为Node1节点宕机了，所以primary shard0、replica shard1、replica shard2三个3shard就丢失了。master node宕机的一瞬间，primary shard0这个shard就没有了，此时就不是active status,所以集群的状态为red.

2. 容错第一步：master选举，自动选举另外一个node成为新的master，承担起master的责任来：
   

3. 容错第二步：新master将丢失的primary shard的某个replica shard提升为primary shard，此时cluster status会变为Yellow，因为所有的primary shard都变成了active status，但是，少了一个replica shard，所以不是所有的replica shard都是active。
   

4. 容错第三步：重启故障的node, new master节点将会把缺失的副本都copy一份到该node上去，而且该node会使用之前已有的shard数据，只是同步一下宕机之后发生的改变。
   

   此时es cluster的状态为green，因为所有的primary shard和replica shard都是active状态。