Distributed system characteristics and solutions
A scalability
Second, the availability of
three consistency
four, high-performance
A large site is a distributed system that contains many components that are part of a distributed system; and I now believe that a large site containing many components, each component is a distributed system, such as distributed storage is a distributed system, the message queue is a distributed system.
Scalability, performance, availability, consistency. This feature is also distributed several indicators to measure the system, precisely in order to meet these characteristics in varying degrees (or to achieve these targets), will design a variety of algorithms, protocols, and then based on business needs balance between these characteristics.
Why settle these characteristics, to meet the needs of these features solve any problem, any good solutions.
First, scalability
Scalability refers to when the system tasks (work) increases, the ability to cope with the task growth by increasing resources. Scalability is an essential characteristic of any distributed system, which is determined by the concept of distributed systems:
Distributed systems communicate over the network by a group of computer nodes in order to complete the tasks common coordination system composed of
the advent of distributed computing systems to solve a single computer can not be completed, storage tasks. Then when the scale of the task increases, inevitably need to add more nodes, this is scalability.
Target scalability is to make the nodes in the system are working in a more stable load, which is load balancing, of course, when the dynamic addition of nodes, the need for the task (may be calculated, may be data storage) migration, in order to achieve a dynamic equilibrium.
So first question to consider is how to split the task, assigning the task to a subset of each node, we call this process issue Partition (Sharding). About Partition, I actually "learning with a problem of fragmented data distributed systems" in the article are described in detail, it is summarized here.
First: fragment fraction, i.e., in what task splitting algorithm
common algorithms include: hash (the hash), consistent hashing (consistency hash), based on the data range (range based). Each method has its own advantages and disadvantages, also have their own application scenarios.
Second: fragment key, partition key
partition key value data is characteristic of any tile method mentioned above are dependent on the partition key, how to select it then
partition key task affects between slice balanced, and some systems (mongodb) almost can not re-select the partition key, so when you have to think clearly designed
Third: the added benefit of fragmentation
increase performance and concurrency: different requests are distributed to different slices
improve usability: a slice hung up without affecting other fragments
Fourth: fragmentation problem caused
if an operation spans multiple fragments, then the efficiency will be very low, the operating data such as join
Fifth: metadata management
metadata records the mapping between fragmentation and node, node status and other core information, distributed systems, a dedicated node (node cluster) to manage metadata, which we call the metadata server. Metadata server has the following characteristics:
High performance: cache
availability: fast failover redundancy plus
strong consistency (only one node at the same time provide services)
Sixth: dynamic balancing tasks
in order to achieve a dynamic equilibrium, the need for data migration, how to provide services guaranteed to remain in the process of migration, this is a complex issue carefully designed needs.
Second, the availability of
availability (Availability) system is the ability to continuously provide services, availability is a question of degree, the highest goal is 7 * 24, that is always online. But in fact impossible, usually with a few 9 to measure the availability of the system, as shown below:
That is, if you want to reach 4 availability 9 (99.99%), then a year only 52.6 minutes are not available, this is a huge challenge
why the distributed system must take into account the availability of it, this is because of the high probability of a distributed system failure. By a large number of heterogeneous distributed systems and network composed of nodes, the nodes may crash, power failure, disk corruption, network packet loss may, latency, network segmentation. Scale system amplifies the probability of failure, so the distributed system, failure is the norm. Then one goal is to design fault-tolerant distributed systems, in the case of partial failure still provide services, which is usability.
Redundancy is to improve the availability, reliability magic.
Redundancy means that more than one node is responsible for the same task, the need to maintain the state of the scene, such as distributed storage is widely used. In a distributed computing, such as MapReduce, when a worker runs abnormally slow, the Master will task the worker rescheduled to another worker, in order to improve system throughput, which is also considered a redundancy. But in terms of storage redundancy compared to many complex computing, so the main consideration of redundant storage.
Multiple nodes maintain the same data is called multiple copies. We consider a problem when writing data to the replica set, how to ensure the consistency of concurrency, data, whether there is a node has decided to update the order, which is the difference between centralized, decentralized, then a copy of the agreement.
Centralization and centralization to
decentralization is one master node (primary Master) responsible for updating the schedule data, the advantage of simple protocol, the concurrent operation into a sequence of operations, the disadvantage is primar may become a bottleneck, and a fault in the primary time there will be a time of re-election is not available.
Decentralization is that all nodes have equal status, can initiate the update data, the advantage of high availability, the disadvantage is complex protocol, to ensure consistency difficult.
Referred to the center, the more famous is the dynamo, cassandra, use a quorum, vector clock algorithms to try to ensure consistency in the decentralized environment. For this one to the center, it is currently not how to learn, so the following discussion of the main centers of the replica set. For centralized copy of the agreement, I "distributed learning center of the replication set with the problem," the article also described in detail in, simply summarized here.
Update Policy node
synchronization primary node to the secondary node or asynchronous data, that is, whether the client needs to wait for data to all nodes in the floor of the replica set.
Synchronization advantage that strong consistency, but the availability and performance (response delay) is poor; asynchronous opposite.
Data flow
that is how the data from the Primary node to the secondary node, and a master-slave mode have a chain.
When the chain take full advantage of the advantages of the network bandwidth and reduces primary pressure, but the drawback is write latency will be larger. GFS, MongoDB (by default) are chained.
Some nodes in writing the
theory, data from multiple nodes of a replica set should be consistent, and therefore theoretically write more data should be a transaction: either all happen or not happen. But the distributed transaction (such as 2pc) is a complex, inefficient process, updating the replica sets are generally best effort 1pc, if that fails, try again, or tell the application themselves.
primary elections
in the center of the copy of the agreement, how the primary node is elected, when the primary node hang up, but also how to choose a new primary node it, there are two ways: autonomous system, dependent on other system components. (Ps, the two names that I invented ...)
the so-called autonomous systems, internal node is its own voting options, such as mongodb, tfs, zookeeper
dependent on other system components, refer to primary to be appointed by the assembly after the replica set such as GFS appointed by the primary master (GFS metadata server), hdfs metadata namenode elected by the zookeeper heartbeat.
whether secondary external services (read the service)
center of the replication set, whether secondary reading services provide external, depending on the system requirements for consistency.
When such as the previously introduced to update the policy node may be asynchronous, so the data on the secondary compared to the primary there is some delay, read data from the secondary if not satisfy strong consistency requirement.
Such as metadata, it requires strong consistency guarantees, it is generally only read data from the primary. Moreover, the general said the master node is active (master), from the node to standby (slave). In this case, by adding fast failover redundancy to ensure availability.
Third, consistency
for high availability, the introduction of redundancy (copy) mechanism, and a copy of mechanisms to bring consistency problems. Of course, if there is no redundancy, redundant or not the data (state), then the consistency will not be problems, such as MapReduce.
The relationship between the consistency and availability in a distributed system, there have been enough studies, formed the CAP theory. That theory CAP distributed data storage, a partition can only satisfy both fault tolerance (P, Partition Tolerance) Consistency (C, Consistency), availability (A, Availability),. But the problem is a degree of consistency and availability of, is 0-1, and not only 0 and 1 two extremes.
From the perspective of the user and the consistency of the system point of view there are different levels.
Angle system consistency
strong consistency, weak consistency, the consistency of the final
The consistency of the user point of
monotony read consistency, consistency monotonous write, write consistency reading, read-after-write consistency
Fourth, high-performance
formal scale up because a single node can not complete the task, so we need to scale out, with a large number of nodes to complete the task, the ideal goal of distributed systems is the task of the node according to a certain proportion of linear growth.
Measure
high concurrent
high-throughput
low-latency
different systems focus on core indicators are not the same, such as MapReduce, itself is off-line calculation, no latency
Feasible way
scaleup single node
slice (Partition)
buffer: metadata such
short transactions