With its complex concepts, distributed systems involve many technologies, theories, and protocols, forming a huge knowledge system that troubles beginners. How to see the leopard in the spot and sort out the context from these complex information is the key to understanding and learning the idea of distributed system.
Here, according to the abstract thinking rules for understanding complex problems, I will analyze my understanding of distributed systems.
1. Abstract Razor Thought
According to Occam's razor philosophy, the way to understand a problem is to use the razor of thinking to cut off all the information that has nothing to do with the essence, in order to understand the essence of a thing. So, how to identify which are the contextual information of irrelevant things?
The key lies in the angle of looking at things. From different angles, different information can be obtained. However, for a system with a huge amount of information, from another angle, it is like a mirror of Wanhua, unable to touch the fundamental idea. So, what we need is abstraction.
Abstraction is actually looking at problems from a higher dimensional perspective. Our basic idea for solving mathematical problems is to use high-dimensional solutions to solve problems with huge low-dimensional computations. Abstraction can make things three-dimensional. In higher dimensions, complex low-dimensional problems and information can be reduced by one dimension. Clearly simplify and grasp the essence of things.
The so-called abstract razor is to first abstract complex problems to a higher level to observe and understand, and then you can get closer to the essence of things.
2. Abstractly understand the idea of distributed systems
The idea of distributed systems involves a lot of technologies and theories. At first glance, it is difficult to understand the whole picture through partial learning. But we understand it abstractly and think about the meaning of distributed systems from a holographic perspective .
Distributed system, in essence, is to solve the dilemma of limited ability to expand a single resource, using clusters to expand system capabilities.
Clustering is not simply a matter of splicing resources. The focus of distributed thinking is how to organize clusters.
The goal of organizing a cluster is to make the parts of the cluster truly become a whole. There are three elements to truly becoming a whole:
1. Whether there is communication (information exchange, registration center)
2. Coordination (distributed locks, distributed transactions)
3. Unified scheduling (a registry that implements distributed locks)
These three aspects contain all the aspects to be solved by distributed thinking.
The first is to communicate with each other. That is, the communication problem between the clusters, which involves the use of the message middleware MQ (Message Queue) technology . The message middleware is used as the message channel of each part to ensure the information exchange of each part of the cluster. Consistency is guaranteed. Common message queue technologies include ActiveMQ, RabbitMQ, kafka, etc.
Or set up a unified rpc registry like the dubbo framework, and communicate between various parts of the microservice cluster through the producer register service and the consumer subscribe service.
Second, coordination. It refers to solving the multi-threaded concurrency problem that occurs in each part of the cluster while using shared resources . That is to ensure that the behavior of using shared resources between clusters has the basic characteristics of transactions (atomicity, consistency, isolation, and durability), and the implementation methods are distributed locks and distributed transactions .
Distributed locks mainly include: distributed locks based on databases; distributed locks based on caches (redis, memcached, tair); distributed locks based on zookeeper.
The main implementation of distributed transactions is TCC transactions (trying, confirm, cancel), or using message queue technology to avoid distributed transactions.
Third, unified scheduling. That is, when using a cluster, how to schedule each part of the cluster and assign tasks. This requires implementing a distributed lock registry, subscribe and register services, and now the popular distributed lock implementation is zookeeper. Ali's open source distributed framework dubbo is a registration center implemented using zookeeper .
In this way, each technology can be roughly understood and its position in the distributed system.