A brief introduction to the implementation principle of dubbo A brief introduction to the implementation principle of dubbo

1. What is dubbo

Dubbo is Alibaba's open source distributed service framework. Its biggest feature is that it is structured in a layered manner, which can decouple (or maximize loose coupling) between layers. From the perspective of service model, Dubbo adopts a very simple model, either the provider provides the service or the consumer consumes the service, so based on this, the service provider (Provider) and the service consumer can be abstracted (Consumer) Two roles. For details about the registry, protocol support, service monitoring, etc., please refer to the following description.

Overall architecture

The overall architecture of Dubbo, as shown in the figure:

The Dubbo framework design is divided into 10 layers. The top Service layer is the interface layer for developers who actually want to use Dubbo to develop distributed services to implement business logic. The light blue background on the left in the figure is the interface used by the service consumer, the light green background on the right is the interface used by the service provider, and the interface located on the central axis is the interface used by both parties.
Below, combined with Dubbo's official documents, let's understand the design points of each level in the framework's layered architecture:

Service interface layer (Service): This layer is related to the actual business logic, and the corresponding interface and implementation are designed according to the business of the service provider and service consumer.
Configuration layer (Config): External configuration interface, centered on ServiceConfig and ReferenceConfig, you can directly create new configuration classes, or you can generate configuration classes through spring parsing configuration.
Service Proxy Layer (Proxy): Transparent proxy of service interface, which generates the client Stub and server Skeleton of the service, with ServiceProxy as the center, and the extension interface is ProxyFactory.
Service registration layer (Registry): Encapsulates the registration and discovery of service addresses, centered on the service URL, and the extended interfaces are RegistryFactory, Registry, and RegistryService. There may be no service registry, in which case the service provider directly exposes the service.
Cluster layer (Cluster): encapsulates routing and load balancing of multiple providers, and bridges the registration center, with Invoker as the center, and the extended interfaces are Cluster, Directory, Router, and LoadBalance. Combining multiple service providers into one service provider realizes transparency to service consumers and only needs to interact with one service provider.
Monitoring layer (Monitor): Monitoring the number of RPC calls and call time, with Statistics as the center, and the extended interfaces are MonitorFactory, Monitor and MonitorService.
Remote call layer (Protocol): Encapsulates RPC calls, centered on Invocation and Result, and extends the interface to Protocol, Invoker, and Exporter. Protocol is the service domain, it is the main function entry exposed and referenced by Invoker, and it is responsible for the life cycle management of Invoker. Invoker is the entity domain, it is the core model of Dubbo, other models rely on it, or convert to it, it represents an executable body, and can initiate invoke calls to it, it may be a local implementation, or it may be Is a remote implementation, and possibly a cluster implementation.
Information exchange layer (Exchange): encapsulates the request response mode, synchronously to asynchronous, centered on Request and Response, and the extended interfaces are Exchanger, ExchangeChannel, ExchangeClient and ExchangeServer.
Network transport layer (Transport): abstract mina and netty as unified interfaces, with Message as the center, and extended interfaces as Channel, Transporter, Client, Server, and Codec.
Data serialization layer (Serialize): Some reusable tools, extended interfaces are Serialization, ObjectInput, ObjectOutput and ThreadPool.

According to the official description, the description of the relationship between the above layers is as follows:

In RPC, Protocol is the core layer, that is, as long as there is Protocol + Invoker + Exporter, non-transparent RPC calls can be completed, and then Filter intercepts points on the main process of Invoker.
The Consumer and Provider in the figure are abstract concepts, and I just want the viewer to understand more intuitively which categories belong to the client and the server. The reason why the Client and Server are not used is that Dubbo uses Provider, Consumer, Registry, Monitor divides logical top nodes and maintains a unified concept.
Cluster is a peripheral concept, so the purpose of Cluster is to disguise multiple Invokers as one Invoker, so that other people only need to pay attention to the Protocol layer Invoker. Adding or removing Cluster will not affect other layers, because only one provides Otherwise, a Cluster is not required.
The Proxy layer encapsulates the transparent proxy of all interfaces, while the other layers are centered on Invoker. Only when it is exposed to the user, the Proxy is used to convert the Invoker into an interface, or the interface implementation is converted into an Invoker, that is, to remove the Proxy Layer RPC can be run, but it is not so transparent, and it does not seem to tune remote services like local services.
The Remoting implementation is the implementation of the Dubbo protocol. If you choose the RMI protocol, the entire Remoting will not be used. The Remoting is divided into the Transport transport layer and the Exchange information exchange layer. The Transport layer is only responsible for one-way message transmission. The abstraction of Netty and Grizzly, it can also extend UDP transport, and the Exchange layer encapsulates Request-Response semantics on top of the transport layer.
Registry and Monitor are not actually a layer, but an independent node, just for a global overview, drawn together in layers.

As a distributed service framework, Dubbo mainly has the following core points:

Service Definition
Services are built around service providers and service consumers. The service provider implements the service, and the service consumer invokes the service.

Service registration
service provider, it needs to publish services, and due to the complexity of the application system, the number and types of services are also expanding; for service consumers, it is most concerned about how to obtain the services it needs, and in the face of complex The application system needs to manage a large number of service calls. Moreover, for service providers and service consumers, they may also have both roles, that is, they need to provide services and consume services.
Through unified management of services, the process and management of service publishing/use by internal applications can be effectively optimized. The service registry can complete the external unification of services through specific protocols. The registration center provided by Dubbo has the following types to choose from:

Multicast Registry
Zookeeper Registry
Redis registry
Simple Registry

service monitoring

Both the service provider and the service consumer need to effectively monitor the actual status of service calls to improve service quality.

Telecommunication and information exchange

Long-distance communication needs to specify the protocol agreed upon by both parties. On the basis of ensuring that both parties understand the semantics of the protocol, efficient and stable message transmission must also be ensured. Dubbo inherits the current mainstream network communication framework, mainly including the following:

Mina
Netty
Grizzly

service call

Let's take it directly from Dubbo's official website and take a look at the calling relationship between the service provider and the service consumer based on the RPC layer, as shown in the figure:

In the above figure, blue indicates interaction with the business, and green indicates only internal interaction with Dubbo. The calling process described in the above figure is as follows:

The service provider publishes the service to the service registry;
The service consumer subscribes to the service from the service registry;
The service consumer invokes the registered available service;
Node role description:

Provider: The service provider that exposes the service.

Consumer: The service consumer that invokes the remote service.

Registry: A registry for service registration and discovery.

Monitor: The monitoring center that counts the invocation times and invocation time of the service.

Container: The service runs the container.

Description of the calling relationship:

0. The service container is responsible for starting, loading, and running the service provider.

1. When the service provider starts, it registers the service it provides with the registry.

2. When the service consumer starts, it subscribes to the registration center for the services it needs.

3. The registry returns the service provider address list to the consumer. If there is a change, the registry will push the change data to the consumer based on the persistent connection.

4. The service consumer, from the provider address list, selects a provider to call based on the soft load balancing algorithm, and if the call fails, select another provider to call.

5. Service consumers and providers accumulate the number of calls and call times in memory, and regularly send statistical data to the monitoring center every minute.

Next, expand the above abstract call flow chart, as shown in detail:

Registration/deregistration service
The registration and deregistration of the service is for the role of the service provider, so the sequence diagram of the registration service and the deregistration service is as shown in the figure:

Service subscription/cancellation
In order to meet the needs of the application system, the service consumer may need to subscribe to the specified service published by the service provider from the service registry. When notified that the service can be used, the service can be called directly. Conversely, if a service is no longer needed, the service can be cancelled. Take a look at the corresponding timing diagram below, as shown in the figure:

Protocol Support
Dubbo supports multiple protocols as follows:

Dubbo Protocol
Hessian Protocol
HTTP protocol
RMI protocol
WebService protocol
Thrift Protocol
Memcached protocol
Redis protocol

In the communication process, different service levels generally correspond to different service qualities, so it is very important to choose an appropriate protocol. You can choose according to the creation of your application. For example, the use of the RMI protocol is generally restricted by the firewall, so for the external and internal communication scenarios, the RMI protocol should not be used, but the HTTP protocol or the Hessian protocol.

Reference supplement

Dubbo organizes each module in a package structure, each module and its relationship, as shown in the figure:

Can be organized by Dubbo's code (managed using Maven), compare to the modules above. Briefly describe the situation of each package:

dubbo-common Common logic modules, including Util classes and common models.
dubbo-remoting remote communication module, equivalent to the implementation of the Dubbo protocol, if the RPC uses the RMI protocol, this package is not required.
The dubbo-rpc remote call module, abstracting various protocols, and dynamic proxy, only includes one-to-one calls, and does not care about cluster management.
The dubbo-cluster cluster module disguises multiple service providers as one provider, including: load balancing, fault tolerance, routing, etc. The address list of the cluster can be statically configured or issued by the registry.
The dubbo-registry registry module, based on the clustering method of addresses issued by the registry, and the abstraction of various registries.
The dubbo-monitor monitoring module, which counts the number of service calls, the call time, and the service tracked by the call chain.
The dubbo-config configuration module is Dubbo's external API. Users can use Dubbo through Config to hide all the details of Dubbo.
The dubbo-container container module is a Standalone container that loads Spring Boot with a simple Main, because services usually do not need the features of web containers such as Tomcat/JBoss, so there is no need to use a web container to load services.

Reference: A brief introduction to the implementation principle of dubbo