architecture design
1.7.4-HOTFIX-T4 version package layout and brief meaning
After reading the brief introduction of the whole package, its core functional modules start from the commonly used project xml configuration, which is easy for us to understand. as follows:
jsf-provider.xml configuration
Take the jsf-provider.xml file of our address service as an example, namely:
As you can see, in the JSF configuration file, we don't see anything about the registration center. After all, as an efficient RPC call framework independently developed by the group, its high-availability registration center is the most important thing, so with this doubt, continue to explore how these tags complete the service without the address of the registration center Sign up, subscribe.
Configuration parsing
In Spring's system, Spring provides support for extensible Schema, that is, custom tag parsing.
1. First, we found the custom xsd file in the configuration file, and found the NamespaceUri link on the tag name http://jsf.jd.com/schema/jsf/jsf.xsd
2. Then load according to SPI, find the defined Spring.handlers file and Spring.schemas file in META-INF, one is the configuration of the specific parser, and the other is the specific path of jsf.xsd
Spring.handlers文件内容：
http\://jsf.jd.com/schema/jsf=com.jd.jsf.gd.config.spring.JSFNamespaceHandler
--------------------------------------------------------
Spring.schemas文件内容：
http\://jsf.jd.com/schema/jsf/jsf.xsd=META-INF/jsf.xsd
3. Therefore, we further inquire about inheriting NameSpaceHanderSupport or implementing the interface class corresponding to NameSpaceHandler. In our jsf framework, JSFNamespaceHandler is implemented by inheriting the former (NameSpaceHanderSupport), namely:
com.jd.jsf.gd.config.spring.JSFNamespaceHandler
[Supplement] The function of NamespaceHandler is to parse our custom JSF namespace. For convenience, we implement NamespaceHandlerSupport, which internally processes specific tags through BeanDefinitionParser, that is, specifically handles the tags we define.
com.jd.jsf.gd.config.spring.JSFBeanDefinitionParser#parse
4. Finally, these configurations (that is, the tag values we configure in xml) will be parsed into ServerConfig and ProviderConfig, and will assign attributes to the corresponding classes according to the configured attributes.
com.jd.jsf.gd.config.ServerConfig
com.jd.jsf.gd.config.ProviderConfig
initialization
OK, let's go back to JSFNamespaceHandler to see how the service is exposed. As we all know, beans in the Spring container must go through an initialization process. Therefore, implement org.springframework.beans.factory.xml.BeanDefinitionParser through com.jd.jsf.gd.config.spring.JSFBeanDefinitionParser to parse XML, and encapsulate the BeanDefinitionRegistry object in ParserContext for BeanDefinition registration. Initialize each bean, namely (com.jd.jsf.gd.config.spring.JSFBeanDefinitionParser#parse).
As follows: the bean class ProviderBean will listen to context events, and when the entire container is initialized, it will call the export() method to expose the service.
com.jd.jsf.gd.config.spring.ProviderBean
service exposure
We went back to the source code and found that its core code logic is as follows:
com.jd.jsf.gd.config.ProviderConfig#doExport
com.jd.jsf.gd.config.ProviderConfig#doExport The overall logic of this method is as follows:
1. First, perform various basic checks and interceptions, such as:
-
[JSF-21200] The alias of provider cannot be empty;
-
[JSF-21202] providerconfig server is empty;
-
[JSF-21203] Multiple providers of the same interface + alias are configured;
-
…
2. Secondly, get all the RegistryConfig. If you can’t get the registered address, then you will go to the default registry address: "i.jsf.jd.com".
com.jd.jsf.gd.config.RegistryConfig
3. Then get the server configured in the provider. If there is a server-related configuration in the provider, it is com.jd.jsf.gd.config.ServerConfig. At this time, the server will be started (serverConfig.start()), and the default corresponding The serialization method (serverConfig.getSerialization(), default msgpack) is used to register the service code.
com.jd.jsf.gd.config.ServerConfig#start (service start)
4. In the start method, the method in ServerFactory will be called to generate the corresponding object, and then the method in Server will be called to start the server. In this process, it will eventually correspond to ServerTransportFactory to generate the corresponding transport layer, which is to locate JSFServerTransport The start() method, here we can see that this part implements the transport layer of the netty framework. When entering this method, it will select whether the generated object is EpollServerSocketChannel or NioServerSocketChannel according to whether the epoll model is configured, and then in Initialize related parameters in ServerBootStrap until the port number is finally bound.
com.jd.jsf.gd.server.ServerFactory#initServer
com.jd.jsf.gd.transport.JSFServerTransport#start
5. Finally, through (this.register();) service registration and exposure, the object of the JsfRegistry class will be connected to the jsf registration center in the constructor of this class. If the registration center is not available, it will be generated and used The local file starts to guard the thread, and uses two thread pools to send heartbeat detection and retry mechanism, and another thread pool to detect whether the connection is successful (com.jd.jsf.gd.registry.JSFRegistry#addRegistryStatListeners).
com.jd.jsf.gd.registry.JSFRegistry
6. The process of service registration (com.jd.jsf.gd.registry.JSFRegistry#register) will convert the corresponding ProviderConfig into the JsfUrl class, where JsfUrl is the core of the entire framework, it saves a series of configurations, and communicates with other It is also important to subscribe to the Url class SubscribeUrl, where JsfUrl belongs to the service Url, which stores important information such as the protocol, port number, ip address, etc., and returns to the upper layer JsfContext to maintain the configuration information (JSFContext.cacheProviderConfig( this);). At this point (this.exported = true;) the provider's service is completed from configuration assembly to service exposure.
com.jd.jsf.gd.registry.JSFRegistry#register
com.jd.jsf.vo.JsfUrl
jsf-consumer.xml configuration
The above is the completion of the exposure of the provider service, so let's go back to the consumer and take a look, as follows:
We found the registration center address i.jsf.jd.com in the above configuration file, that is to say, the configuration related to service registration is not written in jsf-provider.xml, but only configured in jsf-consumer.xml.
Configuration parsing & initialization
The configuration parsing process is the same as above, so I won’t go into details. Finally, these configurations will be parsed into ConsumerConfig and RegistryConfig, and attributes will be assigned to the corresponding classes according to the configured attributes.
com.jd.jsf.gd.config.ConsumerConfig->AbstractConsumerConfig
The final initialization maps to the ConsumerBean class.
com.jd.jsf.gd.config.spring.ConsumerBean
service subscription
However, we found that it implements FactoryBean. As we know, if a bean implements the interface (FactoryBean), it is exposed as an object factory, not directly as a bean instance that exposes itself. This also means that getObject() needs to be called to get the real instantiated object (which may be shared or independent). The reason for this use is that our Consumer side can only call the interface, and the interface cannot be used directly. It needs to be encapsulated by a dynamic proxy to generate a proxy object, and then put it into the Spring container. Therefore, using FactoryBean is actually just for the convenience of creating proxy objects.
In the getObject () method, ConsumerBean will call the refer () method of the subclass consumerConfig, thus starting the initialization process of the client. During the refer() process, the consumer will subscribe to the relevant provider's services. The core code is as follows:
com.jd.jsf.gd.config.ConsumerConfig#refer
The overall logic of this method is as follows:
1. First, perform various basic checks and interceptions, such as:
-
The alias of the consumer cannot be empty, please check the configuration
-
The same interface+alias+protocol is configured locally more than three times, and a startup exception is thrown
-
…
2. The logic of some different configurations, such as whether to generalize calls, whether to use injvm calls, etc.
3. Generate a Client instance through the factory mode. Due to the default cluster policy failover of ConsumerConfig->AbstractConsumerConfig above, FailoverClient will be generated if there is no configuration, and then related Invoke operations will be performed (this.proxyInvoker = new ClientProxyInvoker(this) ;).
com.jd.jsf.gd.client.ClientFactory#getClient
[Supplement] The current cluster strategies supported by JSF are: failover: retry on failure (default); failfast: ignore failure; pinpoint: fixed-point call;
4. In the client, first define its load balancing, and then determine whether it is necessary to delay the establishment of a long link. Otherwise, the initial connection will be performed directly. Secondly, if no routing rules are defined, or there is no connection, the Client will first initialize the relevant routes and initialize the connection, as follows:
5. In the initialization connection (initConnections), the subscribe() in ConsumerConfig will be called to subscribe to the service, and during the initialization process, the consumer will connect to the corresponding Providers.
com.jd.jsf.gd.client.Client#initConnections
6. We see that in the process of obtaining a connection (connectToProviders()), if the connection pool already exists, it will return directly, and if it does not exist, the connection needs to be re-established. As follows: A thread pool named JSF-CLI-CONN-#interfaceId will be initialized, and corresponding tasks will be executed in the thread pool, that is, a ClientTransport object will be obtained.
com.jd.jsf.gd.client.Client#connectToProviders
If connected (com.jd.jsf.gd.transport.ClientTransportFactory#initTransport), then: 1) first determine the relevant protocol;
com.jd.jsf.gd.transport.ClientTransportFactory#initTransport
2) Then BuildChannel establishes a connection (com.jd.jsf.gd.transport.ClientTransportFactory#BuildChannel), where it will correspond to the relevant parameters of the server-side startup code, that is, it will listen to the server-side port number and bind to the corresponding IP, and do data transmission according to the corresponding channel.
com.jd.jsf.gd.transport.ClientTransportFactory#BuildChannel
3) And, for the transport layer, we should pay attention to which pipelines it injects: you can see that JSFEncoder/JSFDecoder is used to decode and encode specific protocols. Handler is a specific channel processor for processing heartbeat packets, client requests, and message routing. Finally, go back to the top layer (com.jd.jsf.gd.config.ConsumerConfig#refer) to maintain the configuration file (JSFContext.cacheConsumerConfig(this);).
com.jd.jsf.gd.transport.ClientChannelInitializer#initChannel
To sum up, I briefly explained the initialization and loading methods of the respective functions, etc., but the reading must be fragmented, and we need to further explore according to the above steps and source code packages. In addition, we can also use the server startup log to see the log records of the JSF core process. Some screenshots are as follows (if you are interested, you can read it in the startup log of the pre-released machine):
Server startup log (JSF)
In general, the relationship between its Provider, Consumer, and Registry should be as shown in the following figure:
Architecture
The first part: Provider starts the service and registers its own service with the registration center Session. The registration service form is as follows: [jsf://192.168.124.73:22000/?safVersion=210&jsfVersion=1691&interface=com.jd.wxt.material.service. DspTaskQueueService&alias=chengzhi36_42459] At this time, in the process of registering the service and exposing the service, the JsfRegistry will initialize the connection at this time. During this process, if there is a corresponding server side configured, the server will also be initialized.
The second part: when Consumer obtains entity information for the first time, because it is a FactoryBean, it must call the getObject() method to obtain the entity. During this process, the refer() method of ConsumerConfig will be called to register the client to jsf Register the center and subscribe to the changes of the Provider under the corresponding alias. In this process, the listeners on the Consumer and Provider sides will be initialized, and event monitoring will be performed on the relevant Consumers and Providers. This process will generate a Client object, and when the object is initialized, it will use a random load balancing strategy by default, and will initialize the route. After the route is initialized, the subscribe method of ConsumerConfig will be called, and then the client will be initialized. Server to connect.
The third part: The registration center will asynchronously notify the Consumer whether to re-subscribe to the Provider.
The fourth part: is to directly call the method invoke.
Part V: Monitor monitors services, manages or downgrades disaster recovery.
flow chart
The final call is as follows:
com.jd.jsf.gd.filter.FilterChain#invoke
ResponseMessage response = this.filterChain.invoke(requestMessage);
References
SPI： http://jnews.jd.com/circle-info-detail?postId=215333
NamespaceHandler: https://www.yisu.com/zixun/447867.html
jsf：https://cf.jd.com/pages/viewpage.action?pageId=132108361
netty：https://www.cnblogs.com/jing99/p/12515149.html
This time I will write it here first. If there are any questions in the article, please leave a message to correct me. I also hope to communicate with more like-minded partners. In the future, we will update the detailed analysis of modules such as JSF heartbeat detection, service governance, service anti-registration, and hook tools, as well as some experience sharing on the initial connection of our platform system to the wormhole platform (middleware mesh). Welcome everyone to like and follow.