Practice sharing from JD.com and Vipshop on microservice orchestration, API gateway, and continuous integration (Part 1)

Architect group exchange meeting: Choose one of the most popular technical topics in each issue to share practical experience.
Phase 3: Microservices. Microservice architecture has quickly attracted the attention of architects and technical decision makers in various fields due to its high elasticity, flexibility and efficiency. Its basic idea is to split a fat system into several small service components, and the communication between the components is done using a lightweight protocol. In this group exchange meeting, we will discuss the current practice of microservices in Internet companies.
Guests: Jingdong Zhang Geng, former Vipshop Shi Tingxin, Qiniu Chen Aizhen
This article is a compilation of this exchange, divided into two articles.

Round 1: Free Communication
Jingdong Zhang Geng: Hello everyone, I am Zhang Geng from the platform middleware of Jingdong Infrastructure Department. I am mainly responsible for Jingdong's service framework, configuration center and other projects. At the end of 2011, Jingdong carried out the technical transformation of .NET to Java, and at that time proposed the concept of SOA of interfaces. At that time, JD.com's business was developing very fast, and there were more and more projects, and service-oriented was an inevitable trend. There are two generations of Jingdong's service-oriented framework. The first generation started in 2012. We used an open source implementation, using Dubbo as the RPC framework and Zookeeper as the registry. At that time, it should be considered a mainstream technology selection. On the basis of open source, we have done some ease of use and functional extensions, such as support for REST, support for serialization such as kryo/thrift, service monitoring, etc. This framework is relatively stable under the business scale and number of nodes at that time. At the beginning of 2014, we started to develop our own service framework. Why do it? One is that there is still a lot of room for improvement in the previous service framework, whether it is performance or some functions of service governance. Because of the number of machines in JD.com, the computer room is constantly being built, and the complexity of the network topology directly requires advanced service governance functions; Another is the increase in the order of magnitude of interface nodes. At that time, the scale of our interfaces has gradually reached several thousand, and the instances of interfaces are also hundreds of thousands; the other is that it is difficult to get through some internal systems with open source, so we have to upgrade. Replacement. At that time, I also considered the choice of using open source to make changes or doing it all by myself. Later, I felt that one was time, and the other was that I had the ability to create a customized service framework that conformed to JD.com, so in 2014, we started to develop our own framework for a whole year. At the beginning of 2015, we launched a new version of the service framework.

Our new registry is a stateless node with eventual consistency based on the database. Why choose a database to replace the previous Zookeeper? Because Zookeeper is a tree structure, it needs to traverse the entire number of trees to query it from some dimensions. The advantage of using a database is that it can be filtered from multi-dimensional query analysis. Whether it is a computer room or IP, you can query, analyze, filter and compare results. Then Zookeeper has a problem when it crosses computer rooms. It can only be used when half of the nodes are alive. Therefore, if it is deployed across computer rooms, at least 3 computer rooms are required to ensure the overall availability of the cluster. There is also Zookeeper, which is strongly consistent and relies more on the network. If the network is not good, if the network is disconnected across the computer room, it is actually unavailable and cannot be read, so it will bring certain problems. Before using zookeeper to register the server, it was to write a temporary node, and the node would disappear automatically when the server died. But when the network is shaking, or there is a network failure between the server and Zookeeper, it will actually remove it accidentally, because Zookeeper thinks it is dead, but it is not necessarily really dead. So at this time, some auxiliary tools are needed to judge whether it is really dead. The first time I encountered it was the temporary node. Later, we changed it to a permanent node, and then telnet its port regularly. For example, Dubbo directly executes the ls command. We also directly execute a command to see if it has any Respond, that's when it was the first generation. There is a sentinel component in the JSF framework. Our Provider will send heartbeats regularly. For the registry, it knows the last heartbeat time, and then periodically brushes it into the database to see the situation of the sentinel. If there is no sentinel, the database It saves the last heartbeat time, and you can use the scheduled task to find it at this time. This is our earliest version. Later we improved, because sometimes the network is disconnected, the heartbeat time is not allowed. Therefore, an independent program is also distributed in each computer room, and it is used to judge whether it is available when there is no heartbeat. If the program in the same computer room cannot be connected, we will set it to an unavailable state. Double guarantee. In addition, when Zookeeper used to be a full list of services, for example, 1,000 units were added, and 1,001 units were issued. In the new version of the registration center, we pushed the changed part, and only added 1 unit, which greatly saved the data. push volume.

RPC framework, our internal name is Jeff, JSF, short for Jingdong Service Framework. We developed it ourselves based on Netty. In order to be compatible with the previous generation version and the previous dubbo protocol, we also use codeless intrusion; we support multiple protocols on the same port, including custom JSF protocol, http protocol, dubbo protocol, etc. At present, we only have C++ and Java clients, and if it is other languages ​​such as Golang, python, and PHP, we will let him send requests to one of our HTTP gateways, which are mainly forwarding. Converting the previous http request to the next JSF request is also based on Netty. Serialization defaults to Message Pack, which is a cross-language protocol and also supports hessian, json, Java, Protobuf, etc. The registry and the RPC framework are directly connected to each other and can carry out a communication.

Vipshop Shi Tingxin: Hello, everyone, I'm Shi Tingxin. I used to be in JD.com and Vipshop, and now I'm in home delivery. I'm basically in warehousing and logistics. In the past, in Asia One, the service was divided into modules according to the entire operation details of the warehouse, and each module was deployed separately. Each warehouse in the warehouse is different according to the type, and the entire operation process is different. But the core things, such as inventory and order management, are the same, but some combinations are different. For example, small items need to be put on and off the shelves, but for large items, such as major appliances, basically no need to put them on the shelves. You don't need to place an order. You only need an order to complete the delivery. In fact, the order is bound at the end. Therefore, each process is different, and the required combination is also different. At that time, we thought of dismantling each module first, mainly using KVM to run the entire node, and using these nodes to connect the entire business part. The core things, such as inventory, orders, and commodity information, basically use this simple module.

Vipshop's warehousing is also done in this way. At that time, Vipshop used Dropwizard. In fact, we used one of its shells, and we still used Spring later. We also made a module to integrate Spring into one. Later, Springboot made it and basically abandoned Dropwizard. Because Dropwizard version upgrade changes are relatively large. But it has an advantage that the Bundle is better. When the service is split, it can be divided and combined according to its size. If it does not need to be split, we will combine the Bundle into one piece. If it needs to be split, It is very easy to separate the Bundle and separate the application.

The home delivery I arrived last year, our service registration, service discovery, and gateway all use this set of Spring Cloud. So far, it's still good, and basically no major problems have been found. The only problem is that if the entire data extraction is not done well in the early stage, the entire report may be more troublesome.

Qiniu Chen Aizhen: Hello everyone, I'm Chen Aizhen from Qiniu. The design concept of the microservice architecture is very suitable for Qiniu's business characteristics, and each service is only responsible for a single responsibility. For example, audio and video processing services: audio and video transcoding, audio and video splicing, video frame thumbnails, and on-demand streaming transcoding are all constructed in the form of microservices, so that each service has an independent operating environment and can be Its own business pressure can be independently expanded, and dynamic elastic expansion can also improve resource utilization. A landing microservice architecture should provide an independent operating environment, scheduling framework, registration center, configuration management center and monitoring platform for microservices. Qiniu adopts the architecture of Mesos+Docker+ self-developed scheduling system. Docker is the environment sealer, Mesos is the resource scheduling, and the self-developed scheduling system is responsible for flexible scheduling of Docker.

We use Consul as a registry to implement service registration and discovery. Consul comes with its own key/value store, can perform service discovery through the DNS interface, and has specific health check functions, and supports service discovery across data centers. API Gateway can query the list of all available instances of the service through the DNS interface provided by Consul, and forward the request. Traffic forwarding has the function of load balancing, using the polling method, and service discovery is based on Consul. After the user's request comes in, the access addresses of all available nodes are queried through Consul, and then the request is sent to the back-end service for processing by polling. The returned result is only forwarded, which is interpreted and used by the requester. And there is a monitoring component in the API gateway, which monitors the number of requests, the number of failures, etc., and transmits it to the Prometheus server. Through monitoring data, it performs corresponding processing such as flow control and service degradation on requests.

When multiple microservices need to be called, we use pipelines to perform serial processing according to the business characteristics of Qiniuyun's data processing. The output of each microservice is the input of the next microservice, and the final data processing is not until the end of the execution of the last microservice. For example, after uploading a video resource, two data processing operations need to be performed: converting to mp4 resource and performing HLS slicing. In this scenario, two data processing operations cannot be performed on the original resource at the same time, and must be performed in sequence.
  
The second round: topic exchange
Moderator : How to manage the dependencies between services? How to do service orchestration? String these service nodes together.

Shi Tingxin from Vipshop: For example, it is not a business, and basic services, such as image servers, are independent. The only thing that is relevant is that there are calls between the business layer below and the basic services below, and there are calls of services between business modules, which are not reflected in the system, but we add between services and services. The default thing similar to fuse. For the dependencies between systems, we have made a simple system for maintenance. Spring cloud eureka is stored in memory, we have improved it, and I have changed it to a database. All our interactions are Rest, and we also wrote a set later, Google protobuf. The internal business modules of our warehouse are done with Google protobuf. We made our own set of virtual ones inside.

Shi Tingxin from Vipshop: I am currently using Apache camel for orchestration, and I can use DSL descriptions to string together services. In fact, it is more about how to organize the business process and how to string together basic services to form a truly large business scenario.

JD Zhang Geng: At present, most of JD’s interface and one method already operate multiple tables. It is generally considered to be a relatively atomic operation. If there is another thing outside to wrap it, for example, three methods are wrapped. In fact, it is of little significance to become a method, so we are not doing this work now. In addition, JD.com has an elastic cloud and uses Docker, but it is not the Docker usage of the micro-service you imagine. JD.com now uses something called "fat container", which is more like a virtual machine. Docker runs an application, and this application may contain multiple interfaces and multiple methods. Maybe everyone understands that the Docker usage of microservices should be in a container, which only runs an independent logic, an operation on data, or an operation on a resource. For example, in these two steps, placing an order and deducting inventory, he may have run two containers. Arranging it into a service, we don't seem to have this.

Moderator: How is the service split done?
JD.com Zhang Geng: JD.com's current service is not too detailed, mainly because the business department controls the service granularity by itself. JD’s business is still relatively complex. Various applications depend on each other and call each other. Each operation basically involves the change of many resources. There is basically no operation on a single resource like the microservices. To be honest, the services in such a large Internet company are not trivial at all. It is impossible to perform a put post operation on a resource like RESTful. We are still the R&D personnel using our framework. Then they are generally combined or split according to some correlation or atomicity or extensibility between the methods. Therefore, we generally call ourselves a service framework, not a microservice framework. In addition, they may also consider whether some services can be deployed independently, which may be considered when dismantling.

Moderator: In fact, many large-scale e-commerce Internet can not be disassembled, nor can it be called micro-services. It is the kind of service that is very strong, and then each service number has several interfaces. In fact, they are highly coupled together. Related The degree is high, and the data is all together.

JD.com Zhang Geng: It's almost the same, all of which are based on business development to divide the granularity of services. Just like I said before placing an order. There must be many tables to be operated at one time. Business development thinks that several table operations are actually an atomic operation of placing an order, so he breaks it down to such a small detail.

Shi Tingxin from Vipshop: Our service is not one for each department. In fact, it is related to the data domain, and they are all put together.

Vipshop Shi Tingxin: According to the function, the function is called between functions. If it is not in the same data domain, it is still called by RPC.

Jingdong Zhang Geng: That sounds very good. You think that we have dismantled it like this, and there are already tens of thousands of interfaces, and there are basically 100,000 methods. If you dismantle it like this, at least multiply the interface by 10, and if you do it like this, the entire company will have to use a very, very large capacity to do it well.

Moderator: If a single unit is disassembled into the kind of microservices advocated by the market, in fact, it will consume a lot of internal energy, because many of our internal calls are actually very frequent. If they are all disassembled If it is deployed independently, it actually requires very high network consumption.

Shi Tingxin from Vipshop: It's the most difficult point. You can't open the data at first.

Moderator: As long as the data can be disassembled, you can do anything. As long as you can break down the data granularity very finely, then there is no problem, and you can do it finely, but you cannot break it apart. In fact, a lot of data are old watches, which are mostly inherited from the original system, so it is difficult to disassemble.

Vipshop Shi Tingxin: So it's okay to make a new system. It is difficult to do this old system plan, and it takes many days to disassemble a watch.

JD.com Zhang Geng: And the amount of data is relatively small, and the logic is relatively simple, such as the kind of start-up companies that use open source solutions, which can be very convenient.

Moderator: During Double 11, various e-commerce companies will definitely have large-scale promotions. At this time, the pressure on services will definitely increase greatly. How to solve the problem of service expansion?

Shi Tingxin from Vipshop: Just like what we said here, it depends on which base point is more stressful, and then you can add a few more base points. Now it's basically the one from China and Thailand, which is the Spring cloud set. Expansion is a step ahead. Automatic expansion is equivalent to renting a monitor similar to a simple one. It is based on his example, and because of microservices, we are now Spring cloud, basically java-jar, and then you can start it yourself. Anyway, you can put that jar in a public place, and the remote java-jar needs to be installed. Just get up, monitor according to his amount, and then you can start it at any time.

JD.com Zhang Geng: JD.com's service release is all linked to the application, and the platform where the application is released will actually be connected to various systems, such as the system authorized by the database, and then automatically linked to some VIP systems, monitoring platforms, log systems, etc. . Then we also have a monitoring platform to monitor some indicators, such as machine conditions, application conditions, and the performance of our own business sites. As for whether the service is under pressure, it is all self-assessment and self-expanding in advance. Generally, a large-scale pressure test will be carried out before the big promotion. JD.com does not have automatic elastic expansion. Basically, they apply for containers in advance before the big promotion, and then expand them in advance.

Shi Tingxin from Vipshop: Businesses with a large amount of concurrency are placed on the public cloud. Double 11 has twice as many machines as usual. We only bought cloud hosts for one month. Temporary use for a few days, and then it's over.