Practical sharing of microservice orchestration, API gateway, and continuous integration from JD.com and Zakisong (Part 2)

Architect group exchange meeting: Choose one of the most popular technical topics in each issue to share practical experience.

Phase 3: Microservices. Microservice architecture has quickly attracted the attention of architects and technical decision makers in various fields due to its high elasticity, flexibility and efficiency. Its basic idea is to split a fat system into several small service components, and the communication between the components is done using a lightweight protocol. In this group exchange meeting, we will discuss the current practice of microservices in Internet companies.

Guests: Zhang Geng from JD.com, Shi Tingxin from Home Delivery, Chen Aizhen from Qiniu
This article is a compilation of this exchange, divided into two articles.

The second round: topic exchange
Moderator : How is the API gateway designed?
Jingdong Zhang Geng: We have an HTTP gateway, but not a gateway for external network services. For external business, there are some gateways. For example, wireless has its own gateway, website has its own gateway, and another public gateway is a gateway called Jingdong Open Platform. Then what is our gateway used for? The main thing is a cross-language support, protocol forwarding and current limiting. The significance of the existence of this gateway is mainly to allow some callers to adjust the services of the server without using the Jeff protocol, or without relying on the server.

The most important thing is the forwarding function of converting HTTP into an internal protocol. Secondly, we have made an interface-level isolation on it, so that you don't want to hang up the gateway with just one interface. Another is the current limit, how many times per minute. There is also authorization. The previous practice of gateway forwarding is generally similar to a VIP, and then a domain name is attached to the front. Then the list of services hanging behind the virtual IP is generally maintained manually. And our online shopping is automatically linked to this gateway, which is a service discovery, and our results. In terms of statistics, we will package it uniformly. Our gateway mainly does this function, and now it should be billions of dollars a day. The number of calls, 90 units, is about the same.

Service discovery is to read the service list in the database, and after reading it from the registry, it will be pushed to our gateway. Gateways are similar to callers. It's actually a caller, but it's a proxy caller. Its service list is also subscribed from our registry. It is not directly connected to the database and can be considered as the caller. The gateway goes to the registration center to read it for the first time, and we can push the changed part later. For example, if you have 1000 units originally, if you add one, you will pull 1001 units according to the idea of ​​pulling before, and then you can compare it yourself and see which one is more. But we are not now, we are now adding one to him in reverse. In this case, a large reduction in that, as well as the push of network IO.

Jingdong Zhang Geng: We want to use nginx+luaw to do a HTTP transfer to our internal JSF protocol, but it is not a worker or a process. There will be a lot of long links, so we gave up later. We are now doing a forwarding based on nitty, that is, HTTP externally, and JSF protocol internally. It also does some authorization, current limiting, and thread isolation between services, service discovery, and one is the packaging of the result, which is packaged into a standard HTTP response. Because those on the Internet actually have their own systems, whether you are wireless or PC, they all have their own systems, and we don't need to do that. For the third party, it also has a development platform of JD.com, as well as a more stringent verification. Our API gateway is mainly for protocol conversion.


Moderator: How did you verify the legitimacy of the request, and what method did you use? Just how do you do that balance of efficiency and security?

JD. Zhang Geng: We have an authorization, that is, an application ID. JD.com has an application ID for each startup. If you come with that application ID, we can also support the token on your head. The kind that JD.com is open to is relatively strict to the outside world. We don’t need to be so strict. Anyway, it depends on who you are and who your gateway is used for.

Moderator: You now have two types, one is for internal calls, and the other is for external calls and internal calls to your system.
JD Zhang Geng: That is an open service. Some suppliers have internal systems. If you want to adjust JD's system, that is JD's open service, which requires Oauth authentication.

Jingdong Zhang Geng: HTTP+keepalive is also very fast, because it is nothing more than a bigger head, and the HTTP head is bigger. But if the background is to adjust redis, it will be more obvious. If the background is a few hundred milliseconds, then you will not feel so obvious. If the background, you just can't read it, read redis, you feel more obvious. Here we use netty to do HTTP and binary support on the same port.


Moderator: How do you divide, which ones use binary and which ones use restful protocol?

JD.com Zhang Geng: We don't have a mandatory requirement, and the business uses what it wants to use.

Jingdong Zhang Geng: For us, it supports these two protocols as soon as it starts the port. Start the same port, both protocols are supported.

Moderator: How do you distinguish between a port and a protocol?

Jingdong Zhang Geng: Isn't each data including the first two digits of the head a modulus? They all have their own modulo bits. Then our own protocol has its own modulo bits, and your HTTP is the H at the beginning, and then our decode is automatically loaded, it does not mean that you can load one at the beginning, that is the adapter decode. When you request, you will automatically load the amount, because we are hyperlinks, no matter if you are HTTP, we generally turn on keepalive by default, which is also a hyperlink. In fact, you can know what protocol this long link corresponds to.


Moderator: It is generally a hyperlink that remains stable. It must be an agreement that continues, and it is impossible to say that it is dynamically deteriorating.

Jingdong Zhang Geng: Yes, depending on the efficiency requirements, in fact, HTTP keepalive is also ok, and its performance is ok. Then the readability may be better when debugging. The biggest problem with binary is still more troublesome. In particular, we use message pack now, and then a bunch of proxy classes and template classes will be generated. Anyway, the problem is more troublesome.

Home delivery Shi Tingxin: We all use the Spring cloud set, and then changed some things ourselves. Like Consul, it seems to have the same problem as Zookeeper, so the database has been modified later. I am using the open source eureka now, but the affiliation has changed. So far, I haven't found a problem, because I don't have a problem across the computer room. If it is across the computer room, it is basically a database synchronization problem, and the synchronization between the two data.

Jingdong Zhang Geng: Generally, we have a functional service downgrade. In fact, the most important thing is to implement the code of the business department. Then we actually provide a mok function, that is, we configure one directly. If this interface is not available There is no switch for what is returned, which is also possible. But this is used less often. Generally, they are also fault-tolerant in their business code, that is, they do not say that they should do it from the perspective of the platform. Generally, they think about it themselves.

Then if there is an eye tracking system, it can generally track the entire call chain, and you will see this. For example, this interface depends on other interfaces, and JD.com actually does not invest so much, because our branch company has not been tracked yet. We are now mainly relying on an internal application management system of ours. We call it JOne, which is a bit like automatic deployment. . When we start each process, we will bring this application ID. On our management side, we can see which application this interface belongs to. We can only see the application level. Which interfaces are adjusted by this application? Which interface dependencies? Who else called those interfaces that were called? to this level.

Home delivery Shi Tingxin: If we use Springcloud, if the downgrade of the circuit breaker mechanism is processed, it has a statistical interface, which basically follows that rule. In terms of calling relationship, one is that we have made a trace ID, which is google zipkin, which has its own tools. Another part is the arrangement of services. We are now using camel to do it, and the whole business is arranged in layers. This is generally the case. At present, the big situation is that there will be some jitters when monitoring. For example, the trace ID part cannot use the stream mode that comes with it. We still use elk to do it now, then type out the trace ID, and then do a simple set of monitoring, similar to imitating its entire trace path. Of course, don't use Google's own monitoring, we found that the monitoring of the machine itself is not very reliable. We all do it in the log, and then collect it with elk. Speaking of the call volume from a monitoring, this is a little bit delayed.

Jingdong Zhang Geng: We are doing this recently, and our thinking is like this. There is a package in the application, it will output logs, and then we have a log collection, we just expanded the original log collection. Put it into a kafka on each machine, and then use a storm to read it out, write it into H base for analysis, and then we have a concept of sampling rate, such as one thousand times, Write it only once, or write it once every 10,000 times, and make a sampling rate. Finally, we are now divided into two parts. I just said that writing H base is an offline data. In fact, we still have some simple examples, which are to do some statistics directly, in real time, with a delay of about one minute.

Moderator: What's going on with the Dockerization of services?

Jingdong Zhang Geng: We are mainly application-level Docker. Now I just say, maybe this release mode will change. Now it is based on a Docker VM. For example, after you get up, the entire image file is actually there. Then you're actually quite slow when you play. For example, if I want to expand, I have to create a Docker VM first, and then copy those things into it, so that there is a process of installation. It's just a little slower, and it may take minutes to lift it up. But in the future, we hope to change it to the form of an image, and generate an image after you go online. Every time you go online, you only need to deploy one machine, and the rest is a process of replication. It will be changed to this in the future. It is estimated that it will be developed this year and pushed next year. Now it is equivalent to deploying 20 nodes, which is equivalent to giving you 20 Docker VMs. You publish 20 times online. In the future, I hope to give you one. After you publish once, the system will automatically copy 19 Docker VMs for you. And anyway, what the service finds later is all native, and it doesn't matter.

Jingdong Zhang Geng: Jingdong's Docker mainly solves the problem of resource scheduling. It is equivalent to the current physical machine, you may need to deploy the machine yourself. However, Docker can allocate resources more evenly, and use algorithms to calculate them. During allocation, they will not be allocated to the same rack, to the same host, or to very busy machines. These will help you think about it.

JD.com Zhang Geng: JD.com has its own deployment system. Although he did not publish it as a mirror as you said, although it is not so fast, it is actually the same for our developers to go online. He only needs to configure it. Then, at one point, his 24 units went up automatically, even if he had a set of tools, it was very fast. However, it needs to be created in advance. For example, if you just said 20 VMs, you need to create 20 VMs in advance. It is definitely slower than the mirror image. If you mirror, you can directly pull it down together, and then you can schedule it to which machine to adjust the Docker API, and he directly mentions it, which is also our future change. direction.

Qiniu Chen Aizhen: Our data processing system runs all CPU-intensive calculations. Obtaining an original file and executing data processing algorithms, such as transcoding a video, requires a lot of The CPU resources are used for processing, and the preset file style can be obtained after processing. Different data processing has different resource requirements. For example, to obtain the hash value of a file, the processing logic is very simple, and there is no large number of operations. The configured resources are relatively small, while video transcoding is very complicated. The configured resources will be relatively more.

Now there are thousands of data processing applications running on our platform, each with different requests. For example, some image processing can reach hundreds of thousands of requests per second, while others may be tens of thousands per second. The peak period of thousands of data processing applications is different, some may be in the morning, some may be in the evening, and there will be burst traffic in each processing, such as some e-commerce customers, which are expanding During promotions, it will lead to a sudden increase in the number of requests for image processing, and some audio and video customers will suddenly increase the processing of audio and video during some activities.

The core problem of this system is how to reasonably allocate hardware resources to the request volume and burst traffic of each application in different peak periods. Unreasonable resource allocation may result in waste of resources or lead to heavy load. Machines will go down and cannot respond to user needs in a timely manner. The resources of the original system architecture are statically planned, that is, the specified resources are used by the specified applications, and the allocation of such resources is often planned according to the business peak of each application. A certain amount of redundancy will be preset, so a lot of resources will need to be prepared. Later, we used containers, because containers can encapsulate the environment, migrate dynamically, and use Mesos for resource scheduling at the bottom layer, which can dynamically allocate the entire environment on demand, which solves the problem of resource utilization.


Moderator: Regarding service testing and continuous integration, please share your practical experience.

Jingdong Zhang Geng: We actually do continuous integration in the compilation process, and we have a grayscale online function on our side. We have a pre-release link, which can directly mark it as pre-release, and then have a test platform to test it as a service. If it is a formal environment, then he has to find a way by himself, because we can't test casually in this link, because I can't judge whether you are reading or writing, and I can't let you test casually.

Moderator: Because you split a business system into many such service-oriented services to run, it must involve such tests of unit testing, integration testing and business flow. How are they all done?

Jingdong Zhang Geng: This is all pre-tested, even if you haven’t tested it, then you can’t mention the step of going online at all. When you go online, you must have a test approved. In fact, it has already been tested offline.

Qiniu Chen Aizhen: We do continuous integration based on Jenkins. After uploading the code to Github, we will do automatic code static checking and unit testing, and then do automatic integration testing. The advantage of this is that the development only needs to develop the code, no need to build the environment, it is all done automatically, and the speed of feedback will be very fast.

Home delivery Shi Tingxin: The test environment is deployed by the developers, and the online formal link is from the test environment to pass the test report and copy it directly. I think Docker solves the whole configuration problem, because the production environment test environment is not the same as the development environment. The configuration environment is very difficult and troublesome to do. As much as possible, it is the UAT environment and test environment, that is, the user test environment and the online environment are the same as possible. Isn't there configuration management now? These three environments can be switched back and forth. For example, spring boot is actually a Jar package command. The only difference between the Jar package command is its configuration. You can see it as soon as you go online and monitor it there. Because he is registered when you start up, basically he can see the configuration of his registration if he wants to adjust the things.

Jingdong Zhang Geng: Jingdong test environment, pre-release, online are all compiled from the source code library. Trunk compilation prevails, so the code should be the same. We manage the online configuration in the configuration center, but currently there is no such thing in our test environment. Pre-release, there is a configuration center online, and this configuration center is also integrated into our publishing platform JOne.