Open source address
https://github.com/XiaoMi/mone/tree/master/gateway-all Welcome friends who are interested in cloud native technology/R&D efficiency to join (Fork, Star) us.
Overview
1. Background
In the era of microservices, the granularity of service splitting is becoming increasingly finer. Each microservice is responsible for its own core functions and provides a series of API interfaces to the outside world. However, as the business expands and there are more and more interfaces, some problems arise. Can these interfaces be managed uniformly in one place? When it comes to the common issue of authentication, does it need to be implemented once for each microservice? Each microservice has its own protocol and code writing style, such as camel case and underline. Can it be unified?
In this case, we need api gateway to solve these problems.
2. What is gateway gateway?
API gateway is a service and a unified entrance to the system. We can implement the public non-business functions of each microservice in API Gateway, such as authentication, monitoring, load balancing, caching, etc., to reduce the responsibilities of each service as much as possible. The API gateway aggregates the services exposed by each system to the outside world. All systems that want to call these services need to be accessed through the API gateway. In this way, the gateway can uniformly control the API.
3. Tesla gateway
Tesla is a high-performance, easily scalable and excellent API gateway platform based on JDK19 open sourced by the Xiaomi Performance Team. It is the product of the Xiaomi Performance Team combined with Xiaomi's years of experience in major promotions. It has already experienced more than ten Xiaomi major promotions. Test, and assume the core role of traffic management, is an indispensable link in Xiaomi's business chain.
Speaking of Java-based API gateways, many people may first think of Spring Cloud Gateway or Zuul, but Tesla Gateway has very obvious advantages in terms of ease of use, scalability and performance.
Ease of use: Tesla has a visual operation interface and supports real-time updates of configurations, which take effect in real time, greatly simplifying learning and use costs and optimizing the operating experience.
Scalability: Tesla provides powerful scalability support through support for custom filters, especially dynamic filter support in Sidecar mode, which can meet the needs of most businesses and projects.
High performance: Tesla gateway itself is developed based on Netty and has very outstanding performance in terms of performance. At the same time, it fully supports Virtual Thread of JDK19, which greatly improves the performance and resource allocation of Tesla gateway.
1. Technical architecture
Tesla gateway mainly consists of two parts: console + gateway. The console mainly maintains API data and Filter data, and can also manage the domain names of different gateway clusters. Tesla gateway centralization is the core component of this product and is responsible for the entire traffic management, security protection, Filter execution, protocol conversion, and traffic forwarding.
2. Core competencies
2.1. Dynamic discovery and load balancing
With the current popularity of microservices and containerization, the number of nodes and IP addresses of business services may change at any time. The Tesla gateway opens up the service registration center, subscribes to relevant service information, and can sense changes in the business service side in real time. And when processing actual requests, it supports multiple load balancing strategies, such as polling, hashing, etc. In this way, when business services are expanded or reduced, or during fault recovery, hot backup, or switchover, the graceful exit of business services can basically make the caller unaware.
2.2. Unified management API, dynamic updates in seconds
We organize the api interfaces by "Tenant->Cluster->Interface Group->Interface". You can view the API interfaces that you have permission to view in the Tesla console, and perform a series of operations on them, such as enabling authentication, interface mocking, etc. The Tesla console manages all api interfaces in a unified manner. When you add or update an interface, the gateway cluster can also detect the changes in the interface in real time, implementing a runtime configuration update mechanism .
2.3. User-defined plug-ins, real-time uploading, dynamic plug-in and unplugging
During the development of Tesla Gateway, we found that it is a great thing to allow users to participate in contributing filter plug-ins. Each filter is a function. Everyone can use their creativity to contribute their own filters to achieve multiple functions, such as It is authentication, grayscale, current limiting, etc. Tesla gateway allows users to upload their own filter code in real time, and the gateway cluster is dynamically loaded without restarting. Each user can reuse other people's filters.
2.4. Multiple protocol conversion
As a unified API entrance, the gateway provides a restful-style http interface to the outside world. But the back-end microservices use a variety of protocols. You can use dubbo, grpc, or http. Here, the gateway encapsulates the different protocols of the back-end and exposes a unified interface style to the front-end.
2.5. Traffic management
Traffic security : Traffic security is a topic that every gateway product cannot avoid. How to deal with security issues such as DDoS, how to authenticate requests, how to desensitize data, etc. Tesla Gateway has complete solutions to this series of security issues and has been verified for many times.
Flow control : Gateway products have a complete set of current limiting and fusing measures, whether in daily operation or special times such as major sales, which can protect the robustness and availability of the back-end business system to a great extent. But Tesla gateway is not only satisfied with this, but also extends many practical functions to better help business systems. These include results caching, traffic record playback and other functions, thus providing more convenience and help to business systems.
Resource isolation : Tesla gateway carries many business systems. Some of them have large traffic, some have high security requirements, and some have complex business. It can be said that almost every business system has its own special requirements, so it is especially important for Tesla gateway to ensure that each business system does not interfere with each other. For example, the business of system A is very complex, resulting in slow return of requests. If resource isolation is not implemented, a single system can seize more system resources, thus affecting other business systems. Tesla gateway can currently achieve system resource isolation and business resource isolation. Each business has its own independent resource pool to ensure "dedicated funds" and protect the business.
3. Design difficulties
As a unified entrance for traffic, the gateway's stability, performance, scalability and scalability are the focus of its design.
In order to ensure stability, we have isolated the cluster, and core services such as transaction links can apply for their own independent clusters. Each interface group in the cluster also has an independent thread pool to handle the reception, distribution and return of requests to ensure that different groups do not affect each other.
For performance, the gateway introduces a web_server developed by Netty to ensure high throughput. The gateway cluster will have a cache of core metadata in the memory, relying on a synchronization mechanism to ensure real-time performance; in addition, an interruption mechanism is implemented for slow requests to avoid resources was occupied. Finally, we also made some optimizations to the dubbo version. For example, dubbo's generalized call did not support setting a timeout at the beginning.
In terms of scalability, the gateway as the traffic entrance must be able to support expansion and contraction at any time to cope with sudden traffic. Therefore, each machine in the gateway cluster does not have its own status, and the admin management platform controls the synchronization of data.
Finally, there is scalability. The gateway encourages everyone to submit their own filters and help each other.
4. Design mind map
4. Regarding the future development direction
At present, our gateway architecture is a centralized gateway, which provides general capabilities such as authentication, current limiting, etc. to process requests, and routes requests to the correct back-end service based on the service identifier; after the service processes the request, the response returns to the original route.
As a unified entrance for traffic, the centralized gateway architecture encounters some difficulties in actual scenarios in the face of diverse business forms and complex network structures. First, as the business develops, the number of gateway cluster machines continues to grow. This intensifies the complexity of the operation and maintenance level, and IT costs are also facing unbearable burdens. Second, some core link services urgently need to be isolated from other services to prevent unpredictable burst traffic from affecting the availability of these high-security services.
So here we start to build and promote the decentralization of the gateway. The Mesh gateway and the back-end service are deployed in the same Pod, that is, the Mesh gateway and the business system are on the same server and in different processes, and have the full capabilities of the gateway. This is a direction we are currently exploring.