API Gateway (API Gateway)

API Gateway (API Gateway)

The reason for the emergence of the API gateway is the emergence of the microservice architecture. Different microservices generally have different network addresses, and external clients may need to call multiple service interfaces to complete a business requirement. If the client directly communicates with each microservice Service communication, there will be the following problems:

• The client will request different microservices multiple times, increasing the complexity of the client.
• There are cross-domain requests, and the processing is relatively complicated in certain scenarios.
• Authentication is complex, and each service requires independent authentication.
• It is difficult to refactor, and as the project iterates, it may be necessary to re-partition microservices.

For example, it is possible to merge multiple services into one or to split one service into several. If clients communicate directly with microservices, refactoring will be difficult to implement.

Some microservices may use firewall/browser-unfriendly protocols, making direct access difficult.

The above problems can be solved with the help of API gateway.

The API gateway is an intermediate layer between the client and the server, and all external requests will first pass through the API gateway layer. That is to say, more business logic is considered in the implementation of the API, while security, performance, and monitoring can be done by the API gateway, which not only improves business flexibility but also does not lack security. A typical architecture diagram is shown in the figure:

The API gateway is between the client and the server, and is the only entrance for the client to access the business system. The existence of the API gateway allows microservices to focus more on business logic, while public services such as security, performance, and monitoring are implemented by the API gateway, such as: unified authentication, load balancing, service fusing, flow control, gray scale publishing, and logging statistics etc.

In fact, before the concept of microservices became popular, the entities of API gateways had already been born. For example, front-end processors commonly used in banking, securities and other fields appeared to solve problems such as access authentication, message conversion, and access statistics.

Regarding the functions of the API gateway, someone made such a vivid analogy. If there is a company, there is a doorman at the entrance of the company's office. Under normal circumstances, the doorman will confirm the identity of the people entering and leaving the company, and will help provide location guidance services when visitors visit, and assist when the company holds events. Personnel management, etc. The role of the API gateway is equivalent to a doorman. Services such as commodities, inventory, orders, and comments are all back-end internal microservices, which are independently maintained by each business team, which is equivalent to each business department of the enterprise. When an external request occurs, the API The gateway first authenticates and authenticates the request, and then distributes the request to the corresponding microservice on the backend. If there is a traffic burst in the external request, it will perform load sharing and flow control on these requests.

Authentication

Under the monolithic application system, the request generally passes through a permission interceptor for permission verification, and the user information is cached in the session session when logging in, and the user information is obtained from the session for subsequent access.

Under the microservice architecture, an application is split into several microservices, and each microservice needs to specify the current access user and its permissions. The authentication method under the monolithic application architecture is not suitable. The API gateway serves as a unified entrance for service access. All user requests go through the API gateway, which is very suitable for face-to-face services such as authentication and authentication.

The API gateway intercepts user requests, obtains the user identity information attached to the request, calls the service of the authentication and authorization center, and performs identity authentication on the requester, that is, confirms that the current visitor is indeed the identity he claims, and checks whether the user has access to the background service permissions.

There are currently two mainstream authentication schemes:

Introduce Redis for distributed sessions. That is, after the user logs in successfully, store the user identity and permission information in Redis, use a unique ID as the key, and set the expiration time of the information in Redis. The Key of this unique ID will be returned to the client, and the client can store it in local storage such as Cookie or sessionStorage. The next time you visit, put this unique ID into the request parameter and send it together (usually put in the Header). The server judges whether the user is logged in by checking whether the ID exists in Redis, and obtains the user identity and permission information. If the client does not operate for a long time, the session information stored in Redis will be automatically deleted when it expires. Every time the client accesses the server, it needs to refresh the expiration time of the session information to avoid the poor user experience caused by the fixed expiration time.

JWT stands for Java Web Token. After the user logs in successfully, the unique ID returned by the server to the client is no longer a meaningless string, but an encrypted string containing information such as user identity, authority, and expiration time, and this string contains a digital signature. The terminal can perform digital signature verification on this string to ensure that the string has not been tampered with or forged. Compared with the distributed session solution, although JWT saves Redis storage, digital signature verification is required for each access, which increases CPU resource consumption.
Composition of JWT

The first part we call it the header (header), the second part we call it the load (payload, > similar to the items carried on the plane), and the third part is the visa (signature). The server will verify the token, if the
verification Passing will return the corresponding resource. The whole process is like this:

Load Balancer (LB)

In actual application deployment, when the application system faces business peaks and the access traffic is too high, we usually expand the number of services horizontally and use cluster load balancing to improve the system's business response capability.

Load balancing is divided into centralized load balancing and in-process load balancing.

Centralized load balancing sets up LB hardware (such as F5) or software middleware (such as Ngnix) between the visitor and the target service. The client request is first sent to the load balancing server, and the load balancing server forwards the request to one of the backend servers according to a certain set balancing algorithm. The server-side load balancing technology is mature and efficient. However, the change of the back-end server (expansion or shrinkage) requires modification of the load balancing configuration, which has poor maintainability and requires high availability of the load balancing itself, which increases deployment and maintenance costs.

In-process load balancing is different. The address list of the backend server is no longer stored and maintained by the load balancing server, but the LB logic is integrated on the client. When the back-end server starts, it automatically registers the service with the registration center, and when the service goes offline, it automatically cancels the service with the service registration center. The client establishes a long-term monitoring of the service registration center. Whenever the back-end service changes, the registration center automatically notifies the client to update the server list in the local cache. When the client accesses the backend service, it obtains which addresses are available from the service registry, and then selects an appropriate server from these addresses according to the load balancing algorithm. The advantage of in-process load balancing is convenience, and changes in backend services are transparent to clients.

flow control

The main purpose of flow control is to prevent large concurrent requests from being forwarded to the backend in a short period of time and cause business overload. A typical example is DDoS (Distributed Denial of Service) attack, where malicious attackers control a large number of zombie machines, Continuously send a large number of requests to the target server, so that the server is too busy to deal with the requests of normal users, so as to achieve the purpose of paralyzing the business system. In order to prevent this from happening, the API gateway can limit the flow of external requests.

Common current limiting algorithms include counter algorithm, leaky bucket algorithm and token bucket algorithm.

The counter algorithm is a simple and crude current-limiting algorithm. Its design idea is to limit the maximum number of requests allowed to pass within one second. For example, set the current-limiting QPS (Queries Per Second, the number of requests per second) to 100. Then, from The first request arrives and starts timing. In the next 1 second, every time a request comes, the count will be increased by 1. If the accumulated number reaches 100, all subsequent requests will be rejected. After 1s is over, the counter is reset to 0 and starts counting again. The counter algorithm has a disadvantage: if 100 requests have been passed in the first 10ms of unit time 1s, then all incoming requests will be rejected in the next 990ms. We call this phenomenon "spurt phenomenon".

In order to eliminate the "spike phenomenon", the leaky bucket algorithm appeared. There is a "container" inside the leaky bucket algorithm, similar to the funnel used in life. When a request comes in, it is equivalent to pouring water into the funnel, and then flowing out slowly and at a constant speed from the small opening at the lower end. No matter how big the above flow is, the speed of the flow below remains unchanged, and a request is processed every 10ms. Because the processing speed is fixed, the speed at which requests come in is unknown. There may be a lot of requests coming in suddenly, and the requests that have not been processed in time will be placed in the bucket first. Since it is a bucket, it must have an upper limit. If the bucket is full, then New incoming requests are discarded. The leaky bucket algorithm also has disadvantages: it cannot cope with short-term burst traffic.

The token bucket algorithm is an improvement to the leaky bucket algorithm. The leaky bucket algorithm can limit the rate of request calls, while the token bucket algorithm can limit the average rate of calls while allowing a certain degree of burst calls. In the token bucket algorithm, there is a bucket used to store a fixed number of tokens. There is a mechanism in the algorithm to put tokens into the bucket at a certain rate. Each request call needs to obtain a token first. Only when the token is obtained, can the execution continue. Otherwise, wait for the available token, or directly reject it. The action of putting tokens is carried out continuously. If the number of tokens in the bucket reaches the upper limit, the tokens will be discarded, so there is such a situation. There are always a large number of available tokens in the bucket. At this time, incoming requests can be directly taken When the token is executed, for example, if the QPS is set to 100, then 1 second after the current limiter is initialized, there will already be 100 tokens in the bucket. At this time, the service has not been fully started. The streamer can withstand 100 requests instantaneously. Therefore, only when there is no token in the bucket, the request will wait, and finally it is equivalent to executing at a certain rate.

Service circuit breaker

In the actual production environment, some services are likely to fail due to certain reasons, such as internal service errors, network delays, etc. If the faulty service is left alone, it may cause an "avalanche effect" due to cascading failures, making the entire The system is paralyzed. For example, service A calls service B, and service B calls service C. If service C becomes unavailable and blocks a large number of upstream requests, the request thread of service B will also be blocked, and then service A will also become unavailable. If used, the entire calling link is dragged down.

The fuse mechanism is a microservice link protection mechanism to deal with the avalanche effect. We will come into contact with the word fuse in various scenarios. In a high-voltage circuit, if the voltage somewhere is too high, the fuse will blow to protect the circuit. In stock trading, if the stock index is too high, a circuit breaker mechanism will also be adopted to suspend stock trading. Similarly, in the microservice architecture, the fuse mechanism also plays a similar role. When a microservice in the call link is unavailable or the response time is too long, the service will be degraded, and then the call of the node microservice will be broken, and the wrong response information will be returned quickly. When it is detected that the microservice call response of the node is normal, the call link is resumed.

For the fuse mechanism, there are generally three states:

1. Closed state (Closed): The state of the circuit breaker when there is no fault in the service, and there is no restriction on the calling of the caller.
2. Fuse opening state (Open): within a fixed time window (for example, 10s), if the interface call error rate reaches a threshold (for example, 50%), it will enter the fuse open state. After entering the fuse state, subsequent calls to the service interface will no longer go through the network, and directly execute the local Fallback (retreat) error method.
3. Half-Open: After entering the fuse-open state for a period of time (for example, 5s), the fuse will enter the half-open state. The so-called semi-fuse is to try to restore the service call, allow limited traffic to call the service, and monitor the success rate of the call. If the success rate meets expectations, it means that the service has been restored and enters the circuit breaker shutdown state; if the success rate is still low, re-enter the circuit breaker shutdown state.

The advantages of using the API gateway are as follows

• Easy to monitor. Monitoring data can be collected at the gateway and pushed to external systems for analysis.
• Easy to authenticate. Instead of authenticating in each microservice, authentication can be done on the gateway before forwarding the request to the backend microservices.
• The number of interactions between clients and individual microservices is reduced.

API Gateway Selection

The situation in the industry:

At this time, we have not seen API Gateway. For example, where IOS, Android, and PC clients call services, multiple URL addresses are required, including orders, products, and users. After micro-service, there must be a unified entrance and exit. In this case, API Gateway appears. API Gateway solves the problems of microservice calls and unified access, as shown in the following figure:

With the API gateway, each API service provider team can focus on their own business logic processing, while API is more focused on security, traffic, routing and other issues.
Seeing the above diagram and description, we may think of another thing similar to the gateway - proxy. The difference between a gateway and a proxy: the proxy is purely data transparent transmission, and the protocol will not change; the gateway will also design protocol conversion in the context of data transparent transmission, for example, the protocol that the user request is transmitted to the gateway in the above figure is HTTP, Transparently transmitted to the downstream through the gateway, it may have been converted into RPC within the enterprise (such as RPC frameworks developed by enterprises such as JSF and Dubbo).

Basic functions covered by API Gateway

The basic functions of an API gateway include unified access, protocol adaptation, traffic management and fault tolerance, and security protection. These four basic functions constitute the core functions of the gateway. The primary function of the gateway is to be responsible for unified access, and then convert the requested protocol into an internal interface protocol. In the process of calling, there are also fault-tolerant methods such as limited flow, downgrade, and fuse to protect the overall stability of the gateway. At the same time, the gateway also needs to Implement basic security protection (anti-brush control), and basic security measures such as black and white lists (such as IP white lists), as shown in the following figure:

Architecture Example of an API Gateway

In addition to the four basic functions, the environment in which the gateway operates well also includes the registration center (for example: ZK reads the dynamic configuration of the published API interface). In order to achieve high performance, all the data is heterogeneously stored in the cache (such as: Redis), and it can also cooperate with the local cache to further improve the performance of the gateway system. In order to improve the throughput rate of the gateway, NIO+Servlet 3 asynchronous method can be used, and the asynchronous feature of Servlet 3 can also be used to separate request threads from business threads, so as to provide basic support for subsequent thread pool isolation. The storage of access logs can be stored in Hbase. If it is to be used as an open gateway, an authorization center that supports OAuth2.0 is required. You can also introduce Nginx + lua to put some basic verification judgments on the application system, so that you can handle access problems more lightweightly. The overall gateway architecture example is as follows:

https://blog.joway.io/posts/kubernetes-gateway/