Java Must-Know Series: Microservice Governance and Gateways

Author: Zen and the Art of Computer Programming

1 Introduction

With the booming development of the Internet, the increasing complexity of applications, and the increasing number of users, it has become increasingly difficult for a single architecture to meet demand. Faced with such a huge system, how can we make it more reliable, secure, and scalable? How to make various services independent of each other and work together through a "multi-point" architecture? In this case, microservice architecture emerged. The emergence of microservice architecture has provided developers with a lot of conveniences. For example, a functional module can be deployed into a cluster as an independent service; it can also be expanded on demand to improve the system's processing capabilities; it can also flexibly adjust the communication methods between services according to business conditions to achieve resource sharing and allocation; and That is, loose coupling, isolation, security and other considerations need to be considered between different microservices. Therefore, in practical applications, the microservice architecture has also experienced various forms of improvement and optimization, and its governance mechanism has become an indispensable part of the enterprise-level microservice architecture. The most important part is the gateway. The so-called gateway is simply a service designed to accept requests and forward them to the background microservice cluster. The main functions of the gateway include the following aspects:

Identity authentication and authentication: The gateway can provide identity authentication, authorization, current limiting, circuit breaker and other functions, so that the services within the microservice cluster only respond to legal requests, avoiding excessive load or service avalanche effect caused by illegal requests. At the same time, the gateway can also selectively route requests to different microservice clusters through different protocols (HTTP/HTTPS, gRPC), different entry domain names, different request headers and other information to achieve transparent communication between microservice clusters.
Service discovery and load balancing: Since microservice clusters may be distributed on different servers, the gateway needs to be able to perceive the existence of these services and distribute requests to the corresponding microservice nodes according to certain policies. When some microservice nodes fail, the gateway can automatically forward corresponding requests to other normal nodes to ensure the availability of the overall service.
Request protocol conversion: The gateway can perform request conversion according to the internal protocol of the microservice cluster, such as converting HTTP protocol requests into Dubbo protocol, thereby allowing clients in different languages to communicate with the microservice cluster.
API gateway: Since the services of the microservice architecture are stateless and shareless, the core task of the API gateway is to forward API requests to specific microservices for execution. The advantage of this is to reduce the coupling within the microservice cluster, simplify the calling logic in the microservice architecture, and improve the consistency of the API.
Traffic control and protection: In addition to functions such as identity authentication, authentication, service discovery, and load balancing, the gateway can also perform functions such as flow control, data caching, current limiting, circuit breaker, and monitoring to help microservice cluster managers protect microservices. Stable operation of the cluster.
Other function expansion: In addition to the functions mentioned above, the gateway can also perform grayscale publishing, AB testing, cross-domain access control and other functions to meet more business scenarios. This article will use easy-to-understand language and examples to introduce readers to what is microservice architecture, what problems exist in microservice architecture and how to solve them, why a gateway is needed and its role, and how to build a simple based on Spring Cloud Gateway gateway. Finally, this article will also share some common questions and answers related to gateways, hoping to be helpful to readers.
2. Overview of microservice architecture

2.1 Introduction to microservice architecture
Microservices Architecture, also known as Service-Oriented Architecture (SOA) and Service-Oriented Architecture, is an application architecture pattern. It advocates dividing a single application into a set of small services, each service is responsible for a single functional area, and the services use lightweight communication mechanisms to communicate with each other. Software under a microservice architecture is usually composed of multiple small services. Each service runs in its own process, and the processes communicate with each other through lightweight communication protocols. Each service is simple enough, with only its own data storage, its own business logic, and an independent development level. Therefore, modifications to a single service will not affect other services. The code base of each service in the system is also independent of each other.

Features of microservices architecture include:

Componentization: Under the microservice architecture, each service can be deployed independently and is highly elastic, adaptable and scalable. In some scenarios, existing services can even be reused. For example, two identical services can be deployed in two different containers, allowing them to share certain infrastructure resources to achieve resource sharing and allocation.
Loose coupling: Under the microservice architecture, services are independent of each other, and the messaging mechanism adopts lightweight communication protocols, which can effectively decouple services.
Fault tolerance: A failed service only affects a certain instance of the service and does not affect the entire system.
Scalability: The performance bottleneck of a single service can be solved by adding machines in the horizontal direction, while the splitting of services in the vertical direction requires adjusting the relationship between services, such as introducing message queues to cut peaks and fill valleys.
Unified deployment: All services can be deployed to the test environment, pre-production environment, and production environment through unified scripts.

2.2 Problems with microservice architecture

However, relying solely on microservice architecture cannot completely solve all problems. Since the microservice architecture does not have a unified service registration center, services cannot communicate directly. Communication between services can only be achieved through the gateway. Another function of the gateway is to serve as a flow control, fuse, and load balancer. , permission verification, etc. roles. Let's take a look at several problems with microservice architecture.

2.2.1 Single point of failure

Under a monolithic architecture, if a certain service fails, the entire system will fail. In order to ensure the high availability of the system, microservice architecture often uses clusters to deploy multiple services. Even if a problem occurs in a certain service, it will only affect a certain instance of the service. In order to improve availability, microservice architecture usually sets up multiple service copies, but single points of failure are still a hidden danger.

2.2.2 Data consistency

Under the microservice architecture, the distributed nature of services can lead to data inconsistency. There may be data synchronization problems between different services. For example, two services insert a record respectively, but due to network delay or other reasons, the insertion order of the two records may be different. In order to ensure data consistency, microservice architecture often uses the event publishing/subscribing (Event Publishing/Subscribing) model to deliver data change notifications between services.

2.2.3 Performance bottleneck

Although the microservice architecture can effectively solve problems such as single points of failure and data consistency, it still has some performance bottlenecks. For example, since each service is deployed independently, it takes up a larger amount of memory space. In addition, if a certain service has performance problems, it may cause the overall performance of the entire system to decrease.

2.2.4 New technology adaptation

For those who are new to microservice architecture, the most troublesome thing is adapting to new technologies. Each service should rely on minimizing external dependencies as much as possible, ensuring maximum cohesion, and avoiding the introduction of new technologies that will affect the whole system.

2.3 Microservice architecture solution - gateway

In order to solve the above problems of microservice architecture, it is necessary to introduce the gateway architecture model. The main responsibility of the gateway is to act as a scheduler, receiving client requests and forwarding the requests to the back-end microservice cluster.

Why introduce a gateway? Suppose there is a microservice architecture with three services A, B, and C. Each service is deployed on a different server, and the client accesses the microservice cluster through the gateway. If there is no gateway, service A sends a request, which requires three network transmissions to reach service C, as shown in the following figure:

To send a request to services A, B, and C, the client needs to go through three network transmissions. Then if service A fails, the entire request will time out. In order to improve the availability of the system, the client needs to re-request the failed service until it succeeds. As a result, the client needs to maintain a lot of state, not only needs to manage requests for service A, but also needs to manage requests for service B and service C, and the client needs to be deployed on multiple servers. The gateway is like a centralized request portal. It can shield the microservice cluster and only retain client requests, reducing the client's state management burden and improving the system throughput. As shown below:

Since the microservice architecture does not have a unified service registration center, services A, B, and C cannot communicate directly, and communication between services needs to be achieved through a gateway. The client sends a request to the gateway, and the gateway forwards the request to the corresponding microservice cluster. For the client, there is no need to manage multiple service clusters, only one gateway is required.

2.4 Introduction to Spring Cloud Gateway

Spring Cloud Gateway is a sub-project of Spring Cloud. It is a gateway framework built on the Spring Framework. Based on the non-blocking asynchronous features provided by Spring WebFlux and Project Reactor, Spring Cloud Gateway can support high-performance routing, filtering and traffic management.

Spring Cloud Gateway, as the gateway module of Spring Cloud, has the following advantages:

Easy to use: Spring Cloud Gateway is based on Spring Boot Starter and Spring WebFlux, and can use a simple and flexible way to build a customized API GateWay. It supports many routing Predicates and Filter Handlers, and also provides additional Gateway integration modes such as Zuul, Apigee, etc., which users can choose freely.
High performance: Spring Cloud Gateway uses the Netty framework as the network communication engine, making full use of multi-core CPU and asynchronous IO to improve processing capabilities and request response speed.
Dynamic routing: Spring Cloud Gateway provides dynamic routing Predicate and Filter Handler, allowing users to configure routing rules by writing code and flexibly match and modify request parameters.
Plug-in support: Spring Cloud Gateway supports plug-in development model, which can easily integrate Zuul plug-ins and custom plug-ins to achieve function expansion and custom configuration.
Integrated Spring Security: Spring Cloud Gateway integrates the Spring Security security framework, which can securely protect requests from microservice clusters, including authentication, authorization, circuit breaker, current limiting, etc.
Open interface: In addition to serving as a service gateway, Spring Cloud Gateway also provides a complete RESTful API interface definition specification, allowing third-party systems to be integrated into the API gateway to achieve open interface capabilities.

2.5 Basic use of Spring Cloud Gateway

Below we use a simple example to demonstrate the basic use of Spring Cloud Gateway. First, create a Spring Cloud project and add the Spring Cloud Gateway module:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>

Then, add annotations to the startup class @EnableGatewayto enable the gateway:

@SpringBootApplication
@EnableDiscoveryClient // Spring Eureka Discovery Client enables discovery of services in a specific zone
@EnableCircuitBreaker // Hystrix Circuit Breaker to prevent cascading failures and enable fallback
public class GatewayApplication {

    public static void main(String[] args) {
        SpringApplication.run(GatewayApplication.class, args);
    }

    @Bean
    public RouteLocator routeLocator(RouteLocatorBuilder builder) {
        return builder.routes()
           .route(r -> r
               .path("/api/**")
               .uri("http://localhost:8080"))
           .build();
    }

}

Above, we configured a routing rule to /api/**forward requests under the path to http://localhost:8080. Next, start the Spring Cloud application. After the startup is successful, open the browser and input http://localhost:8080, and you can see Hello World!the output.

At this point, we have successfully started a Spring Cloud Gateway instance and forwarded the request to a microservice cluster.