Microservices in Practice (2): Using API Gateway
[Editor's Note] The first article in this series introduced the microservices architecture pattern. It discusses the advantages and disadvantages of adopting microservices, except for some complex microservices, this pattern is also ideal for complex applications.
When you decide to treat your application as a set of microservices, you need to decide how your application clients will interact with the microservices. In a monolith, there is usually only one set of redundant or load-balanced service providers. In a microservice architecture, each microservice exposes a set of fine-grained service delivery points. In this article, we look at how it affects client-to-server communication and present an API Gateway approach.
For example, the image below shows the information you see when you swipe the final page of a product on the Amazon Android client.
Although this is a smartphone app, this product final page shows a lot of information. For example, not only is there basic product information (name, description, and price), but also the following:
When using a monolithic application architecture, a mobile client will fetch this data via a REST request (GET api.company.com/productdetails/productId). A load balancer distributes requests to one of multiple application instances. The application will query various databases and return requests to the client.
In contrast, if the microservice architecture is adopted, the data on the final page will be distributed in different microservices. Listed below are some microservices that might be relevant to product final page data:
We need to decide how mobile clients can access these services. Please see the following methods
Unfortunately, this scheme has many difficulties and limitations. One of the problems is the mismatch between client demand and the number of fine-grained APIs exposed by each microservice. As shown in the figure, the client needs 7 separate requests. In more complex scenarios, more requests may be required. For example, Amazon's final product page requests hundreds of microservices. Although a client can initiate many requests through the LAN, it is very inefficient on the public network, and this problem is particularly prominent on the mobile Internet. This solution also results in very complex client code.
Another problem is that the protocol that the client directly requests the microservice may not be web friendly. One service might use Thrift's RPC protocol, while another service might use the AMQP messaging protocol. None of them are browsing or firewall friendly, and are best for internal use. Applications should use protocols like HTTP or WEBSocket outside the firewall.
Another disadvantage of this approach is that it is difficult to refactor microservices. Over time, we may need to change the current sharding scheme of system microservices. For example, we may need to merge two services or split one service into multiple. However, this refactoring is difficult to implement if the client interacts directly with the microservice.
Due to the above three problems, the way that the client directly communicates with the server is rarely used in practice.
API Gateway is responsible for request forwarding, composition, and protocol conversion. All requests from clients go through API Gateway, and then route these requests to the corresponding microservices. API Gateway will often process a request and aggregate the results of multiple services by calling multiple microservices. It can convert between web protocols and non-web friendly protocols used internally, such as HTTP protocol, WebSocket protocol.
API Gateway can provide a customized API to the client. It exposes a coarse-grained API to mobile clients. Take the usage scenario of the product final page as an example. API Gateway provides a service provider (/productdetails?productid=xxx) so that mobile clients can retrieve all data on the final page of the product in one request. API Gateway processes this request by calling multiple services and returns results, involving product information, recommendations, reviews, etc.
A good API Gateway example is Netfix API Gateway . The Netflix streaming service offers hundreds of different microservices, including TVs, set-top boxes, smartphones, gaming systems, tablets, and more. Initially, Netflix full-scene API. However, they found this form difficult to use because of the variety of devices involved and their unique needs. Now, they use an API Gateway to provide a fault-tolerant API, with corresponding code for different types of devices. In fact, an adapter calls an average of 6 to 8 backend services to process a request. Netflix API Gateway handles billions of requests every day.
API Gateway also has some disadvantages. It is a highly available component that must be developed, deployed and managed. There is also a problem, it can become a bottleneck in development. Developers must update API Gateway to provide new service providers to support newly exposed microservices. Updates to API Gateway must be as lightweight as possible. Otherwise, developers will be queued for updating Gateway. However, in addition to these shortcomings, for most applications, the API Gateway approach is effective.
Code that leverages the traditional synchronous callback approach to API merging can lead you into a callback nightmare. Such code would be very difficult and difficult to maintain. An elegant solution is to implement reactive programming patterns. Similar reactive abstraction implementations are Scala's Future , Java8's CompletableFuture , and JavaScript's Promise . There are Reactive Extensions (Rx) based on Microsoft .Net platform . Netflix created RxJava for the JVM environment to use their API Gateway. Likewise, the JavaScript platform has RxJS, which can run on the browser and Node.js platform. Adopting a reactive programming approach can help quickly implement an efficient API Gateway code.
When the cache is valid, API Gateway should be able to return the cache. For example, since product prices do not change frequently, API Gateway should return the cached value when the price service is unavailable. This kind of data can be cached by API Gateway itself, or it can be implemented by external caches such as Redis or Memcached. By returning cached data or default data, API Gateway ensures that system errors do not affect the user experience.
Netflix Hystrix is a very useful library for implementing remote service call code. Hystrix logs calls that exceed pre-set limits. It implements the circuit break mode, which makes it possible to stop the client from endless waiting for an unresponsive service. If the error rate of a service exceeds a preset value, Hystrix will interrupt the service, and all requests will be invalidated immediately for a period of time. Hystrix can define a fallback operation for request failure, such as reading the cache or returning the default value. If you're using the JVM, you should consider using Hystrix. If you are using a non-JVM environment, you should consider using a library with similar functionality.
Link to the original text: Building Microservices: Using an API Gateway (Translation: Chen Jie; Reviewer: Yang Feng)
=============================== ===================
The translator introduces
Chen Jie , a Ph.D candidate at the School of Computer Science, Beijing Institute of Technology, whose research direction is the application of natural language processing in corporate network reputation evaluation. to realize some unexpected ideas. I discovered Docker when I was tired of configuring the system environment, and learned, used and researched Docker with everyone.
When you decide to treat your application as a set of microservices, you need to decide how your application clients will interact with the microservices. In a monolith, there is usually only one set of redundant or load-balanced service providers. In a microservice architecture, each microservice exposes a set of fine-grained service delivery points. In this article, we look at how it affects client-to-server communication and present an API Gateway approach.
introduce
Suppose you are developing a native mobile client for an online shopping application. You need to implement a product final page to display product information.For example, the image below shows the information you see when you swipe the final page of a product on the Amazon Android client.
Although this is a smartphone app, this product final page shows a lot of information. For example, not only is there basic product information (name, description, and price), but also the following:
- Number of items in cart
- Order history
- user comment
- Low stock warning
- Express Options
- A variety of recommendations, including products that are frequently purchased with this item, products purchased by other customers who purchased the item, and what other products customers who purchased this product have viewed.
- Optional shopping options
When using a monolithic application architecture, a mobile client will fetch this data via a REST request (GET api.company.com/productdetails/productId). A load balancer distributes requests to one of multiple application instances. The application will query various databases and return requests to the client.
In contrast, if the microservice architecture is adopted, the data on the final page will be distributed in different microservices. Listed below are some microservices that might be relevant to product final page data:
- shopping cart service -- the number of items in the shopping cart
- Order Service -- Order History
- Classification service -- basic product information such as name, picture, and price
- Review Service -- User Reviews
- Inventory Service -- Low Inventory Warning
- Courier Services -- courier options, cut-off times, costing from different courier APIs
- Recommended Services -- Recommended Products
We need to decide how mobile clients can access these services. Please see the following methods
Client to Microservice Direct Communication
In theory, a client can make a request directly to any of the multiple microservices. Each microservice will have an external server ( https://serviceName.api.company.name ). This URL may be mapped to the load balancer of the microservice, which in turn forwards the request to the specific node. In order to search for product details, the mobile terminal needs to make requests to the above microservices one by one.Unfortunately, this scheme has many difficulties and limitations. One of the problems is the mismatch between client demand and the number of fine-grained APIs exposed by each microservice. As shown in the figure, the client needs 7 separate requests. In more complex scenarios, more requests may be required. For example, Amazon's final product page requests hundreds of microservices. Although a client can initiate many requests through the LAN, it is very inefficient on the public network, and this problem is particularly prominent on the mobile Internet. This solution also results in very complex client code.
Another problem is that the protocol that the client directly requests the microservice may not be web friendly. One service might use Thrift's RPC protocol, while another service might use the AMQP messaging protocol. None of them are browsing or firewall friendly, and are best for internal use. Applications should use protocols like HTTP or WEBSocket outside the firewall.
Another disadvantage of this approach is that it is difficult to refactor microservices. Over time, we may need to change the current sharding scheme of system microservices. For example, we may need to merge two services or split one service into multiple. However, this refactoring is difficult to implement if the client interacts directly with the microservice.
Due to the above three problems, the way that the client directly communicates with the server is rarely used in practice.
Adopt an API Gateway
Generally speaking, a better solution is to use API Gateway. API Gateway is a server, or the only node that enters the system. This is very similar to the Facade pattern in object-oriented design patterns. API Gateway encapsulates the architecture of the internal system and provides APIs to various clients. It may also have other functions such as authorization, monitoring, load balancing, caching, request fragmentation and management, static response processing, etc. The following diagram shows an API Gateway adapted to the current architecture.
API Gateway is responsible for request forwarding, composition, and protocol conversion. All requests from clients go through API Gateway, and then route these requests to the corresponding microservices. API Gateway will often process a request and aggregate the results of multiple services by calling multiple microservices. It can convert between web protocols and non-web friendly protocols used internally, such as HTTP protocol, WebSocket protocol.
API Gateway can provide a customized API to the client. It exposes a coarse-grained API to mobile clients. Take the usage scenario of the product final page as an example. API Gateway provides a service provider (/productdetails?productid=xxx) so that mobile clients can retrieve all data on the final page of the product in one request. API Gateway processes this request by calling multiple services and returns results, involving product information, recommendations, reviews, etc.
A good API Gateway example is Netfix API Gateway . The Netflix streaming service offers hundreds of different microservices, including TVs, set-top boxes, smartphones, gaming systems, tablets, and more. Initially, Netflix full-scene API. However, they found this form difficult to use because of the variety of devices involved and their unique needs. Now, they use an API Gateway to provide a fault-tolerant API, with corresponding code for different types of devices. In fact, an adapter calls an average of 6 to 8 backend services to process a request. Netflix API Gateway handles billions of requests every day.
Advantages and disadvantages of API Gateway
As you might expect, there are pros and cons to using API Gateway. One of the biggest benefits of API Gateway is to encapsulate the internal structure of the application. Compared to calling the specified service, it is simpler for the client to interact with the gateway directly. API Gateway provides each client with a specific API, which reduces the number of communications between the client and the server and simplifies the client code.API Gateway also has some disadvantages. It is a highly available component that must be developed, deployed and managed. There is also a problem, it can become a bottleneck in development. Developers must update API Gateway to provide new service providers to support newly exposed microservices. Updates to API Gateway must be as lightweight as possible. Otherwise, developers will be queued for updating Gateway. However, in addition to these shortcomings, for most applications, the API Gateway approach is effective.
Implement an API Gateway
Now that we know the motivations and pros and cons of adopting API Gateway, let's take a look at what to consider when designing it.Performance and Scalability
Only a handful of companies need to handle the scale of Netflix, with billions of requests per day. However, for most applications, the performance and scalability of API Gateway is also very important. Therefore, it makes sense to create an API Gateway that supports synchronous, non-blocking I/O. There are already different techniques that can be used to implement a scalable API Gateway. On the JVM, use a framework based on NIO technology, such as Netty, Vertx, Spring Reactor or JBoss Undertow. Node.js is a popular non-JVM platform, it is a platform built on top of Chrome's JavaScript engine. An alternative is NGINX Plus . NGINX Plus provides a mature, scalable, high-performance web server and reverse proxy that are easy to deploy, configure and re-develop. NGINX Plus manages authorization, permission control, load balancing, caching and provides application health checks and monitoring.Adopt a reactive programming model
For some requests, API Gateway can handle the request by directly routing the request to the corresponding backend service. For other requests, it needs to call multiple backend services and combine the results to process. For some requests, such as product final page requests, the requests to the backend service are independent of each other. To minimize response time, API Gateway should process independent requests concurrently. However, sometimes there are dependencies between requests. API Gateway may need to authenticate the request through the authorization service before routing to the backend service. Similarly, in order to obtain a customer's product wish list, it is necessary to first obtain the user's profile, and then return the information of the products on the list. One such API component is the Netflix Video Grid .Code that leverages the traditional synchronous callback approach to API merging can lead you into a callback nightmare. Such code would be very difficult and difficult to maintain. An elegant solution is to implement reactive programming patterns. Similar reactive abstraction implementations are Scala's Future , Java8's CompletableFuture , and JavaScript's Promise . There are Reactive Extensions (Rx) based on Microsoft .Net platform . Netflix created RxJava for the JVM environment to use their API Gateway. Likewise, the JavaScript platform has RxJS, which can run on the browser and Node.js platform. Adopting a reactive programming approach can help quickly implement an efficient API Gateway code.
service call
A microservice-based application is a distributed system and must employ mechanisms for inter-thread communication. There are two methods of inter-thread communication. One is to use an asynchronous mechanism, a message-based approach. Such implementations are JMS and AMQP. Others, such as Zeromq, belong to direct inter-service communication. There is also a synchronization mechanism for inter-thread communication, such as Thrift and HTTP. In fact, a system will use both synchronous and asynchronous mechanisms. Since there are many ways to implement it, API Gateway needs to support multiple communication methods.service discovery
API Gateway needs to know the IP and port of each microservice. In traditional applications, you might hardcode these addresses, but in today's cloud-based microservice applications, this would be an easy problem. Basic services usually use static addresses, which can be specified using operating system environment variables. However, probing the addresses of application services is not so easy. Application services typically assign addresses and ports dynamically. Likewise, instances of services can change dynamically due to expansion or upgrades. Therefore, API Gateway needs to use the system's service discovery mechanism, either server -side discovery or client -side discovery . A follow-up article will cover this part in more detail. If client-side discovery services are used, API Gateway must query the service registry , which is the database of microservice instance addresses.Processing part failed
In the process of implementing API Gateway, another issue to consider is partial failure. This problem occurs in distributed systems when one service calls another service that times out or is unavailable. API Gateway should not be blocked and wait indefinitely for downstream services. However, how to handle this failure depends on the specific scenario and specific service. For example, if the recommendation service module on the product details page is not responding, then API Gateway should return the rest of the information to the user, because this information is also useful. The recommendation section can return empty, or it can return a fixed top 10 to the user. However, if the product information service is unresponsive, API Gateway should return an error to the client.When the cache is valid, API Gateway should be able to return the cache. For example, since product prices do not change frequently, API Gateway should return the cached value when the price service is unavailable. This kind of data can be cached by API Gateway itself, or it can be implemented by external caches such as Redis or Memcached. By returning cached data or default data, API Gateway ensures that system errors do not affect the user experience.
Netflix Hystrix is a very useful library for implementing remote service call code. Hystrix logs calls that exceed pre-set limits. It implements the circuit break mode, which makes it possible to stop the client from endless waiting for an unresponsive service. If the error rate of a service exceeds a preset value, Hystrix will interrupt the service, and all requests will be invalidated immediately for a period of time. Hystrix can define a fallback operation for request failure, such as reading the cache or returning the default value. If you're using the JVM, you should consider using Hystrix. If you are using a non-JVM environment, you should consider using a library with similar functionality.
Summarize
For most microservice-based applications, it makes sense to implement an API Gateway, which acts as a service provider point into the system. API Gateway is responsible for request forwarding, request synthesis, and protocol conversion. It provides a custom API to application clients. API Gateway can mask errors in backend services by returning cached or default values. In the next article in this series, we will discuss the problem of communication between services.Link to the original text: Building Microservices: Using an API Gateway (Translation: Chen Jie; Reviewer: Yang Feng)
=============================== ===================
The translator introduces
Chen Jie , a Ph.D candidate at the School of Computer Science, Beijing Institute of Technology, whose research direction is the application of natural language processing in corporate network reputation evaluation. to realize some unexpected ideas. I discovered Docker when I was tired of configuring the system environment, and learned, used and researched Docker with everyone.
Reprinted from: http://dockone.io/article/482