Author: Xu Jingfeng
I have followed Alibaba Cloud's cloud-native official account, and I often see tweets related to MSE-Higress. It just so happened that Alibaba Cloud held an MSE-Higress cloud-native gateway evaluation event this time, and took this opportunity to experience a cloud-native gateway. function.
Purchase process experience
When purchasing a gateway, the page clearly states that the cost does not include the cost of public network and private network SLB. It should be noted here that additional costs will be incurred during the evaluation. It is also recommended that the MSE-Higress purchase page give specific pricing and refer to the ACK purchase experience. .
Routing management experience
After purchasing through the purchase page, the instance was created shortly after waiting. It was very fast and the experience was good. The first evaluation content is to experience the routing and forwarding capabilities of the most important functions of the gateway, configure routing & services for MSE-Higress, and access httpbin.org, a public network service. Students who are familiar with HTTP interface testing will definitely not be impressed by httpbin Strange, it has many built-in endpoints and supports rich HTTP testing scenarios.
The product design and domain model of MSE-Higress are not much different from some open source API gateways I have come into contact with before, so it is easy to get started. First, create the httpbin service:
Then create a route, and I plan to test the route by forwarding ${gateway ip}/httpbin/get to httpbin.org/get.
The matching methods support prefix matching, exact matching, and regular matching, which basically cover the common requirements of gateway routing scenarios. Another thing to note is that the routing path must be configured as /httpbin/ instead of /httpbin, otherwise there will be problems when the configuration path is rewritten later. I also did not understand the design of MSE-Higress at the beginning. It was mismatched to /httpbin, causing routing failure.
Reference document: "Configuration Rewrite Strategy" [ 1]
Next, associate the service you just created.
Finally, in the routing policy configuration, configure the rewrite policy so that the gateway removes the /httpbin prefix used for route matching when requesting the upstream service.
MSE-Higress provides a debugging interface that can easily debug routing. Just when I was confident and ready to complete the first test, an error occurred during the debugging:
The steps are not complicated. I spent a little time searching for the precautions, and I found the problem. It turned out that there was a prompt when configuring the service: "DNS domain name configuration, such as www.aliyun.com, the public domain name needs to be configured in the VPC. "Network NAT gateway, intranet domain name is not supported yet", so the NAT gateway was configured for the VPC where MSE-Higress is located, and the call was finally successful.
➜ ~ curl 101.xx.166.xx/httpbin/get
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "101.xx.166.xx",
"Original-Host": "101.xx.166.xx",
"Req-Start-Time": "1691746441214",
"User-Agent": "curl/7.64.1",
"X-Amzn-Trace-Id": "Root=1-64d60089-5f09b9560522afd56f11b4e0",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-External-Address": "140.xx.11.xx",
"X-Envoy-Original-Path": "/httpbin/get"
},
"origin": "140.xx.11.xx, 121.xx.116.xx",
"url": "http://101.xx.166.xx/get"
}
There was also an episode during this period. Repeatedly saving the service would trigger a front-end bug. The save button kept spinning in circles. It reappeared stably during the evaluation period:
Routing strategy-current limiting function experience
When I was just testing the routing function, I already used a strategy of MSE-Higress: the rewrite strategy. MSE-Higress supports a total of 6 routing strategies, namely: current limiting, rewriting, header settings, cross-domain, timeout, and retry. In the second evaluation, I plan to give another commonly used function in gateway scenarios – current limiting.
When creating the current limiting policy, I found that the interface has traces of embedded components, and the configuration interaction experience is somewhat different from other policies. I blindly guessed whether there is some existing interface embedded in the front end. The product supports current limiting based on QPS. To facilitate evaluation, it is set to 1 to make it easier to trigger current limiting.
Through behavior management, you can jump to the management interface of the application high-availability service AHAS. It seems that the application high-availability service AHAS is integrated internally, and its current limiting capability is reused. Professional things are left to professional products.
Conduct current limiting testing through a shell script:
#!/bin/bash
for i in {1..5}
do
curl 101.xx.166.xx/httpbin/get &
done
wait
Verify that the current limit is successful.
sentinel rate limited
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "101.xx.166.xx",
"Original-Host": "101.xx.166.xx",
"Req-Start-Time": "1691747565429",
"User-Agent": "curl/7.64.1",
"X-Amzn-Trace-Id": "Root=1-64d604ed-6dc526617e735d4f0f083e86",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-External-Address": "140.xx.11.xx",
"X-Envoy-Original-Path": "/httpbin/get"
},
"origin": "140.xx.11.xx, 121.xx.116.163",
"url": "http://101.xx.166.xx/get"
}
sentinel rate limited
sentinel rate limited
sentinel rate limited
Problem record: The current limit monitoring page is not stable, and console request errors appear intermittently. It needs to be optimized.
EDAS microservice integration experience
MSE-Higress's integration of microservice capabilities is one of its highlights. In addition to the HTTP protocol family, it also supports Dubbo and gRPC protocols. EDAS is often used to host microservice applications. MSE-Higress has also adapted EDAS. The content of this evaluation case is to deploy an application in EDAS that integrates SpringCloud Alibaba (for testing HTTP protocol) and Dubbo (using (For testing the microservice application of Dubbo protocol), use MSE-Higress to perform interface proxy for the application.
EDAS application deployment
Prepare a Dubbo service:
package moe.cnkirito.sca.provider;
import java.util.List;
import java.util.Map;
public interface IHelloService {
String sayHello(String str);
String sayHello();
String sayHello(List<String> name);
String sayHello(List<String> name, Integer age);
String sayHello(List<People> name, String first);
String sayHello(People name);
String sayHello(Map<String, Integer> map);
String sayHello(Integer age);
}
Prepare a RestController:
@RestController
public class DemoController {
@Autowired
DemoService demoService;
@RequestMapping(value = "/echo", method = RequestMethod.GET)
public String echo() {
return "Hello MSE-Higress";
}
}
Configure application information and connect to the EDAS registration center:
spring:
application:
name: sc-dubbo-mixed-app
cloud:
nacos:
discovery:
server-addr: edas-registry:8848 # EDAS会自行替换该连接串
dubbo:
application:
id: sc-dubbo-mixed-app
name: sc-dubbo-mixed-app
protocol:
id: dubbo
port: 20880
registry:
id: nacos
address: nacos://edas-registry:8848 # EDAS会自行替换该连接串
To deploy to EDAS, confirm that the application is started in the service list menu of EDAS application management.
MSE-Higress creation source
In order to better support the service discovery of microservices, MSE-Higress abstracts the domain model of "source", which corresponds to the registration center in the microservice architecture.
The source of MSE-Higress supports container service, MSE Nacos, MSE Zookeeper, EDAS registration center, and SAE registration center. Selecting the EDAS registration center can be associated with the microservice space deployed by the sc-dubbo-mixed-app application.
MSE-Higress creation service
The management and control of MSE-Higress can directly access the EDAS registration center and obtain the two services sc-dubbo-mixed-app and providers:moe.cnkirito.sca.provider.IHelloService:1.0.0:default.
Check the protocol of the service, and it is correctly identified as the Dubbo protocol.
This inevitably makes me have some doubts. When importing the service, MSE-Higress did not have the opportunity for me to specify the protocol type of the service, but it correctly identified the protocol of the service in the protocol details. The guess is that the naming format of the service was judged, because The Dubbo type service is registered in Nacos in the format of providers:xx:xx:xx. The product adopts a design where convention is greater than configuration.
MSE-Higress creates route
The SpringCloud service provides the standard HTTP protocol. The above routing management evaluation has been covered, so we will not evaluate it again. Let's focus on how HTTP2Dubbo configures routing. There is no way to interpret this part of the content interactively. You still have to follow the document step by step: "Configuring the Conversion from HTTP to Dubbo Protocol" [ 2]
The configuration is as follows:
MSE-Higress routing debugging
Test Dubbo routing:
Test SpringCloud routing:
EDAS microservice integration summary
This evaluation introduces the SpringCloud and Dubbo applications hosted by EDAS, which can be easily integrated by MSE-Higress, and MSE-Higress acts as a gateway agent to expose microservices to be accessed outside the cluster. Although this evaluation does not involve it, the theory You can also rely on the current limiting, authentication, and security protection provided by MSE-Higress to protect the microservice system, and there is a certain amount of room for imagination.
At the same time, record the areas that personally feel can be improved during the evaluation.
Improvement suggestion 1: Optimization of service model
As mentioned above, the presence of services in MSE-Higress is very weak. In my opinion, services should be the same as routing, with strong customization attributes, including:
- agreement type
- The unique identifier of the service discovery layer
- The unique identifier of the communication layer
- Load balancing method
- Health check configuration
Currently, when MSE-Higress creates a service, it only supports specifying the "unique identifier of the service discovery layer". Other attributes do not support specifying at the time of creation. The protocol type and load balancing method can be modified on the service details page. Modify the list page.
For the "unique identification of the communication layer", take Dubbo as an example, providers:moe.cnkirito.sca.provider.IHelloService:1.0.0:default is its unique locator in Nacos namespace for service discovery, The service name is moe.cnkirito.sca.provider.IHelloService, version 1.0.0, and the group default is its unique identifier in the communication layer, which should also be an attribute of the service, but in MSE-Higress, it is the service binding route The current configuration feels a bit like a routing model, which is not properly designed.
Improvement suggestion 2: Optimization of Dubbo protocol conversion
In the above evaluation process, a configuration process of Dubbo protocol conversion is introduced. Now that the format of Dubbo service has been recognized, the triplet of Dubbo service can be automatically parsed for filling.
In addition, the design of method mapping gave me some doubts. I am not sure whether it is due to technical reasons or intentional product design, because in my understanding, the method level can be dynamically specified in the request. Imagine that an application has n Service, a service has m interfaces, and needs to be configured nxm times for complete exposure. From a technical perspective, the generalized calling provided by Dubbo can support dynamically specified methods without configuring the parameter list type. The conceivable benefit of maintaining this design is that it allows some interfaces to be exposed, which returns to the eternal topic: the trade off of security and ease of use.
Referring to MSE-Higress's support for gRPC protocol conversion, it is another form: the request path is: {package name}.{service name}/{method name}, and gRPC itself is not included in MSE-Higress with a The type of service is reflected in the product design. MSE-Higress has the ability to support Dubbo and gRPC type protocol conversion, but there is still room for unified optimization in product design.
For MSE-Higress's support for gRPC, please refer to: "Implementing routing and forwarding of gRPC services based on cloud native gateway" [ 3]
Improvement suggestion 3: EDAS registration center type support optimization
There are two forms behind the EDAS microservice space, one is the form of a shared registration center, and the other is the form of binding an MSE Nacos instance. In the above demonstration, the first form was mainly tested. For the second form, There are some compatibility issues with the support of MSE-Higress. The specific manifestations are as follows: MSE Nacos bound to the EDAS microservice space is located in vpc-a, and MSE-Higress is located in vpc-b. The source creation can be successful, but when importing the service, the page reports an error. :
The reason behind this should be that when supporting the EDAS registry, the MSE Nacos bound to it was not considered to be located in other vpcs. It is recommended to judge this case when importing the source.
Plug-in market experience
The plug-in system has many functions and my personal energy is limited. I only selected some plug-ins and used them, and the performance was in line with expectations. I selected the plug-in support of APISIX to compare with MSE. Since APISIX is an open source product, I intentionally selected some relatively common capabilities so that they have comparative value.
In addition to the plug-ins shown in the table, the two gateway products also support many other plug-ins. You can find plug-ins required for basically common gateway scenarios, and MSE-Higress supports them. Different from the design of APISIX, MSE-Higress does not pile all functions on the concept of plug-ins. For example, mocking and redirection are controlled by routing configuration, cross-domain and flow limiting are controlled by routing policies, and quite a few functions are also controlled by plug-ins. Yes, at this point, I approve of the design of MSE-Higress, which can reduce the understanding cost for gateway users.
But at the same time, in terms of policy configuration flexibility, there is still room for optimization in the design of MSE-Higress. Taking current limiting as an example, since it is abstracted into the routing policy model, and this model does not support configuration to the consumer level, this This makes MSE-Higress lose the ability to limit consumer-level traffic.
In terms of commercial integration, since MSE-Higress is a gateway product officially provided by Alibaba Cloud, it also provides additional integrations such as: waf security protection, edas authentication plug-in, IDaaS authentication and authentication, etc., and is developed in a public cloud assembly style. In this mode, it can be better linked with existing cloud products, which is the biggest advantage compared to the plug-in capabilities provided by open source gateways.
EDAS x MSE-Higress canary release experience
MSE-Higress allows to be associated with multiple services when configuring routing. With this feature, many grayscale practices can be completed. This evaluation will verify the scenario where MSE-Higress and EDAS cooperate to complete canary release.
The meaning of canary is to first divert a small amount of traffic to the new version of the service, and most of the traffic remains in the old version.
The previous sc-dubbo-mixed-app application is still used, but it needs to be slightly modified to introduce the concept of version for SpringCloud and Dubbo services. Refer to the section "Managing Service Versions" [4], we can see that in the Nacos service discovery scenario , MSE -Higress is routed through node labels. The following is my modification.
EDAS deployment V1 version
SpringCloud service introduction version:
spring:
application:
name: sc-dubbo-mixed-app
cloud:
nacos:
discovery:
server-addr: edas-registry:8848 # EDAS会自行替换该连接串
metadata:
x-version: v1
@RestController
public class DemoController {
@Autowired
DemoService demoService;
@RequestMapping(value = "/echo", method = RequestMethod.GET)
public String echo() {
return "Hello MSE-Higress V1";
}
}
Dubbo service introduction version:
@DubboService(group = "default", version = "1.0.0",parameters = {"x-version:v1"})
public class IHelloServiceImpl implements IHelloService {
@Override
public String sayHello() {
return "Hello MSE-Higress V1";
}
}
Deploy the above version on EDAS and expand it into two copies. At this time, the contents of the two copies are consistent.
EDAS deploys V2 version in batches
New version of SpringCloud service:
@DubboService(group = "default", version = "1.0.0",parameters = {"x-version:v1"})
public class IHelloServiceImpl implements IHelloService {
@Override
public String sayHello() {
return "Hello MSE-Higress V1";
}
}
@RestController
public class DemoController {
@Autowired
DemoService demoService;
@RequestMapping(value = "/echo", method = RequestMethod.GET)
public String echo() {
return "Hello MSE-Higress V2";
}
}
New version of Dubbo service:
@DubboService(group = "default", version = "1.0.0",parameters = {"x-version:v2"})
public class IHelloServiceImpl implements IHelloService {
@Override
public String sayHello() {
return "Hello MSE-Higress V2";
}
}
Publish in batches in EDAS:
Let me explain here why EDAS canary publishing is not used, because EDAS canary publishing is mainly used for calls between microservices, not ingress traffic, and this evaluation is exactly MSE-Higress's evaluation of EDAS applications. In this case, what EDAS needs to do is publish in batches, ensuring that the backend has both v1 and v2 versions.
This completes the preparations for the canary release. There are two versions of the application, v1 and v2, and the rest is to configure MSE-Higress to drain the two versions according to a specific ratio (just imagine , if there is no canary release, since v1 and v2 are both the same machine, the traffic ratio should be 1:1).
MSE-Higress configures service version and label routing
In the service details, you can add the service version. The experience of MSE-Higress here is very good. Because it is associated with the registration center, the corresponding tag name and tag value can be automatically obtained, and the corresponding number of nodes can be calculated in real time without worrying about configuration. wrong.
It is necessary to modify the route-associated service method from single service to label route, and configure the traffic ratio between v1 and v2 to be 80:20.
At this point, I found that the Dubbo service could not be selected in the label routing, and then I noticed the prompt above "Multi-service and label routing functions do not support adding Dubbo service"! In other words, all my previous preparations for labeling the Dubbo service were in vain, but I still recorded the evaluation process.
Traffic ratio test
Observe the return value by debugging /sc-dubbo-mixed-app/echo 10 times:
Hello MSE-Higress
Hello MSE-Higress
Hello MSE-Higress
Hello MSE-Higress V2
Hello MSE-Higress
Hello MSE-Higress
Hello MSE-Higress
Hello MSE-Higress V2
Hello MSE-Higress
Hello MSE-Higress
In line with expectations.
Test summary
Through the above examples, it can be found that it is very simple for MSE-Higress and EDAS applications to implement canary release in the Nacos service discovery scenario, but some problems are also seen from it, that is, the product only tells users how to achieve canary release. The verification state has not completed the last mile, that is, at what stage canary release verification can be considered completed. After the release is completed, how to complete the batch release of EDAS, how to modify the label routing, and reach a final running state. Moreover, the configuration of this process is still very complicated. It needs to be integrated into the user's operation and maintenance system, which requires certain integration work. At least a best practice based on the MSE-Higress canary publishing portal application should be provided in a system like EDAS.
Experience summary
I probably browsed the other functions of MSE-Ingress, but due to limited energy and limited space, I can’t list them one by one. Let’s briefly summarize them.
In addition to the purchase process at the beginning of the article, MSE-Higress also supports the Ingress gateway as an ACK cluster, which benefits from its cloud-native genes and can be compared to Nginx Ingress, which is a good choice for companies willing to embrace the cloud-native ecosystem Good news, I won’t review this feature separately this time.
In terms of document support, my evaluation this time was completed with reference to the console copywriting and documentation. It can be seen that the documentation system is relatively complete, and some common questions are also highlighted in the document. Thumbs up. It should be noted that after some new functions are launched, existing relevant documents need to be updated. Taking "Migrating from Spring Cloud Gateway to Cloud Native Gateway" [5] as an example, the EDAS shared registration center source is currently supported. For For the case of using the EDAS shared registry in the document, there is no need to migrate first, which may cause misunderstandings for SpringCloud Gateway migration users.
MSE-Higress can play the role of security gateway and traffic gateway very well, but I think it remains to be discussed whether it can play the role of microservice gateway/business gateway well. Because the direct demand of the business gateway is to expose a large number of APIs within the enterprise through the gateway, the domain model of MSE-Higress is the routing/service model, which limits its abstraction of business capabilities. Most of the time, routing is still The role of a generic interface is often used to undertake a back-end application model. Judging from the user pattern, it may be that users on the operation and maintenance side will pay attention to the current MSE-Higress form instead of development.
If you use MSE-Higress in depth, you may have a demand for refined management of API interfaces. The current product design of MSE-Higress does not seem to be able to meet this demand well, which is specifically reflected in the routing model of MSE-Higress and the refined management of APIs. Contradictions between needs. If the routing model of MSE-Higress is configured as the prefix matching mode of pan-routing /order/*, all interfaces of the application will be exposed; if configured as the exact matching mode of /order/createOrder, the demand for refined management can be achieved. However, common requirements at the interface level, such as API input and output parameter definition, parameter mapping, and error code management, cannot be well adapted to the routing model. This may be a problem that most R&D users may face in the future when using MSE-Higress.
Overall, I am still very optimistic about the product MSE-Higress. The product interface interaction is also fashionable, and most operation processes are smooth; in terms of product integration, from its integration with WAF, EDAS, ACK and other products, we can see that Alibaba Cloud positions it not just as a gateway component, but I hope to use it to complete the construction of a product ecosystem. Serverless is quite popular on the cloud native official account. MSE-Higress does not yet support serverless services, which is a bit surprising. At the same time, it also has the open source attribute of Higress, which also solves some concerns about being tied up by Alibaba Cloud when selecting models.
Participate in the cloud-native gateway MSE-Higress evaluation and win awards
From August 10, 2023 to September 15, 2023, by experiencing MSE-Higress and conducting evaluation and creation around three major themes, you will have the opportunity to win a 30 yuan Mao Super Card, Mijia Desk Lamp Lite, and CHERRY Mechanical Keyboard MX3.0S Waiting for the grand prize.
Details of the evaluation activity: Aliyun product evaluation wins the award丨Cloud native gateway MSE-Higress
Related Links:
[1] "Configuration rewrite strategy"
https://help.aliyun.com/zh/mse/user-guide/configure-a-rewrite-policy
[2] "Configuring Conversion from HTTP to Dubbo Protocol"
https://help.aliyun.com/zh/mse/user-guide/configure-http-to-dubbo-protocol-conversion**
[3] "Realizing Routing and Forwarding of gRPC Services Based on Cloud Native Gateway"
https://help.aliyun.com/zh/mse/getting-started/route-the-traffic-of-grpc-applications-based-on-cloud-native-gateways
[4] "Management Service Version"
https://help.aliyun.com/zh/mse/user-guide/manage-service-versions
[5] "Migrating from Spring Cloud Gateway to Cloud Native Gateway"
https://help.aliyun.com/zh/mse/user-guide/migrate-services-from-spring-cloud-gateway-to-cloud-native-gateways