Not long ago, on November 16, the Australian Securities Exchange (ASX) launched a new trading system, but it was forced to shut down due to a malfunction. This was the most serious accident after the market was closed due to hardware failure in 2016.
After testing for more than a year, it crashed the day it went online
At noon on November 16, ASX issued a statement stating that the market will be closed that day and will reopen at normal hours the next day. The reason for the closure given by the exchange is that "software issues limited to the trading of multiple securities (combined transactions) in a single transaction order have led to inaccurate market data."
The upgraded system of ASX is the latest generation trading system developed by Nasdaq and is currently widely used worldwide. In order to ensure safe operation after the launch, ASX, technology provider Nasdaq (Nasdaq), customers and third-party independent experts have done extensive testing for more than a year, including four rehearsals.
How to test microservices?
After watching the excitement, also take a look at our own system. With the popularity of the microservice architecture represented by Spring Cloud and Dubbo, many companies now adopt the microservice architecture. With more and more services, how should these services be tested? How to prevent the system risks mentioned above?
Let's take a look at the risks of the online system, and then make a corresponding test plan for the corresponding risks. Take the following architecture as an example:
Generally speaking, the risks of a business system come from two sources:
- One is the risk of changes , such as the launch of the new system mentioned earlier, or we fix a bug in the shopping cart service in the picture above.
- The second is daily risks , such as the underlying database, host, network and other software and hardware problems.
As shown:
First of all, we can use the following test methods to verify the risks caused by changes:
Development self-test
After developing a feature, the developer needs to make every effort to ensure that the logic is correct before submitting it for testing. Tools such as IDEA or Postman can test the HTTP interface locally and can be used to test Spring Cloud services:
For Dubbo service, because it is based on tcp protocol, development needs to write a simple conumer by yourself to verify the function of the service:
Test environment verification service
After the development and testing are submitted, testers also need to test the service to ensure that the service can work normally. The test at this time is generally carried out in a separate test environment.
For Spring Cloud services, testers can fill in the destination IP, url, and parameters on tools such as Postman to initiate requests and verify services:
For Dubbo services, Dubbo Admin also provides a service test function, which can initiate calls on the page to verify the service:
For security reasons, the general test environment is isolated from the public and office networks. For example, the test environment is a separate VPC on Alibaba Cloud. In this case, if you need to use postman or dubbo admin, you need to get through the network, such as a dedicated line.
If your service is connected to Alibaba Cloud's Microservice Engine (hereinafter referred to as MSE), then you can directly initiate a test request on the MSE console, regardless of network and permission issues.
On the MSE console, click the Microservice Governance Center -> Microservice Test -> Service Test menu, and after selecting the service and method, you can select the node to be called and fill in the parameters of the call on the service test page:
It can be seen that for complex structures in the input parameters, such as the ProductItem in the figure, the service test function of MSE will also generate sample data, so the tester does not need to go through the code to see how to fill in the input parameters.
After filling in, you can click Execute to execute the test:
The service testing function of MSE also supports Spring Cloud interface testing:
Overall regression testing
After a service is released, testers need to verify the more important product processes. For example, in the architecture diagram, several services are called from browsing the product to the final order, and the tester needs to return to this function as a whole.
For simple scenarios, in the case of infrequent launches, testers can manually complete the entire process to see if the entire business process is normal. If the business scenario is too complex, or the online is frequent, then automated testing is required. Generally speaking, each company will build its own CI system to complete this integration test.
Taking Jenkins as an example, you need:
- Build Jenkins, configure the network and other environments
- Create a test script warehouse and add test scripts
- Configure pipeline in Jenkins
- Execute and view results
Similarly, network issues in different environments must be handled here, especially regression testing in a production environment, which requires strict control of permissions.
As an overall solution for microservices, MSE also provides automated regression capabilities to complete regression testing with one click. First, click the Microservice Governance Center -> Microservice Test -> Service Test menu to create a new automation use case. At each step, a Spring Cloud and Dubbo service can be called, and assertions can be added to verify that the test passes:
After clicking "Execute Now", you will see the results of the automated regression on the execution history page:
Service inspection
In addition to the risks caused by changes, we also need to deal with daily risks, such as problems with components such as databases and networks. In order to deal with these risks, we need to periodically verify whether this service can work normally. Usually, many companies will use the timing execution function of the CI system to construct a scheduled task. If the task execution fails, an alarm will be automatically triggered.
Take Jenkins as an example. In addition to building Jenkins and writing test scripts, you also need to set Jenkins tasks to be triggered regularly:
For example, if we need to check whether the commodity service is normal, we can write a Jenkins timing task to call the commodity service and regularly check whether the commodity service is normal. Of course, if your service is connected to MSE, you can also use the service inspection function provided by MSE to periodically check the service. If it does not work as expected, then immediately send an alert to notify the subscriber of the alert.
Click the Microservice Governance Center->Microservice Test->Service Inspection Menu to create a new inspection task:
Then click the start button of the corresponding inspection task in the list to start the inspection:
If there is an error in the inspection, there will be a corresponding alarm. Take the Dingding alarm as an example:
Of course, here also supports setting email and SMS alerts. You can set different alarm receiving methods according to the importance of the service.
Service stress test
The flow of the system is constantly changing. In order to cope with the sudden flow that may appear, we need to evaluate the system pressure in time to decide whether to expand. On the other hand, the code is constantly being revised, so it is necessary to pressure test each time before going online to see if the code performance has significantly deteriorated. In terms of stress testing, Apache JMeter can test Spring Cloud services and Dubbo services well.
For Spring Cloud services, you can use the HTTP sampler that comes with JMeter to pressure test:
For the stress test of Dubbo service, jmeter-plugins-for-apache-dubbo adds Dubbo sampler, which can be used to test Dubbo service.
Just set the registry, interface, method, etc. on the configuration page of the Dubbo sampler to create a pressure test task.
After creating the stress test task, you can execute the stress test task through the jmeter command line and get the stress test result:
jmeter -n -t ./rest-order-thread-group.jmx -l ./result.txt -e- o ./webreport
Finally, if your service has been connected to MSE, MSE also provides out-of-the-box stress testing capabilities.
Click the Microservice Governance Center -> Microservice Test -> Service Stress Test menu, create a new stress test scenario, and configure the stress test parameters:
You can also configure parameters such as pressure model on the pressure configuration tab:
After the configuration is complete, you can start the stress test on the details page of the corresponding stress test scenario. You can also view the results on the details page of the stress test scenario. If you need more detailed results, please click the details button of the running record.
to sum up
Microservice testing, whether you build your own test system or test through MSE, is to deal with the risks and daily risks caused by changes. Only by understanding the risks can we respond in time to ensure high service availability.
Compared with the self-built test system, the Alibaba Cloud product MSE provides the same function, which not only avoids users from dealing with the problem of interoperability between different networks, but also provides a complete authority management function to ensure stable and safe online operation. In addition, MSE also provides functions such as registration configuration center hosting, microservice management, etc. Welcome to experience.
This article is the original content of Alibaba Cloud and may not be reproduced without permission.