Apache JMeter is a 100% pure Java open source software designed to load test functional behavior and measure performance. It can be used to test the performance of static and dynamic resources such as static files, Java Servlets, CGI Scripts, Java Objects, databases and FTP servers. JMeter can be used to simulate large loads to test the robustness of a server, network or object or to analyze overall performance under different loads. Compared with other performance testing software, JMeter has the following advantages:
■ Open source : JMeter is an open source free software that supports interface and performance testing of multiple protocols;
■ Lightweight : JMeter's software package is small and lightweight, and it only needs to run in the JDK environment, without any additional deployment and installation (stand-alone version);
■ Powerful function : JMeter was designed as a simple Web performance testing tool at the beginning, but after continuous updating and expansion, it can now complete the testing of database, FTP, LDAP, WebService and so on. Based on JMeter's own open source, users can extend its functions according to their own needs.
Limitations of JMeter Distributed Performance Stress Testing Solution
In recent years, with the development of online businesses such as Internet+ and online e-commerce, performance testing has gradually become one of the types of high-frequency testing for enterprises. As the most widely used open source performance stress testing tool, JMeter has been used by many enterprises in the actual environment.
However, the stand-alone version of JMeter can support up to a few hundred to a thousand simulated request threads under the general press configuration, because it is limited by JMeter's own mechanism and hardware configuration. After the scale is expanded again, it is easy to cause freezes and no response. Sometimes, in order to simulate business scenarios as much as possible, enterprises need to simulate a large number of concurrent requests. At this time, a single JMeter press seems powerless.
In response to this situation, JMeter's solution is to support distributed performance stress testing, that is, to distribute a large number of simulations to multiple presses concurrently to meet this high-traffic concurrent request scenario. The principle of JMeter's native distributed performance stress testing solution is as follows:
1. In the distributed test, select one machine as the master (Controller), and other machines as the test execution slaves (Agent);
2. When executing the test, the Controller sends the test script to the Agent through the command line, and then the Agent executes the test and sends the test result to the Controller;
3. After the test is completed, you can see the test results sent by the Agent in the listener on the Controller, which is a summary of the test results of multiple Agents.
However, although JMeter's native master-slave solution can support large concurrent pressure measurement requirements, its solution also has certain limitations, which are reflected in the following aspects:
1. The JMeter master-slave solution is cumbersome to deploy. In addition to ensuring that the Controller and Agent machines are in the same subnet, it is also necessary to ensure the consistency of the JMeter version, Java version, and plug-in version;
2. If JMeter pressure test needs to use CSV or additional JAR package, etc., it needs to copy one copy on each Agent, and ensure that all path configurations are consistent;
3. If the CSV of JMeter pressure measurement requires different data read by each Agent pressure measurement machine, it needs to be manually divided, cut and placed in advance;
4. JAR packages, JMX files, and stress test reports that JMeter relies on during stress testing are all locally saved and managed, and cannot be managed and retained in a unified manner, which will cause inconvenience and confusion in management;
5. JMeter can cover basic CPU, memory and other monitoring through plug-ins during stress testing, but cannot cover application-level monitoring, such as the number of MySQL connections, JVM usage, etc., and cannot support user-defined monitoring.
Optimization of MeterSphere Based on JMeter Distributed Performance Stress Measurement
In view of the above shortcomings of the JMeter distributed performance stress testing solution, the MeterSphere open source continuous testing platform optimizes the performance stress testing of JMeter from the perspective of technology and management, while ensuring the compatibility of the use of JMeter. At the same time, MeterSphere also supports invoking the Kubernetes environment and cloud environment for stress testing, easily supporting high-concurrency and large-scale performance testing. The MeterSphere performance test overview dashboard is shown below:
The MeterSphere distributed performance test solution and the components involved are as follows (Note: For component introduction, refer to the MeterSphere official website link: https://metersphere.io/docs/system_arch/):
■ Node Controller: Provides a test resource pool of independent node type for performance testing, receives performance testing tasks from the system, and dynamically starts the JMeter container to complete performance testing;
■ MySQL: The main data of the MeterSphere project is stored in the MySQL database;
■ Kafka: Receive performance test result data generated by JMeter;
■ Data Streaming: Obtain performance test result data from Kafka, and store it in MySQL database after processing;
■ Docker Engine: Provides the JMeter container runtime environment for Node Controller.
The solution architecture diagram of the MeterSphere distributed performance stress measurement solution is shown in the following figure.
1. The user sets the performance stress testing parameters (concurrency, time, gradient, stress testing resource pool, file segmentation, etc.) on the MeterSphere interface;
2. According to the pressure measurement configuration parameters set by the user, MeterSphere automatically selects the resource pool, distributes the pressure measurement scripts and files, and automatically starts the JMeter pressure measurement node (started through Docker) for pressure measurement;
3. According to the distributed scripts and parameters, the pressure measurement node pushes the pressure measurement data to the Kafka cluster after real-time or local calculation;
4. The Data Streaming cluster consumes the pressure measurement data in the Kafka cluster in real time, and then dynamically writes it into the MySQL database after calculation;
5. If the relevant Prometheus monitoring is configured, MeterSphere will automatically collect the performance monitoring data of the system under pressure during the pressure test;
6. After the stress test is over, all data will be saved in the database, which is convenient for subsequent analysis, sharing and query.
Combining the above-mentioned principles of the MeterSphere distributed stress testing solution, we found that the MeterSphere distributed stress testing solution has the following advantages over the JMeter native distributed stress testing solution:
1. Management advantages
■ All stress testing scripts and dependent packages, such as JAR packages and other files, are stored and managed uniformly in MeterSphere. During stress testing, MeterSphere is uniformly distributed to different stress testing clusters, and supports automatic segmentation of CSV files during distribution. ;
■ After each pressure test of MeterSphere is completed, the report will be persistently stored, which is convenient for subsequent query, sharing and analysis, and supports the comparison of different test reports at the same time;
■ Support the management of teams, projects, personnel and permissions. In some large-scale performance tests, different team members may be involved, such as operation and maintenance personnel, DBAs, developers, etc. MeterSphere can well manage the collaborative management of different team members.
2. Technical advantages
■ MeterSphere is fully compatible with JMeter's plug-ins and protocol stacks. In addition to supporting JMeter's most common HTTP (HTTPS) protocol, MeterSphere also fully supports WebSocket, MQTT, FTP, SSH, database and other performance tests;
■ All stress testing clusters use Docker to automatically start JMeter when receiving the MeterSphere stress testing task, without the need to manually deploy JMeter machines, and at the same time ensure the consistency of the JMeter version and Java environment of all stress testing nodes;
■ All JMeter stress testing hosts do not have network dependencies on the same subnet. Based on this feature, Internet traffic access in different regions can be easily simulated in MeterSphere. For example, the hosts of public clouds such as Shanghai area, Hangzhou area, and Beijing area can be dispatched through MeterSphere to perform stress test on the target system;
■ Introducing Kafka message queue and Data Streaming computing cluster, all pressure measurement data is not directly sent back to MeterSphere, which reduces the pressure on MeterSphere (the pressure measurement data in JMeter's native solution is sent back to the main controller);
■ Built-in integrated Prometheus monitoring system. At the same time of stress test, users can customize the monitoring indicators. In addition to the commonly used CPU, memory, disk, etc., it can also support monitoring indicators such as the number of MySQL connections and JVM;
■ All components (Kafka, Node Controller stress testing cluster, Kubernetes stress testing cluster, and Data Streaming computing cluster) can be dynamically expanded and expanded. When the concurrency of stress testing needs to be higher, different components can be dynamically expanded, thereby improving the performance of stress testing. Concurrency capability.
MeterSphere distributed performance testing practice
1. Deploy the MeterSphere platform
For the platform deployment method of MeterSphere, please refer to the official website (https://metersphere.io/docs/quick_start/quick_start/). The official website gives a detailed one-click deployment script:
curl -sSL https://github.com/metersphere/metersphere/releases/latest/download/quick_start.sh | sh
2. Deploy the Node-Controller
To deploy the pressure division test node, you can install the entire MeterSphere service first, and then modify the configuration file. The specific operations are as follows:
① Install MeterSphere: curl -sSL https://github.com/metersphere/metersphere/releases/latest/download/quick_start.sh | sh
② Stop the service: msctl down
③ Modify the installation mode: vim /opt/metersphere/.env
④ Start the service: msctl reload
3. Add a voltage divider node to the interface
Log in to the MeterSphere platform, select "System Settings" → "Test Resource Pool" in turn, edit the NODE-LOACL resource pool, add a stress test node and save it.
4. Upload the local JMX stress test script and dependency files
① Select "Performance Test" → "Test" → "Create Performance Test" in turn.
② Select the "Load JMX file" option. It is recommended not to use pre- and post-scripts in the JMX stress test script, so as not to affect the accuracy of the performance test.
If the JMX script of the stress test contains dependent files, third-party plug-ins, or custom JAR packages, you need to modify the address in the JMX script to an absolute path.
There are two common places that need to be modified, as follows:
■ The JAR package directory in the test plan
■ CSV address in CSV file settings
③ If the JMX script of the stress test contains dependent CSV files, or third-party plug-ins or JARs, you need to select the "Load File" option to upload at the same time.
5. Set stress test parameters
Enter the "Pressure Configuration" page, and you can set the corresponding pressure measurement configuration parameters (Note: This parameter will override the configuration in the JMeter thread group).
The default allocation strategy is equal distribution. If you want different pressure measurement nodes to distribute different pressures, you can select the "custom" distribution method. This distribution method can be customized to set different pressure distribution weights.
6. CSV segmentation and configuration monitoring
If the stress test JMX script contains dependent CSV files that require different stress test nodes to read different data, you can enable the "CSVDataSet" option under the "Advanced Configuration" page to perform automatic CSV segmentation, as shown in the following figure:
For details on the configuration and design of monitoring, please refer to the official MeterSphere tutorial: https://blog.csdn.net/FIT2CLOUD/article/details/119673564.
The monitoring configuration added this time is as follows:
7. Viewing, comparison and analysis of reports
After all configurations are completed, select the "Save and Execute" option, and MeterSphere will automatically refresh and display the stress measurement data report.
At the same time as the pressure test, you can view it through the background, and the added pressure test node has automatically run the JMeter container for the pressure test.
]
The test report generated by MeterSphere supports the display of test details. Different display items and time ranges can be selected on the "Test Details" page. Once a time range is selected, the reported values are recalculated for the selected time range.
On the "Monitoring Details" page, you can view the monitoring data of the terminal under pressure during the pressure test.
For a performance test, you can select different reports on the "Report Comparison" page and conduct comparative analysis.
Summarize
It can be seen that the distributed performance stress test of the MeterSphere open source continuous testing platform is more suitable for test teams and testers to use than JMeter's native distributed performance stress test.
First, the deployment scheme of MeterSphere is simpler. MeterSphere stress test execution nodes support on-demand creation and installation, and components can be dynamically expanded to easily support large-scale performance testing;
Secondly, MeterSphere is easier to manage. The MeterSphere open source continuous test platform adopts B/S architecture design, and performance test scripts, performance test tasks and performance test reports can be effectively isolated and shared and collaborated in the dimensions of projects and teams.