I have been doing testing for more than a year. Although I can complete my usual work very well, I suddenly found that I have a lack of understanding of the overall knowledge system about testing. Therefore, I have done some testing in my spare time. knowledge supplement. For the shortcomings, please communicate and learn from each other.
Now it seems that although the usual work involves automated testing, it is more about functional testing. Today I learned about performance testing.
At the same time, I also clearly realize that for testing tools, whether or not familiarity with them is a matter of time, as long as you use them frequently, but mastering the basics of testing and understanding some testing ideas and concepts will enable us The benefits are endless.
Here's a summary of what I've learned about performance testing:
Performance Testing ( Performance Testing ): It is to test various performance indicators of the system by simulating various normal, peak and abnormal load conditions through automated testing tools. Both load testing and stress testing are performance testing. Through load testing, determine the performance of the system under various workloads. The goal is to test the changes in various performance indicators of the system when the load gradually increases. Stress testing is a test to obtain the maximum service level that the system can provide by determining the bottleneck or unacceptable performance point of a system.
Load Testing ( Load Testing ): It is a system load that simulates the load conditions of the actual software. Through continuous loading (such as gradually increasing the number of simulated users) or other loading methods to observe the response time and data throughput of the system under different loads, The resources occupied by the system (CPU, memory, etc.) are used to test the behavior and characteristics of the system to discover possible performance bottlenecks, memory leaks, and real-time synchronization problems in the system. The load test reflects more of a method or a technology.
Stress testing ( stress testing ): test under strong load (large data volume, a large number of concurrent users, etc.), to check the operating behavior of the application system under peak usage conditions, so as to effectively discover a certain functional hidden danger of the system, and whether the system has Good fault tolerance and recoverability. Stress tests can be divided into long-term (such as more than 24 hours) stability stress tests under high load and destructive stress tests that cause system crashes under extreme load conditions.
The difference between the three : From the purpose of testing and the needs of users, it is easier to distinguish between performance testing, load testing and stress testing. Performance testing is to obtain the performance index data of the system under certain conditions (including specific load conditions), while load testing and stress testing are to discover problems in the software system, including performance bottlenecks, memory leaks, etc. . Passing the load test is also to obtain the maximum load that the system can withstand when it works normally, and the load test at this time becomes a capacity test. Through stress testing, we can know under what extreme conditions the system will crash, whether the system is self-recovery, etc., but more to determine the stability of the system.
Performance testing tools (I only know of these tools at present, and I will summarize their usage methods later in the process of use):
(1) Apache Jmeter: User ManualApache JMeter - User's Manual
(2) Load Runner
(3) QTP(Quick Test)
(4) Web Polygraph
Performance testing is not only about being familiar with testing tools, but also about the testing ideas. Here are three concepts about performance testing:
- Precise and Fuzzy
ie How much gasoline does a car need to drive 100 kilometers?
Make assumptions, there are 3 stages of assumptions:
a. Make assumptions without knowing that they are making assumptions
Some people do tests based on their own experience, and then write a test report for others to see. The key is that you think your test is correct, but not everyone is in your environment. Do it the same way you do it when you test Exactly the same thing, so the test results will only mislead others. For example, the problem of fuel consumption mentioned above, some people's practice is that I drive 100 kilometers to see, and I will get as much as I can.
b. Make too many assumptions
"When the road surface is smooth, all the way is green, the wind speed is 5km/h, there is only one 70kg passenger, the speed is stable at 70km/h, good driving habits,... , the fuel consumption is 7.1L/100km." This may be very rigorous, But to the readers of your report, such data doesn't mean much.
c. Make necessary and reasonable assumptions
There are times in life where compromises and compromises are required, if they are necessary and reasonable. Because jumping out, our tests need to provide valuable information, so it is acceptable to make necessary reasonable assumptions for such valuable information.
- macro and micro
It's also an interesting counterpoint. When doing performance testing, especially the performance testing of the entire product, we see the core functions of the product and major functional modules, such as databases, web servers, core daemons , and so on. In our minds, we have an architecture diagram, even if you don't draw it. So sometimes, we think that performance testing has a macro perspective for products, looking at large components rather than specific details.
Is that really the case? Take a look at the example below:
1. After changing the log level of the daemon to debug (log_level changed from 2 to 5), the performance dropped by almost half.
2. Turn off a cache option
3. Turn on the keepalive option
4. Open DNS reverse lookup
......
The above are all minutiae settings, just a configuration item, hidden in a certain table or ini of the DB. However, after changing it, the performance results obtained may be very different.
At this time, in fact, whether to consider the minutiae mainly depends on whether he is critical or not. As for what is crucial, this needs to be considered and summarized in future work.
- projects and tasks
Performance testing is certainly a task in itself, both for the person who is assigned to do it, or who arranges it. But sometimes it is like a project, for those who do it. why?
First of all you need to deal with a lot of people.
Product manager or customer: Get requirements, set goals.
QA manager/lead: Discuss resource and schedule. Including the required machines, environment, software, and the entire plan.
Developers: Finding issues, tuning, etc.
The owner of the functional test: The performance tester may not know all the functions well.
Admin: Lab, network, DB, etc.
Secondly, it is a work with a long cycle and a large span. Especially for a relatively large product. You need to prepare a detailed test design, including goals, scope, possible methods, and the resources and time plans mentioned above; then invite many people to review the plan; next, prepare tools, environments, and test data. Then execute and record the analysis results. If there is a problem, it is necessary to repeatedly adjust and regression. Finally, organize the report and answer questions.
It is said that it is a project because of the complexity of the work due to the above reasons, and some reasons are because performance testing has the nature of evaluation, because you are trying to measure, measure or evaluate something, and there is a comparison Absolute result. This leads to the inevitable introduction of some authoritative questions to performance testing, although you don't necessarily expect this. This makes a lot of things like any other project, with expectations management and the need for good external communication and coordination. So sometimes, I prefer to treat it as a small project, so that it may be more comprehensive.
[Performance Test] Finally, there is a set of comprehensive performance test tutorials! Real enterprise performance testing full-process project actual combat!