I have been doing testing for more than a year. Although I can complete my usual work well, I suddenly found that my understanding of the overall knowledge system about testing is very lacking. Therefore, I also did some testing in my spare time. knowledge supplement. If there are any shortcomings, please communicate more and learn from each other.
Now it seems that although in daily work, it involves automated testing, but more of it is functional testing. Today I learned about performance testing.
At the same time, I also clearly realize that for testing tools, it is a matter of time whether you know it or are familiar with it. As long as you use it frequently, mastering the basic knowledge of testing and understanding some testing ideas and concepts will help us better. The benefits are endless.
Here is a summary of the performance testing I learned:
Performance Testing : It uses automated testing tools to simulate a variety of normal, peak and abnormal load conditions to test various performance indicators of the system . Load testing and stress testing are both performance tests. Through load testing, the performance of the system under various workloads is determined. The goal is to test the changes in various performance indicators of the system when the load gradually increases. Stress testing is a test that determines the bottlenecks or unacceptable performance points of a system to obtain the maximum service level that the system can provide.
Load Testing : It is a system load that simulates the load conditions endured by actual software. Through continuous loading (such as gradually increasing the number of simulated users) or other loading methods, the response time and data throughput of the system under different loads are observed . The resources (CPU, memory, etc.) occupied by the system are used to test the behavior and characteristics of the system to discover possible performance bottlenecks, memory leaks, inability to synchronize in real time, etc. in the system. Load testing more reflects a method or a technology.
Stress testing : Testing under strong load (large data volume, large number of concurrent users, etc.) to check the operating behavior of the application system under peak usage conditions, thereby effectively discovering certain functional risks of the system and whether the system has Good fault tolerance and recoverability. Stress testing can be divided into long-term (such as more than 24 hours) stability stress testing under high load and destructive stress testing that causes system collapse under extreme load conditions.
The difference between the three : Starting from the purpose of testing and the needs of users, it is easier to distinguish between performance testing, load testing and stress testing. Performance testing is to obtain performance indicator data of the system under certain specific conditions (including specific load conditions), while load testing and stress testing are to discover problems in the software system, including performance bottlenecks, memory leaks, etc. . Passing the load test is also to obtain the maximum load that the system can withstand during normal operation. The load test at this time becomes a capacity test. Through stress testing, we can know under what extreme circumstances the system will crash, whether the system is self-restorable, etc., but more importantly, it is to determine the stability of the system.
Performance testing tools (currently I only know that these tools exist, and I will summarize their usage later in the process):
(1) Apache Jmeter: User's ManualApache JMeter - User's Manual
(2) Load Runner
(3) QTP(Quick Test)
(4) Web Polygraph
Performance testing is not only familiar with the testing tools, but also pays attention to the testing ideas. Below you can learn about three concepts about performance testing:
- precise and vague
ie How much gasoline does a car need to drive 100 kilometers?
Make assumptions. There are three stages of assumptions below:
a. Making assumptions but not knowing that they have made assumptions
Some people do tests based on their own personal experience, and then write test reports for others to see. The key is that you feel that your tests are correct, but not everyone is in the environment you are in and do the same things you do when testing. Exactly the same thing, so the test results will only mislead others. For example, regarding the fuel consumption issue mentioned earlier, some people's approach is to drive 100 kilometers to see what the result is.
b. Made too many assumptions
"When the road is flat, the light is green, the wind speed is 5km/h, there is only one 70kg passenger, the speed is stable at 70km/h, good driving habits,..., the fuel consumption is 7.1L/100km." This may be very strict, But for readers of your report, such data will not mean much.
c. Make necessary and reasonable assumptions
There are times in life when compromises are necessary and reasonable. Because out of the blue, our tests need to provide valuable information, it is acceptable to make necessary and reasonable assumptions for such valuable information.
- Macro and Micro
It's also an interesting dichotomy. When doing performance testing, especially the performance testing of the entire product, what we see are the core functions and main large functional modules of the product, such as databases, web servers, core daemons , etc. In our mind, we have an architecture diagram, even if you don't draw it. So sometimes, we think that performance testing has a macro perspective on products, looking at large components rather than specific details.
Is that really the case? Take a look at the following example:
1. After changing the daemon's log level to debug (log_level changed from 2 to 5), the performance dropped by almost half.
2. Turn off a cache option
3. Turn on the keepalive option
4. Turn on DNS reverse query
......
The above are all detailed settings, just a configuration item, hidden in a certain table or ini in the DB. But after making the change, the performance results obtained may be very different.
At this time, whether you should consider the details or not depends mainly on whether he is critical. As for what is crucial, this still needs to be thought and summarized in future work.
- projects and tasks
Performance testing is definitely a task in itself, both for the person assigned to do it, and for the person assigned to do it. But sometimes it's like a project, for the people who do it. why?
First you need to deal with a lot of people.
Product manager or customer: Obtain needs and set goals.
QA manager/lead: Discuss resources and schedule. Including the required machines, environment, software, and the entire plan.
Developers: Find issues, tune, etc.
Owner of functional testing: Performance testers may not know all the functions.
Admin: Lab, network, DB, etc.
Secondly, it is a long-term and long-span work. Especially for a relatively large product. You need to prepare a detailed test design, including goals, scope, possible methods, as well as the resources and time plan mentioned above; then invite many people to review the plan; and then prepare tools, environments, and test data. Then there is execution, recording the analysis results. If there are problems, you need to make repeated adjustments and regressions. Finally, the report must be compiled and questions answered.
It is said to be a project because of the complexity of the work for the above reasons, and also because performance testing is of an evaluation nature, because you are trying to measure, measure or evaluate something, and it involves comparison. Absolute results. This leads to performance testing inevitably introducing some authority issues, although you don't necessarily expect this. This makes a lot like any other project, with expectations management and the need for good external communication and coordination. So sometimes, I prefer to treat it as a small project, so that it may be more comprehensive.
Thank you to everyone who reads my article carefully. Looking at the increase in fans and attention, there is always some courtesy. Although it is not a very valuable thing, if you can use it, you can take it away for free:
① More than 2,000 software testing e-books (mainstream and classic books should be available)
② Software testing/automated testing standard library information (the most complete Chinese version)
③ Project source code (forty or fifty interesting and classic practice projects and source code)
④ Python programming language, API interface automated testing, web automated testing, App automated testing (suitable for beginners)