Stability testing is one of the core contents in the field of automated testing . What aspects should be considered in stability test design? How to maximize test output on limited samples? Combining some experience and understanding of automation, the author summarizes the five core elements of stability test design: pressure, randomness, concurrency, interaction, and time. Why not, let's take a look together.
pressure
In English, stability is called Stability; and pressure is called Stress. In the field of testing, many people equate stability testing with stress testing. actually not. One of the most important ways to achieve stability is through stress, but stress is not the only way to test stability. The connotation of pressure is to do the same thing repeatedly, through long-term and multiple rounds of execution, in order to achieve the goal of making the function break or the system crash. In reality, this is often a more effective and feasible way.
However, one of the most important problems of stress testing is that the effect of repeating a simple function 1000 times may be very different from executing 1000 simple functions in turn. The former may not be able to find the problem, but the latter is very likely to find the problem. why? This involves the following elements.
random
In the stability test, if the cases are executed step by step, the output may not be very good. Because your sequence of events is fixed and predictable without too many "surprises".
Imagine that there are several paths between point A and point B, but you take the same path every time, then you may know the unevenness of this path well, but you know nothing about the situation on other paths. known.
If you want to find out all the pits between A and B, ideally, you can traverse every path between A and B. But the reality is that you don't know how many paths there are, and it's very difficult to traverse them all. Therefore, it is necessary to add some randomness. Random allows you to construct as many combinations as possible within limited time and space, thus creating more "accidental" discoveries.
concurrency
The real world is often complex and changeable. For example, a call may come in while you are swiping WeChat, and the network may change when you answer the call, and so on. Most of this kind of concurrency is impossible to know in advance, which can be called "interruption", which is the concurrency between the test object and the surrounding environment; but in real user scenarios, there are still a large number of natural concurrency scenarios, such as listening to music. Then surf the Internet, answer calls while playing games and so on. This is the concurrency between different functional modules inside the test object.
Why does concurrency help to find stability problems? Firstly, because it is a scene that exists in a large number in the real world; secondly, because it involves competition and coordination of system resources, the internal state of the system is much more complicated than a single scene, which is more likely to trigger problems.
The number of concurrent scenes can be two or more. The more concurrent its state becomes, the more complex it will be.
interact
Interaction and concurrency are not exactly the same. Concurrency is generally multiple independent functions or states running at the same time. But interaction refers to the collaborative work between multiple subsystems involved in completing the same operation.
For example, if you open Alipay to scan your face, then you need to call the camera; if you project your mobile phone to the TV through WIFI to play games, then it involves the collaboration and interaction of multiple parts such as GPU, WIFI, and Display. Considering subsystem interactions when designing automation can greatly improve test output.
time
Generally speaking, after considering the above four elements (pressure, randomness, concurrency, and interaction), you can basically find many stability problems. If you have noticed all the above, but still can't find the problem, then it may be that you haven't executed it long enough.
Time is a very important factor, but it is also the most easily overlooked factor. As time goes by, the state of the device will change more, such as the storage space is getting smaller and smaller, and the system is getting slower and slower. In these cases, the device is more prone to problems.
The only consequence of changing test execution strategies frequently is that each strategy's execution time is compressed and under-executed. So, do the first four points well, and then just wait for the flowers to bloom.
To sum up, a good stability test must fully consider the impact of these five elements: appropriate pressure + sufficient randomness + targeted concurrency and interaction + sufficient time, then it can better cover the product in Quality requirements in terms of stability.
Finally, I would like to thank everyone who has read my article carefully. Reciprocity is always necessary. Although it is not a very valuable thing, you can take it away if you need it:
These materials should be the most comprehensive and complete preparation warehouse for [software testing] friends. This warehouse has also accompanied tens of thousands of test engineers through the most difficult journey, and I hope it can help you! Partners can click the small card below to receive