Abstract : Software testing is often regarded by practitioners as a work link that does not directly create value in the software field. The industry has also been promoting software testing practitioners to pursue high-quality, efficient and low-cost testing. With the rapid development of AI & big data technology, it has also brought good news to software testing. Baidu MEG quality and efficiency platform has been exploring the application scenarios of AI technology in the field of software testing since 2018. With the continuous breakthrough of exploration, it is found that software testing contains extremely rich AI scenarios. We define intelligence testing as a test method that uses data and algorithms to enable quality activities; after discussion and exploration, we divide intelligence testing into three stages: computational, perceptual, and cognitive intelligence.
01The first stage of intelligent test development: computational intelligence
This stage uses the behavior data + algorithm + machine computing power generated by the software process to embed in the quality activities, assist and intervene in the testing behavior, and then achieve the goal of improving quality and efficiency; this stage does not pursue high-precision algorithms, the purpose is to prove data And algorithms can play a huge role in software testing.
At this stage, we use genetic algorithm and task priority algorithm to shorten the queuing time of test tasks, apply memory leak detection based on DTW algorithm, realize accurate playback of tens of billions of traffic based on Pearson correlation coefficient and bucketing algorithm, and use case screening based on JC distance. The technology implements scenarios such as a significant reduction in use case execution.
We have spent nearly two years mining 50+ application scenarios in software testing activities, and all of them have achieved very positive results. However, in the process of practice, we also recognize the limitations of computational intelligence, such as the use case recommendation based on coverage, if the underlying code is modified, most of the use cases will be selected, but the actual use cases still have no ability to uncover errors, resulting in extreme Big waste, in order to solve such problems, perception intelligence came into being.
02The second stage of intelligent test development: perceptual intelligence
Confidence at this stage comes from two realities in the software testing field: 1. Not all software change behaviors bring risks; 2. Not all software testing activities can reveal risks. Therefore, there is a huge waste of resources and problems in the field of software testing, which prompts us to try our best to explore perceptual intelligence.
This stage uses behavioral data generated by the software process + higher-level algorithms + machine computing power to perceive risks and make decisions like humans. The main purpose is to be able to perceive, identify risks and make decisions like humans. Guide quality activities. The exploration at this stage includes: the application of vision technology in front-end automation use case writing, pop-up window removal, UIDIFF and other fields; use case recommendation technology based on risk estimation Bayes + catboost to achieve use case recommendation ratio from 50% to 10% improvement; 70% low-risk unattended launch of an autonomous testing system based on LR model to estimate project risk; white-box code defect detection based on deep learning, etc. The breakthroughs in these scenarios and technical applications prove that AI can completely replace humans in software testing to perceive, identify risks, and decide on quality activities, making software testing activities efficient enough. Baidu Smart Test is currently mainly conducting continuous exploration at this stage.
03The third stage of intelligence test development: cognitive intelligence
This stage is a continuation of perceptual intelligence. The perceptual intelligence stage can perceive and identify risks and decide what quality activities to perform, but the quality activities may still be completed by humans, such as use case writing, scene design, manual clicks, etc. Cognitive intelligence It is hoped that after perceiving the risk, the machine can actually react to reveal the risk. At this stage, we explored the AST-based intelligent exception unit test code generation technology to realize C++ exception single test recall core, infinite loop and other issues, based on UCB priority policy traversal technology achieves higher page coverage per unit time, intelligent task failure location technology reduces manual inspection costs, and delivers continuous integration pipeline self-healing technology; this stage is still in its infancy, but we believe that with AI technology With the continuous maturity and innovation, the cognitive intelligence stage will also usher in a big explosion.