1. Background and concept
With the rapid development of the software testing industry, testing concepts and technologies are undergoing rapid changes. Therefore, a complete set of automated test cases is indispensable for every software company. However, although there is such a large-scale automated case set resource investment and a large amount of manpower input, when the version is updated, it is impossible to know exactly which ones. The function is affected, and the test cases cannot be accurately selected.
And financial technology is developing rapidly. There are more and more software serving the banking industry, demand iterations are getting faster and faster, the system architecture is becoming more and more complex, and the testing cycle is getting shorter and shorter. The banking business system has a long business chain, The system structure is complex and cannot be completely decoupled. Therefore, it is impossible to accurately select test cases based on experience, which is not enough to ensure the accuracy of the test scope.
In order to solve these difficulties, precision testing was proposed in 2012. The so-called precision testing is a traceable software testing technology, which refers to the process of visualizing, analyzing and optimizing the traditional software testing process with the help of certain technical means and algorithms. , its core idea is to monitor the activities of the testing process, analyze the collected monitoring data, and obtain accurate quantitative data for quality evaluation.
Minor changes to the program will also bring new testing work. Quality assessment through experience can easily introduce problems, and full regression testing is also difficult to load. The picture below shows the current status of the testing work.
Therefore, accurate data is used to judge, so that testers can accurately know program changes, test scope, test coverage integrity, etc., and finally provide more effective test analysis results. The goals of precision testing are shown in the figure below.
Goal one is to capture the change procedure by identifying the change code. Accurate testing can more precisely delineate the range of use cases that require regression testing.
The second goal is to accurately test and provide test measurement results based on the change program or call chain dimension to judge whether the change program code of this test is completely covered, thereby helping the test to more accurately evaluate the test risk and form a test standard.
The third goal is to accurately locate the cause of the error. When the test case is executed, the precise test provides a visual call relationship for the failure case, helps the test to quickly locate the problem, and improves the collaborative efficiency of the development test.
To sum up, precise testing provides analysis methods in terms of test scope analysis, test measurement and positioning errors, which makes the testing work more effective.
2. Industry practice
(1) iQIYI Accurate Testing System
iQIYI Accurate Test supports multi-terminal use of client and server, and supports manual and automated scenarios. Multiple internal platforms are linked to open up the overall use process and can accurately obtain code coverage for a single use case. The server can achieve system-level series connection, while the client can realize multi-component integration, and the association between use cases and code branches or code blocks has been realized.
The figure below shows the overall architecture of iQIYI’s precision testing system: the precision testing service is the core, responsible for establishing a knowledge base of use cases and code mapping relationships, and calculating the scope of testing and scope of influence based on git submission records. Different terminals use different code coverage tools. In terms of integration with peripheral platforms, the server can be combined with the environment platform, and the code coverage tool can be installed through the environment platform; the client can be integrated with the construction platform. integrated.
(2) ByteDance Precise Test Practice
The picture above shows the common R&D process of the ByteDance R&D team. Requirements R&D is divided into five stages, and precise testing is mainly in self-testing, functional acceptance testing and regression testing nodes. In the three nodes, the business can obtain the self-test recommended use cases, functional test recommended use cases and regression test recommended use cases by requesting the SmartEye service. Among them, self-test use cases and functional test recommended use cases are used to verify whether code changes cause abnormal functions of existing programs, and to pre-discover functional defects, so as to gain more repair time for the R&D team. Regression test cases are recommended to simplify regression testing manpower and help businesses save regression testing manpower.
The picture above shows the overall solution for precise testing. The Code Graph platform provides data support for SmartEye. The Code Graph platform’s dual-end static code call chain analysis capability and overall call link topology data complete MR-based change DIFF analysis and upstream and downstream call chain analysis. , and for the changed DIFF part, complete the statistics of the code change rate.
3. Current status and development in the industry
Generally speaking, the accurate testing in the industry (ICBC) currently lacks use case recording and use case recommendation functions.
1. Coverage collection type
Our bank has made in-depth explorations in coverage collection, and currently supports multi-language coverage collection, including JAVA, stored procedures, CTP and MYBATIS.
2. Use the coverage tool code coloring system to assist testing
Obtain the dimension of the change method from the JAVA change submitted by GIT, and support the collection and storage of changes, change the Mybatis method, and change the CTP file. The current coverage tool visually displays the incremental coverage of configuration applications in multiple languages through code coloring, providing support for reducing the risk of system launch.
Configure the application of access code coloring, and you can intuitively obtain the coverage rate of the change program method level corresponding to the monthly version, and use this as the practical data for accurate testing. In the future, more in-depth analysis can be carried out, and the scale and depth of promotion will be continuously expanded to enhance The application effect of accurate test.
3. It supports static call chain analysis, and the whole link is accurate . ICBC explores the relationship between heterogeneous call chains, and the test analysis given is more comprehensive.
4. Static call chain analysis includes three categories: CTP, storage, and JAVA
5. Open up the deployment of the platform in the same industry
On top of the integration of the industry environment, the PAAS platform has been embedded, and the transformed application only needs to configure three PAAS parameters to download the toolkit and use the service. If it is a non-cloud environment, we will get the configuration information from cmdb for automatic deployment at night.
The schematic diagram shows the data flow direction. The bottom-level Docker image analysis service obtains the bytecode content by parsing the .dockfile file; the upper part is data collection, and the coverage results collected by the client are sent to the message middleware branch through the coverage cluster. At the same time, the static call chain analysis service will analyze a static file; then up is the data storage layer, which stores the coverage data on redis, stores the massive coverage data and call chain data in hbase, and stores the environment-related method information in oracle ; Then there is the service layer, which provides real-time change method coverage collection and display according to the owned data, three functions: the coverage report of the production day dimension, and the first-level in-degree call chain coverage report.
4. Further construction
1. The current change impact analysis is mainly based on static call chain analysis, but static code has limitations, such as reflection. In the future, it is necessary to further modify and supplement the call chain based on the results of dynamic code analysis.
2. Accurate test intelligence is mainly reflected in the recommendation of test cases. Accurately recommend cases and form a closed test loop of "accurate analysis -> automated test -> coverage measurement". At present, the Industrial and Commercial Bank of China is also promoting development in this area. Shanghai Research Institute has proposed an "intelligent specification push tool" to intelligently extract information based on existing demand data, case content, and test specification database. Without additional investment, Realize intelligent case recommendation to improve test efficiency.
3. Expand on the basis of the three services of the service layer, based on the data support provided by the data base layer, provide a full link to accurately open up the relationship between heterogeneous programs, support the scheduling between micro-services, and comprehensively measure including calling Chain test analysis.
Finally: The following complete software testing video learning tutorial has been sorted out and uploaded. Friends can get it for free if they need it [Guaranteed 100% free]
Software Testing Interview Documentation
We must study to find a high-paying job. The following interview questions are the latest interview materials from first-tier Internet companies such as Ali, Tencent, and Byte, and some Byte bosses have given authoritative answers. Finish this set The interview materials believe that everyone can find a satisfactory job.
