REVIEW: This article is based on actual usage, a brief analysis of the traditional white box testing and precision tools Jacoco and other similarities and differences in design, features and application scenarios, and explains how technology coverage in the development of new enterprises in the system, should play important role.
Coverage technology can be said to test the theory in the most basic technology system, but due to the traditional coverage is not well adapted to the new software development model, resulting in more and more narrow application scenarios. For example: Jacoco and other similar tools, still remains in the technical aspects of the evolution of the traditional white box cover technology, currently only available in the basic development system waterfall model. The latest test dark horse Technology - "precision test" function is the enterprise-class coverage, Agile iterative scenes coverage of new technology. It clearly put forward the concept of use case coverage levels, and is widely used intelligence test coverage analysis algorithms use case level technology.
One patent precise test: test bidirectional traceability code, can be simply understood as: all analyzes and calculations depends on the dimensions of test coverage information. Its innovative coverage statistics will show the dimension from the global dimension, dimensionality reduction to the details of this test case, the coverage of amplification far beyond the original capacity. It is like a magnifying glass, the test analysis, test algorithms and test data truly glance, do not miss any important details.
Jacoco is the traditional white box coverage tool, do not have the coverage function associated with the use case, we regret not have all the advanced features accurate testing.
Far below "coverage" this feature, the accurate testing and Jacoco represented by the traditional white box cover than a simple tool to:
1 coverage analysis capabilities PK
Jacoco: As a representative conventional white-box function, its application is deployed in the background mode, the acquisition of all execution code coverage, covering all the data requested by the user and mixed statistical functions, only in the range around watching coverage. But this kind of malpractice coverage information is: system-wide dimension to it from the statistics, leading to particle size is too large to locate the real problem in detail and in-depth analysis of the coverage.
Precision test: can the same as adjusting the focus, in the background, serving concurrent access, the isolated execution path of a test case out of a variety of detailed analysis. It is covered with Example stage bidirectional traceability information, with execution timing, an execution path can quickly locate defects in the time code (similar to stepping reproducing scene), at the same time, also the implementation of the use cases testers , very precise tracking code level.
2 for agile responsiveness test of PK
Jacoco: not suited for agile iterative process for coverage of enterprise-level requirements. When the code is changed, Jacoco will re-acquisition coverage, results for each release version collection exist in isolation. Modern organizations often publish multiple versions of every day, no island-style coverage of the results of comparative analysis, of little value. Because the test is usually performed each run part of the use cases in each version, instead of using all of the test cases in a version of the waterfall model.
Precision test: cumulative coverage in the coverage calculation and application, incremental coverage-related coverage, high risk coverage analysis, variable number of innovative approaches such as the denominator cover, ready to respond to the needs of agile iteration.
1) Cumulative Reach: accurate test multiple test coverage can be accumulated and projected, so it does not matter how many versions middle published, how many use cases run on each version, you can choose to see overall coverage of a stage.
2) incremental reach: Due to the huge amount of enterprise software body, so the tests are in any case difficult to meet the requirement of 100% coverage. So companies are usually more concerned with incremental reach, that is compared to the newly released version of the code of a modified version of the test coverage. Since the precise time of each test releases have detailed code static analysis data to support, so it can intelligently identify the version differences, differences between versions of the code and cover were highlighted to help companies focus on new / adjustment to cover part of the code rate.
3) Related Coverage: accuracy test innovative technology is proposed related to coverage, the code that is associated with a functional module as the denominator calculation of coverage. This is very beneficial to analyze in an agile iteration scenario, only some of the features of the test coverage. Related coverage can be used to support the functional module (use case classification) level of coverage, statistical coverage of the relevant code for a range of functional modules from a business point of view, clear that a certain range of functions of the full test coverage, rather than the traditional global code coverage.
4) high-risk coverage analysis: precise test supports static and dynamic data detected high-risk module, to guide the user to focus on the highest risk into modules covering supplementary logic.
5) Variable denominator covered: multiple series of precise tests to support enterprise computing coverage requirements, for example, by setting the interface, or a certain determined that no code can not be covered (e.g. escrow invalid code) from the calculation result Coverage discharging, the whole coverage recalculated. A support of a high level code path characteristic (block certain degree) like code coverage range.
3 coverage visualization capabilities PK
Jacoco: Based on bytecode instrumentation, no comprehensive program static analysis process, and therefore can not be coverage through visual graphics combined with static analysis to get the clear impression reach. In addition, Jacoco must provide the source code to understand coverage.
Precision Test: a plurality of call graph, the control flow chart showing the coverage information, without the source code can be based on accurate analysis, in conjunction with Dynamic Coverage view, clearly show coverage and program execution path information.
4 coverage of statistical capacity PK
Jacoco: rows coverage and branch coverage based. Jacoco traditional row cover, substantially each row insertion required. In view of the aspects covered by the combination of code shows, you can not get static in-depth information about the program, so in general can only express color code to overwrite another view.
Precision test: support coverage visualization, multi-standard algorithm coverage and depth of data analysis.
1) support coverage visualization, are very clear expression of how the coverage is calculated (program elements corresponding to the contribution of the numerator and denominator of the coverage, quantity), user-friendly information to in-depth understanding of the meaning and coverage.
2) support a more accurate test conditions and the depth of the combination of conditions aerospace grade level and MC / DC coverage standard. Precise statement coverage testing is provided to cover basic block, a code sequence with a support section because the static analysis results, only one plug decorate.
3) analytical data accurately test the static structure of a program of in-depth, so it is very clear view of the code shows, program structure can be drawn, code coverage view looks very clear. For example, basic block with no jump will be drawn on the display of results as a whole, and whether there is identity cover, each row must not be identified.
5 Height of product characteristics PK
Jacoco: open category belongs, using bytecode instrumentation (instruction level), instrumentation code is not visible can not be maintained, the problem is difficult to troubleshoot the reason, it is difficult to locate and repair, higher risk commercial use. Every time covering data acquisition needs access to regenerate a web address displayed.
Jacoco application mode is deployed background collection, no front-end display. When an exception occurs, the testers can not know their own test case execution is valid.
Precision test: leading research and development by the nebula test (www.teststars.cc), self-control. In the characteristics of the products have unparalleled advantages:
1) based on the source code instrumentation, instrumentation code visible and open, very conducive to enterprise applications and troubleshooting for process integration. In case there are special grammatical problems caused instrumentation problem, the user can view and handle, because extreme cases will not affect product application.
2) a summary of the coverage information in real time, through a client can have permission to view by multiple users (including development, testing, and management staff) to view real-time.
Software oscilloscope can transfer real-time data during testing. 3), for the transmission of the test execution coverage during acquisition is valid, it can be visualized troubleshooting.