table of Contents
0 defined coverage
As a tester, software quality assurance products is the primary objective of its work, to this end, testers often by many means or tools to be guaranteed coverage is one of the more important part of a ring.
We will usually test coverage is divided into two parts, namely "Demand coverage" and "code coverage."
- Requirements coverage: means testers level of understanding of the needs, according to the testability needs to be split into individual sub-demand point to prepare the corresponding test cases, the eventual establishment of a demand and the mapping relation embodiment used to test cases using needs to verify the results achieved can be understood as a black box cover.
- Code coverage: For a more complete coverage, we may also need to understand the logic of the program under test, you need to take into account the implementation of the input and output of each function, logic branch code, this time we test it with the implementation of the code coverage to measure, as it will be appreciated white-box cover.
Both the above can complement each other to cover the results of the checks needs to cover the reverse (use cases) to test whether a full and complete code.
more popular java code coverage tools EMMA, Cobertura, jacoco and so on. In fact, at present situation, the use jacoco crowd is more, a little trend of feeling.
1 EMMA
EMMA had previously used a lot of people, but the development of this tool pit father team since 2006 and never updated after the
can has been understood as EMMA die.
- maven repository also lacks updated
doubtless will be eliminated
but EMMA really too simple to use, novice choice.
By querying the ancient books, it had found that IDEA is also supported, and now is no longer supported.
1.1 maven integration
mostly java project with maven management, if we want statistical unit test coverage, then by emma with maven integration is the most simple enough. Unlike jacoco so much trouble, emma configuration is very simple. Only you need to add the following dependence:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>emma-maven-plugin</artifactId>
<version>1.0-alpha-3</version>
<inherited>true</inherited>
</plugin>
Then run the command: mvn emma: emma. Then you can see the report
1.2 Integration with jenkins
If you use jenkins as a CI tool, in fact even simpler. You are no longer dependent increase pom file, installed after EMMA plug-in, run the command directly above example just fine.
1.3 Instrumentation
Statistics can only project itself test described above, that is, in engineering src / test / java package the following test script. If our interface testing, UI testing it? How we can do code coverage statistics do? This requires some means of.
First, you need to download from the official website emma.jar to your test environment, and then copy to your jre the ext folder. This is to extend the java command, as you can perform operations directly in the form of java emma.
Then you need to test the instrumentation package. Emma will then open a service, the default port 47653. This service will monitor the project being tested. Examples of instrumentation are as follows. Specific command parameters refer to the official website just fine.
java emma instr -m overwrite -cp simba-1.0.jar -out coverage.em
The final step is to collect reports. It should be two commands, one for data collection, a report is generated. Example is as follows. Focus on talk about a few parameters. -sp is the path to your source code, so in order to get the code emma information shows a more detailed report. -in is to generate metadata information reporting needs. It is a collection of data and instrumentation on the generated intermediate file. -r is the report format. These parameters are commonly used.
java emma ctl -connect localhost:47653 -command coverage.get,coverage.ec
java emma report -r html -Dreport.out.encoding=utf-8 -sp /opt/web/simba/src/main/java -in coverage.em,coverage.ec
As long as you do not delete the file metadata and data collection instrumentation generated words. You can generate reports accumulated. There is also a merge mode can merge report detailing the things we can look at Quguan network. EMMA advantage is simple to use. Finally, I made a report generated drawing.
pit
EMMA has a pit in jdk1.6 version is more likely to be a problem, there will be a run classformat abnormal after instrumentation. The parameter is incorrect number of tips you probably mean it. In fact, this is used in jdk 1.7 after verification is not the same. The EMMA pit father goods too long without update, and did not cover the new version of the JDK. So we need to add a parameter when starting the jvm. -XX: -UseSplitVerifier. that's it. The pit was really pit us hours.
to sum up
- Even if 100% coverage can not guarantee your product quality
- But the quality is too low coverage of products must be not be guaranteed
- The real importance of code coverage, it added to the quality assurance process. It will be very easy to become a decoration
- Improve code level testers familiar with the product code. Otherwise it is easy to become decoration
- Do not attempt to be 100%, it is impossible.
- Maintaining a balanced coverage: 50% of test unit 70,% the interface 20 is 40% UI10%
- Persuade developers to write unit tests as possible. The more the bottom of the test, the lower the cost to achieve coverage. In an attempt to achieve high coverage testers here it is almost impossible
- If there is no unit tests. Try to improve coverage in the interface testing
2 jacoco
jacoco is developed EMMA team they have all turned to jacoco
JaCoCo advantage
- JaCoCo support branch coverage, introduces Agent
- EMMA official website is no longer maintained, JaCoCo is his team developed, it can be understood as an upgraded version
- JaCoCo community more active, on github is constantly updated to maintain
it for the development of language is java, its use is very flexible and can be embedded into Ant, Maven in; available as an Eclipse plug-in, you can use the technology to monitor its JavaAgent Java programs and so on .
Many third-party tools to provide an integrated JaCoCo, such as sonar, Jenkins and so on.
JaCoCo contains multiple scales coverage counter stage comprising instructions cover (Instructions, C0coverage), branches (Branches, C1coverage), cyclomatic complexity (CyclomaticComplexity), covered lines (Lines), covering method (non-abstract methods), covering class (classes)
JaCoCo basic concepts
Line coverage: each line of code program under test whether the metric is performed, if at least one instruction to be executed criterion line.
Class Coverage: metric calculation class if the class file is executed.
Branch coverage: metric if and switch statements branch coverage, a method of calculation which
The total number of branches, number of branches and determined execution is not carried out.
Method Coverage: The method of implementation of the program under test metric, whether at least one instruction to be executed depending on whether to perform the method.
Instructions cover: single counting unit java binary code instructions, instruction coverage is provided if the code information is performed completely independent measurement source format.
Cyclomatic complexity: the (linear) combination, in which a minimum number of calculation method for all possible paths
JaCoCo principle
Injection method
ASM bytecode modification techniques used in the method JaCoCo Byte Code, can modify the Jar file, class file byte code file.
JaCoCo support both instrumented mode on-the-fly and the offline
On-the-fly instrumentation:
Specific jar file JVM startup Instrumentation Agent -javaagent specified by parameters, loading agent prior to determining whether a class converts the modified class files Class Loader, statistical code into class, test coverage analysis can be performed by the JVM code test the process is completed.
Offline mode:
After the files to be instrumented before the test, and then generate inserted through piles of class or jar package, test inserted through piles of class and jar package, will generate dynamic overlay information to a file, and finally unified coverage information processing, and generates report.
On-the-fly and offline comparison:
On-the-fly mode is more convenient and simple code coverage analysis, without prior instrumented byte code, regardless of the setting classpath.
There is a case not appropriate on-the-fly, in advance of the need offline bytecode instrumentation:
(1) operating environment does not support java agent.
(2) the deployment environment does not allow to set JVM parameters.
(3) byte code is to be converted to other virtual machines such as Android Dalvik VM.
(4) dynamic modification process bytecode agent and other conflicts.
(5) can not load custom user class.
JaCoCo minimum required execution of Java5
JaCoCo to modify and generate java bytecode by implantation, using the ASM library.
use
ant
No longer
Project maven plugin
This approach is suitable Maven project.
Call flow:
(1) project is packaged jar package, introduced junit and jacoco.
Performing instrument, report, check the time (2) Build.
(3) to generate coverage target / jacoco.exec
3 summary
reference
- https://cloud.tencent.com/developer/article/1038055
