code coverage

When doing unit testing, code coverage is often used as an indicator to measure the quality of testing, and even code coverage is used to assess the completion of test tasks. For example, code coverage must reach 80% or 90%. As a result, testers put great effort into designing case coverage code. Measured by code coverage, there are pros and cons. In this article, we discuss the code coverage rate, and welcome comments from students.

First, let's take a look at what's called "code coverage". I found the so-called definition:

code coverage = code coverage, a measure.

The above short and concise text describes the meaning of code coverage very accurately. There are many ways to measure the degree of code coverage. Here are the most commonly used ones:
1. Statement Coverage,

also known as LineCoverage, SegmentCoverage, and BasicBlockCoverage, This is the most common and common method of coverage, which is to measure whether each executable statement in the code under test has been executed. It's talking about "executable statements", so it doesn't include header file declarations like C++, code comments, blank lines, etc. Very easy to understand, only count how many lines of code that can be executed are executed. Note that curly braces {} on a single line are also often counted. Statement coverage is often accused of "weakest coverage", which only covers the executed statements in the code, but does not consider the combination of various branches and so on. If your boss only requires you to achieve statement coverage, then you can save a lot of effort, but in exchange, the actual test effect is not obvious, and it is difficult to find more problems in the code.

Here's an example that couldn't be simpler. Let's look at the following code under test:
int foo(int a, int b)
{
   return a / b;
}


Suppose our testers write the following test case:
TeseCase: a = 10, b = 5


Tester's test results will tell you that his code coverage is 100% and all test cases are passed. Unfortunately, our statement coverage has reached the so-called 100%, but we have not found the simplest bug. For example, when I let b=0, a division-by-zero exception will be thrown.

Because of this, if the above only requires the tester to achieve the statement coverage, the tester can easily meet the requirements of the supervisor as long as he takes advantage of the loopholes and writes test cases specifically for how to cover the lines of code. Of course, this also shows several problems:

    1. There is a problem with the supervisor only using statement coverage to assess testers.

    2. The purpose of testers is to test the code well, and it is unprofessional to exploit such loopholes.

    3. Should better assessment methods be used to assess the work of testers?

In order to seek better assessment standards, we must first understand what the code coverage is. If your supervisor only knows statement coverage and line coverage, then you should take the initiative to introduce him to more coverage methods. For example:
2. Decision Coverage (DecisionCoverage),

also known as BranchCoverage, All-EdgesCoverage, BasicPathCoverage, and Decision-Decision-Path. It measures whether each decided branch in the program has been tested. This sentence needs to be further understood and should be easily confused with the conditional coverage mentioned below. Therefore, we directly introduce the third coverage method, and then compare it with the judgment coverage to understand what is going on.
3. Condition Coverage

It measures whether each subexpression in the predicate has been tested for true and false results. To illustrate the difference between decision coverage and conditional coverage, let's take an example, if our code under test is as follows:

int foo(int a, int b)
{
    if (a < 10 || b < 10) // decision
    {
        return 0; // Branch 1
    }
    else
    {
        return 1; // Branch 2
    }
}



When designing a decision coverage case, we only need to consider two cases where the decision result is true and false. Therefore, we design the following case to achieve decision coverage Rate 100%:
TestCaes1: a = 5, b = any number Covers branch one
TestCaes2: a = 15, b = 15 Covers branch two When


designing conditional coverage cases, we need to consider the result of each conditional expression in the decision, To achieve 100% coverage, we designed the following cases:
TestCase1: a = 5, b = 5 true, true
TestCase4: a = 15, b = 15 false, false


Through the above example, we should be very clear about the difference between decision coverage and condition coverage. It should be noted that condition coverage is not a combination of the results of each conditional expression in the judgment, but as long as the results of each conditional expression are tested for true and false, it is OK. Therefore, we can infer that complete conditional coverage does not guarantee complete decision coverage. For example, in the above example, if the case I designed is:
TestCase1: a = 5, b = 15 true, false Branch 1
TestCase1: a = 15, b = 5 false, true Branch 1


We see that although we have done it completely Condition coverage, but we did not achieve complete decision coverage, we only covered branch one. It can also be seen from the above example that these two coverage methods do not seem to be very good. We next look at the fourth coverage method.
4. Path Coverage is

also known as PredicateCoverage. It measures whether each branch of the function was executed. This sentence is also very easy to understand, that is, all possible branches are executed once. When multiple branches are nested, multiple branches need to be arranged and combined. It is conceivable that the test path increases exponentially with the number of branches. For example, the following test code has two decision branches:

int foo(int a, int b)
{
    int nReturn = 0;
    if (a < 10)
    {// branch one
        nReturn += 1;
    }
    if (b < 10)
    {// Branch two
        nReturn += 10;
    }
    return nReturn;
}



For the above code, we design test cases for our first three coverage methods:

a. Statement coverage

TestCase a = 5, b = 5 nReturn = 11

Statement coverage 100%

b . Decision coverage

TestCase1 a = 5, b = 5 nReturn = 11

TestCase2 a = 15, b = 15 nReturn = 0

Decision coverage 100%

c. Condition coverage

TestCase1 a = 5, b = 15 nReturn = 1

TestCase2 a = 15, b = 5 nReturn = 10

Condition Coverage 100%



We see that the above three coverage results look cool! All at 100%! The supervisor may be very happy, but let's take a closer look. In the tested code above, the result of nReturn has a total of four possible return values: 0, 1, 10, 11, and our above is for each The test case of this coverage design only covers part of the return value. Therefore, it can be said that using any of the above coverage methods, although the coverage rate reaches 100%, it is not completely tested. Next, let's take a look at the test cases designed for path coverage:


TestCase1 a = 5, b = 5 nReturn = 0

TestCase2 a = 15, b = 5 nReturn = 1

TestCase3 a = 5, b = 15 nReturn = 10

TestCase4 a = 15, b = 15 nReturn = 11

100% path coverage is



awesome! Path coverage tests all possible return values. This is why it is considered by many to be the "strongest coverage".

There are some other coverage methods, such as: Loop Coverage (LoopCoverage), which measures whether the loop body is executed zero, one and more than one time. The rest of the other coverage methods will not be introduced.
Summary

Through the above learning, let's look back and think about how meaningful the coverage data is. I have summarized the following points for discussion:

a. Coverage data can only represent which codes you have tested, not whether you have tested them well. (such as the first divide-by-zero bug above)

b. Don't trust coverage data too much.

c. Don't judge your testers only by statement coverage (line coverage).

d. Path coverage > Decision coverage > Statement coverage

e. Testers should not blindly pursue code coverage, but should find ways to design more and better cases, even if the more designed cases have no impact on coverage.

Author: CoderZh (CoderZh's technical blog - Blog Park)
Weibo: http://t.sina.com.cn/coderzh
Source: http://coderzh.cnblogs.com
Article address: http://www.cnblogs.com/coderzh/archive/2009/03/29/1424344.html