Code coverage test (I)

Code coverage （ English ：Code coverage） yes software test One of the Measure , describe Program in Source code The proportion and degree of being tested , The resulting ratio is called Code coverage . Code coverage ＝ Code coverage , A measure .

1. Statement override (StatementCoverage)

Also known as line coverage (LineCoverage), Segment coverage (SegmentCoverage), The basic block covers (BasicBlockCoverage), This is the most commonly used and common coverage method , Is to measure whether each executable statement in the tested code has been executed to . This is about “ Executable statement ”, So it won't include things like C＋＋ The header file declaration of , code annotation , Blank line , wait . Very easy to understand , Only count how many lines of executable code have been executed . It should be noted that , Curly braces on a single line ｛｝ It is often counted . Statement coverage is often accused of “ Weakest coverage ”, It just overwrites the execution statements in the code , Without considering the combination of various branches, etc . If your boss only asks you to achieve sentence coverage , Then you can save a lot of effort , however , In exchange, the test effect is not obvious , It's hard to find more problems in your code .

Here's an example that can't be simpler , Let's look at the tested code below ：

int foo(int a, int b)
{
   return  a / b;
}

Suppose our testers write the following test cases ：

TeseCase: a = 10, b = 5

The tester's test results will tell you , His code coverage has reached 100％, And all test cases passed . But unfortunately , Our statement coverage has reached the so-called 100%, But I didn't find the simplest Bug, such as , When I let b=0 when , Will throw a divide by zero exception .

Because of this , If only the tester's statement coverage is required , The tester just needs to make a hole , Write test cases specifically on how to cover lines of code , It's easy to meet the requirements of the supervisor . Yes, of course , This also illustrates several problems ：

    1. The supervisor only uses the statement coverage to evaluate the tester, which is problematic .

    2. The tester's goal is to test the code , It lacks professional ethics to exploit such loopholes .

    3. Should a better assessment method be adopted to assess the work of testers ？

In order to seek better assessment standards , We must first understand what code coverage is , If your supervisor only knows statement overrides , Line overlay , Then you should take the initiative to introduce him to more coverage methods . such as ：

2. Determine coverage (DecisionCoverage)

Branch coverage (BranchCoverage), All boundary coverage (All-EdgesCoverage), Basic path coverage (BasicPathCoverage), Determine path coverage (Decision-Decision-Path). It measures whether each determined branch in the program has been tested . This sentence needs further understanding , It should be very easy to be confused with the conditional coverage mentioned below . Therefore, we directly introduce the third coverage method , Then compare it with the decision coverage , I'll see what's going on .

3. Conditional coverage (ConditionCoverage)

It measures the result of each subexpression in the decision true and false Is it tested . To illustrate the difference between decision coverage and conditional coverage , Let's take an example , If our tested code is as follows ：

int foo(int a, int b)
{
    if (a < 10 || b < 10) //  determine 
    {
        return 0; //  Branch one 
    }
    else
    {
        return 1; //  Branch two 
    }
}

Design Determine coverage In case , We just need to think about it The judgment result is true and false Two cases , therefore , We design the following cases to achieve the judgment coverage 100％：

TestCaes1: a = 5, b ＝  Arbitrary number    Covered branch one
TestCaes2: a = 15, b = 15           Covered branch 2

 
Design Conditional coverage In case , We need to consider the Result of each conditional expression , In order to achieve coverage 100％, We designed the following case ：

TestCase1: a = 5, b = 5       true,  true
TestCase4: a = 15, b = 15   false, false

Through the example above , We should be clear about the difference between decision coverage and conditional coverage . Here's the thing to watch out for ： Conditional coverage does not arrange and combine the results of each conditional expression in the decision , Instead, just the result of each conditional expression true and false When the test comes OK 了 . therefore , We can infer this ： Complete conditional coverage does not guarantee complete decision coverage . Take the example above , If the case I designed is ：

TestCase1: a = 5, b = 15  true,  false    Branch one
TestCase1: a = 15, b = 5  false, true     Branch one

 
We see , though However, we have completely achieved conditional coverage , However, we have not achieved complete decision coverage , We only covered one branch . The above example can also be seen , Neither of these coverage methods seems to be very good . Let's look at the fourth coverage method .

4. Path coverage (PathCoverage)

Also known as assertion override (PredicateCoverage). It measures whether each branch of the function is executed . This sentence is also very easy to understand , All possible branches are executed once , When multiple branches are nested , Multiple branches need to be arranged and combined , As one can imagine , The test path increases exponentially with the number of branches . For example, the following test code has two decision branches ：

int foo(int a, int b)
{
    int nReturn = 0;
    if (a < 10)
    {//  Branch one 
        nReturn += 1;
    }
    if (b < 10)
    {//  Branch two 
        nReturn += 10;
    }
    return nReturn;
}

a. Statement override

TestCase a = 5, b = 5   nReturn = 11

  Statement coverage 100％

b. Determine coverage

TestCase1 a = 5,   b = 5     nReturn = 11

TestCase2 a = 15, b = 15   nReturn = 0

Determine coverage 100％

c. Conditional coverage

TestCase1 a = 5,   b = 15   nReturn = 1

TestCase2 a = 15, b = 5     nReturn = 10

Conditional coverage 100％

We see , All three coverage results look cool ！ All have reached 100％！ The supervisor may be very happy , however , Let's take a closer look , In the tested code above ,nReturn There are four possible return values for the result of ：0,1,10,11, The above test cases designed for each coverage only cover part of the return values , therefore , It can be said to use any of the above coverage methods , Although the coverage has reached 100%, But not completely tested . Next, let's take a look at the test cases designed for path coverage ：

TestCase1 a = 5,    b = 5     nReturn = 0

TestCase2 a = 15,  b = 5     nReturn = 1

TestCase3 a = 5,    b = 15   nReturn = 10

TestCase4 a = 15,  b = 15   nReturn = 11

Path coverage 100％

summary

Learn from the above , Let's look back , What is the significance of coverage data . I summarize the following points , Welcome to discuss ：

a. Coverage data only represents what code you've tested , It doesn't mean that you test the code .（ For example, the first division above zero Bug）

b. Don't trust coverage data too much .

c. Don't just take statement coverage ( Line coverage ) To assess your testers .

d. Path coverage > Determine coverage > Statement override

e. Testers should not blindly pursue code coverage , We should try to design more and better cases , Even more designed cases have no impact on coverage at all .

original text ：https://www.cnblogs.com/coderzh/archive/2009/03/29/1424344.html