Overview

What is unit testing
Why do unit testing?
Some misunderstandings about unit testing
What are the mainstream unit testing frameworks?
Appreciation of examples of each framework
Comparison of various frameworks
How to do unit testing
Problem thinking

1. What is unit testing?

Wikipedia describes it this way: In computer programming, unit testing, also known as module testing, is a test for correctness testing of program modules. A program unit is the smallest testable component of an application. In procedural programming, a unit is a single program, function, process, etc.; for object-oriented programming, the smallest unit is a method, including methods in base classes, abstract classes, or derived classes.

The difference between unit testing and integration testing

The testing frameworks and tools used for unit testing and integration testing are mostly the same.

The first thing that needs to be agreed upon is that whether they are unit tests or integration tests, they are automated tests. In order to better distinguish, we can understand it this way: it is in the same code warehouse as the production code and unit test code, written by the development students themselves, and has real-life effects on the external environment (database, file system, external system, message queue, etc.) The tests that are called are integration tests.

The following table also compares the differences between unit testing, integration testing and system-level testing (including end-to-end testing, link testing, automated regression testing, UI testing, etc.) from various perspectives.

	unit test	Integration Testing	System level testing
Writers	develop	develop	development/testing
writing venue	In the production code warehouse	In the production code warehouse	Inside the production code warehouse/Outside the production code warehouse
Writing time	Before code release	Before code release	Before code release/After code release
writing cost	Low	middle	high
Writing difficulty	Low	middle	high
feedback speed	Extremely fast, seconds	Slower, minute level	Slow, day level
Coverage area	Code line coverage 60-80% Branch coverage 40-60%	Functional level coverage	core assurance link
environment dependence	Code level, independent of environment	Rely on daily or local environment	Depends on staging or production environment
External dependency simulation	All simulations	partial simulation	No simulation, completely use the real environment

Second, why do we need unit testing?

benefit:

Improve system stability and facilitate iteration.
Conducive to in-depth understanding of technology and business.
The single test cost is low and the speed is fast.
Single test is the best, automated, executable document.
Single test-driven design improves code simplicity and standardization, ensuring safe refactoring. After the code is modified, the single test can still pass, which can enhance developer confidence.
Quick feedback, discover problems faster, locate defects faster and more accurately than integration testing, and reduce repair costs.

Low development costs:

1，

The most intuitive idea:

2，

This idea is indeed the most intuitive. But this only thinks of the first level. If we add all the steps of the development process , we will find that it is like this:

Behind the development process, bugs may be thrown in almost every process. The more bugs are thrown later in the process, the more time and business the programmer has to invest than in the development stage, and the risks they bear are also the highest.

The picture below also illustrates two problems: first, 85% of defects are generated in the code design stage; second, the later the stage when bugs are discovered, the higher the cost, which increases exponentially. This kind of "exponential cost" case often happens. When we correct a bug, three more bugs may follow, commonly known as: the correction crashes.

Therefore, bugs can be found in early unit testing, which not only saves time and effort, improves efficiency in the development process, but also reduces the risk and time cost of repeated modifications.

3. Some misunderstandings about unit testing

Zhihu https://zhuanlan.zhihu.com/p/547068206 2, 5, 6, 9, 11

Myth 1: Unit testing slows down the development process

The truth is: like any new tool, getting used to unit testing takes a little time, but overall, unit testing saves time and wastes less time. In fact, performing regression testing allows you to move the development process forward continuously and without any worries. If unit testing is performed during daily builds, such testing will not take up development time.

Misconception 2: Once the project is over, the work invested in unit testing is wasted

Not at all. If you've ever reused code, you'll realize that everything you do is an asset.

The fact is: when you use code that you previously wrote for another project in one project, or edit this code, you can use the same unit tests, or you can edit these unit tests. There is no problem in using similar test code snippets in the same project.

Myth 3: Unit testing is a waste of time

You need to figure out what is a waste of time?

Fixing the same vulnerabilities over and over again

Write or rewrite validation code throughout the development process

One bug is patched, only to have another bug appear somewhere else out of nowhere

I was accidentally interrupted while writing code and I have no idea what to do.

Resistance to unit testing is understandable, but many developers won't praise how good unit testing is until they complete a project using it.

The truth is: you only need to write a unit test once but run it many times. This has nothing to do with changes you make to other code. The initial investment will pay off in the long run.

Misunderstanding 4: Unit testing does not help in debugging the program, or it cannot prevent the occurrence of vulnerabilities.

That's absolutely not the case. Unit testing can make program debugging easier because you can focus on problematic code, fix the problems, and then merge the modified code again. It also prevents the introduction of vulnerabilities when adding functionality, and prevents problems from recurring frustratingly, especially when programming using an object-oriented approach. Unit testing cannot ensure 100% elimination of vulnerabilities, but it is a good way to reduce vulnerabilities.

The fact is: although unit testing cannot solve all the problems you encounter during debugging, when you find a vulnerability, the isolated code in the unit test can make it easier to fix the vulnerability. According to die-hard fans of unit testing among developers, the biggest benefit of unit testing is that it makes debugging the program very easy and simple.

Misunderstanding 5: Using unit tests for program debugging does not provide comprehensive coverage

Just because you can't debug the entire code doesn't mean debugging coverage isn't comprehensive. Using unit tests for program debugging is at least better than other types of debugging. In fact, unit testing has a very significant advantage: (if not greatly eliminating, then) greatly reducing the number of reported vulnerabilities I mentioned above. Reproducing bugs can be very frustrating when developing and debugging programs. With unit testing, you can reduce the frequency of introducing new vulnerabilities when adding, modifying, and removing functionality. Debugging has always been "full coverage", especially when the devices or systems on which the program runs are very different.

The truth is: especially when dealing with vulnerabilities, unit testing can ensure that you find vulnerabilities that have never been reported. And when you debug the program, you don't need to check the entire code, you only need to modify the places where vulnerabilities occur.

4. What are the mainstream unit testing frameworks?

Junit: Junit is one of the most commonly used Java unit testing frameworks. It provides a set of simple and easy-to-use APIs to easily write and run unit tests. The main advantages of Junit include easy to learn and use, wide use, rich ecology, etc., but it lacks some advanced functions, such as simulated objects, etc.
Mockito: Mockito is a Java unit testing framework for mock objects, which helps developers create and manage mock objects for real unit testing. The main advantages of Mockito include powerful functions, easy to learn and use, support for extensions, etc., but it may bring some performance issues.
Spock: Spock is a Java unit testing framework based on the Groovy language. It provides a set of concise and readable DSL (domain specific language) that allows developers to easily write and run unit tests. The main advantages of Spock include being easy to read and maintain, providing a rich assertion library, supporting data-driven testing, etc., but it requires additional Groovy compiler support. However, its compatibility is relatively poor. If the dependent version is slightly wrong, some inexplicable errors will be reported, and the prompts are not obvious, making it difficult to troubleshoot problems.
TestNG: TestNG is a Java testing framework, similar to Junit. It provides a set of powerful testing functions, including support for multi-threaded testing, data-driven testing, group testing, etc. The main advantages of TestNG include rich functionality, easy expansion, and the ability to integrate with various continuous integration tools. However, it lacks some advanced features, such as mock objects.
PowerMock: PowerMock is an extension framework for Java unit testing that helps developers write more flexible unit tests. The main advantages of PowerMock include support for testing static methods, private methods, etc., easy to learn and use, and can be used with other testing frameworks, etc., but it may introduce more complexity and maintenance costs.

5. Examples of usage of each framework

JUnit：

import org.junit.Test;
import static org.junit.Assert.*;

public class MyTest {
    
    
    @Test
    public void testSomething() {
    
    
        // 执行测试代码
        assertEquals(2 + 2, 4);
    }
}

Mockito：

import static org.mockito.Mockito.*;

public class MyTest {
    
    
    @Test
    public void testSomething() {
    
    
        // 创建模拟对象
        MyObject mockObject = mock(MyObject.class);
        // 设置模拟对象的行为
        when(mockObject.someMethod()).thenReturn("Hello World");
        // 执行测试代码
        String result = mockObject.someMethod();   
        // 断言结果是否符合预期
        assertEquals(result, "Hello World");
    }
}

Spock：

import spock.lang.Specification
import spock.lang.Subject

class CalculatorSpec extends Specification {
    
    

    @Subject
    Calculator calculator = new Calculator()

    def "test add method"() {
    
    
        given:
        int a = 2
        int b = 3

        when:
        int result = calculator.add(a, b)

        then:
        result == 5
    }

    def "test subtract method"() {
    
    
        given:
        int a = 5
        int b = 2

        when:
        int result = calculator.subtract(a, b)

        then:
        result == 3
    }
}

class Calculator {
    
    
    int add(int a, int b) {
    
    
        return a + b
    }

    int subtract(int a, int b) {
    
    
        return a - b
    }
}

TestNG：

import org.testng.annotations.Test;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.AfterMethod;
import static org.testng.Assert.assertEquals;

public class MyTestNGTest {
    
    
    
    @BeforeMethod
    public void setUp() {
    
    
        // 在测试方法执行前执行的代码
    }
    
    @AfterMethod
    public void tearDown() {
    
    
        // 在测试方法执行后执行的代码
    }
    
    @Test
    public void testAddition() {
    
    
        int result = Calculator.add(2, 3);
        assertEquals(result, 5);
    }
    
    @Test
    public void testSubtraction() {
    
    
        int result = Calculator.subtract(5, 3);
        assertEquals(result, 2);
    }
}

class Calculator {
    
    
    public static int add(int a, int b) {
    
    
        return a + b;
    }
    
    public static int subtract(int a, int b) {
    
    
        return a - b;
    }
}

PowerMock：

import org.junit.Test;
import org.junit.runner.RunWith;
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.junit4.PowerMockRunner;
import static org.powermock.api.mockito.PowerMockito.*;

@RunWith(PowerMockRunner.class)
@PrepareForTest(Example.class)
public class ExampleTest {
    
    

    @Test
    public void testPrivateMethod() throws Exception {
    
    
        Example spy = spy(new Example());
        doReturn("mocked value").when(spy, "privateMethod");

        String result = spy.publicMethod();
        assertEquals("mocked value called from publicMethod", result);
    }
}

class Example {
    
    
    public String publicMethod() throws Exception {
    
    
        return privateMethod() + " called from publicMethod";
    }

    private String privateMethod() {
    
    
        return "privateMethod";
    }
}

The above is just a simple example. In actual use, it is necessary to configure and write test code according to the specific situation.

6. Comparison of various frameworks

Framework and features	Mock function	Support private methods and static methods	code readability	learning cost	grammatical concepts	Scalability and customization	Integration and compatibility	Documentation and community support	Performance and stability
Junit	×	×	High readability	Low	Simple and easy to understand	Poor	spring-test is integrated by default	good	good
Mockito	√	×	High readability	Low	Simple API, excellent documentation	Poor	spring-test is integrated by default	good	good
TestNg	×	√	Average readability	medium	Simple syntax, but cumbersome dependency configuration	Provides a wealth of expansion points and plug-in mechanisms	good	Generally, there is relatively little documentation	good
PowerMock	√	√	The code is long and complex and not very readable	medium	The syntax is relatively complex	Provides a wealth of expansion points and plug-in mechanisms	good	good	good
Spock	√	√	Need to know Groovy syntax	high	Based on Groovy language, complex syntax	Provides a wealth of expansion points and plug-in mechanisms	Poor compatibility	good	good

Summary: It is recommended to use Junit+Mockito

Junit and TestNG are mostly used for integration testing. PowerMock and Spock have high learning costs, long codes, and complicated readability, which are not conducive to reading.
It has a simple API, excellent documentation, and a large number of examples, which makes it easy to get started and the code complexity is not high.
It is highly readable, the code is simple and easy to understand, it has good community support, and it is easy to find solutions when problems arise.
spring-boot-starter-test integrates the Junit and Mockito frameworks by default.
For private and static methods, they can be mocked using a combination of Junit, PowerMock and Mockito.

7. How to conduct a single test?

1. Usage scenarios of single test

Code reuse rate . The higher the code reuse rate, the more necessary it is to implement single testing, and the more necessary it is to improve the requirements for single testing. Therefore, these codes are referenced by many businesses, so once they have problems, they will affect many business parties. It is more profitable to implement single testing on such codes.
Business change rate . The faster the business changes, the less suitable it is to use single testing. If the business changes very quickly and the content of a single test needs to be modified within a few days after it goes online, then you not only need to modify the business code, but also the single test code, which is double the workload.
Personnel turnover rate . The personnel change rate refers to the changes in the person in charge of a certain module. If the person in charge of a certain module changes frequently, it is not suitable for single testing. Because the new person in charge needs to spend a lot of time getting familiar with the content of the single test, this will cause the requirements development time to become very long.
Business importance . The more core the business, the more necessary it is to implement single testing, and the more necessary it is to meet high standards. Because the stability and robustness of the core business are definitely very important to the company, and single testing can indeed improve system stability and system robustness in the smallest unit.

We cannot look at the four measurement dimensions mentioned above in isolation, but must make comprehensive judgments based on the actual situation to arrive at the most suitable standard!

2. Good unit testing must abide by the principles.

①AIR principle

Note: When the unit test is run online, it feels like air (AIR), but it is very critical to ensure the quality of the test. From a macro perspective, good unit testing has the characteristics of automation, independence, and repeatable execution.

A : Automatic unit testing should be fully automated and non-interactive. Test cases are usually executed regularly, and the execution process must be fully automated to be meaningful. A test whose output requires manual review is not a good unit test.
I : Independent (independence) maintains the independence of unit tests. In order to ensure that unit tests are stable, reliable and easy to maintain, unit test cases must not call each other, nor can they rely on the order of execution. Counter example : method2 needs to rely on the execution of method1, and the execution result is input as a parameter of method2.
R : Repeatable (repeatable) unit tests can be executed repeatedly and cannot be affected by the external environment. Note : Unit tests are usually placed in continuous integration, and unit tests will be executed every time code is checked in. If a single test depends on the external environment (network, service, middleware, etc.), it will easily lead to the unavailability of the continuous integration mechanism. Positive example : In order to not be affected by the external environment, it is required to change the SUT dependency to injection when designing the code, and use a DI framework such as spring to inject a local (memory) implementation or Mock implementation during testing.

②FIRST principle

1. F-Fast (fast)

Unit tests should be able to run quickly. Among various testing methods, unit tests run the fastest. Unit tests for large projects should usually run within a few minutes.

2. I-Independent (independent)

Unit tests should be able to run independently. Unit test cases have no dependence on each other or any external resources.

3. R-Repeatable (repeatable)

Unit tests should be able to run stably and repeatedly, and the results of each run should be stable and reliable.

4. S-SelfValidating (self-verifying)

Unit testing should be automatically verified by use cases and cannot rely on manual verification.

5. T-Timely

Unit tests must be written, updated and maintained in a timely manner to ensure that use cases can dynamically guarantee quality as the business code changes.

3. Write unit test code in compliance with BCDE principles to ensure the delivery quality of the tested module.

B : Border, boundary value testing, including loop boundaries, special values, special time points, data order, etc.
C : Correct, correct input, and get the expected results.
D : Design, combined with design documents to write unit tests.
E : Error, forcing the input of error information (such as illegal data, abnormal processes, non-business allowed input, etc.), and obtaining the expected results.

4. Which scenarios require writing single tests?

The incremental code of core business, core applications, and core modules ensures that unit tests pass.

Unit testing is the most basic means of software development, and one of the most important ideas in software development is the idea of layering. Each high-level module is composed of multiple low-level modules. If it is assembled with some unstable low-level modules, the high-level modules will also become unreliable. Just like the kingdom of mathematics, there are several basic axioms, which are strictly constructed layer by layer through proofs. In addition, when writing business, we should pay more attention to the layered thinking of the traditional three-layer model of controller, service, and dao, rather than dogmatically saying that all businesses have these three layers. Taking the hierarchical idea as the guiding ideology, there are more complex business levels and less simple business levels.

Controller -> service -> Manager external interface

— - - - - - - - - - - - - - -> Dao database

Controller: Responsible for accepting requests and returning responses, as well as simple verification of parameters. For logic without verification, no single test is required. Complex verification logic requires single testing. Mainly used for integration testing

Service: serves as a building block. It is responsible for arranging business logic, processing business logic, processing requests from the Controller layer, and accessing the DAO layer and manager. It needs to write single tests.

Manager: ① Responsible for coordinating multiple Service layer components and processing interactions between service layers, requiring single testing. ② Encapsulate the external interface, and without additional processing logic, no single test is required. ③Interacting with the Dao layer and controlling transactions requires single testing.

Dao: Perform database related operations. Complex logic requires writing single tests, which are purely for fetching data or updating, and basically no single tests are done.

How to perform single testing of the DAO layer without polluting the test database?

Use embedded databases: You can use embedded databases, such as H2, HSQLDB, etc., for unit testing. These embedded databases can run in memory and therefore do not pollute the data in the production database.

Use transaction rollback: You can use transaction rollback to ensure that the test does not pollute the data in the database. Before the test method starts, a transaction is started. After the test is completed, the transaction is rolled back so that all modified data will be restored to the state before the test, thus avoiding data pollution. (Recommended Use)

@RunWith(SpringRunner.class)
@SpringBootTest
public class UserServiceTest {
      
      

    @Autowired
    private UserService userService;

    @Test
    @Transactional
    public void testAddUser() {
      
      
        User user = new User();
        user.setName("John Doe");
        userService.addUser(user);

        // perform assertion
        User savedUser = userService.getUserByName(user.getName());
        Assert.assertNotNull(savedUser);
    }
}

Use database migration tools: You can use database migration tools, such as Flyway, Liquibase, etc., for unit testing. These tools can automatically create a new database instance before each test and use database migration scripts to initialize the data, thereby avoiding the problem of data pollution.
Create a docker image and start a database.

In short, the purpose of writing unit tests is to ensure the correctness and reliability of each module. Only when each module has been verified by unit tests can it be combined into a stable and reliable whole.

8. Discussion questions:

1. In legacy projects, the code is confusing and it is difficult to write single tests, and it cannot be overturned and rewritten. How to elegantly add single tests?

2. After writing the unit test, if the function changes, should you change the code first or write the unit test?

Testing and coding can be compared to two human legs. So the question becomes, should I walk with my left leg or my right leg first? I think everyone will think that the answer to this question is meaningless, but if you think about walking carefully, you will find that the coordination of the left and right legs is the key to our walking. If the steps are long, one leg is lame, or one foot is skipping, They are all in bad shape. Analogy to unit testing, the same is true, test-driven coding, coding optimization testing. Bloated coding is like taking too big a step; a bad single test means one leg is lame; if you don't write a single test, it's a one-legged game.

Detailed explanation of unit testing