[The Beauty of Design Patterns Design Principles and Ideas: Specification and Refactoring] 29 | Theory 3: What is code testability? How to write testable code?

In the last class, we introduced unit testing and talked about "what is unit testing? Why should we write unit testing? How to write unit testing? Why is it difficult to implement unit testing in practice?" These questions.

In fact, writing unit tests is not difficult and does not require too many skills. On the contrary, writing testable code is a very challenging thing. So, today, let’s talk about the testability of the code again, mainly including the following questions:

• What is code testability?
• How to write testable code?
• What are some common bad-tested code?

Without further ado, let's officially start today's study!

Write testable code case practice

I am going to explain the few issues about code testability just mentioned through a practical case. The specific tested code is as follows.

Among them, Transaction is a transaction class of an e-commerce system after my abstraction and simplification, which is used to record the status of each order transaction. The execute() function in the Transaction class is responsible for performing the transfer operation, transferring money from the buyer's wallet to the seller's wallet. The real transfer operation is done by calling the WalletRpcService RPC service. In addition, the code also involves a distributed lock DistributedLock singleton class, which is used to avoid the concurrent execution of Transaction, causing the user's money to be transferred out repeatedly.

public class Transaction {
  private String id;
  private Long buyerId;
  private Long sellerId;
  private Long productId;
  private String orderId;
  private Long createTimestamp;
  private Double amount;
  private STATUS status;
  private String walletTransactionId;
  
  // ...get() methods...
  
  public Transaction(String preAssignedId, Long buyerId, Long sellerId, Long productId, String orderId) {
    if (preAssignedId != null && !preAssignedId.isEmpty()) {
      this.id = preAssignedId;
    } else {
      this.id = IdGenerator.generateTransactionId();
    }
    if (!this.id.startWith("t_")) {
      this.id = "t_" + preAssignedId;
    }
    this.buyerId = buyerId;
    this.sellerId = sellerId;
    this.productId = productId;
    this.orderId = orderId;
    this.status = STATUS.TO_BE_EXECUTD;
    this.createTimestamp = System.currentTimestamp();
  }
  
  public boolean execute() throws InvalidTransactionException {
    if ((buyerId == null || (sellerId == null || amount < 0.0) {
      throw new InvalidTransactionException(...);
    }
    if (status == STATUS.EXECUTED) return true;
    boolean isLocked = false;
    try {
      isLocked = RedisDistributedLock.getSingletonIntance().lockTransction(id);
      if (!isLocked) {
        return false; // 锁定未成功，返回 false，job 兜底执行
      }
      if (status == STATUS.EXECUTED) return true; // double check
      long executionInvokedTimestamp = System.currentTimestamp();
      if (executionInvokedTimestamp - createdTimestap > 14days) {
        this.status = STATUS.EXPIRED;
        return false;
      }
      WalletRpcService walletRpcService = new WalletRpcService();
      String walletTransactionId = walletRpcService.moveMoney(id, buyerId, sellerId, amount);
      if (walletTransactionId != null) {
        this.walletTransactionId = walletTransactionId;
        this.status = STATUS.EXECUTED;
        return true;
      } else {
        this.status = STATUS.FAILED;
        return false;
      }
    } finally {
      if (isLocked) {
       RedisDistributedLock.getSingletonIntance().unlockTransction(id);
      }
    }
  }
}

Compared with the code of the Text class in the previous lesson, this code is much more complicated. If you were asked to write a unit test for this code, how would you write it? You can try to think about it first, and then come to my analysis below.

In the Transaction class, the main logic is concentrated in the execute() function, so it is the key object of our test. In order to cover various normal and abnormal situations as comprehensively as possible, I designed the following 6 test cases for this function.

• Under normal circumstances, the transaction is executed successfully, and the walletTransactionId used for reconciliation (transaction and wallet transaction flow) is backfilled, the transaction status is set to EXECUTED, and the function returns true.
• If buyerId and sellerId are null and amount is less than 0, InvalidTransactionException will be returned.
• The transaction has expired (createTimestamp is more than 14 days old), the transaction status is set to EXPIRED, and false is returned.
• The transaction has been executed (status==EXECUTED), and the transfer logic will not be repeated, and true will be returned.
• The wallet (WalletRpcService) fails to transfer money, the transaction status is set to FAILED, and the function returns false.
• The transaction is being executed and will not be executed repeatedly, and the function returns false directly.

The test case design is finished. Now it looks like everything is going well. However, the fact is that when we implement the test cases into specific code implementations, you will find that there are many places that do not work. For the above test cases, the second one is very simple to implement, so I won't introduce it. We focus on 1 and 3 of them. Test cases 4, 5, 6 are similar to 3 and are left to you to implement.

Now, let's look at the code implementation of test case 1. Specifically as follows:

public void testExecute() {
  Long buyerId = 123L;
  Long sellerId = 234L;
  Long productId = 345L;
  Long orderId = 456L;
  Transction transaction = new Transaction(null, buyerId, sellerId, productId, orderId);
  boolean executedResult = transaction.execute();
  assertTrue(executedResult);
}

The execution of the execute() function depends on two external services, one is RedisDistributedLock and the other is WalletRpcService. This leads to the following problems in the unit test code above.

• To make this unit test work, we need to set up the Redis service and the Wallet RPC service. The cost of construction and maintenance is relatively high.
• We also need to ensure that after sending the fake transaction data to the Wallet RPC service, we can correctly return the results we expect. However, the Wallet RPC service may be a third-party (developed and maintained by another team) service, which is not under our control . In other words, it doesn't return whatever data we want it to return.
• The execution of Transaction communicates with Redis and RPC services, which needs to go through the network, which may take a long time and affect the execution performance of the unit test itself.
• Network interruption, timeout, unavailability of Redis and RPC services will all affect the execution of unit tests.

Let's go back to the definition of unit testing. Unit testing is mainly to test the correctness of the code logic written by the programmer itself, not an end-to-end integration test. It does not need to test the logical correctness of the external systems (distributed locks, Wallet RPC services) it depends on. Therefore, if the code relies on external systems or uncontrollable components, for example, it needs to rely on databases, network communications, file systems, etc., then we need to de-depend on the code under test from the external system, and this de-dependency method is called Make "mock". The so-called mock is to replace the real service with a "fake" service. The mock service is completely under our control, simulating and outputting the data we want.

So how to mock the service? There are two main ways of mocking, manual mocking and framework mocking. The use of framework mock is only to simplify code writing, and the mock method of each framework is different. We only show manual mocking here.

We implement the mock by inheriting the WalletRpcService class and rewriting the moveMoney() function. The specific code implementation is as follows. By mocking, we can make moveMoney() return any data we want, which is completely within our control and does not require real network communication.

public class MockWalletRpcServiceOne extends WalletRpcService {
  public String moveMoney(Long id, Long fromUserId, Long toUserId, Double amount) {
    return "123bac";
  } 
}
public class MockWalletRpcServiceTwo extends WalletRpcService {
  public String moveMoney(Long id, Long fromUserId, Long toUserId, Double amount) {
    return null;
  } 
}

Now let's look again, how to replace the real WalletRpcService in the code with MockWalletRpcServiceOne and MockWalletRpcServiceTwo?

Because WalletRpcService is created by new in the execute() function, we cannot replace it dynamically. That said, the execute() method in the Transaction class is poorly testable and needs to be refactored to make it more testable. How to refactor this code?

In Section 19, we said that dependency injection is the most effective means of achieving code testability. We can apply dependency injection to reverse the creation of the WalletRpcService object to the upper layer logic, and then inject it into the Transaction class after it is created externally. The code of the Transaction class after refactoring is as follows:

public class Transaction {
  //...
  // 添加一个成员变量及其 set 方法
  private WalletRpcService walletRpcService;
  
  public void setWalletRpcService(WalletRpcService walletRpcService) {
    this.walletRpcService = walletRpcService;
  }
  // ...
  public boolean execute() {
    // ...
    // 删除下面这一行代码
    // WalletRpcService walletRpcService = new WalletRpcService();
    // ...
  }
}

Now, we can easily replace WalletRpcService with MockWalletRpcServiceOne or WalletRpcServiceTwo in unit tests. The unit test corresponding to the refactored code is as follows:

public void testExecute() {
  Long buyerId = 123L;
  Long sellerId = 234L;
  Long productId = 345L;
  Long orderId = 456L;
  Transction transaction = new Transaction(null, buyerId, sellerId, productId, orderId);
  // 使用 mock 对象来替代真正的 RPC 服务
  transaction.setWalletRpcService(new MockWalletRpcServiceOne()):
  boolean executedResult = transaction.execute();
  assertTrue(executedResult);
  assertEquals(STATUS.EXECUTED, transaction.getStatus());
}

The mock and replacement problem of WalletRpcService is solved, let's look at RedisDistributedLock again. Its mock and replacement are more complicated, mainly because RedisDistributedLock is a singleton class. A singleton is equivalent to a global variable, which we cannot mock (methods cannot be inherited and overridden), nor can it be replaced by dependency injection.

If RedisDistributedLock is maintained by ourselves and can be freely modified and refactored, then we can change it to a non-singleton mode, or define an interface, such as IDistributedLock, and let RedisDistributedLock implement this interface. In this way, we can replace RedisDistributedLock with MockRedisDistributedLock in the same way as the previous replacement of WalletRpcService. But if RedisDistributedLock is not maintained by us, we have no right to modify this part of the code, what should we do at this time?

We can repackage the logic of locking the transaction. The specific code implementation is as follows:

public class TransactionLock {
  public boolean lock(String id) {
    return RedisDistributedLock.getSingletonIntance().lockTransction(id);
  }
  
  public void unlock() {
    RedisDistributedLock.getSingletonIntance().unlockTransction(id);
  }
}
public class Transaction {
  //...
  private TransactionLock lock;
  
  public void setTransactionLock(TransactionLock lock) {
    this.lock = lock;
  }
 
  public boolean execute() {
    //...
    try {
      isLocked = lock.lock();
      //...
    } finally {
      if (isLocked) {
        lock.unlock();
      }
    }
    //...
  }
}

For the refactored code, our unit test code is modified to look like this. In this way, we can isolate the logic of the real RedisDistributedLock distributed lock in the unit test code.

public void testExecute() {
  Long buyerId = 123L;
  Long sellerId = 234L;
  Long productId = 345L;
  Long orderId = 456L;
  
  TransactionLock mockLock = new TransactionLock() {
    public boolean lock(String id) {
      return true;
    }
  
    public void unlock() {}
  };
  
  Transction transaction = new Transaction(null, buyerId, sellerId, productId, orderId);
  transaction.setWalletRpcService(new MockWalletRpcServiceOne());
  transaction.setTransactionLock(mockLock);
  boolean executedResult = transaction.execute();
  assertTrue(executedResult);
  assertEquals(STATUS.EXECUTED, transaction.getStatus());
}

At this point, test case 1 is written. Through dependency injection and mock, the unit test code does not depend on any uncontrollable external services. You can follow this line of thought and write test cases 4, 5, and 6 yourself.

Now, let's look at test case 3 again: the transaction has expired (createTimestamp is more than 14 days old), the transaction status is set to EXPIRED, and false is returned. For this unit test case, we still write the code first, and then analyze it.

public void testExecute_with_TransactionIsExpired() {
  Long buyerId = 123L;
  Long sellerId = 234L;
  Long productId = 345L;
  Long orderId = 456L;
  Transction transaction = new Transaction(null, buyerId, sellerId, productId, orderId);
  transaction.setCreatedTimestamp(System.currentTimestamp() - 14days);
  boolean actualResult = transaction.execute();
  assertFalse(actualResult);
  assertEquals(STATUS.EXPIRED, transaction.getStatus());
}

The above code doesn't seem to have any problems. We set the transaction creation time createdTimestamp to 14 days ago, that is to say, when the unit test code runs, the transaction must be in an expired state. However, what if the set method for modifying the createdTimestamp member variable is not exposed in the Transaction class (that is, the setCreatedTimestamp() function is not defined)?

You may say, if there is no set method for createTimestamp, I will add one again! In effect, this violates the encapsulation properties of classes. In the design of the Transaction class, createTimestamp is the system time that is automatically obtained when the transaction is generated (that is, in the constructor), and should not be easily modified artificially. Therefore, although the set method of exposing createTimestamp brings flexibility, But it also brings uncontrollability. Because we can't control whether the user will call the set method to reset the createTimestamp, and resetting the createTimestamp is not our expected behavior.

If there is no set method for createTimestamp, how can test case 3 be implemented? In fact, this is a relatively common type of problem, that is, the code contains "pending behavior" logic related to "time". Our general approach is to repackage this pending behavior logic. For the Transaction class, we only need to encapsulate the logic of whether the transaction expires into the isExpired() function. The specific code implementation is as follows:

public class Transaction {
  protected boolean isExpired() {
    long executionInvokedTimestamp = System.currentTimestamp();
    return executionInvokedTimestamp - createdTimestamp > 14days;
  }
  
  public boolean execute() throws InvalidTransactionException {
    //...
      if (isExpired()) {
        this.status = STATUS.EXPIRED;
        return false;
      }
    //...
  }
}

For the refactored code, the code implementation of test case 3 is as follows:

public void testExecute_with_TransactionIsExpired() {
  Long buyerId = 123L;
  Long sellerId = 234L;
  Long productId = 345L;
  Long orderId = 456L;
  Transction transaction = new Transaction(null, buyerId, sellerId, productId, orderId) {
    protected boolean isExpired() {
      return true;
    }
  };
  boolean actualResult = transaction.execute();
  assertFalse(actualResult);
  assertEquals(STATUS.EXPIRED, transaction.getStatus());
}

Through refactoring, the testability of Transaction code is improved. All the test cases listed before have now been successfully implemented. However, the design of the constructor of the Transaction class is a bit off. For your convenience, I re-copied the code of the constructor and posted it here.

  public Transaction(String preAssignedId, Long buyerId, Long sellerId, Long productId, String orderId) {
    if (preAssignedId != null && !preAssignedId.isEmpty()) {
      this.id = preAssignedId;
    } else {
      this.id = IdGenerator.generateTransactionId();
    }
    if (!this.id.startWith("t_")) {
      this.id = "t_" + preAssignedId;
    }
    this.buyerId = buyerId;
    this.sellerId = sellerId;
    this.productId = productId;
    this.orderId = orderId;
    this.status = STATUS.TO_BE_EXECUTD;
    this.createTimestamp = System.currentTimestamp();
  }

We found that the constructor does not only contain simple assignment operations. The assignment logic of the transaction id is a little more complicated. We'd better test it too to ensure the correctness of this part of the logic. For the convenience of testing, we can abstract the logic of id assignment into a function. The specific code implementation is as follows:

public Transaction(String preAssignedId, Long buyerId, Long sellerId, Long productId, String orderId) {
    //...
    fillTransactionId(preAssignId);
    //...
  }
  
  protected void fillTransactionId(String preAssignedId) {
    if (preAssignedId != null && !preAssignedId.isEmpty()) {
      this.id = preAssignedId;
    } else {
      this.id = IdGenerator.generateTransactionId();
    }
    if (!this.id.startWith("t_")) {
      this.id = "t_" + preAssignedId;
    }
  }

So far, we have refactored Transaction from untestable code to well-testable code step by step. However, you may still have questions, does the isExpired() function in the Transaction class not need to be tested? For the isExpired() function, the logic is very simple, you can judge whether there is a bug with the naked eye, and you don't need to write a unit test.

In fact, codes with poor testability are not designed well enough, and many places do not follow the design principles and ideas we mentioned before, such as the idea of ​​"programming based on interfaces rather than implementation" and the principle of dependency inversion. The refactored code not only has better testability, but also follows the classic design principles and ideas from the perspective of code design. This also confirms what we said before that the testability of the code can reflect whether the code design is reasonable from the side. In addition, in the usual development, we have to think more about whether it is easy to write unit tests when writing code in this way, which is also conducive to us designing good code.

Other Common Anti-Patterns

We just used a practical case to explain how to use dependency injection to improve the testability of code, and the most complicated part of writing unit tests: how to de-depend on external services through mocking and secondary packaging. Now, let's summarize what are the typical and common codes with poor testability, which is what we often call Anti-Patterns.

1. Pending actions

The so-called pending behavior logic is that the output of the code is random or uncertain, for example, code related to time and random numbers. Regarding this point, we have already mentioned in the actual combat case just now, you can use the method just mentioned, try to refactor the following code, and write unit tests for it.

public class Demo {
  public long caculateDelayDays(Date dueTime) {
    long currentTimestamp = System.currentTimeMillis();
    if (dueTime.getTime() >= currentTimestamp) {
      return 0;
    }
    long delayTime = currentTimestamp - dueTime.getTime();
    long delayDays = delayTime / 86400;
    return delayDays;
  }
}

2. Global variables

As we said earlier, global variables are a process-oriented programming style with various drawbacks. In fact, misuse of global variables also makes writing unit tests difficult. Let me explain with an example.
RangeLimiter represents a range of [-5, 5], the position is initially at 0, and the move() function is responsible for moving the position. Among them, position is a static global variable. The RangeLimiterTest class is a unit test designed for it, but there are big problems here, you can analyze it yourself first.

public class RangeLimiter {
  private static AtomicInteger position = new AtomicInteger(0);
  public static final int MAX_LIMIT = 5;
  public static final int MIN_LIMIT = -5;
  public boolean move(int delta) {
    int currentPos = position.addAndGet(delta);
    boolean betweenRange = (currentPos <= MAX_LIMIT) && (currentPos >= MIN_LIMIT);
    return betweenRange;
  }
}
public class RangeLimiterTest {
  public void testMove_betweenRange() {
    RangeLimiter rangeLimiter = new RangeLimiter();
    assertTrue(rangeLimiter.move(1));
    assertTrue(rangeLimiter.move(3));
    assertTrue(rangeLimiter.move(-5));
  }
  public void testMove_exceedRange() {
    RangeLimiter rangeLimiter = new RangeLimiter();
    assertFalse(rangeLimiter.move(6));
  }
}

The above unit test may fail to run. Assume that the unit testing framework executes the two test cases testMove_betweenRange() and testMove_exceedRange() sequentially. After the execution of the first test case is completed, the value of position becomes -1; when the second test case is executed, the position becomes 5, the move() function returns true, and the assertFalse statement fails. So, the second test case fails to run.

Of course, if the RangeLimiter class has a function to expose the reset (reset) position value, we can reset the position to 0 before each execution of the unit test case, which can solve the problem just now.

However, the way each unit testing framework executes unit test cases may be different. Some are executed sequentially and some are executed concurrently. For concurrent execution, even if we reset the position to 0 every time, it doesn't work. If two test cases are executed concurrently, the four lines of code 16, 17, 18, and 23 may be executed interleaved, affecting the execution result of the move() function.

3. Static methods

We also mentioned earlier that static methods, like global variables, are also a process-oriented programming thinking. Calling static methods in code sometimes makes the code difficult to test. The main reason is that static methods are also difficult to mock. However, this depends on the situation. Only when the static method takes too long to execute, depends on external resources, has complex logic, and pending behavior, we need to mock this static method in the unit test. In addition, if it is just a simple static method like Math.abs(), it will not affect the testability of the code, because it does not require a mock.

4. Complex inheritance

As we mentioned earlier, compared with the composition relationship, the code structure of the inheritance relationship is more coupled, inflexible, and less easy to expand and maintain. In fact, inheritance relationships are also more difficult to test. This also confirms the correlation between code testability and code quality.

If the parent class needs to mock a dependent object for unit testing, then all subclasses, subclasses of subclasses... must mock this dependent object when writing unit tests. For an inheritance relationship with a deep hierarchy (shown as vertical depth in the inheritance relationship class diagram) and complex structure (shown as horizontal breadth in the inheritance relationship class diagram), the lower the subclass may have more objects to mock, This will lead to the fact that when the underlying subclasses write unit tests, they need to mock many dependent objects one by one, and they also need to check the parent class code to understand how to mock these dependent objects.

5. Highly coupled code

If a class has heavy responsibilities and needs to rely on more than a dozen external objects to complete the work, and the code is highly coupled, then we may need to mock these more than a dozen dependent objects when writing unit tests. This is unreasonable both from the point of view of code design and from the point of view of writing unit tests.

key review

Well, that's all for today's content. Let's summarize and review together, what you need to focus on.

1. What is code testability?

Roughly speaking, the so-called testability of the code is the ease of writing unit tests for the code. For a piece of code, if it is difficult to write a unit test for it, or the unit test is very laborious to write and needs to rely on very advanced features in the unit test framework, it often means that the code design is not reasonable enough and the testability of the code is not good.

2. The most effective means of writing testable code

Dependency injection is the most effective means of writing testable code. Through dependency injection, when we write unit tests, we can de-depend on external services through the mock method, which is also the most technical challenge in the process of writing unit tests.

3. Common Anti-Patterns

Common test-unfriendly codes include the following five types:

• Code contains pending action logic
• Misuse of mutable global variables
• Abuse of static methods
• Use complex inheritance relationships
• highly coupled code

class disscussion

• The void fillTransactionId(String preAssignedId) function in the actual combat case contains a static function call: IdGenerator.generateTransactionId(), will this affect the testability of the code? When writing unit tests, do we need to mock this function?
• We talked about today that dependency injection is the most effective means to improve code testability. Therefore, dependency injection means not to create objects through new inside the class, but to create them externally and pass them to the class for use. Does that mean that all objects cannot be created inside the class? What type of objects can be created inside a class without affecting the testability of the code? Can you give some examples?
  Welcome to write down your answers in the message area, communicate and share with your classmates. If you gain something, you are welcome to share this article with your friends.

If we use composition instead of inheritance to organize the relationship between classes, the structural hierarchy between classes is relatively flat. When writing unit tests, we only need to mock the objects that the class depends on.