I believe that readers who have written unit tests in Java will be familiar with Mockito. As for what Mockito is and why Mockito is used, this article will not go into details.
Mockito.mockStaticThis article records a troubleshooting process of occasional unit test errors caused by improper use in the Apache ShardingSphere project .
Article directory
foreword
Mockito has added a new method since 3.4.0 Mockito.mockStatic, which supports mocking of static methods.
I also answered a question on Stack Overflow, showing my Mockito.mockStaticcase of using mock singleton in the unit test code of Apache ShardingSphere Mockito.mockStatic. Students who are not particularly familiar with the method can learn about it:
How to use Mockito mock singleton Mocking a singleton with mockito
mockStaticWhat precautions are used to implement? Let's take a look at the description of the official Mockito documentation: 48. Mocking static methods (since 3.4.0)
When using the inline mock maker, it is possible to mock static method invocations within the current thread and a user-defined scope. This way, Mockito assures that concurrently and sequentially running tests do not interfere. To make sure a static mock remains temporary, it is recommended to define the scope within a try-with-resources construct.
The general meaning is: mockStaticthe scope of the method is the current thread and the scope defined by the user. To ensure that mockStaticit only takes effect temporarily, it is recommended to wrap it with a try-with-resources code block mockStatic.
Interpreting the example provided by the Mockito docs:
assertEquals("foo", Foo.method()); // 静态方法 Foo.method() 原本行为
try (MockedStatic mocked = mockStatic(Foo.class)) {
// 对 Foo 类进行 mockStatic
mocked.when(Foo::method).thenReturn("bar"); // 通过 mock 改变静态方法 Foo.method() 行为
assertEquals("bar", Foo.method()); // 进行测试断言
mocked.verify(Foo::method);
}
assertEquals("foo", Foo.method()); // 离开 mockStatic 作用域，Foo.method() 恢复原本行为
Now let's think about what would happen if
mockStaticthe method wasn't wrapped in a try-with-resources block and the object wasn't closed manually ?MockedStatic
According to the description in the document, if it is not closed mockStatic, will the behavior of the mocked static class on this thread always be changed?
The unit test of Apache ShardingSphere once had Mockito.mockStaticthe problem that the unit test failed sporadically because it was not released after use.
Troubleshooting process
Apache ShardingSphere will run CI through GitHub Actions for each PR or commit merged into master - the standard Maven clean install process, which includes running unit tests during the install process.
For a while, the CI of ShardingSphere occasionally failed. I asked other students who are also involved in the development of ShardingSphere. Local install or unit testing may also fail.
Due to a long time, GitHub Actions logs have been cleaned up.
If the unit test of a project cannot be guaranteed to pass stably, it must be a problem with the test code or a hidden danger in the production code .
Problem recurrence
Let's look at a unit test under the ShardingSphere infra-common module. There is a use case in ShardingSphereMetaDataTest as follows:
@Test
public void assertGetMySQLDefaultSchema() throws SQLException {
MySQLDatabaseType databaseType = new MySQLDatabaseType();
ShardingSphereDatabase actual = ShardingSphereDatabase.create("foo_db", databaseType, Collections.singletonMap("", databaseType), mock(DataSourceProvidedDatabaseConfiguration.class), new ConfigurationProperties(new Properties()), mock(InstanceContext.class));
assertNotNull(actual.getSchema("foo_db"));
}
Running this test case alone is passed.
However, this use case fails if all tests under the infra-common module are run.
Among them, ShardingSphereDatabase.createthe final static method called is roughly as follows. There are only ShardingSphereDatabasetwo possibilities in the code to return an instance normally or throw an exception, and there is no return nullof .
private static ShardingSphereDatabase create(final String name, final DatabaseType protocolType, final DatabaseConfiguration databaseConfig, final Collection<ShardingSphereRule> rules, final Map<String, ShardingSphereSchema> schemas) {
// 省略中间过程代码
return new ShardingSphereDatabase(name, protocolType, resourceMetaData, ruleMetaData, schemas);
}
However, such a simple unit test does report a null pointer, and it is still actual( ShardingSphereDatabase.createthe return result of the static method) null.
java.lang.NullPointerException: Cannot invoke "org.apache.shardingsphere.infra.metadata.database.ShardingSphereDatabase.getSchema(String)" because "actual" is null
at org.apache.shardingsphere.infra.metadata.ShardingSphereMetaDataTest.assertGetMySQLDefaultSchema(ShardingSphereMetaDataTest.java:109)
From the code point of view, a static method that cannot return nullis returned in the unit test null, I don't understand!
Since the local environment can continue to have certain problems for the time being, you can interrupt the Debug.
The reason why failures are sporadic rather than inevitable is that the order in which unit tests under a module are run is not constant. Some test codes that may pollute other test cases happen to be relatively late in the running sequence, and the test operation appears to pass normally.
I have also solved another occasional problem affected by the execution order of unit tests. For details, please refer to my previous article: Troubleshooting and repairing sporadic failures of shardingsphere-jdbc-core unit tests caused by a ThreadLocal leak
debug code
Put a breakpoint, run the full module test, and run to the code before the assertion failed.
Come to a quick expression calculation, indeed ShardingSphereDatabase.createthe method returns null.
Then go inside the method to see:
Find clues & solve
A strange phenomenon appeared! You can see the animation below: After
entering ShardingSphereDatabaes.createthe method, click Step Into, and normally you should continue to enter createthe method with the first line of code in the method DatabaseRulesBuilder.build, but the debugger jumps directly to createthe method return, and clicking Step Intodoes not continue to enter createthe method!
This strange phenomenon, based on experience, may be that the actual running bytecode and source code do not match. I searched the code globally for mockStaticthe use of the method, and found that some unit test codes use mockStaticthe method, but neither try-with-resources nor manual release.
Therefore, I mockStaticrepaired the improperly used code, and added the requirements for using mockStaticand in the code specification of ShardingSphere mockConstruction.
Specifically, you can see:
- Fix ShardingSphere unit test mockStatic leak: Fix mockStatic leak in unit tests #19077
- Update the code specification used by ShardingSphere about Mockito: Update Code of Conduct about Mockito #19083
dig a hole
Issues with unit testing have been identified and resolved in the previous steps, but this was a matter of personal experience and luck.
mockStaticIf I am a developer who has never used methods such as , and has no relevant experience, I cannot directly drawmockStaticconclusions about leaks based on the Debug phenomenon of IDEA. How can I troubleshoot such leaks?
Find time to continue digging deeper into this question.