Article Directory
Problem scenario
When developing a microservice project, I found that the database link creation process fell into a blocking state. The manifestation is: after the program outputs the log, it is stuck and no longer displays the new log. So use arthasfor problem tracking, and based on the problems found, targeted solutions!
Problem environment
| software | version |
|---|---|
| JDK | 1.8 |
| springboot | 2.1.1.RELEASE |
| hutool | 5.5.5 |
problem causes
The following are arthasthe relevant commands used, readers can have a look:
[arthas@22366]$ dashboard
ID NAME GROUP PRIORITY STATE %CPU TIME INTERRUPTED DAEMON
46 Timer-for-arthas-dashboard-0cd6726c-c0c3-4fa0- system 10 RUNNABLE 91 0:0 false true
26 SimplePauseDetectorThread_0 system 9 TIMED_WAITING 8 0:0 false true
31 Abandoned connection cleanup thread main 5 TIMED_WAITING 0 0:0 false true
16 AsyncResolver-bootstrap-0 main 5 TIMED_WAITING 0 0:0 false true
35 AsyncResolver-bootstrap-executor-0 main 5 WAITING 0 0:0 false true
36 Attach Listener system 9 RUNNABLE 0 0:0 false true
17 DiscoveryClient-0 main 5 TIMED_WAITING 0 0:0 false true
34 DiscoveryClient-1 main 5 WAITING 0 0:0 false true
33 DiscoveryClient-CacheRefreshExecutor-0 main 5 WAITING 0 0:0 false true
15 Eureka-JerseyClient-Conn-Cleaner2 main 5 TIMED_WAITING 0 0:0 false true
3 Finalizer system 8 WAITING 0 0:0 false true
2 Reference Handler system 10 WAITING 0 0:0 false true
5 Signal Dispatcher system 9 RUNNABLE 0 0:0 false true
25 Thread-5 system 9 WAITING 0 0:0 false true
29 Thread-7 main 5 BLOCKED 0 0:0 false false
30 Timer-0 main 5 TIMED_WAITING 0 0:0 false true
[arthas@22366]$ thread --state BLOCKED
Threads Total: 29, NEW: 0, RUNNABLE: 7, BLOCKED: 1, WAITING: 11, TIMED_WAITING: 10, TERMINATED: 0
ID NAME GROUP PRIORITY STATE %CPU TIME INTERRUPTED DAEMON
29 Thread-7 main 5 BLOCKED 0 0:0 false false
Affect(row-cnt:0) cost in 102 ms.
[arthas@22366]$ thread 29
"Thread-7" Id=29 BLOCKED on java.lang.Class@5a6f639c owned by "main" Id=1
at java.sql.DriverManager.registerDriver(DriverManager.java:334)
- blocked on java.lang.Class@5a6f639c
at com.microsoft.sqlserver.jdbc.SQLServerDriver.<clinit>(SQLServerDriver.java:903)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at java.sql.DriverManager$2.run(DriverManager.java:603)
at java.sql.DriverManager$2.run(DriverManager.java:583)
at java.security.AccessController.doPrivileged(Native Method)
at java.sql.DriverManager.loadInitialDrivers(DriverManager.java:583)
at java.sql.DriverManager.<clinit>(DriverManager.java:101)
at oracle.jdbc.driver.OracleDriver.<clinit>(OracleDriver.java:188)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at com.zaxxer.hikari.HikariConfig.setDriverClassName(HikariConfig.java:501)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.zaxxer.hikari.util.PropertyElf.setProperty(PropertyElf.java:146)
at com.zaxxer.hikari.util.PropertyElf.lambda$setTargetFromProperties$0(PropertyElf.java:57)
at com.zaxxer.hikari.util.PropertyElf$$Lambda$587/493239805.accept(Unknown Source)
at java.util.Hashtable.forEach(Hashtable.java:879)
- locked cn.hutool.setting.dialect.Props@647b312d
at com.zaxxer.hikari.util.PropertyElf.setTargetFromProperties(PropertyElf.java:52)
at com.zaxxer.hikari.HikariConfig.<init>(HikariConfig.java:134)
at cn.hutool.db.ds.hikari.HikariDSFactory.createDataSource(HikariDSFactory.java:57)
at cn.hutool.db.ds.AbstractDSFactory.createDataSource(AbstractDSFactory.java:127)
at cn.hutool.db.ds.AbstractDSFactory.getDataSource(AbstractDSFactory.java:92)
- locked cn.hutool.db.ds.hikari.HikariDSFactory@2be25459
[arthas@22366]$ thread --state RUNNABLE
Threads Total: 29, NEW: 0, RUNNABLE: 7, BLOCKED: 1, WAITING: 11, TIMED_WAITING: 10, TERMINATED: 0
ID NAME GROUP PRIORITY STATE %CPU TIME INTERRUPTED DAEMON
49 as-command-execute-daemon system 10 RUNNABLE 100 0:0 false true
36 Attach Listener system 9 RUNNABLE 0 0:0 false true
5 Signal Dispatcher system 9 RUNNABLE 0 0:0 false true
1 main main 5 RUNNABLE 0 0:15 false false
39 nioEventLoopGroup-2-1 system 10 RUNNABLE 0 0:0 false false
44 nioEventLoopGroup-2-2 system 10 RUNNABLE 0 0:0 false false
40 nioEventLoopGroup-3-1 system 10 RUNNABLE 0 0:0 false false
Affect(row-cnt:0) cost in 103 ms.
[arthas@22366]$ thread 1
"main" Id=1 RUNNABLE
at java.sql.DriverManager.registerDriver(DriverManager.java:358)
- locked java.lang.Class@5a6f639c
at java.sql.DriverManager.registerDriver(DriverManager.java:334)
- locked java.lang.Class@5a6f639c
at com.sybase.jdbc4.jdbc.SybDriver.registerWithDriverManager(SybDriver.java:708)
- locked java.lang.Class@5a6f639c
at com.sybase.jdbc4.jdbc.SybDriver.<init>(SybDriver.java:139)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at com.zaxxer.hikari.HikariConfig.setDriverClassName(HikariConfig.java:501)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.zaxxer.hikari.util.PropertyElf.setProperty(PropertyElf.java:146)
at com.zaxxer.hikari.util.PropertyElf.lambda$setTargetFromProperties$0(PropertyElf.java:57)
at com.zaxxer.hikari.util.PropertyElf$$Lambda$587/493239805.accept(Unknown Source)
at java.util.Hashtable.forEach(Hashtable.java:879)
- locked cn.hutool.setting.dialect.Props@657e32f4
at com.zaxxer.hikari.util.PropertyElf.setTargetFromProperties(PropertyElf.java:52)
at com.zaxxer.hikari.HikariConfig.<init>(HikariConfig.java:134)
at cn.hutool.db.ds.hikari.HikariDSFactory.createDataSource(HikariDSFactory.java:57)
at cn.hutool.db.ds.AbstractDSFactory.createDataSource(AbstractDSFactory.java:127)
at cn.hutool.db.ds.AbstractDSFactory.getDataSource(AbstractDSFactory.java:92)
- locked cn.hutool.db.ds.hikari.HikariDSFactory@69e98e1f
at cn.hutool.db.ds.DSFactory.get(DSFactory.java:111)
at cn.hutool.db.Db.use(Db.java:44)
From the result of the above command, it can be seen that there is a thread trapped 死锁, causing the program to be unable to continue execution.
So, now the question is here, what caused it to 死锁happen. We are going to talk about a theory first, about 类初始化the 加锁scene:
Use the same class or interface at the same time. There is also a possibility that the initialization of a class or interface can be requested recursively as part of the initialization of the class or interface; for example, a variable initializer in class a can call a method of an unrelated class B, and an unrelated class B can call the method of class a. The implementation of the Java virtual machine is responsible for synchronization and recursive initialization using the following procedures.
The process assumes that the class object has been validated and prepared, and the class object contains a state that represents one of the following four situations:
1. The class object has been validated and prepared, but has not been initialized.
2. This class object is being initialized by a specific thread T.
3. This type of object has been fully initialized and can be used.
4. Such objects are in an error state, possibly because they tried to initialize but failed.For each class or interface C, there is a unique initialization lock LC. The mapping from C to LC is determined by the implementation of the Java virtual machine. The process of initializing C is as follows:
1. Synchronize the initialization lock LC of C. This includes waiting for the current thread to acquire the LC.
2. If the class object of C indicates that other threads are initializing C, then release LC and block the current thread until it is notified that the ongoing initialization has been completed, and this step is repeated at this time.
3. If the class object of C indicates that the current thread is initializing C, then this must be a recursive initialization request. Release the LC and complete it normally.
4. If the class object of C indicates that C has been initialized, no further operations are required. Release the LC and complete it normally.
5. If the C class object is in an error state, it cannot be initialized. Release the LC and throw NoClassDefFoundError.
6. Otherwise, record that the current thread is initializing the C class object, and then release the LC. Then, initialize the final class variables and fields of the interface whose initial value is a constant expression at compile time
Here we can deal with the cause of the problem:
-
Thread 29 needs to initialize the database link, and the database is a
Oracledatabase. During the initialization, theDriverManagerinstance was obtained. At this time, because it wasDriverManagernot initialized, theDriverManagerstatic code block was initialized. We can see the thread call chain above:at java.sql.DriverManager.loadInitialDrivers(DriverManager.java:583) at java.sql.DriverManager.<clinit>(DriverManager.java:101) at oracle.jdbc.driver.OracleDriver.<clinit>(OracleDriver.java:188)java.sql.DriverManager.loadInitialDriversThe function will find all jdbc driver implementation classes under the classpath. -
Thread 1 starts
Sybasethe creation of the database link, and when it is called, it needs toDriverManager.classbe locked. At this time, because it isDriverManagerin the initialization process and the initialization has not been completed yet, the program pauses here, waiting for other threads to notify the completion of the initialization of this class. code show as below:protected void registerWithDriverManager() { try { Class var1 = DriverManager.class; synchronized(DriverManager.class) { DriverManager.registerDriver(this); Enumeration var2 = DriverManager.getDrivers(); while(var2.hasMoreElements()) { Driver var3 = (Driver)var2.nextElement(); if (var3 instanceof com.sybase.jdbcx.SybDriver && var3 != this) { DriverManager.deregisterDriver(var3); } } } } catch (SQLException var6) { } } -
The thread 29 in the first step needs to load all the driver classes in. At this time, all the driver classes have not been loaded yet, and they are being loaded
com.microsoft.sqlserver.jdbc.SQLServerDriver. In the process of loading this class, you need to call a methodDriverManager.registerDriver(new SQLServerDriver());, check the source code, you can know that the method needs to be locked. And because the second stepDriverManagerwas locked, and theDriverManagerlock could not be released because the initialization was not completed. So I got into a死锁situation where I got the corresponding locks on both sides ! ! !
solution
Don't use the concurrent form to JDBCload the driver class. You can also use clever ways to delay the loading of the database in a thread. In this way, the problem can be avoided to a large extent.
result
The deadlock problem is solved and the program runs normally! ! !
to sum up
From point to surface, you can learn a lot. Only by knowing the reason and knowing the reason can we continue to make progress! ! !
Expand
Class initialization official website description
Ask for praise
If my article is helpful to everyone, you can click like or favorite at the bottom of the article;
if there is a good discussion, you can leave a message;
if you want to continue to view my future articles, you can click Follow
You can scan the following QR code to follow me 'S public account: Fengye Zhixuege, check out my latest share!
