Author : Senior Mingming Ruyue, CSDN blog expert, senior Java engineer of Dachang, author of "Performance Optimization Methodology", "Unlocking Dachang Thinking: Analysis of "Alibaba Java Development Manual"", "Relearning Classics: Exclusive Analysis of "EffectiveJava"" Columnist.
Recommended popular articles :
- (1) "Why do many people work for 3 years but only have 1 year of experience?" 》
- (2) "From Disappointment to Mastery: Mastery and Application Skills of AI Large Models"
- (3) "In the AI era, where is the future for programmers?" 》
- (4) "How to write high-quality articles: from strategy to tactics"
- (5) "My Technology Learning Methodology"
- (6) "My Performance Methodology"
- (7) "Learning Methods in the AI Era: Dialogue with Documents"
- (8) "The artificial intelligence terminal is coming, are you still using the outdated iterm?" 》
I. Introduction
Programmer Xiao Ming encountered a very strange problem. It was obvious that the data status had been updated successfully before, but some data (not all) were not found when subsequent query data was based on the updated status. As a result, the defense code judged to be empty and returned directly. No subsequent synchronization operations were performed.
After checking for a long time, I couldn't find her sister.
So the programmer Xiao Ming asked for help from his senior brother, who said: "It's a long story, just read Senior Mingming Ruyue's article..."
2. Scene recurrence
Here is a code that reproduces the problem:
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
@Service
public class MyService {
@Autowired
private MyRepository myRepository;
private final ExecutorService threadPool = Executors.newFixedThreadPool(5);
public void updateAndSyncData(List<Long> ids, String newState) {
log.info("更新状态和同步数据, ids:{}, newState:{}" ,ids, newState)
// Step 1: 修改数据状态
List<MyEntity> entities = myRepository.findByIds(ids);
entities.foreach(entity->{
entity.setState(newState);
myRepository.save(entity);
});
// Step 2: 在新的线程中查询数据状态并调用新接口来同步数据
threadPool.submit(() -> {
ids.foreach(id->{
try{
// 根据新的状态查询数据
MyEntity entity = myRepository.findByIdAndState(id,newState);
if(entity == null){
log.info("未查询到数据, id:{}, newState:{}" ,id, newState)
return;
}
//调用下游接口同步
log.info("执行下游同步, id:{}, newState:{}" ,id, newState)
callNewInterfaceToSyncData(entity, newState);
}catch(Exception e){
log.error("执行下游同步失败, id:{}, newState:{}" ,id, newState)
}
});
});
}
private void callNewInterfaceToSyncData(MyEntity entity,String state) {
// 在这里调用新接口来同步数据到其他系统
}
}
Note: This is just to demonstrate the problem, please do not get into details, such as performance issues.
Give you 2 minutes to think about the possible reasons?
[2 minutes]
Experienced programmers will have a hunch that it may be related to multi-threading.
But the source code is very confusing. Although the thread pool is newly started to execute the task and the data is queried according to the new status, the status of the thread pool has been updated before the task is submitted? !
unless…
3. Problem analysis
To investigate the problem, we need to make bold guesses and verify carefully.
3.1 Conjecture 1: Is the code logic wrong?
There may be a problem with the code logic, such as a problem with the statement to update the status, a problem with the query SQL based on ID and status, etc.
After re-reviewing the code, I found that the logic and underlying SQL statements were fine.
3.1 Conjecture 2: Is there an error report, causing status modification to fail or query to succeed and synchronization to fail?
Through the logs, it was found that there were no errors reported. After verification, there will be abnormal logs in places where errors may occur, so they were eliminated.
3.2 Conjecture 3: Was it modified by other threads before querying?
There is a possibility that the status was changed by other threads before the asynchronous query.
The data update time in the log and database proves that it has not been modified by other threads.
3.3 Conjecture 4: Is there something going on in the outer layer?
Judging from the above code, we do not see any open transactions.
Continue to scroll up, and after four or five floors, I found that the transaction has indeed been opened! !
So, the truth comes out.
A transaction is opened externally and the status is modified. When querying some data based on the new status in the thread pool, the transaction has not yet been submitted and cannot be found using the new status, resulting in subsequent synchronization tasks not being updated.
Some people may say, this is not difficult, right? ?
Indeed, when you see this, it seems very simple, but when you write code with too many layers, it is easy to forget that external transactions are started.
In addition, many students who have made similar mistakes will know it if you ask them, but they may not notice it when writing.
4. Relevant knowledge points
4.1 Four characteristics of transactions
- Atomicity : All operations in a transaction either succeed or fail and are rolled back, and there will be no partial execution.
- Consistency : A transaction must transform the database from one consistency state to another. That is to say, the data in the database satisfies predefined rules and constraints before and after the transaction is executed.
- Isolation : Transactions are isolated from each other, and the execution of one transaction will not be affected by another transaction. Different isolation levels can prevent concurrency problems such as dirty reads, non-repeatable reads, and phantom reads.
- Durability : Once a transaction is submitted, its changes to the data in the database are permanent. Even if a system failure or database crash occurs, the submitted data will not be lost.
The data state was modified in a transaction, but the transaction had not yet committed when you created a new thread to query for these changes. Therefore, the query in the new thread cannot see these uncommitted changes because it is in a different transactional or non-transactional state.
4.2 The relationship between transactions and threads
- A transaction is a set of logically related operations in a database that are either all executed or none of them are executed. The four major characteristics of transactions are atomicity, consistency, isolation and durability.
- A thread refers to an execution path in a program that can execute multiple tasks concurrently. Memory and resources can be shared between threads, but synchronization and coordination are also required.
- The relationship between transactions and threads mainly depends on the way database connections and transaction management are performed. Database connection refers to the communication channel between the program and the database, and transaction management refers to the process of controlling the start, submission and rollback of transactions.
- A common approach is based on thread-bound database connections and declarative transaction management . In this way, each thread obtains an independent database connection when executing a transaction, and declares the boundaries and properties of the transaction through annotations or configurations. This ensures that each thread has its own transaction context and will not interfere with each other.
- Another approach is based on shared database connections and programmatic transaction management . In this way, multiple threads can use the same database connection and manually control the start, commit, and rollback of transactions through code. This can save database connection resources, but you also need to pay attention to thread safety and transaction isolation issues.
5. Solutions
There are many solutions, the most common ones are as follows:
- Query it out in the transaction before asynchronous execution (if the transaction is rolled back later, the asynchronous logic may also be executed normally).
- Commit the transaction before executing asynchronous logic.
- Can be used
TransactionSynchronizationManagerto register a callback that will be executed after the current transaction is successfully committed. This allows you to perform specific logic after the transaction commits (more rationally). - Remove asynchronous logic and change everything to synchronous logic.
Since the specific implementation is not difficult, no code demonstration is needed here. The specific strategy to be adopted needs to be decided based on the actual situation.
6. Enlightenment
6.1 Pay attention to code review
This problem can still be seen if the code is reviewed carefully.
For example, if the person being reviewed explains the code logic layer by layer from the Facade to the Dao layer, the reviewing students will most likely see this problem when they see transactions and asynchrony.
Of course, this cannot rely solely on code review. Everyone should proactively think about possible problems when using thread pools.
6.2 Make bold guesses and verify carefully
I think the bad question should be: "Bold guesses, careful verification."
Don't make random guesses, as guessing can easily waste a lot of time.
You need to reversely speculate on possible causes based on the performance of the problem and your own professional capabilities, and demonstrate your guesses based on code, logs, database data, etc.
Of course, there are many "weird problems" because "you don't know the true face of Mount Lu, just because you are in this mountain". Sometimes it is easier to find the cause earlier by asking the classmates around you to help.
6.3 Integrate knowledge with action and apply what you have learned (misunderstanding of "Eight Part Essay")
During the interview, ask job seekers: "The four major characteristics of affairs", and most people can "recite them backwards and forwards." Many people even think that this is an "eight-legged essay" and is meaningless.
However, during the actual coding process, it is easy to forget this knowledge, leading to a disconnect between knowledge and application.
The purpose of learning is to apply what you have learned, as Mr. Gu Jin said: "memory, understanding, expression, and integration."
In fact, memory does not mean mastery. Being able to integrate knowledge and action, and being able to express comprehensive understanding means that you have truly mastered knowledge.
7. Summary
This article explains the situation where the asynchronous query cannot find data when the transaction is not submitted, causing the code effect to be inconsistent with expectations, and provides a solution.
Everyone should pay special attention to this issue when using asynchronous threads to perform tasks in transactions.
Everyone should strengthen code review, and there is a high probability that some problems can be avoided. At the same time, when investigating problems, everyone must use "evidence as the basis" and "bold guesses and careful verification."
Creation is not easy. If this article is helpful to you, please like, collect and follow it. Your support and encouragement are the biggest motivation for my creation.
Welcome to join my Knowledge Planet, Knowledge Planet ID: 15165241 (has been operating for 5 years and will continue to operate) to communicate and learn together.
https://t.zsxq.com/Z3bAiea marked it as coming from CSDN when applying.