background
This article mainly shares some problems found during stress testing (high concurrency). The previous two articles have described some summaries and optimizations of message queues and database connection pools in the case of high concurrency. If you are interested, you can read them on my official account. Without further ado, let's get to the point.
Transactions, presumably CRUDthe kings are not unfamiliar with them. Basically, there are multiple write requests that need to use transactions, and Spring's use of transactions is very simple, only one @Transactionalannotation is needed, as shown in the following example:
@Transactional
public int createOrder(Order order){
orderDbStorage.save(order);
orderItemDbStorage.save(order.getItems());
return order.getId();
}
When we create an order, we usually need to put the order and the order item in the same transaction to ensure that it satisfies ACID. Here we only need to write a transaction annotation on the method we create the order.
Fair Use of Transactions
For the above code for creating an order, if you now need to add a requirement, send a message to the message queue or call an RPC after the order is created, what would you do? Many students will first think of calling directly in the transaction method:
@Transactional
public int createOrder(Order order){
orderDbStorage.save(order);
orderItemDbStorage.save(order.getItems());
sendRpc();
sendMessage();
return order.getId();
}
This kind of code will appear in the business written by many people. In the transaction, rpc is nested, and some non-DB operations are nested. Under normal circumstances, there is no problem in writing this way. Once the non-DB write operations appear slower, or the traffic is relatively large , there will be a problem of large transactions. Since the transaction has not been committed, the database connection will be occupied. At this time, you may ask, is it okay if I expand the database connections? If 100 is not enough, I will increase to 1000. As mentioned in the previous article, the size of the database connection pool will still affect the performance of our database. Therefore, database connections are not Expand as much as you want.
So how should we optimize it? Here you can think about it carefully, our non-db operation does not meet the ACID of our transaction, so why write it in the transaction, so we can extract it here.
public int createOrder(Order order){
createOrderService.createOrder(order);
sendRpc();
sendMessage();
}
In this method, the creation order of the transaction is called first, and then other non-DB operations are called. If we now want more complex logic, such as sending a successful RPC request if the order is created successfully, and sending a failed RPC request if it fails, we can do the following transformation from the above code:
public int createOrder(Order order){
try {
createOrderService.createOrder(order);
sendSuccessedRpc();
}catch (Exception e){
sendFailedRpc();
throw e;
}
}
Usually, we will catch exceptions, or perform some special processing according to the return value. The implementation here needs to explicitly catch exceptions and throw them at the next time. This method is not very elegant, so how can we write this kind of logic better? ?
TransactionSynchronizationManager
In Spring's transaction, some tool methods are provided to help us fulfill this requirement. The TransactionSynchronizationManagermethod that lets us register callBack with the transaction is provided in:
public static void registerSynchronization(TransactionSynchronization synchronization)
throws IllegalStateException {
Assert.notNull(synchronization, "TransactionSynchronization must not be null");
if (!isSynchronizationActive()) {
throw new IllegalStateException("Transaction synchronization is not active");
}
synchronizations.get().add(synchronization);
}
TransactionSynchronization is the callBack of our transaction, which provides us with some extension points:
public interface TransactionSynchronization extends Flushable {
int STATUS_COMMITTED = 0;
int STATUS_ROLLED_BACK = 1;
int STATUS_UNKNOWN = 2;
/**
* 挂起时触发
*/
void suspend();
/**
* 挂起事务抛出异常的时候 会触发
*/
void resume();
@Override
void flush();
/**
* 在事务提交之前触发
*/
void beforeCommit(boolean readOnly);
/**
* 在事务完成之前触发
*/
void beforeCompletion();
/**
* 在事务提交之后触发
*/
void afterCommit();
/**
* 在事务完成之后触发
*/
void afterCompletion(int status);
}
We can use the afterComplettion method to implement our business logic above:
@Transactional
public int createOrder(Order order){
orderDbStorage.save(order);
orderItemDbStorage.save(order.getItems());
TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() {
@Override
public void afterCompletion(int status) {
if (status == STATUS_COMMITTED){
sendSuccessedRpc();
}else {
sendFailedRpc();
}
}
});
return order.getId();
}
Here we directly implement afterCompletion, which is determined by the status of the transaction, which RPC we should send. Of course, we can further encapsulate TransactionSynchronizationManager.registerSynchronizationthe Util that encapsulates it as a transaction, which can make our code more concise.
In this way we don't have to write all non-DB operations outside the method, so the code is more logically coherent, more readable, and elegant.
The pit of afterCompletion
The callback code of this registered transaction often appears in our business logic, such as refreshing the cache after a transaction is completed, sending message queues, sending notification messages, etc. In daily use, everyone uses this basically There was no problem, but in the process of suppressing, it was found that there was a bottleneck in this area, which took a long time. Through a series of monitoring, it was found that it took a long time to obtain a connection from the database connection pool, and finally we located afterCompeltion This action, actually did not return the database connection.
In Spring's AbstractPlatformTransactionManager, the code for commit processing is as follows:
private void processCommit(DefaultTransactionStatus status) throws TransactionException {
try {
boolean beforeCompletionInvoked = false;
try {
prepareForCommit(status);
triggerBeforeCommit(status);
triggerBeforeCompletion(status);
beforeCompletionInvoked = true;
boolean globalRollbackOnly = false;
if (status.isNewTransaction() || isFailEarlyOnGlobalRollbackOnly()) {
globalRollbackOnly = status.isGlobalRollbackOnly();
}
if (status.hasSavepoint()) {
if (status.isDebug()) {
logger.debug("Releasing transaction savepoint");
}
status.releaseHeldSavepoint();
}
else if (status.isNewTransaction()) {
if (status.isDebug()) {
logger.debug("Initiating transaction commit");
}
doCommit(status);
}
// Throw UnexpectedRollbackException if we have a global rollback-only
// marker but still didn't get a corresponding exception from commit.
if (globalRollbackOnly) {
throw new UnexpectedRollbackException(
"Transaction silently rolled back because it has been marked as rollback-only");
}
}
// Trigger afterCommit callbacks, with an exception thrown there
// propagated to callers but the transaction still considered as committed.
try {
triggerAfterCommit(status);
}
finally {
triggerAfterCompletion(status, TransactionSynchronization.STATUS_COMMITTED);
}
}
finally {
cleanupAfterCompletion(status);
}
}
Here we only need to pay attention to the penultimate lines of code. We can find that our triggerAfterCompletion is the penultimate execution logic. When all the codes are executed, our cleanupAfterCompletion will be executed, and our return database connection is also in this section. In the code, this makes it slower for us to get the database connection.
How to optimize
How to optimize the above problem? There are three options for optimization here:
- The non-DB operation is mentioned outside the transaction. This method is the most primitive method above. For some simple logic, it can be extracted, but for some complex logic, such as the nesting of transactions, afterCompletion is called in the nesting, Doing so would be a lot more work and could be prone to problems.
- By doing it asynchronously with multiple threads, the return speed of the database connection pool is improved. This is suitable for registering afterCompletion and writing it at the end of the transaction, and directly putting what needs to be done on other threads to do it. But if the registration of afterCompletion occurs between our transactions, such as nested transactions, it will cause the subsequent business logic we have to do to be parallel to the transaction.
- Imitates Spring transaction callback registration and implements new annotations. The above two methods have their own drawbacks, so in the end we adopted this method, implemented a custom annotation
@MethodCallBack, marked this annotation on the transaction, and then performed it through a similar registration code.
@Transactional
@MethodCallBack
public int createOrder(Order order){
orderDbStorage.save(order);
orderItemDbStorage.save(order.getItems());
MethodCallbackHelper.registerOnSuccess(() -> sendSuccessedRpc());
MethodCallbackHelper.registerOnThrowable(throwable -> sendFailedRpc());
return order.getId();
}
Through the third method, we basically only need to replace all the places where we registered the transaction callback and it can be used normally.
Let's talk about big business
After talking about big business for so long, what exactly is big business? The simple point is that the transaction time runs for a long time, then it is a big transaction. Generally speaking, the factors that lead to long transaction time running time are nothing more than the following:
- A lot of data operations, such as inserting a lot of data into a transaction, then the transaction execution time will naturally become very long.
- The competition for locks is large. When all connections operate on the same data at the same time, there will be queuing and waiting, and the transaction time will naturally become longer.
- There are other non-DB operations in the transaction, such as some RPC requests. Some people say that my RPC is very fast and will not increase the running time of the transaction, but the RPC request itself is an unstable factor, affected by many factors, network fluctuations, Downstream services are slow to respond. If these factors occur, there will be a large number of transactions that take a long time, which may cause Mysql to hang up and cause an avalanche.
In the above three cases, the first two may not be particularly common, but there are many non-DB operations in the third transaction, which is very common for us. Usually, the reason for this situation is often our own habits and norms, beginners Or some inexperienced people write code, they often write a big method first, add transaction annotations directly to this method, and then add it to it, no matter what logic it is, just a shuttle, just like the picture below Same:
Of course, some people want to do distributed transactions, but unfortunately they use the wrong method. For distributed transactions, you can pay attention to Seata, and you can also use an annotation to help you achieve distributed transactions.
finally
In the end, think about it, why does this happen? Generally, everyone thinks that it is done after completion, and the database connection must have been released long ago, but this is not the case. Therefore, when we use many APIs, we can't take it for granted. If there is no detailed doc, then you should learn more about its implementation details.
Of course, in the end, I hope that everyone should try not to use a shuttle before writing code, and take every code seriously.
If you think this article is helpful to you, your attention and forwarding are the greatest support for me, O(∩_∩)O: