Remember a performance optimization process of settlement procedures

Written in 2017-10-6

Background: a project clearing process, the system generates daily business day of the full amount of the transaction data and upload to FTP, analytical data after the transaction settlement procedures to obtain files from FTP, perform a balance change operation and registration of funds flowing water.

A first wheel pressure measurement results: TPS = 3

After analyzing the source discovering, developing shoes single serial process, simplified core code is as follows:

//遍历每条交易数据
foreach(){
    insertLogAndUpdateBalance();
}

//启用事务
insertLogAndUpdateBalance(){
    //插入资金流水
    insertLog();
    //更新余额
    updateBalance();
}

The first round improved method of:
1, changed into serial multi-process single serial process pen
2, a balance change update switch insert, i.e. credited to the balance of the water table, and then back to an asynchronous refresh Balance Table
3, introduces a new problem where : how to ensure that the balance is not deducted as negative? The problem behind the other from the presentations.
Simplified core code

//遍历每批交易数据，每500条一批
foreachBatch(){
    insertLogAndUpdateBalanceBatch();
}

//启用事务
insertLogAndUpdateBalance(){
    //插入资金流水
    insertLogBatch();
    //更新余额
    updateBalanceBatch();
}

The second wheel pressure measurement results: TPS is unstable between 15 - 45

The second round improved process:
After the developers to find a way to exclude the timing of the task, (SpringBoot item)

@EnableScheduling
public class ExceptionStatusChangeNotifyScheduler {
 @Scheduled(cron = "0 */1 * * * ?")
    public void doUpdateStatus() {
        //业务逻辑处理
    }
}

EnableScheduling look at the source code you can see the class description which has been very clear description, the default is to use a single thread, and we have multiple projects simultaneously timed tasks, so this also explains why TPS will be very unstable phenomenon.

 * In all of the above scenarios, a default single-threaded task executor is used.
 * When more control is desired, a {@code @Configuration} class may implement
 * {@link SchedulingConfigurer}. This allows access to the underlying
 * {@link ScheduledTaskRegistrar} instance. For example, the following example
 * demonstrates how to customize the {@link Executor} used to execute scheduled
 * tasks:

For that matter, I deliberately a bit Baidu also saw some books and article describes SpringBoot timed tasks are the same demo, but did not mention this way is single-threaded ......
So here I say the code on the network and can refer to books, but there should be relatively detailed understanding helpless when a problem as not appears.
The solution is simple, it is to achieve SchedulingConfigurer interfaces, for example:

public class XXX implements SchedulingConfigurer{
    @Override
    public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
        taskRegistrar.setScheduler(taskExecutor());
    }    
    @Bean(destroyMethod="shutdown")
    public Executor taskExecutor() {
        return Executors.newScheduledThreadPool(30);
    }
}

The third round pressure test results: TPS = 30
third round of the improved method:
has been progress, is not it, at least compared to the first round results, we have increased by 10 times, but this reasonable? We are all insert operations, plus log after investigation, the final positioning bulk insert 4500 records need 10S above, we insert statements like this. (Using MyBatis 3.1.1)

<insert id="insertBatch">
       insert into xxx (msg_id, balance_type,
           amount, account_id, user_id,
           is_handle, create_time
           ) values
     <foreach collection="list" item="item" index="index" separator=",">
       (#{item.msgId,jdbcType=VARCHAR}, #{item.balanceType,jdbcType=VARCHAR},
           #{item.amount,jdbcType=BIGINT}, #{item.accountId,jdbcType=BIGINT}, #{item.userId,jdbcType=VARCHAR},
           #{item.isHandle,jdbcType=INTEGER}, #{item.createTime,jdbcType=TIMESTAMP}
           )
     </foreach>
     </insert>

In the case of using MyBatis foreach grammar, generates insert into xxx values (aa, bb), (cc, dd) such sql statement, tested, each of length 500K sql, network latency is also substantially excluded, and can not help but begin to doubt foreach grammar MyBatis, so SQL generated by the above code, and then performed via MyBatis, the following sample code:

<insert id="insertBySql" parameterType="com.moext.SqlVo" >
    ${sql}
 </insert>

The fourth round of pressure test results: TPS = 300, and start the second batch of 100 down to about
the fourth round Improvement: negative first significant data gaps and the data amount subsequent batches case, the target issues Mysql, after positioning the original field msg_id (value UUID) there is a unique index, there are two problems to use UUID as the only index:
1, is too long
2, disorderly, each insert operation, the index rebuild is not efficient
in this field is due to historical reasons there is, in fact, has no access, it is deleted.

Fifth wheel pressure measurement results: TPS = 300 +

to sum up:

The performance optimization of the process, from the first batch operations into serial serial single operation, each time increasing the cross-process (service to DB) the amount of interactive data; then discovered the existence of a single-threaded problem SpringBoot by TPS instability in EnableScheduling ;
Finally, based on historical experience and test comparison and locate performance problems using Mybatis of foreache; TPS suddenly last fall according to locate the UUID as the only index of the problems, which will eventually stand-alone serial processing TPS increased to 300 or more. So far only a single performance. How to use multi-machine further enhance the TPS? The reader to consider scalability design. Follow-up to make presentations.

Remember a performance optimization process of settlement procedures

Guess you like