DBPack read-write separation function release announcement

In the v0.1.0 version, we released the distributed transaction function and provided a preview of the read-write separation function. In the v0.2.0 version, we added the function of custom query request routing through hint, and fixed some bugs. In addition, in this version, we also provide a preview of the audit log function, which will be officially released in v0.3.0. UseDB 

Fix bug situations

1. PHP 8.0 pdo throws exception transaction not active 

   When the Mysql client sends the sql execution result to the user, if there is no abnormality in the execution, the first packet sent is OKPacket, and there is a flag in this packet to identify whether the sql request is in a transaction. As shown below:

   

   The contents of this package are:

   07 00 00 // 前 3 个字节表示 payload 的长度为 7 个字节
   01 // sequence 响应的序号，前 4 个字节一起构成了 OKPacket 的 header
   00 // 标识 payload 为 OKPacket
   00 // affected row
   00 // last insert id
   03 00 // 状态标志位
   00 00 // warning 数量
   

   The previous version of dbpack set the flag bit to 0, the mysql driver before java, golang, .net core, and php 8.0 can correctly coordinate transactions, and the pdo driver of php 8.0 will check the flag bit, so the version above php 8.0 is in An exception is thrown when using dbpack to coordinate distributed transactions . transaction not active 

2. Load balancing algorithm deserialization exception

   

   This exception causes read-write split query requests to be executed between DBs with a random algorithm.

3. other bugs.

new features

Use hint to customize query request routing UseDB 

E.g:

SELECT /*+ UseDB('employees-master') */ emp_no, birth_date, first_name, last_name, gender, hire_date FROM employees WHERE emp_no = ?

Add an annotation to the query request. The parameter of the annotation is the name of the data source, and you can specify which data source the SQL request is routed to for execution. UseDB 

Preview function

This version adds the audit log function. It can be enabled by adding it to the configuration , for example: AuditLogFilter 

filters:
  - name: auditLogFilter
    kind: AuditLogFilter
    conf:
      audit_log_dir: /var/log/dbpack/
      # unit MB
      max_size: 300
      # unit Day
      max_age: 28
      # maximum number of old log files to retain
      max_backups: 1
      # determines if the rotated log files should be compressed using gzip
      compress: true
      # whether to record the audit log before or after the sql request is actually executed
      record_before: true

When enabled, DBPack records audit logs in the following format:

[timestamp],[username],[ip address],[connection id],[command type],[command],[sql text],[args],[affected row]

The record is as follows:

2022-06-14 07:15:44,dksl,172.18.0.1:60372,1,COM_QUERY,,SET NAMES utf8mb4,[],0
2022-06-14 07:15:45,dksl,172.18.0.1:60372,1,COM_STMT_EXECUTE,INSERT,INSERT INTO employees ( emp_no, birth_date, first_name, last_name, gender, hire_date ) VALUES (?, ?, ?, ?, ?, ?),['100000' '1992-01-07' 'scott' 'lewis' 'M' '2014-09-01'],1
2022-06-14 07:15:45,dksl,172.18.0.1:60372,1,COM_STMT_EXECUTE,DELETE,DELETE FROM employees WHERE emp_no = ?,['100000'],1
2022-06-14 07:15:45,dksl,172.18.0.1:60372,1,COM_STMT_EXECUTE,INSERT,INSERT INTO employees ( emp_no, birth_date, first_name, last_name, gender, hire_date ) VALUES (?, ?, ?, ?, ?, ?),['100001' '1992-01-07' 'scott' 'lewis' 'M' '2014-09-01'],1
2022-06-14 07:15:45,dksl,172.18.0.1:60372,1,COM_STMT_EXECUTE,SELECT,SELECT emp_no, birth_date, first_name, last_name, gender, hire_date FROM employees WHERE emp_no = ?,['100001'],0

illustrate

Next version plans:

• V0.3.0

  Audit log, SQL request link trace

• V0.4.0

  Sensitive data encryption and decryption, current limiting, fusing

• V0.5.0

  Sub-library and sub-table function

The v0.2.0 version released this time can solve the user's distributed transaction requirements and read-write separation requirements. The two functions can be used in combination without invading user business, and the experience is very smooth.

Open source enthusiasts are welcome to join us in building the DBPack community, join groups or participate in community building, please contact via WeChat: scottlewis.

Link

• dbpack: https://github.com/CECTC/dbpack
• dbpack-samples: https://github.com/CECTC/dbpack-samples
• dbpack-doc: https://github.com/CECTC/dbpack-doc
• Event-driven distributed transaction design: https://mp.weixin.qq.com/s/r43JvRY3LCETMoZjrdNxXA
• Video introduction: https://www.bilibili.com/video/BV1cg411X7Ek