Sub-database sub-table combat and middleware (3)
foreword
Sharding also supports read-write separation. Read-write separation can greatly improve the throughput and availability of the system, but it also brings about data inconsistency, including the data consistency between multiple main databases and the main database
. The problem of data consistency with the slave library. Moreover, read-write separation also brings the same problems as data fragmentation, which also makes the operation and maintenance of the database more complicated for application developers and operation and maintenance personnel.
read-write separation
In the case of a small amount of data, we can separate the read and write of the database to meet the high concurrency requirements, and relieve the query pressure by horizontally expanding the slave database.
Sub-table + read-write separation
If the amount of data is still large on the basis of read-write separation, you can consider sub-database and sub-table storage under the foreseeability of the future.
read-write separation
- Master-slave architecture
Separation of read and write, the purpose is high availability, read and write expansion. The content of the master-slave library is the same, and routing is performed according to SQL semantics. - Sub-database and sub-table architecture
Data fragmentation, the purpose of reading and writing expansion, storage expansion. The library and table contents are different, and are routed according to the shard configuration.
The combination of horizontal sharding and read-write separation can improve system performance more effectively. The following figure shows the complex topological relationship between the application and the database cluster when the sub-database and table are used together with read-write separation.
Sharding read-write separation configuration
Sharding supports read-write separation configuration, so that the user can use the master-slave database cluster as much as possible as a database is the main design goal of the ShardingSphere read-write separation module.
Sharding read-write separation supports master library, slave library, master-slave synchronization, load balancing
Core functions
- Provides a read-write separation configuration with one master and multiple slaves.
It only supports a single master library, and can be used independently or in conjunction with sub-databases and sub-tables - Independently use read-write separation and support SQL transparent transmission. No need for SQL rewriting process
- In the same thread and the same database connection, data consistency can be guaranteed. If there is a write operation, subsequent read operations are all
read from the main library. - Hint-based mandatory main library routing. You can force the route to query real-time data through the master database to avoid delays in master-slave synchronization data.
Item not supported
- Data synchronization between master library and slave library
- Data synchronization delay between master and slave
- Double write or multiple write in the main library
- Data inconsistency across transactions between master and slave. It is recommended that in the master-slave architecture, the read and write in the transaction are all operated by the master library.
shortcoming
Due to the delay caused by the separation of reading and writing, sharding supports mandatory routing, and can be forced to go to the main database to operate data. You can refer to the actual combat and middleware of sub database and sub-table (4)
Read and write separation in practice
Here is just the read-write separation configuration of a single table
#打印shardingsphere sql
spring.shardingsphere.props.sql.show=true
## 读写分离
# 分库信息配置
spring.shardingsphere.datasource.names=test0,test1
#配置数据库信息
#配置连接池
spring.shardingsphere.datasource.test0.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.test0.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.test0.jdbc-url=jdbc:mysql://localhost:3306/test0
spring.shardingsphere.datasource.test0.username=root
spring.shardingsphere.datasource.test0.password=root
#配置连接池
spring.shardingsphere.datasource.test1.type=com.zaxxer.hikari.HikariDataSource
spring.shardingsphere.datasource.test1.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.test1.jdbc-url=jdbc:mysql://localhost:3306/test1
spring.shardingsphere.datasource.test1.username=root
spring.shardingsphere.datasource.test1.password=root
# master-slave
spring.shardingsphere.masterslave.name=datasource
#主库数据源名
spring.shardingsphere.masterslave.master-data-source-name=test0
#从库数据源名 多个可以,隔开
spring.shardingsphere.masterslave.slave-data-source-names=test1
#从库负载均衡配置
spring.shardingsphere.masterslave.load-balance-algorithm-type=ROUND_ROBIN
# 表 雪花算法id
spring.shardingsphere.sharding.tables.position_master_slave.key-generator.column=id
#对应主键类getType返回内容
spring.shardingsphere.sharding.tables.position_master_slave.key-generator.type=SNOWFLAKE
复制代码