Invited to enjoy science classroom Author: Gu old
reprint please state the source! ! !
Preface
In the previous article, Gu introduced the basic concepts of sharding-jdbc . Today, Gu will introduce how to use it.
I often encounter the question of some small partners, that is, at what magnitude will we divide the database and the table ?
Sub-database sub-table experience value
MySQL single table experience
-
300W Mysql can easily resist
-
600W data start card, optimization can be solved (table structure, index design)
-
800W ~ 1000W awesome DBA optimization will encounter bottlenecks
Generally, the data of about 1000W in a single MySQL table can be divided into tables without consideration. Of course, in addition to considering the current data volume and performance, we need to consider the business growth of the system in half a year to about a year in advance . But to avoid over-design (considering many needs in the next few years, for example, a table in the next few years is expected to reach tens of millions of data, this is a transitional consideration)
According to the growth rate of data volume, choose the implementation steps
Step 1: Regardless of database or table
Step 2: Sub-tables in the same library
Step 3: Sub-database and sub-table
Don’t over-design, as soon as you come up to play the big ones, you will sub-databases and tables
If multiple instances of the sub-database exist on the same server, it only solves the problem of the maximum number of connections to the database , but io (database data is stored on the hard disk, and each time you get it, you need to go to the hard disk to fish out the data), CPU and other servers The resource bottleneck has not been resolved . The performance of the database depends on the performance of the server.
Set up the environment
We use SpringBoot + MybatisPlus + Shrading-JDBC + Druid connection pool.
POM.xml dependency
Old Gu used a relatively new version, SpringBoot 2.2.9, Sharding-Jdbc4.1.1.
Horizontal sub-table, create database table
-
Create database course_db
-
Create two tables course1, course2
Table structure of course1 and course2
Sub-table rules
If the added course cid is an even number, insert the data into course1, if it is an odd number, insert the data into course2.
Persistent mybatis
We are referencing mybatis-plus
Define entity classes
Operational data mapper
Start class, MapperScan
At this point, the environment is set up.
Configure sub-table strategy
Configure data source
#配置数据源
Data source address
#配置数据源具体内容，
Note that the name of the data source is consistent with the data source configured in the data source address.
Table distribution
Specify the course table distribution, which database the configuration table is in, and what is the table name #m1.course1;m1.course2 spring.shardingsphere.sharding.tables. course .actual-data-nodes=m1.course_$->{1... 2}
The above expression m1.course_$->{1...2} represents the distribution of the course table , using the row expression algorithm to specify the real table name; the course table is a logical table.
The primary key definition of the table
Specify the key cid generation strategy in the course table spring.shardingsphere.sharding.tables.course.key-generator.column=cid spring.shardingsphere.sharding.tables.course.key-generator.type=SNOWFLAKE
The primary key is designated as cid above, and it is generated by the snowflake algorithm. If you don't know the snowflake algorithm, you can check the previous article by Gu.
Strategy
Specify the sharding strategy cid as an even number and add it to the course1 table, and an odd number to add it to the course2 table spring.shardingsphere.sharding.tables.course.table-strategy.inline.sharding-column=cid spring.shardingsphere.sharding.tables.course. table-strategy.inline.algorithm-expression=course_$->{cid% 2 + 1}
The table-strategy strategy of the course logic table is defined above. According to the cid field, the expression course_$->{cid% 2 + 1}, you can see that cid% 2 + 1, that is, if you take the modulus of 2 + 1, it will fall. To 1, 2.
sql log
Open the sql output log spring.shardingsphere.props.sql.show=true
Enter the execution sql log
Don't forget to allow duplicate beans to be overridden spring.main.allow-bean-definition-overriding=true
Test class
Repeat the above 10 times to insert the course
Execution error
Let's see what the error message is
BeanCreationException: Error creating bean with name ‘dataSource’ defined in class path resource [com/alibaba/druid/spring/boot/autoconfigure/DruidDataSourceAutoConfigure.class]: Invocation of init method failed; nested exception is org.springframework.boot.autoconfigure.jdbc.DataSourceProperties$DataSourceBeanCreationException: Failed to determine a suitable driver class at org.springframework.beans.factory.
As shown above, DruidDataSourceAutoConfigure Failed to determine a suitable driver class, that is, Druid cannot find the mysql driver, but there is no problem with the mysql driver package, so click directly into DruidDataSourceAutoConfigure to view the source code.
DruidDataSourceWrapper class source code
As above, the red mark indicates that druid finds the jdbc attribute according to spring.datasource.druid . If not found, then finds the jdbc attribute according to spring.datasource . Generally speaking, there will be no error. But I used shardingjdbc here, and did not configure spring.datasource. According to spring.datasource.druid or spring.datasource , I can't find it because my structure is spring.shardingsphere.datasource .
How to solve it?
Exclude automatic configuration of Druid data source
Solution 1:
If the jar package we use is druid-spring-boot-starter, then automatic configuration of druid will be excluded from the startup class.
@SpringBootApplication(exclude = {DruidDataSourceAutoConfigure.class})
Solution 2:
No need to
<dependency>
To
<dependency>
Execute again
Lao Gu uses method 1 here
Let's take a look at the implementation, the implementation was successful
We found that there are 2 tables with 10 data, inserted according to odd and even numbers. We can also see the output sql log of the console ; there are logical sql and real sql in the log . According to the odd and even number, the real sql will be different.
to sum up
Today, Lao Gu introduced the level table of shrading-jdbc. The core is the strategy configuration of the table and the points of attention to the configuration. It is relatively simple. A follow-up article will introduce the library and read-write separation. Thank you for your continued attention! ! !
