With the increasing amount of data to the Internet, a lot of single table of the amount of data already hundreds of millions, or even more, this single data table has reached a bottleneck query, we need to split the database.
How effective it will be split database, and the Internet company shut down for database processing is not very realistic, because the impact of the volume of business. Then we need a better way to resolve.
First, we should first get ready database, then the table structure is set up the same database. Continue the old database CRUD operations continued through the service be doubled, then the new data additions and deletions to the operation. Recorded on a sign inside the database, then the data before the flag of the old database by writing a tool to synchronize data from the old database tables are synchronized to the new database. Results as shown:
Such newly generated data is written to the new library, while the data from the old database are synchronized in time.
Data synchronization needs further examination after the completion of all the data, there is no problem if you apply two virtual IP or domain name, and then were linked to the two machines.
Such a database the database becomes two, and the two database fields in the hash table, hash result to reach the first database of 0, hash result of a database come. Results are as follows:
During handover, the database needs to be switched in this manner directly ZK configuration. At this point need to observe about a week, while the two databases can continue to read, write double, to prevent emergence of new library data issues is timely to switch to the old database.
If there are no problems after a week, it will be doubled removed, to achieve the separation of the data.
This time the database is not completely separated, has (key) = 0 in the database, there are still data hash (key) = 1, then the need to delete such data out. In this way hundreds of millions of data into two pools, and then each of the two databases each 50 million.
After separate databases, but also the results of the subsequent statistics corresponding changes need to calculate the result in memory, and then the result of the calculation into Taken together, the results obtained and the statistical analysis.
This basic realization of the database without downtime switching service carried out.
We should also note that the figure above, in order to achieve high availability, a main libraries with a main library, between two data library through task synchronization, virtual IP access time if there is a problem of a database, then direct He received another database.
未来的话，需要再需要继续扩容的话，还需要以2*n的库进行扩容，这样has(key) = 0 和 hash(key) = 2数据保持一致， hash(key) = 1 和 hash(key) = 3数据一致。当数据同步完成之后0，2就可以按照上述的方法进行拆分，然后拆为两个虚拟IP，同时将hash(key)=2的数据从数据库 hash(key）= 0的数据库删除掉，1、3相同的原理，这样数据库就实现了同步。
同学们有没有更好的方法？可以和我一起讨论。当然咱们还有一些分库分表比较成熟的工具比如ShardingSphere和MyCAT，这些工具都是比较好的分库分表解决方案，当然在使用之前一定要做好功课，避免使用的时候采坑。
有问题欢迎来拍~