After the number of users of the application system continues to increase, performance bottlenecks are brought about. At this time, the performance can be improved by adding load balancing on the server side of the application, but the number of system accesses is still increasing. One day, it is found that the load of the application layer will be increased. Balance has not achieved the expected effect. The performance bottleneck at this time is likely to be in the database. A single database has reached the upper limit of performance, but it is very troublesome to load balance the database. At this time, you should consider using the distributed middle of the database. The software product MYCAT provides a solution to the problem of performance bottlenecks when the database access volume continues to increase.
1. The origin of MYCAT
Alibaba launched Amoeba in 2008. At that time, it happened to be in the wave of Alibaba going to IOE. Amoeba provided functions such as load balancing and SQL filtering. In the same year, on the basis of Amoeba, Ali opened up the product Cobar, which replaced Amoeba, but it was not maintained later. Cobar itself has many problems. In 2013, Ali launched the open source product MYCAT, and MYCAT is used in the Cobar system. Based on the upgrade and transformation, many problems have been fixed (for example, Cobar suspended animation under high concurrency), and the open source community is very active. In 2017, MYCAT launched version 1.6.
2. What is MYCAT?
MYCAT is an intermediate layer of a distributed database system based on Java, which provides solutions for the distribution of databases under high concurrency.
3. What is the main function of MYCAT?
The database middle tier is a layer between the front-end application and the back-end database.
- Traditional Java projects use database connection pools to connect to the database, so that multiple projects connect to the same database, and redundant connections will increase linearly with the increase of projects, which is not conducive to the control of the number of database connections, but when using the middle layer After that, the front-end applications will obtain the connection through the middle layer uniformly, and will not need to maintain the connection pool by themselves, which can avoid the problem of increasing the number of connections.
- The database middle layer can also shield some changes in the back-end database, so that the front-end application is not affected, for example, the database is divided horizontally or vertically.
- MYCAT natively supports MySQL. It can also connect to other relational databases such as Oracle and Sqlserver through jdbc, and it can also connect to non-relational databases such as mongodb. This function is not available in other products of the same type.
- The read-write separation of the database can be realized. In the back-end master-slave replication database cluster, through the MYCAT configuration, the front-end write operations are routed to the master database, and the read operations are routed to the slave database.
- MYCAT can achieve load balancing of read operations under the separation of read and write, and balance a large number of read operations to different slave libraries, mainly in the case of one master and multiple slaves.
- MYCAT can achieve high availability of the database. When the master node of the database is available, configure a writable slave node. Both nodes are configured in MYCAT. When the master node goes down, MyCAT will automatically route write operations to the standby node. On the node, but does not support continued master-slave synchronization after switchover.
- When the read-write separation can no longer meet the continuously increasing traffic, MYCAT can realize the vertical splitting of the database, dividing all database tables according to modules, and splitting different types of tables to different database servers.
- With the growth of business volume, if there is a database performance problem after vertical splitting, horizontal splitting is required, which is commonly known as sub-database and sub-table. The table data with a large amount of data is divided into different server libraries, the table structure is the same, and the use of MYCAT to achieve horizontal division is completely transparent to the front-end application, and there is no need to adjust the front-end logic.
4. MYCAT usage scenarios
When the read and write operations of the business system are all in one database, the database has performance problems, and the read load of all operations on the database is significantly higher than the write load, the MYCAT read-write separation scheme should be used at this time.
When the data volume of the data table reaches 10 million, all the optimization operations have been used up. At this time, the sub-database and sub-table function of MYCAT should be used.
In the statistical reporting system, MYCAT can also be used to accelerate the statistical analysis of data.
MYCAT can also be used in scenarios where multiple databases need to be queried at the same time. MYCAT supports connecting to different types of databases through JDBC.
5. Advantages of MYCAT
MYCAT is developed based on Alibaba Cobar and has experienced the test of Alibaba's big data business.
MYCAT is developed based on Java, open source and free, data files are relatively easy to find, and cross-platform porting is more convenient.
The MYCAT community is active and the number of users is large, indicating that the function is relatively stable.
Supports multiple relational and non-relational databases.
6. Other concepts of MYCAT
Logic library, the logic library is a database definition that exists in MYCAT, which is equivalent to a collection view after the backend database is sharded. Front-end applications only need to pay attention to the logic library and not its sharding structure. MYCAT will save the logic The definition of the library does not save the data of the logical library. The same concept is the logical table. The database table is divided horizontally and distributed in different servers. The front-end application only needs to operate the logical table.
ER relationship sharding strategy, when performing vertical segmentation of database tables, MYCAT can segment related tables into the same database according to the ER relationship to improve query efficiency.
Global sequence number. After the database is fragmented, the same table may generate the same ID, and the global sequence number may provide a sequence number across shards.