Business scene
The main function of a system is as follows: it connects to the customer's mail server, automatically receives emails sent to several specific customer service mailboxes, and automatically generates a work order every time a customer service email is received. Then the system will assign the work order to different customer service specialists according to some rules.
This media group customer has generated nearly 20 million work orders over two years, and the operation records of work orders are nearly 100 million. Usually, when the customer service is operating on the work order page, it takes about 10 seconds to open or refresh the work order list. Optimization is required:
the data situation at that time is as follows:
1) The work order table has reached 30 million pieces of data.
2) The processing record of the work order table reaches 150 million pieces of data.
3) The work order table is growing with a daily data volume of 100,000.
Before the customer put forward the demand, the project team had already improved the response speed of the system by optimizing the table structure, business code, index, SQL statement, etc., and the system finally supported the table query of 30 million data. This time I can only try other options.
Implementation idea of hot and cold separation phase 1: MySQL is used for both hot and cold data
After deciding to use cold and hot separation, the project team began to consider using a most cost-effective cold and hot separation solution. Due to limited resources and short construction period, there is a leading principle for the first phase of cold and hot separation, that is, the hot data and cold data use the same storage (MySQL) and data structure, so that the workload is the least, and the cold and hot separation will be done when there is time in the future. Expect.
Questions to consider
1) How to judge whether a data is cold data or hot data?
2) How to trigger hot and cold data separation?
3) How to separate hot and cold data?
4) How to use hot and cold data?