pressure backend

Read-write separation-separate read operations and write operations to avoid mutual influence
Horizontal split, because the single table is too large and the query performance is too poor, reduce the query atmosphere and improve the response speed, split according to business dimensions, such as the read and write of transaction data in the last week Balanced, and a week ago reads were far greater than writes. At this time, two tables are required for data transfer [backup first]. Then a few merchants will conduct most of the transactions, so the merchants with frequent transactions are naturally separated separately, and the real-time query scope of low-frequency merchants is naturally smaller and will not be affected by the former.

- First read and write separation, basic operations, separate read and write

- Then do vertical split, horizontal split from a narrow technical perspective - Horizontal split from a

broad business perspective -> Horizontal split will cause sorting problems, and partial sorting will not Equal to global

sorting Paging and sorting problems caused by horizontal splitting: After horizontal splitting, if you want to paginate retrieval, traditional thinking select count(id) from table, this number of queries is time-consuming, but we do not necessarily need complete To display the number of pages, for example, on TB, we can directly display the first 2000 pieces of data, 100 pages, 20 pieces/page. For merchants, if you need to retrieve transaction data, it is generally more frequent to view the first 10 pages. At the same time, if you really need to view all data and obtain transaction data for a certain period of time, we will open a separate thread in the background to provide download. The full query is time-consuming, so displaying it on the web is not an ideal solution.
For sorting problems, you can rely on some strategies to retrieve data from multiple database tables and merge them at the service layer. Table design problems with
horizontal split:

- Primary key auto-increment, in order to control the balance of data distribution, for auto-increment types, you can use 1, 3, 5, 7, and 9 another table is suitable for an auto-increment sequence similar to 2, 4, 6, 8, and 10, or if machine A undertakes 1E pieces of data is a better upper limit, then machine B starts from 1E+1. ID. The former cannot solve the upper limit problem in the future, and the latter cannot solve the problem of uneven data distribution.

- Integer hash remainder, if the primary key is the unique field of the business, the access location of the data can be determined in this way.

- Consistent hashing to solve the problem of expansion and single server failure [Memcached cluster will also use this solution]

Vertical splitting: For example, some data is not large but the number of read and write connections is large.
Splitting will bring many problems, such as join query failure, splitting is to reduce the pressure on the database, so that the DB can read and write at ease, and the calculation is handed over to the service layer. At the same time, the split will cause changes in the data structure and distribution. If you do not want the split to cause major changes to the business code in the future, you need to define the data service interface to reduce later changes.
Caching technology: use Memcached Redis to write to the persistent layer every time period. The key point of
retrieval is the scope. The smaller the search scope, the faster the speed. Therefore, if the table is split horizontally, the split will cause query problems. We need to Merge and sort the query results of multiple tables. Splitting tables and splitting tables changes the database structure. During this period of time, the monitoring requirements have increased, and backup alternatives need to be in hand, so some people naturally want to start over.
Single database database-->Database read-write separation-->Cache technology [Striping value is not high data, real-time data is not high]-->Search technology-->Vertical splitting of data-->Horizontal splitting of

data The read and write performance is generally the calm place of a system. If the performance problem cannot be solved by sub-database and sub-table at ordinary cost, NoSQL needs to be used.