Query optimization mainly involves two parts.
1. Data query optimization of order table
The order database is separate, and each record display involves user name (WeChat ID, nickname, app nickname), business name (business name, phone), device name (device number, license plate number), etc.
According to the current table design, the order database is independent of the user database, the merchant database, and the device database. The tables are all related through the primary key ID. The order table stores userId, merchantId, deviceId, etc. This design approach brings a bottleneck, that is, when the order record is displayed, the user, merchant, and device information must be displayed through the ID correlation query and displayed on the front page. At the beginning, in the development process, due to the relatively short time, no too many detailed optimizations were done. The back-end order micro-service was used to cyclically query users, merchants and equipment through ID and then displayed on the front-end. It was good at the beginning, after all. Only 10 pieces of data are displayed on each page, about 3 seconds in the process of opening the page. The following page was adjusted to display 20 items, and as a result, the order microservice on this page occasionally reported timeouts. In order to solve the timeout problem, the data is assembled on the front end, and the front end requests specific users, merchants, and device microservices to display relevant information according to the ID. Although the microservice timeout problem is solved, a large number of pages appear every time the front end visits the page. Ajax requests have a great experience impact on front-end performance. There has also been a communication with the business, that is, when the order page is displayed, can it not display the information of the user, the merchant, and the device, but only display the order record related, and then enter the details page by clicking the order details to display the user, merchant, and device information. But the product experience of the business is not very good, so we still have to find a way to solve this problem. In fact, when this order table was originally designed, it was too much to pursue the design paradigm of the database, which led to too many queries. In fact, the design of the order table should consider the first-time presentation method. After optimization, the simplest solution is actually Several related fields are redundant in the order form, such as user name, merchant name, and device name. There may also be a data update issue. In general business scenarios, the primary key ID cannot be modified. This is affirmative, but some users can modify the user name. At this time, you may need to update the redundant field information in the order table. Pay attention to this Just click.
2. Big data single table multi-condition query optimization
Orders are physically divided by month, such as t_orderhistory201801, t_orderhistory201802, t_orderhistory201803, etc. The query time on the query conditions is also controlled. Only the first day of the current month to the last day of the current month can be selected. As the platform's business volume increases, Now there is another problem. When the selected query conditions are too many, and the time interval selected is larger, such as selecting the 1st to the 30th, the result query speed is very slow and the result is waited for a long time, and even the query abnormal timeout phenomenon may occur. The cause of the problem is also clear to us. It is that the amount of order data in a month is too large, the query conditions are too large, the interval is too large, and then the table is scanned globally, which leads to slowness and timeouts. There are two ways to continue optimization. . The first is to continue to divide the month table into tables, for example, by week or even by day to compress the query warehouse as much as possible. The second is to limit the query time interval, for example, only three days of data or less can be checked each time. In addition, if there are many query conditions, a joint index (combined index) should be required. At present, we use the fastest time to solve the time interval of temporarily limiting the query.
In addition, the database architecture table design is actually inseparable from the product design. The two sides must be combined to design. Sometimes the product detail design tends to lean toward the database table design.