1. SQL statement optimization
1. Optimize count
Each paging operation has to get a count(), and a large number of rows need to be scanned (meaning a large amount of data needs to be accessed) to obtain accurate results, so you can add a summary table, or redis cache to specifically record the records corresponding to the table In this case, the query of summary data can be easily achieved, and the efficiency is very high, but this kind of statistics cannot guarantee 100% accuracy.
Create a table to record the total data volume of the log table:
create table log_counter(
logcount bigint not null
)
After each data insertion, update the table:
<update id="updateLogCounter" >
update log_counter set logcount = logcount + 1
</update>
When performing a paging query, get the total number of records and query from this table:
<select id="countLogFromCounter" resultType="long">
select logcount from log_counter limit 1
</select>
2. Optimize limit
The later the page number is searched, the slower the search efficiency is:
select * from operation_log 1 limit 3000000 , 10;
Optimize the above SQL as:
select * from operation_log t , (select id from operation_log order by id limit
3000000,10) b where t.id = b.id ;
3. Conditional query optimization
For conditional queries, you need to index the query conditions and sort fields.
create index idx_id_username_roleid on t_user(id,username,role_id);
create index idx_username_roleid on t_user(username,role_id);
create index idx_roleid on t_user(role_id);
Use the explain statement to view the optimization effect:
you can see that no matter which combination of queries are all indexed, the query efficiency is significantly improved.
4. Optimize sorting
When querying data, if the business requirements require us to sort the result content, at this time, we also need to establish an appropriate index on the sorted fields to improve the efficiency of sorting.
Two, server optimization
1. MySQL master-slave replication and separation of read and write
Server master-slave replication refers to transferring the DDL and DML operations of the master database to the slave database server through binary logs, and then re-execute these logs on the slave database, thereby keeping the data of the slave database and the master database synchronized. On the basis of Mysql master-slave replication, read-write separation can be used to reduce the pressure on a single Mysql node, thereby improving access efficiency. For the realization of read-write separation, Spring AOP can be used to dynamically switch data sources.
2. Application server optimization
(1) Caching: Redis can be used as caching in the business system to cache some basic data to reduce the pressure on relational databases and improve access efficiency.
(2) Full-text search: If the amount of data in the business system is relatively large (reaching tens of millions), at this time, if you query the database, especially the paging query, the speed will become very slow (because the first Requires count to calculate total). In order to improve access efficiency, at this time, you can consider adding Solr or ElasticSearch full-text search services to improve access efficiency.
(3) Non-relational database: You can also consider storing non-core (important) data in MongoDB, which can improve the efficiency of insertion and query.
Note: The structure of the joint index
create index idx_id_username_roleid on t_user(id,username,role_id);
When sorting the B+ tree, compare id first, then username, and finally role_id.