Database query - index optimization query method

Database query - index optimization query method

This article explains a method to optimize the query when querying the database. This method is index optimization, explaining the principle and implementation method.

Introduction

First of all, this is a simple database. This database is normal. If you use mybatis-plus for query, it is the same as the code in this article: CRUD display of Java book catalog management system (springboot+vue+mybatis-plus)

/*
 Navicat MySQL Data Transfer

 Source Server         : localhost_3306
 Source Server Type    : MySQL
 Source Server Version : 80028
 Source Host           : localhost:3306
 Source Schema         : projectdatabase

 Target Server Type    : MySQL
 Target Server Version : 80028
 File Encoding         : 65001

 Date: 31/01/2023 22:24:02
*/

SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;

-- ----------------------------
-- Table structure for book
-- ----------------------------
DROP TABLE IF EXISTS `book`;
CREATE TABLE `book`  (
  `isbn` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci NOT NULL,
  `title` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci NOT NULL,
  `author` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci NOT NULL,
  `publisher` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci NOT NULL,
  `pubdate` datetime NOT NULL,
  `price` decimal(10, 2) NOT NULL,
  `id` int NOT NULL AUTO_INCREMENT,
  PRIMARY KEY (`id`, `isbn`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 5 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_0900_ai_ci ROW_FORMAT = DYNAMIC;

-- ----------------------------
-- Records of book
-- ----------------------------
INSERT INTO `book` VALUES ('9780439227148', 'The Call of the Wild', 'Jack London', 'Scholastic Press', '2001-01-01 00:00:00', 39.40, 1);
INSERT INTO `book` VALUES ('9787501592401', 'The Old Man and the Sea', 'Ernest Hemingway', 'Knowledge Press', '2023-01-30 00:00:00', 25.80, 2);
INSERT INTO `book` VALUES ('9787501592401', 'The Old Man and the Sea', 'Ernest Hemingway', 'Knowledge Press', '2023-01-31 13:02:42', 25.80, 3);
INSERT INTO `book` VALUES ('9780439227148', 'The Call of the Wild', 'Jack London', 'Scholastic Press', '2023-01-30 16:00:00', 34.90, 6);
INSERT INTO `book` VALUES ('9781772262902', 'Oliver Twist', 'Charles Dickens', 'Engage Books', '2023-01-03 16:00:00', 45.00, 7);

SET FOREIGN_KEY_CHECKS = 1;

Principles of Index Optimization

Index Optimization : Using appropriate indexes can speed up queries. For example, in the book table, the isbn, title, author, and publisher columns are commonly used as search fields, and corresponding indexes should be established for these fields.

Indexes can greatly improve the efficiency of database queries. For example, establishing corresponding indexes on the isbn, title, author, and publisher columns in the book table can greatly optimize the CRUD operations of the library management system.

The following is the SQL statement for indexing the isbn, title, author, and publisher columns in the book table:

ALTER TABLE `book` ADD INDEX `idx_isbn`(`isbn`);
ALTER TABLE `book` ADD INDEX `idx_title`(`title`);
ALTER TABLE `book` ADD INDEX `idx_author`(`author`);
ALTER TABLE `book` ADD INDEX `idx_publisher`(`publisher`);

Execute the above SQL statement to create an index for the specified column in the book table. When querying, MySQL will first use the index to retrieve qualified records and return the results, thereby avoiding full table scanning and improving query speed.

One thing to note is that building an index will also increase data maintenance costs and storage space. Therefore, when selecting index fields and types, it should be determined according to the situation, and excessive use should be avoided, so as not to affect system performance and space occupation. In addition, for frequently updated tables, excessive use of indexes will also reduce the update speed, and will also generate additional overhead such as locks, which need to be selected and configured according to actual needs.

After adding the index, it looks like this for the book class

@Data
@AllArgsConstructor
@NoArgsConstructor
@TableName("book")
public class Book {
    @TableId(type = IdType.AUTO)
    private Long id;
    @TableIndex(name = "idx_isbn", type = IndexType.BTREE)
    private String isbn;
    @TableIndex(name = "idx_title", type = IndexType.BTREE)
    private String title;
    @TableIndex(name = "idx_author", type = IndexType.BTREE)
    private String author;
    @TableIndex(name = "idx_publisher", type = IndexType.BTREE)
    private String publisher;
    private Date pubdate;
    private Double price;
}

Finally, a summary:
Simply put, an index is a data structure used in a database to improve query efficiency. It can be compared to the directory of the dictionary, which can help us locate the records that need to be queried faster. In databases, indexes are generally implemented with B-trees (balanced trees) or hash tables. Among them, B-tree is a multi-stage balanced tree, which is usually used to process data retrieval based on disk storage; hash table is mainly aimed at in-memory databases, which can make read operations reach O(1) time complexity.

In Mybatis-Plus, the @TableIndex annotation can be used to create corresponding indexes for the columns in the entity class. For example, the four fields isbn, title, author, and publisher in the above-mentioned Book entity class are added with the names idx_isbn, idx_title, and idx_author respectively. and idx_publisher's BTREE type index.

When performing a query operation, if the query statement contains these indexed fields, MySQL will choose to use the relevant index to locate the records that meet the conditions. Since indexes can greatly speed up query operations, in actual development, we often need to add corresponding indexes to fields that need to be queried frequently, so as to improve the response speed of the system.

It should be noted that in the process of indexing, it is necessary to choose which fields to index according to actual business needs, and at the same time, it is necessary to weigh the maintenance and storage costs that the index will bring. Excessive or unnecessary indexes can actually slow down system queries and writes, so they need to be carefully optimized. At the same time, when the amount of database data is large, it can also be optimized by means of partitioning to improve performance and availability.

In short, indexing is an important means of database optimization, which can help us find and locate the required data faster, but it also needs to be optimized and managed according to the actual situation.

actual case

Suppose we have a business requirement: we need to query book information based on the author name. By default, we can query through the following SQL statements:

SELECT * FROM book WHERE author = '张三';

But if the amount of data in the book table is very large, such a query operation may take a long time. In order to optimize query efficiency, we can create an index for the author field so that it can query matching records faster.

In Mybatis-Plus, we can add the @TableIndex annotation to the author attribute in the Book entity class, the code is as follows:

@Data
@AllArgsConstructor
@NoArgsConstructor
@TableName("book")
public class Book {
    
    
    @TableId(type = IdType.AUTO)
    private Long id;
    private String isbn;
    private String title;
    @TableIndex(name = "idx_author", type = IndexType.BTREE)
    private String author;
    private String publisher;
    private Date pubdate;
    private Double price;
}

Then, execute the following SQL statement in the database to create a corresponding BTREE type index for the author field:

CREATE INDEX idx_author ON book(author);

In this way, when we query book information through the following SQL statement, the MySQL database can use the idx_author index to improve query efficiency:

SELECT * FROM book WHERE author = '张三';

Since indexes can help us locate eligible records faster, when the amount of data in the book table is large, using indexes will greatly improve query efficiency and reduce the pressure on the database.

Of course, in actual development, we also need to optimize according to specific business needs. For example, if you need to perform a joint query on multiple fields, you can create a composite index for these fields, and so on.

• Principle analysis

Adding an index to improve query efficiency mainly comes from the data structure and location of the index, which can effectively reduce the amount of data that needs to be scanned in the database.

In MySQL, each index is a B-tree or Hash table. After we create an index for the author field, MySQL will store the value of the Autor field according to the data structure of the index. When reading, you only need to find the records that meet the conditions, instead of traversing the entire table.

Assuming that the book table has 10 million records, if the author field is not indexed, MySQL needs to traverse the entire table to find records that meet the conditions, and this operation is very time-consuming. And when we create an index for the author field, MySQL only needs to search on this B-tree. Since it uses the data structure of a balanced tree, it needs to compare log(N) times at most each time, of which N is the number of leaf nodes of this B-tree. Therefore, the speed of using index queries is very fast, and can even reach a time complexity of approximately O(1).

It should be noted that adding indexes to table fields in actual applications is not always better. Too many indexes may also slow down the writing operation of the database and consume a lot of disk space. Therefore, we need to decide whether to add indexes and which fields need to be indexed after analyzing and optimizing specific business requirements.