What should I do if the interview asks about MySQL module division and architecture system?

What should I do if the interview asks about Mysql module division and architecture system?

Image source https://www.rathishkumar.in/2016/04/understanding-mysql-architecture.html

1. Application layer

This is the top layer in the MySQL architecture, as it is called in some articles 网络接入层, and can be seen in many client-server architectures. This layer includes some common services for most client-server applications. At its core, it does three things: connection handling, authentication, and security.

Connection Manager

When a client connects to a server, the client gets its own thread for its connection. All queries from this client are executed within the specified thread. Threads are cached by the server so they do not need to be created and destroyed for each new connection.

Security and Privilege Module

After the client successfully connects to the MySQL server, the server will check whether the client has permission to issue certain queries to the MySQL server.

2. MySQL server layer

This layer is responsible for all logical functions of the MySQL relational database management system. The brain of the MySQL server resides in this layer. MySQL's logical layer is divided into several sub-components, including: service interface and tool set, SQL interface, SQL parser, optimizer, cache and buffer.

2.1. Service support and toolset

MySQL provides a fairly extensive service support and toolset. This is one of the main reasons for MySQL's popularity. This layer provides services and utilities for the management and maintenance of the MySQL system, supporting auxiliary and high-availability extension support functions such as backup and recovery, security, replication, clustering, partitioning, and workbench.

2.2. SQL Interface

Structured Query Language (SQL) is a query language used to query the MySQL server. It is a tool for interaction between MySQL client users and servers. Some SQL components include data manipulation language (DML), data definition language (DDL), stored procedures, views, and triggers.

2.3. Parser

The parser is an important component of the MySQL service layer. It is responsible for parsing and syntax analysis of SQL statements submitted by users for subsequent optimization and execution.

The main functions of the parser

1. Lexical analysis: The parser first performs lexical analysis on the SQL statement, and cuts the input string according to delimiters such as spaces, brackets, commas, etc. to form tokens.

2. Syntax analysis: The parser performs syntax analysis on the tokens obtained by lexical analysis according to the syntax rules defined by MySQL, and builds a syntax tree (Parse Tree) or syntax analysis tree (Abstract Syntax Tree, AST). During the syntax analysis process, the parser will check whether the structure of the SQL statement conforms to the syntax rules. If an error is found, the corresponding error message will be thrown.

3. Semantic analysis: The parser will perform semantic analysis on the syntax tree obtained by syntax analysis to determine the meaning and execution plan of the SQL statement. For example, the parser will check whether the table name and column name exist, check whether the data type matches, and parse the options of the query statement (such as DISTINCT, GROUP BY, HAVING, etc.).

4. Optimization: The parser hands the parsed syntax tree to the optimizer (Optimizer) for further optimization. The optimizer will consider different execution plans, select the optimal execution plan, and generate the corresponding execution plan tree (Execution Plan Tree).

The job of the parser is to convert the SQL statement entered by the user into an executable internal representation, and provide the necessary information and instructions for subsequent executor (Executor) execution. Through the parser, MySQL is able to understand and process SQL statements submitted by users, and convert them into actual operations on the underlying storage engine.

Let’s take an example of a parser to understand

Suppose there is the following SQL statement: SELECT * FROM customers WHERE age > 30;

1. Lexical analysis: Cut the input SQL statement according to spaces and keywords, and obtain the following token sequence: SELECT, *, FROM, customers, WHERE, age, >, 30, ;

2. Syntax analysis: According to the syntax rules of MySQL, the token sequence is constructed into a syntax tree. Syntax tree structure
   

3. Semantic analysis: The parser performs semantic analysis to check whether the table names and column names in the syntax tree exist, whether the data types match, etc. For example, the parser checks whether a table named "customers" exists and whether the "age" column exists.

4. Optimization: The parser hands the syntax tree to the optimizer for optimization and selects the optimal execution plan. In this example, the optimizer might consider whether to use an index to speed up the query and generate a corresponding execution plan tree.

After the parser completes the above process, it passes the generated internal representation to the executor, which performs specific operations according to the execution plan and returns the query results to the user.

2.4. Optimizer

The main task of the optimizer is to select the optimal execution plan to execute the query operation based on the semantics of the query statement and the statistical information of the database. The goal of the optimizer is to improve query execution efficiency and performance by selecting the optimal execution plan. The optimizer's work is performed before the query is executed, and its output is an optimized execution plan, which is used to guide the specific execution operations of the database engine.

Optimizer workflow

1. Query rewrite: The optimizer will rewrite the query statement and convert it into a logical query tree or other internal representation. During this process, the optimizer performs syntax analysis, semantic analysis, and application of query optimization rules to generate an optimized query plan.

2. cost estimate: The optimizer will estimate the cost of each possible query plan and evaluate the time, CPU and memory overhead required to execute each plan. This process mainly estimates based on the statistical information of the database and the characteristics of the query operation in order to select the lowest-cost execution plan.

3. Execution plan generation: The optimizer will select an optimal execution plan based on the cost estimation results. An execution plan is a specific sequence of operations that describes the order and manner in which query operations are executed in the database engine. The optimizer will consider the order and method of execution of operations such as indexing, join operations, sorting, and aggregation to minimize the execution cost of the query.

4. Execution plan optimization: After generating the execution plan, the optimizer will further optimize the execution plan. This includes optimizing the order of operations, selecting the most appropriate algorithms and data structures, selecting the appropriate degree of parallelism, etc.

Let’s take an example of an optimizer to understand

Suppose there is an e-commerce platform that needs to query users' shopping carts and product information. There are the following two tables:

Table A: User shopping cart table (id, user_id, product_id, quantity)
Table B: Product information table (product_id, product_name, price)

Now there is a query statement:

SELECT A.user_id, B.product_name, B.price, A.quantity
FROM A
JOIN B ON A.product_id = B.product_id
WHERE A.user_id = 12345
ORDER BY B.price DESC

In this example, the MySQL optimizer can perform the following optimizations:

1. Query rewriting: The optimizer can rewrite the query statement into an internal representation, such as converting the keywords "JOIN" and "ON" into a form suitable for optimization.

2. Optimization rule application: The optimizer can apply a series of query optimization rules based on the structure and semantics of the query statement. For example, the optimizer can convert "A.user_id = 12345" in the WHERE clause into a filter condition to filter out rows that do not meet the condition in advance.

3. Statistics collection: The optimizer can estimate the cost of performing each operation based on the statistics of table A and table B, such as index selectivity, number of rows, etc. For example, the optimizer can estimate the cost of a join operation based on index selectivity and row count to decide which table to use as the driver and driven table.

4. Execution plan generation: Based on the results of cost estimation, the optimizer can generate an execution plan. In this example, the optimizer may choose to use an index to perform table join operations to reduce disk IO overhead, and select an appropriate index for filtering operations based on the filter conditions of the WHERE clause.

5. Execution plan optimization: After generating the execution plan, the optimizer can further optimize the execution plan. For example, the optimizer can optimize the order of operations, choose the most appropriate algorithms and data structures, and choose the appropriate degree of parallelism.

Through the above optimization steps, the MySQL optimizer can select the optimal execution plan to improve query performance and efficiency. In this example, the optimizer may choose to use user ID as the index for filtering operations, and perform table join operations through product IDs to obtain the shopping cart product information of the specified user and sort them in descending order by price. This reduces disk IO overhead and quickly obtains the desired result set.

2.5. Caching

MySQL cache refers to the memory cache of the MySQL server, also called query cache (Query Cache). MySQL cache means that the MySQL server stores the executed query results in the memory, and returns the results directly from the memory the next time the same query is requested, without the need to execute the query statement again. This can greatly improve query performance and efficiency.

MySQL cache can be divided into two types:

1. Query Cache

When a query is executed, the MySQL server stores the query results in memory. If there is the same query request next time, MySQL will return the results directly from the cache without executing the query statement. Query cache is very effective for repeated queries of the same query request, but for tables that are frequently updated and tables with large data changes, it will lead to a low cache hit rate and affect performance.

MySQL version 8.0 has removed the query cache function. The main reason is that the query cache will cause performance bottlenecks for highly concurrent query and update operations, and the query cache will cause cache hits for frequently updated tables and tables with large data changes. The rate is low, affecting performance.

3. Storage engine layer

The pluggable storage engine feature makes MySQL the first choice for developers. This is a feature that gives MySQL an edge among the big players. MySQL allows us to choose various storage engines according to different situations and requirements. We will discuss the features of each storage engine in upcoming articles, here is just a list of supported storage engines.
MyISAM, InnoDB, Federated, Mrg_MyISAM, Blackhole, CSV, Memory, Archive, Performance_schema.

3.1. Pluggable features

This means that users can choose the appropriate storage engine according to their needs and even switch storage engines without stopping the MySQL server.

The reason why it is pluggable is that MySQL provides an interface called the storage engine API, which allows third-party developers to develop their own storage engines and integrate them into MySQL. Through this interface, users can customize the storage engine to meet various data storage needs.

Users can switch storage engines in MySQL by following these steps:

1. View the currently used storage engine: You can use the following command to view MySQL's current default storage engine and installed storage engines:

   SHOW ENGINES;
   

2. Select and install a new storage engine: Select the appropriate storage engine according to your needs, and then install it according to the installation instructions provided by the engine.

3. Modify the MySQL configuration file: Set the default_storage_engine parameter in the MySQL configuration file (usually my.cnf) to the new storage engine.

4. Restart the MySQL server: Restart the MySQL server to make the new storage engine take effect.

5. Create and use a new storage engine table: Use the CREATE TABLE statement to create a new storage engine table, and specify the new storage engine to use when querying.

3.2. Common MySQL storage engines include:

1. InnoDB engine: InnoDB is MySQL's default storage engine. It supports features such as transactions, row-level locks, and foreign key constraints. The InnoDB engine is suitable for most application scenarios, especially those that require transaction support and high concurrent reading and writing.

2. MyISAM engine: MyISAM is another commonly used storage engine for MySQL. It does not support transactions and row-level locks, but has fast read performance and a small storage space footprint. The MyISAM engine is suitable for read-intensive application scenarios, such as data warehousing and log analysis.

3. Memory engine: The Memory engine stores table data in memory, with fast read and write performance and low latency. However, since the data is stored in memory, data loss will occur when the MySQL server is restarted or the server crashes. The Memory engine is suitable for temporary tables and cache tables that require fast reading and writing.

4. NDB Cluster engine: The NDB Cluster engine is MySQL's distributed storage engine, which can distribute data across multiple nodes and provide high availability and fault tolerance. The NDB Cluster engine is suitable for scenarios that require high availability and large-scale data storage, such as telecommunications, finance, and Internet applications.

In addition to the above common storage engines, there are other storage engines such as Archive, CSV, Blackhole, etc., which have their own characteristics and applicable scenarios.

Reference documentation

https://www.geeksforgeeks.org/architecture-of-mysql/