Past Series Review
Many friends have used GreatSQL, but they don't know much about the underlying principles of GreatSQL. Today, I will take you to uncover the mystery of the GreatSQL architecture!
First, let’s review a classic architecture diagram:
Figure 1_GreatSQL5.7 version architecture diagram
It can be found that GreatSQL5.7 consists of the following parts
- Connection pool component
- System Management and Control Tools
- SQL interface component
- query parser
- query optimizer
- caching component
- Pluggable storage engine
- System and log files
One feature that distinguishes GreatSQL databases from other databases is its pluggable table storage engine. It should be noted that the storage engine is based on tables, not databases .
However, classic also means that the picture is quite old. In GreatSQL8.0 and later versions, the function of query cache has been removed.
Figure 2_GreatSQL8.0 version architecture diagram
Overall, GreatSQL8.0 can be divided into connection layer, service layer, and storage engine layer .
1. Connection layer (Client Connectors)
The connection layer, also known as 客户端连接器（Client Connectors）the role is to provide support for establishing with the GreatSQL server.
The client establishes a connection with the GreatSQL server through the TCP/IP protocol, and each connection corresponds to a thread. Connection management also includes connection pool technology to reuse established connections and reduce the overhead of repeatedly establishing connections.
And it supports almost all mainstream server-side programming technologies, and mainly completes some similar connection processing, authorization authentication, and related security solutions.
Identity authentication and authority acquisition will be performed on the account password transmitted from TCP
- If the user name or password is incorrect, an error will be received
Access denied for user, and the client program will end execution
For example:
$ mysql -uroot -p
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)
- After the user name and password authentication is passed, the permissions owned by the account will be found out from the permission table and the connection is associated, and the subsequent permission judgment logic will depend on the permissions read at this time
Second, the service layer (GreatSQL Server)
The service layer is the core of GreatSQL Server, mainly including connectors, analyzers, optimizers, executors, etc., covering most of GreatSQL's core service functions, as well as all built-in functions (such as date, time, math and encryption functions, etc.), All cross-storage engine functions are implemented at this layer, such as stored procedures, triggers, views, etc.
Ⅰ.SQL Interface: SQL interface
Receive the user's SQL command and return the result that the user needs to query. For example , SELECT … FROMcalling SQL Interface, GreatSQL supports DML, DDL, stored procedures, views, triggers, custom functions and other SQL language interfaces
It also supports NoSQL , which generally refers to non-relational databases and data storage. With the rapid development of Internet platforms, traditional relational databases are increasingly unable to meet the needs. Since version 5.6, GreatSQL has started to support simple NoSQL storage functions. GreatSQL version 8.0 has optimized this function to implement NoSQL functions in a more flexible manner without relying on schemas.
Ⅱ.Parser: Parser
Perform syntax analysis and semantic analysis on SQL statements in the parser. Decompose the SQL statement into a data structure, and pass this structure to the subsequent steps. The transmission and processing of the SQL statement will be based on this structure, and judge whether the SQL statement you input meets the GreatSQL syntax.
Ⅲ.Optimizer: query optimizer
Before starting to execute, it must be processed by the optimizer.
After the SQL statement is parsed and before querying, the query optimizer will be used to determine the execution path of the SQL statement and generate an execution plan, which can be EXPLAINviewed by using commands.
This execution plan indicates which indexes should be used for query (full table retrieval or index retrieval), what is the connection order between tables, and finally, the method provided by the storage engine will be called according to the steps in the execution plan to actually execute the query, and will Query results are returned to the user.
For example the following JOIN statement:
SELECT * FROM tb1 JOIN tb2 USING(ID) WHERE tb1.a=1 and tb2.a=2;
Then there are two ways to choose:
- The first one is
先取表 tb1the ID value of the record where a=1, and then associate the table tb2 according to the ID, and then judge whether the value of a in tb2 is equal to 2 - The second type,
先取表 tb2the ID value of the a=2 record in it, is associated with tb1 according to the ID value, and then judges whether the value of a in tb1 is equal to 10
The execution results must be consistent, but the efficiency is quite different, so we have to choose to use small data sets to drive large data sets, that is, small tables drive large tables .
Ⅳ.Caches & Buffers: query cache components
GreatSQL maintains some internally Cacheand Buffer, for example, Query Cacheto cache the execution result of a SELECT statement. If the corresponding query result can be found in it, then there is no need to go through the whole process of query parsing, optimization and execution, and the result is directly fed back to the customer. end.
However, the query cache function is deleted in GreatSQL 8.0 and above, because the query cache must have two SQL statements exactly the same, otherwise the query cache cannot be triggered, which is very tasteless~
3. Engine layer (Storage Engines)
Ⅰ. Storage engine layer
It is really responsible for the storage and extraction of data in GreatSQL, and performs operations on the underlying data maintained at the physical server level, and the server communicates with the storage engine through the API.
The advantage of storage engines is that all kinds of storage engines have unique characteristics, so that different storage engine tables can be created for specific application requirements.
The storage engines supported by GreatSQL are as follows:
greatsql> SHOW ENGINES;
+--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+
| Engine 引擎名称 | Support 支持情况 | Comment 引擎的说明 | Transactions 事务支持 | XA 分布式事务支持 | Savepoints 保存点 |
+--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+
| FEDERATED | NO | Federated MySQL storage engine | NULL | NULL | NULL |
| PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO |
| InnoDB | DEFAULT | Percona-XtraDB, Supports transactions, row-level locking, and foreign keys | YES | YES | YES |
| MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |
| MyISAM | YES | MyISAM storage engine | NO | NO | NO |
| MRG_MYISAM | YES | Collection of identical MyISAM tables | NO | NO | NO |
| BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO |
| CSV | YES | CSV storage engine | NO | NO | NO |
| ARCHIVE | YES | Archive storage engine | NO | NO | NO |
+--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+
9 rows in set (0.00 sec)
Thanks to the open source feature of the GreatSQL database, users can write their own personalized storage engines based on the storage engine interface. When you have doubts about the performance or function of a certain storage engine, you can optimize the code to achieve the required features, which just shows the convenience and power that open source gives us.
Ⅱ. Storage layer
All the data, the definition of the database, the table, the content of each row of the table, and the index are stored on the file system in the form of files, and complete the interaction with the storage engine. Of course, some storage engines, such as InnoDB, also support direct management of raw devices without using a file system, but the implementation of modern file systems makes this unnecessary. Under the file system, local disks can be used, and various storage systems such as DAS, NAS, and SAN can be used.
Summarize
Therefore, the architecture diagram of GreatSQL can be simplified as follows:
Keep the architecture diagram firmly in mind, it will be of great help to understand the GreatSQL database in the future!
Enjoy GreatSQL :)
About GreatSQL
GreatSQL is a MySQL branch maintained by Wanli Database. It focuses on improving the reliability and performance of MGR, supports InnoDB parallel query features, and is a MySQL branch version suitable for financial-level applications.
Related Links: GreatSQL Community Gitee GitHub Bilibili
GreatSQL community:
Community rewards and suggestions: https://greatsql.cn/thread-54-1-1.html
Community blog prize call for paper details: https://greatsql.cn/thread-100-1-1.html
Community 2022 Medal Winner List: https://greatsql.cn/thread-184-1-1.html
(If you have any questions about the article or have unique insights, you can go to the community official website to raise or share~)
Technical exchange group:
WeChat & QQ group:
QQ group: 533341697
WeChat group: add GreatSQL community assistant (WeChat ID: wanlidbc) friends, and wait for the community assistant to pull you into the group.