1. What is TIDB
TiDB is a distributed NewSQL database. It supports horizontal elastic expansion, ACID transactions, standard SQL, MySQL syntax, and MySQL protocol, and has high-availability features with strong data consistency. It is a hybrid database that is not only suitable for OLTP scenarios but also for OLAP scenarios.
1.1 What is NewSQL
- SQL, a traditional relational database such as MySQL
- noSQL, such as MongoDB, Redis
- newSQL
1.1.1 Problems with traditional SQL
The Internet began to develop rapidly at the beginning of this century. The user scale and data volume of Internet applications are increasing, and 7X24 hours online is required. Traditional relational databases have become a bottleneck in this environment, and there are usually two solutions:
①Upgrade server hardware: Although the performance has been improved, there is always a ceiling.
②Data sharding: When data involves cross-database joins and cross-database transactions, it is very troublesome. Many people simply
process it at the business layer by themselves, which is more complicated.
1.1.2 Problems with NoSQL
Later, noSQL appeared, abandoning the strong transaction guarantee and relational model of traditional SQL, and focusing on the high availability and scalability of the database.
advantage:
- High availability and scalability, automatic partitioning, easy expansion
- Strong consistency is not guaranteed, and performance is greatly improved
- Extremely flexible without the constraints of the relational model
shortcoming
- Strong consistency is not guaranteed, which is fine for ordinary applications, but there are still many enterprise-level applications like finance that require strong consistency.
- It does not support SQL statements, and compatibility is a big problem. Different NoSQL databases have their own APIs to operate data, which is more complicated.
1.1.3 NewSQL features
NewSQL provides the same scalability as noSQL, and is still based on the relational model. It also retains the extremely mature SQL as a query language and guarantees ACID transaction characteristics.
Simply put, NewSQL integrates the powerful scalability of NoSQL on traditional relational databases.
Key Features of NewSQL
- SQL support for complex queries and big data analysis.
- Support ACID transaction, support isolation level.
- Elastic scaling, capacity expansion and contraction are completely transparent to the business layer.
- High availability and automatic disaster recovery.
1.1.4 Comparison of three kinds of SQL
1.2 Core features of TIDB
1.2.1 Horizontal elastic expansion
The horizontal expansion of TiDB can be achieved by simply adding new nodes, and the throughput or storage can be expanded on demand to easily cope with high concurrency and massive data scenarios.
Thanks to the design of TiDB's storage-computing-separated architecture, online expansion or reduction of computing and storage can be performed on demand, and the process of expansion or reduction is transparent to application operation and maintenance personnel.
1.2.2 Distributed transaction support
TiDB 100% supports standard ACID transactions
1.2.3 Financial-level high availability
Compared with the traditional master-slave (MS) replication scheme, the Raft-based majority election protocol can provide financial-level 100% data strong consistency guarantee, and can realize automatic failure recovery without losing most copies ( auto-failover), without manual intervention
The data is stored in multiple copies, and the data copies are synchronized with the transaction log through the Multi-Raft protocol. Most of the transactions can only be submitted after successful writing, ensuring strong data consistency and the availability of data will not be affected when a small number of copies fail. Policies such as geographical location and number of replicas can be configured on demand to meet the requirements of different disaster recovery levels.
1.2.4 Real-time HTAP
As a typical OLTP row-storage database, TiDB also has powerful OLAP performance. With TiSpark, it can provide a one-stop HTAP solution. One storage can handle OLTP & OLAP at the same time without the traditional and cumbersome ETL process
Provides two storage engines: row storage engine TiKV and column storage engine TiFlash. TiFlash copies data from TiKV in real time through the Multi-Raft Learner protocol to ensure strong data consistency between row storage engine TiKV and column storage engine TiFlash. TiKV and TiFlash can be deployed on different machines as needed to solve the problem of HTAP resource isolation.
1.2.5 Cloud-native distributed database
TiDB is a database designed for the cloud. It is deeply coupled with Kubernetes and supports public clouds, private clouds, and hybrid clouds, making deployment, configuration, and maintenance very simple. The design goal of TiDB is 100% OLTP scenarios and 80% OLAP scenarios. More complex OLAP analysis can be done through the TiSpark project. TiDB is not intrusive to the business, and can gracefully replace traditional database
middleware, database sub-database and sub-table and other sharding solutions. At the same time, it also allows development and maintenance personnel to focus on business development without paying attention to the details of database scale, which greatly improves the productivity of research and development
1.4.6 Highly compatible with MySQL
Compatible with the MySQL 5.7 protocol, common functions of MySQL, and the MySQL ecosystem, applications can be migrated from MySQL to TiDB without or with a small amount of code modification.
Rich data migration tools are provided to help applications complete data migration conveniently. In most cases, it is easy to migrate from MySQL to TiDB without modifying the code. MySQL clusters after sub-database and table division can also be migrated in real time through TiDB tools.
1.3 OLTP&OLAP
1.3.1 OLTP (Online Transaction Processing)
OLTP (Online Transactional Processing) is online transaction processing. OLTP is the main application of traditional relational databases. It is mainly for basic and daily transaction processing. Records are added, deleted, modified, and checked in real time, such as depositing and withdrawing a sum in a bank. Payment is a business transaction
Online transaction processing is a very transactional system. It is generally a highly available online system. It mainly focuses on small transactions and small queries. When evaluating its system, it generally depends on the number of ransaction and Execute SQL executed per second. . In such a system, a single database often processes hundreds or thousands of Transactions per second, and the execution volume of Select statements is thousands or even tens of thousands per second. Typical OLTP systems include e-commerce systems, banks, securities, etc., such as the business database of eBay in the United States, which is a very typical OLTP database
1.3.2 OLAP (Online Analytical Processing)
OLAP (Online Analytical Processing) is the core of the data warehouse. It supports complex analysis operations, focuses on decision support, and provides intuitive and easy-to-understand query results.
A typical application is a complex dynamic reporting system. In such a system, the execution volume of the statement is not the assessment standard, because the execution time of a statement may be very long, and the data read is also very large. Therefore, in such a system, the assessment standard is often the throughput (bandwidth) of the disk subsystem, such as how many MB/s traffic can be achieved.
1.3.3 Feature Comparison
Comparison of the characteristics of OLTP and OLAP
1.3.4 Differences in design angles
2. The overall architecture of TiDB
2.1 Advantages of TiDB
Compared with traditional stand-alone databases, TiDB has the following advantages:
- Purely distributed architecture, with good scalability, supports flexible expansion and contraction
- Supports SQL, exposes MySQL's network protocol to the outside world, and is compatible with most MySQL syntaxes, and can directly replace MySQL in most scenarios
- High availability is supported by default. In the case of a small number of replicas failing, the database itself can automatically perform data repair and failover, which is transparent to the business
- Supports ACID transactions, and is friendly to some scenarios with strong consistency requirements, such as bank transfers
- Has a rich tool chain ecology, covering data migration, synchronization, backup and other scenarios
2.2 Components of TiDB
To gain a deep understanding of TiDB's horizontal expansion and high availability features, you first need to understand the overall architecture of TiDB. The TiDB cluster mainly includes three core components: TiDB Server, PD Server, and TiKV Server. In addition, there is a TiSpark component for solving complex OLAP needs of users.
In terms of core design, the TiDB distributed database splits the overall architecture into multiple modules, and each module communicates with each other to form a complete TiDB system. The corresponding architecture diagram is as follows:
2.2.1 TiDB Server
TiDB Server is responsible for receiving SQL requests, processing SQL-related logic, finding the TiKV address for storing data required for calculation through PD, interacting with TiKV to obtain data, and finally returning the result. TiDB Server is stateless, it does not store data itself, it is only responsible for computing, it can be expanded horizontally indefinitely, and can provide a unified access address externally through load balancing components (such as LVS, HAProxy or F5).
2.2.2 PD (Placement Driver) Server
Placement Driver (referred to as PD) is the management module of the entire cluster. It has three main tasks:
one is to store the meta information of the cluster (which TiKV node a Key is stored on);
the other is to schedule and load balance the TiKV cluster (such as data migration, Raft group leader migration, etc.);
the third is to assign a globally unique and incremental transaction ID.
PD ensures data security through the Raft protocol. The leader server of Raft is responsible for handling all operations, and the remaining PD servers are only used to ensure high availability. It is recommended to deploy an odd number of PD nodes
2.2.3 TiKV Server
TiKV Server is responsible for storing data . From the outside, TiKV is a distributed Key-Value storage engine that provides transactions. The basic unit for storing data is Region, and each Region is responsible for storing the data of a Key Range (the left-closed right-open interval from StartKey to EndKey), and each TiKV node is responsible for multiple Regions. TiKV uses the Raft protocol for replication to maintain data consistency and disaster recovery. Replicas are managed in units of Regions, and multiple Regions on different nodes form a Raft Group, which are replicas of each other. The load balancing of data among multiple TiKVs is scheduled by PD, which is also scheduled in units of Regions.
2.2.4 TiSpark
TiSpark, as the main component in TiDB to solve users' complex OLAP needs, runs Spark SQL directly on the TiDB storage layer, and at the same time integrates the advantages of TiKV distributed clusters and integrates into the big data community ecology. So far, TiDB can support both OLTP and OLAP through one system, eliminating the trouble of user data synchronization.
2.2.5 TiFlash
TiFlash is a special type of storage node. Different from ordinary TiKV nodes, inside TiFlash, data is stored in the form of columns, and its main function is to accelerate analytical scenarios.
2.3 TiKV Overall Architecture
Different from the traditional full-node backup method, TiKV divides the data into approximately equal slices (hereinafter collectively referred to as Regions) according to the range of the key. Each slice has multiple copies (usually 3), one of which is Leader, providing read and write services.
TiKV schedules these Regions and replicas through PD to ensure that data and read and write loads are evenly distributed on each TiKV. This design ensures full utilization of the entire cluster resource and can be scaled horizontally as the number of machines increases.
2.3.1 Region split and merge
When the size of a Region exceeds a certain limit (144MB by default), TiKV will split it into two or more Regions to ensure that the sizes of each Region are roughly similar, which is more conducive to PD's scheduling decision. Similarly, when a Region becomes smaller due to a large number of deletion requests, TiKV will merge two smaller adjacent Regions into one.
2.3.2 Region Scheduling
Data consistency is maintained between the Region and the replica through the Raft protocol. Any write request can only be written on the Leader, and it needs to be written to the majority of replicas (the default configuration is 3 replicas, that is, all requests must be written to at least two replicas success) will return the client write success.
When PD needs to schedule a copy of a Region from one TiKV node to another, PD will first add a Learner copy (copy Leader data) for this RaftGroup on the target node. When the progress of the Learner copy roughly catches up with the Leader copy, the Leader will change it to a Follower, and then remove the Follower copy of the operation node, thus completing a scheduling of the Region copy.
The scheduling principle of the Leader copy is similar, but after the Learner copy of the target node becomes a Follower copy, a Leader Transfer is executed again, so that the Follower initiates an election to become the new Leader, and then the new Leader is responsible for deleting the old Leader copy.
2.3.3 Distributed transactions
TiKV supports distributed transactions. Users (or TiDB) can write multiple key-values at one time without caring whether these key-values are in the same data slice (Region). TiKV guarantees these read and write requests through two-phase commit ACID constraints.
2.4 High availability architecture
High availability is another major feature of TiDB. The three components of TiDB/TiKV/PD can tolerate failure of some instances without affecting the availability of the entire cluster. The following describes the availability of these three components, the consequences of a single instance failure, and how to recover.
2.4.1 TiDB High Availability
TiDB is stateless. It is recommended to deploy at least two instances, and the front end provides external services through the load balancing component. When a single instance fails, it will affect the ongoing Session on this instance. From the application point of view, there will be a single request failure, and the service can continue to be obtained after reconnecting. After a single instance fails, it can be restarted or a new instance deployed.
2.4.2 PD High Availability
PD is a cluster that maintains data consistency through the Raft protocol. When a single instance fails, if the instance is not the leader of Raft, the service will not be affected at all; if the instance is the leader of Raft, a new Raft leader will be re-elected , to automatically restore service. PD cannot provide external services during the election process, and this time is about 3 seconds. It is recommended to deploy at least three PD instances. After a single instance fails, restart the instance or add a new instance.
2.4.3 TiKV High Availability
TiKV is a cluster that maintains data consistency through the Raft protocol (the number of copies is configurable, and three copies are saved by default), and uses PD for load balancing scheduling. When a single node fails, it will affect all Regions stored on this node. For the Leader node in the Region, the service will be interrupted and wait for re-election; for the Follower node in the Region, the service will not be affected. When a
TiKV node fails and cannot be recovered within a period of time (10 minutes by default), PD will migrate the data on it to other TiKV nodes.
2.5 Application scenarios
2.5.1 MySQL fragmentation and merging
The first type of scenario where TiDB is applied is the sharding and merging of MySQL. For businesses that already use MySQL, sub-databases, tables, shards, and middleware are commonly used methods. With the increase of shards, cross-shard query is a big problem.
TiDB is compatible with the MySQL access protocol at the business layer. PingCAP has made a data synchronization tool - Syncer, which can use TiDB as a MySQL Slave, and connect TiDB as the slave library of the existing database behind the main MySQL library. The first layer connects the data, and can directly perform complex cross-database, cross-table, and cross-business real-time SQL queries.
2.5.2 Directly replace MySQL
On a TiDB database, all business scenarios do not need to be divided into databases and tables, and all distributed work is done by the database layer. TiDB is compatible with the MySQL protocol, so it can directly replace MySQL, and it basically works out of the box. There is no need to worry about the heavy workload and complicated maintenance costs brought by the traditional database and table division scheme. The friendly user interface allows regular technicians to can efficiently
for maintenance and management. In addition, TiDB has a capacity similar to that of NoSQL, and can improve the business support capability of the system through horizontal expansion when the amount of data and access traffic continue to grow, and the response delay is stable.
2.5.3 Data warehouse
TiDB itself is a distributed system, and the third usage scenario is to use TiDB as a data warehouse. TPC-H is a test set in the field of data analysis. The performance of TiDB 2.0 in OLAP scenarios has been greatly improved. Some complex queries that can only be run in the data warehouse can be run in TiDB 2.0, and the time can basically be controlled. within 10 seconds. Of course, because the scope of OLAP is very large,
TiDB's SQL is also uncertain. For this reason, PingCAP has open sourced TiSpark. TiSpark is a Spark plug-in. Users can directly use Spark SQL to do big data analysis on TiKV in real time.