Author: PingCAP Feng Xiaoming
Connecting global extreme scenarios and innovative scenarios through TiDB is PingCAP's long-term internationalization strategy. Currently, more than 3,000 companies around the world have chosen TiDB. No matter in industries such as games, finance, logistics, Internet or intelligent manufacturing, based on application scenarios such as large-scale OLTP expansion and real-time HTAP analysis, PingCAP service users have extended from China to the world, covering Japan, Asia Pacific, North America and Europe, etc. area.
This article is the second part of the topic "Best Practices of Using TiDB in Global Extreme Scenarios and Innovative Scenarios". It shares how TiDB provides super scalability and elasticity for Bolt's 100 million users, ensuring the global operation of its services.
Bolt is a travel Internet company headquartered in Estonia that provides urban transportation and delivery services. Its business areas include online car-hailing, motorcycle rental, car sharing and food delivery. It is a strong competitor of Uber in Europe. Bolt has more than 100 million customers in more than 500 cities in 45 countries, has established partnerships with more than 3 million suppliers worldwide, and employs more than 3,000 employees.
The main pain points faced by the MySQL solution
As an Internet company, data has great value and role in business decision-making and personalized services. In the past, Bolt has been using MySQL as its main database solution and it has been working well for a long time, but with the rapid growth of Bolt's global business, MySQL began to be unable to cope with its ever-increasing data volume.
There are many product teams within Bolt, developing and managing thousands of microservices. These microservices are processing hundreds of terabytes of data in more than 300 schemas, and the data volume is constantly growing. MySQL is difficult to scale to handle this much data. In order to try to solve these challenges, Bolt started to manually split tables and add columns. But this job is time consuming, loading a 1TB table takes a week or more. Also, there are operational issues when they run commands like ALTER (DDL) or make backups.
To support its hundreds of millions of customers, Bolt has very stringent requirements for system uptime and disaster recovery. But old solutions based on MySQL were too fragile to run smoothly in a disaster. Even though they deployed a multi-master cluster like Galera on top of MySQL to increase the availability of the system, they still occasionally lost data.
Why choose TiDB?
In order to solve the above pain points, Bolt spent a lot of time exploring database solutions including Vitess. But Vitess required many changes to Bolt's applications, complex business transformations -- and it had similar operational issues as MySQL. Also, Vitess has poor compatibility with MySQL. In the end, Bolt chose TiDB, an open source distributed NewSQL database.
open source
TiDB has been an open source database from day one. Every line of code is open and transparent, so Bolt can change, modify and customize the code according to its own needs, while TiDB can still work.
horizontal scalability
TiDB has almost unlimited horizontal scalability. Easily handle data spikes or troughs of up to hundreds of terabytes simply by automatically adding or removing storage nodes. In addition, TiDB adopts a computing and storage separation architecture, so Bolt can expand or shrink computing and storage resources respectively according to its changing business needs, which helps Bolt control its database and operation and maintenance costs.
high availability
TiDB uses the Raft algorithm to ensure that data is highly available and safely replicated throughout the entire storage of the Raft group. TiKV is the storage node of TiDB. Data is redundantly replicated between TiKV nodes and placed in different availability zones to prevent machine or data center failures. TiDB provides automatic failover, thus ensuring system uptime. In addition, TiDB provides a variety of disaster recovery solutions, each of which is suitable for different scenarios and has flexible costs.
MySQL Compatibility
TiDB is highly compatible with MySQL, so Bolt's engineering team can easily migrate data to TiDB without modification. Greatly save time and cost. In addition, TiDB can be well integrated with major technology stacks such as Kafka and Kurbenetes, and can also run perfectly on AWS.
cloud native
Based on cloud-native architecture, TiDB takes full advantage of the scalability and elasticity of the cloud, and is designed to work in cloud, private or hybrid environments. TiDB's cloud-native capabilities allow users to expand as needed to meet changing business needs, while costs are always kept under control.
In addition, when using TiDB with Kubernetes, users can also:
- Ensure the consistency of development and deployment environments;
- Instantiate a new node as needed, either to accommodate growth, or to replace a missing node.
very cost effective
At the same time, TiDB is also an excellent cost-effective solution:
- Powerful data compression reduces storage costs. For example, a 4TB MySQL table can be compressed to less than 2TB after being migrated to TiDB. According to Bolt's experience, the average compression ratio of TiDB is about 3 times. Depending on the data type, the rate may vary slightly - sometimes more, sometimes less;
- Has a simplified HTAP architecture that can handle both transactional and analytical workloads.
- Computing and storage resources can be scaled up or down horizontally according to business needs .
- Easy to operate and maintain , greatly reducing maintenance costs.
- With good MySQL compatibility , the migration process is smooth and easy.
- Supports flexible deployment on Amazon Elastic Kubernetes Service (EKS) using Graviton2 (Arm64) to achieve better cost performance.
TiDB technical support
During the process of deploying, migrating, and using TiDB, Bolt received efficient support from TiDB's original engineers, and carried out a special customized transformation of TiDB.
As of October 2022, Bolt has deployed seven TiDB clusters, storing tens of terabytes of data and hundreds of schemas.
- 4 TiDB clusters are running in the production environment, including balance, billing, ordering and Internet of Things (IoT) services.
- 2 TiDB clusters for development or pre-production alpha and beta clusters.
- A TiDB cluster is deployed in the downstream cluster in Stockholm for disaster recovery.
future outlook
In the future, Bolt plans to rely more on TiDB for its business, including migrating more clusters from MySQL to TiDB, using TiDB's change data capture tool TiCDC to set up multi-regional clusters on AWS for failover, and trying TiDB's columnar storage TiFlash for Analyze queries in real time to obtain more real-time data insights.