Redis is currently the most popular key-value pair storage database. With its high performance and rich data types, it can not only be used as a cache, but also as a persistent database storage. With the development of business and iteration of versions, a series of problems such as insufficient memory, insufficient cluster nodes and bugs will inevitably be encountered. In order to prevent system failures caused by these problems, operations such as expansion and contraction of memory, cluster nodes, and version upgrades are often considered important items in work assessments. These operations all involve data migration, so it is very necessary to provide an efficient and safe non-stop data migration solution.
Now, NineData has implemented a data migration service that is simple to configure, stable, efficient, and secure without interrupting business, and can well meet the needs for data migration and synchronization in scenarios such as version upgrades, expansion, and shrinkage. After actual testing, NineData can complete the migration of 20 million keys (5GB) in 2 minutes, with an average migration speed of 164,398 keys/second, and its performance is more than twice that of open source tools.
1. Traditional migration plan
Currently, the main methods of data migration are: using RDB migration or some open source tools for data migration. There will be some problems with migration in these ways:
-
It requires downtime . Migration by copying RDB files cannot guarantee online business and is not compatible with major version upgrades.
-
Accuracy is difficult to guarantee , data detection capabilities are not supported, and the quality of data after migration is difficult to guarantee.
-
The reliability is poor . For open source tools, the process exits directly after the migration is abnormal.
-
The operation and maintenance is poor , and operations such as suspension, current limiting, and alarming cannot be performed.
2. High-performance migration solution
The data replication provided by NineData includes both data migration and data synchronization capabilities , providing efficient, stable, and safe migration capabilities without affecting the business. Compared with traditional migration, NineData's Redis data migration capability has the following advantages:
▶︎ Simple and easy to use
Task configuration can be completed in one minute, and task migration is fully automated.
▶︎Powerful performance
Through core technologies such as dynamic batch accumulation, queue optimization, and streaming memory management, the migration performance reaches 160,000 keys/second, which is twice as fast as open source migration tools, effectively ensuring migration efficiency.
▶ ︎High reliability
Combined with new breakpoints, exception diagnosis and rich repair methods, it provides complete disaster recovery capabilities for software and hardware failures that may occur during the migration process, greatly improving the success rate of migration.
Through the above advantages, NineData's leadership in Redis migration scenarios is guaranteed. In addition, NineData also provides comparison functions, including full, fast and inconsistent re-inspection comparison methods, and also supports different comparison frequencies . After the migration or copy is completed, the quality of the data can be effectively guaranteed through comparison.
3. Operation and use
NineData not only provides powerful migration capabilities, but also ensures simplicity of use. It only takes 1 minute to complete the configuration of the migration task and realize a fully automated data migration process. Let's take a look at the configuration process of the entire task:
3.1 Migration link configuration
1. Configure the task name and select the source and target instances to be migrated.
2. Select the copy type, select structure and full copy (data migration) for data migration.
3. Select an appropriate conflict handling strategy as needed.
3.2 Select migration objects
Select migration objects: You can select different DBs for migration.
3.3 Configure mapping objects
Configuration mapping: Multiple databases (0~15) of the source instance can be mapped to one or more specified databases of the target instance. Through this mapping capability, scenarios similar to MySQL multi-source replication can be realized.
3.4 Pre-check
Through comprehensive check items, the stability of the migration task is ensured. At this point, we have completed the configuration of an efficient and safe Redis migration task. After completing the configuration and starting the task, NineData will automatically start the full copy and incremental copy processes to achieve fully automated data migration.
At the same time, in order to provide a better migration experience, NineData provides complete observation and intervention capabilities for the migration process. It not only provides detailed status, progress, and details of object migration, but also reveals the internal execution of background threads through monitoring and logs, helping users track migration progress in an all-round way. At the same time, it also provides basic diagnosis and migration current limiting capabilities for abnormal situations that may occur during operation, allowing users to independently and quickly diagnose and repair links to ensure migration stability. Information during migration:
4. Summary
Based on full replication and incremental log replication technology, NineData provides an efficient, safe and reliable Redis non-stop migration solution. Of course, in addition to Redis, NineData already supports the migration and replication of dozens of common databases, realizing business scenarios such as database migration, data disaster recovery, data active-active, and real-time integration of data warehouses. At the same time, in addition to the SAAS mode, it also provides enterprise-specific cluster mode to meet the highest data security compliance requirements of enterprises. At present, NineData has completed large-scale application practice in many industries such as operators, finance, manufacturing, real estate, and e-commerce. If you are interested, you can log in to the official website https://www.ninedata.cloud/dbmigration and start using it for free immediately.