1. preliminary knowledge base
Online data migration, data refers to is providing online services, migrating from one place to another, the entire migration process requires non-stop service will not be affected. The data hierarchy which can be divided into cache memory migration and migration; data migration before and after the change, can be divided into a translation and metastasis.
Translation refers to the organization before and after the migration of data unchanged. For example Mysql extended from Examples 1, 4 instances, Redis extended from the four ports to 16 ports, HBase 20 extend from machine to machine 30 and the like. If on the reduction in the initial design capacity to provide for future expansion in the convenience, the data migration work will be much simpler, such as Mysql has done a sub-library sub-table, when extended instance, you only need a few more from the library, switched access, and finally the extra database table can be deleted. Still further, in the realization of automatic data migration has been done, such as HBase, even simpler: adding machine, automatically or manually modify the configuration found, then the system waits for the completion of the migration.
Transfer means before and after the data migration, data organization has changed. Years ago, a social platform ever done an upgrade for the ID data migration, modify the ID from the initial increment algorithm UUID algorithm cleverly designed, the relocation of the biggest challenges is to modify the primary key data, it could have been the primary key data unique identification, it changes, it means that the original data no longer exists, new data come out of nowhere, the whole system in all business processes, surrounding facilities, the upstream and downstream sectors will have great compatibility challenges. However, most of the data migration project will not modify the primary key, and even does not modify the data itself, the only change in organizational form data. Such as a social platform counters the original in order to save storage space, use redis hash is stored, then in order to improve the performance of batch queries, migrate into KV form; another example forwarding list and the fan list a social platform, initially using Mysql storage, and later to better scalability and cost, have migrated to HBase storage.
2. Data migration steps
- On-line double write, the data is written to the new / old two systems.
- The relocation of historical data offline, historical stock data will be moved to a new system from the old system.
- Cut to read, about to read requests are routed to the new system.
- Cleanup precipitation, including cleaning up old data, resource recovery, and clean up the old code logic, supporting the old system and so on, the lessons learned in the process of migrating to summarize precipitation, or use the process to develop common tools for transformation, for the next use.
Note: In some cases, steps one and two may also be reversed, do first move historical data before writing new data, this time on the need to carefully deal with the relocation of new data generated during that time, the general use of queue fashion cache writes, known as "chasing data."
Transfer: https://blog.csdn.net/zlfprogram/article/details/80735933