Migration is one of the topics that customers are most concerned about during the database selection process. After extensive communication and research, we have summarized three main expectations of customers in database migration: migration will not affect business operations (peace of mind), and data cannot be lost during migration (peace of mind). ), migration is easy to implement (save worry).
This article starts from three questions from the customer's perspective to learn about the core technologies of Huawei Cloud's GaussDB database migration solution and how to make customers feel at ease, assured, and worry-free during the migration process.
Question 1: Can it be moved?
Each database has its own specific "temper". For example, Oracle's language may not necessarily work if it is run directly on GaussDB. This is the so-called "language barrier". This scenario is called "heterogeneous database migration" , "Language barrier" has been the biggest technical obstacle to the migration and replacement of heterogeneous databases for a long time. Let’s look at it from two aspects: technology and effect.
In terms of technology, how did Huawei solve the problem?
There is a tool called UGO in Huawei's migration suite. It is like a "translation master" and divides this problem into two parts:
Part 1 Understand the Oracle language: Oracle’s syntax corresponds to huge tree structures and is composed of some key elements, such as An Oracle table creation statement, what is the table name, how many columns are there, what is the name of each column, etc. UGO has a dedicated syntax analyzer that can detect the elements of each Oracle statement along the syntax tree and enumeration, so that it can finally understand the semantics and understand the Oracle language.
Part 2 "Translation" into GaussDB based on the knowledge base: According to the description just now, this technology is very similar to "translation" "Work and life experience tell us that the level of translation depends on whether the training scenarios and vocabulary samples exposed are large enough. It relies on training in process IT and ICBC, the two most complex Oracle business environments in the world. Up to now, Huawei Cloud UGO has been injected with tens of millions of lines of code training, which is the highest record announced by the industry. There is no relevant introduction in this regard for similar products.
What is the effect?
"Translation Master" UGO relies on Huawei's unique training scenarios. After connecting to Oracle, it can usually generate a detailed evaluation report in a few minutes, making the feasibility of migrating to GaussDB clear at a glance. In the most challenging scenario of ICBC's migration battle, in the migration of tens of millions of stored procedures, after UGO processing, 95% of the Oracle code can be run directly in GaussDB after conversion, and de-O is very close to automation.
Question 2: How to migrate so that the business is imperceptible?
The database is a very busy software. Thousands of data may change every second. How to migrate the database during the dynamic process of highly changing data? The easiest way to think of is to shut down the business to ensure Data migration is not lost, but this is unacceptable for a core business. This raises a technical question: Is there a way to migrate the database to GaussDB during business operation, which is often called hot in the industry? Migration, or online migration.
In terms of technology, Huawei's migration suite includes a tool called DRS, which is like a "data copy machine" and can migrate to GaussDB while Oracle is running. The DRS tool technically divides this problem into three parts:
Part 1 Stock data migration, DRS uses row-level parallelism to extract data, and the extraction performance can reach up to 100+MB/s. At the same time Customers can set the migration speed if the source resources allow it, so that they can migrate as fast as they want or as slowly as they want;
Part 2 Incremental data migration, the database has a component called log, which records the data in the database in a fast and orderly manner. Every data change is a key technology that prevents the database from losing data during a power outage. DRS uses technical means to replay the events recorded in the log. This process is like recording the Oracle data changes and performing the same operation again on GaussDB. , this technology industry is generally called CDC technology (change data capture). Not only that, DRS also uses technical means to optimize the operation of each piece of data into parallel operation, so that the data on both sides can be ensured not to be lost, and the synchronization speed can be greatly improved. Usually, the newly generated data of the Oracle database can be stored in GaussDB within 1 second. It can be seen that even a very large and busy Oracle system (for example, the log volume reaches TB level in a day) can achieve second-level synchronization.
Part 3 Dynamic data comparison, DRS has a built-in patented algorithm to identify incremental logs every x minutes (configurable) Changed data rows in the data row, and based on the primary key value of the data row, query and compare the data of the source database and the target database, thereby verifying the data consistency of Oracle and GaussDB in stages. This loop operation continues to refresh the consistency comparison results. Get the final consistency conclusion. There is no need to stop the business access to the Oracle database and conduct long-term data verification.
How's the effect?
"Data copy machine" DRS realizes online migration through CDC technology, so that Oracle can migrate data to GaussDB during operation. Data can be compared with zero loss. Businesses can freely choose the cutover time, and during the migration process Not interested in business. To sum up, Business is worry-free ->Peace of mind, data will not be lost->Peace of mind, automatic data synchronization->Safety of worry.
Question 3: How to ensure stable business operation after migration?
The above answers the questions of whether it can be migrated and how to migrate. Then, after migrating to GaussDB, can it smoothly survive the test of business peaks? Technically, with the above foundation of UGO and DRS, we can automatically collect data on the Oracle database. Each business request is sent from the application to the Oracle database. There are query requests and data modification requests. The sum of these requests is the business traffic. After the business traffic is collected during the peak period of the business, it is syntax converted through UGO, and combined with The real production environment data constructed by DRS can form a business simulation stress testing environment based on the GaussDB database.
Through simulation stress testing, customers can directly predict whether GaussDB can stably survive business peaks, and identify in advance whether specifications need to be adjusted. For the business itself, it also supports the migration of fine-grained switching. DRS can allow data to be transmitted in both directions, and can be combined Lines are gradually cut, and GaussDB's business carrying capacity is observed, with the ability to roll back at any time.
Finally, let’s summarize Huawei Cloud’s full set of GaussDB migration solutions. The “Translation Master” UGO solves the problem of grammatical conversion of different languages, and the “data replicator” DRS solves the problem of non-stop data migration. At the same time, the DRS tool also provides simulation stress testing. , data consistency comparison and other supporting capabilities.
Through these capabilities, Huawei Cloud GaussDB has been widely used in major financial projects. At the same time,Huawei Cloud database migration solution is the first to win the best in the database migration evaluation of the Academy of Information and Communications Technology. As a manufacturer of advanced migration capabilities, Huawei Cloud GaussDB database is committed to providing customers with a set of secure, secure, and worry-free migration solutions through technology.
That’s it for today’s sharing, thank you all, welcome to leave messages for discussion and exchange~