Overview of data migration testing:
I often hear that applications have been moved to other servers, technology has been changed, updated to the next version or moved to other database servers, etc.
What does this mean?
In this case, what are the expectations of the test team?
From a testing perspective, this all means that the application must be thoroughly tested end-to-end and successfully migrated from the existing system to the new system.
In this case, all data must be systematically tested, and these data must be used in the new data in the old application. Existing functions need to be verified together with new/modified functions.
The migration test can also be called the data migration test, in which all the user's data will be migrated to the new system.
Therefore, migration testing includes testing with old data, new data or a combination of both, old features (unchanged features) and new features.
Legacy applications are often referred to as "legacy" applications. Along with the new/upgraded application, the old application must also be tested until the new/upgraded application becomes stable and consistent. Extensive migration testing of the new application will reveal new problems not found in the old application.
What is migration testing?
Migration testing is a verification process for migrating the old system to the new system to achieve data integrity and no data loss with minimal interruption/downtime, while ensuring that all specified functions and non-functions of the application are met after the application is running Requirements.
A simple representation of the migration system:
Why do we need to conduct a migration test?
As we all know, there may be many reasons for applications to migrate to a new system, including system integration, technology obsolescence, optimization or any other reason.
Therefore, although the system in use needs to be migrated to the new system, the following points must be ensured:
It is necessary to avoid/minimize any type of interference or inconvenience caused to users due to migration, such as downtime, data loss.
Try not to cause damage to the user or only a small amount of damage to ensure that the user can continue to use all the functions of the software. For example: function change, deletion of a specific function.
It is also important to predict and eliminate all possible failures/obstacles that may occur during the actual migration of the real system.
Therefore, in order to ensure the smooth migration of real-time systems by eliminating these defects, it is very necessary to conduct migration tests in the laboratory.
Technically, it also needs to be executed for the following purposes:
Ensure that the new/upgraded application is compatible with all possible hardware and software supported by the old application. In addition, new compatibility should be tested against new hardware and software platforms.
To ensure that all existing functions can work properly in the old version of the application. Compared with the old application, the way the new application works should not change in any way.
The possibility of a large number of defects due to migration is very high. Many defects are usually related to data, so they need to be identified and repaired during the testing process.
Ensure that the system response time of the new/upgraded application is equal to or less than the response time of the old application.
Ensure that the connections between servers, hardware, software, etc. are intact and will not be interrupted during the test. Under no circumstances should the data flow between different components be interrupted.
When is this test required?
Testing must be performed before and after the migration.
The different stages of migration testing conducted in the testing laboratory can be classified as follows.
1. Test before migration
2. Migration test
3. Test after migration
In addition to the above, the following tests will also be performed as part of the entire migration activity.
Backward compatibility verification
Rollback test
Before performing this test, any tester must clearly understand the following points:
Changes that occur as part of a new system (server, front end, database, schema, data flow, function, etc.).
Understand the actual migration strategy developed by the team. How the migration occurs, the changes that occur gradually at the back end of the system, and the scripts responsible for these changes.
Therefore, it is necessary to conduct in-depth research on the old and new systems, and then plan and design test cases and test scenarios accordingly as part of the above-mentioned test phase and develop test strategies.
Data migration test strategy
The test strategy designed for migration includes a set of activities to be performed and several aspects to consider. This is to minimize errors and risks due to migration and perform migration testing effectively.
Activities in this test:
1. Professional team composition
Form a test team with members who have the required knowledge and experience, and provide training related to the system to be migrated.
2. Business risk analysis and possible error analysis
The current business should not be hindered after the migration, so a "business risk analysis" meeting should be held, including appropriate stakeholders (test managers, business analysts, architects, product owners, business owners, etc.) and determine the risks And mitigation measures that can be implemented. Testing should include scenarios where these risks are discovered and verify that appropriate mitigation measures have been implemented.
Use appropriate "error guessing methods" for "possible error analysis", and then design tests around these errors in order to find errors during the test.
3. Analysis and identification of migration scope
Analyze the clear scope of migration testing, when and what needs to be tested.
4. Determine the appropriate migration tool
When defining automated or manual testing strategies, determine the tools that will be used. For example: automated tools for comparing source and target data.
5. Determine the appropriate migration test environment
Identify separate environments for the pre-migration and post-migration environments to perform any verification required for testing. Understand and record the technical aspects of the old and new migration system to ensure that the test environment is configured accordingly.
6. Migrate and review test specification documents
Prepare the migration test specification document, which clearly describes the test requirements, test scope, test technology (automatic, manual), test method (black box, white box), number of test cycles, test schedule, data creation and use of real-time data ( Need to cover up sensitive information) methods, test environment specifications, test personnel qualifications, etc., and review with stakeholders.
7. Production start of the migration system
Analyze and record the to-do list for production migration, and publish it in advance.
Different phases of migration testing
Phase 1: Testing before migration
Before the data is migrated, a set of testing activities will be performed as part of the pre-migration testing phase. In simpler applications, ignore or ignore this. However, when complex applications are to be migrated, pre-migration activities must be performed.
The following is a list of actions taken at this stage:
Set a clear data range: what data must be included, what data must be excluded, data that needs to be converted, etc.
Perform data mapping between the old application and the new application: For each data type in the old application, compare its related types in the new application, and then map.
If there is a required field in the new application, which is not the case in the old application, make sure that the old application does not set the field to null.
Carefully study the data pattern of the new application, including field names, types, minimum and maximum values, length, required fields, field level verification, etc.
Many tables in the old system will be recorded. If tables are deleted and added after migration, verification is required.
In old applications, you should pay attention to multiple records in each table and view.
Investigate the interfaces and connections in the new application. The data flowing in the interface should be highly secure and not damaged.
Prepare test cases, test scenarios, and use cases for the new conditions in the new application.
Use a set of users to execute a set of test cases and scenarios, and save the results and logs. After the migration, verification is also required to ensure that the remaining data and functions are intact.
The counts of data and records should be clearly recorded and verified after migration to ensure that no data is lost.
Phase 2: Migration test
When performing migration activities, you must strictly follow the "migration guide" prepared by the migration team. Ideally, the migration activity starts with backing up the data to the hard drive so that the old system can be restored at any time.
Validation of the documentation part of the "Migration Guide" is also part of the data migration test. Verify that the document is clear and easy to understand. All scripts and steps must be recorded correctly, without any ambiguity and any type of documentation error, and mismatches in the order of execution of the steps also need to be considered important so that they can be reported and resolved.
The migration scripts, guides and other information related to the actual migration need to be obtained from the version control repository for execution.
Recording the actual migration time from the start of the migration to the successful recovery of the system is one of the test cases to be executed. Therefore, the "time used to migrate the system" needs to be recorded in the final test report, which will be submitted as part of the migration test results, and This information will be useful during product launch. Extrapolate the downtime recorded in the test environment to calculate the approximate downtime in the field system.
It will perform migration activities on the old system.
During this testing process, all components in the environment are usually shut down and removed from the network to perform migration activities. Therefore, it is necessary to pay attention to the "downtime" required for migration testing. Ideally, it will be the same as the migration time.
Generally, the migration activities defined in the "Migration Guide" document include:
Actually migrate the application;
The firewall, port, host, hardware, and software configuration are all modified according to the new system to be migrated from the old version;
Perform data leakage and security inspections;
Check the connections between all components of the application;
It is recommended that testers verify the above content at the back end of the system or through white box testing.
After the migration activities specified in the guide are completed, all servers will be started and basic tests related to successful migration verification will be performed to ensure that all end-to-end systems are properly connected, all components are communicating with each other, and the DB is up and running , The front end and the back end communicate successfully. These tests need to be determined earlier and recorded in the migration test specification document.
The software may support multiple different platforms. In this case, the migration needs to be verified on each platform separately.
Migration script verification will be part of the migration test. Sometimes, in an independent test environment, "white box testing" can also be used to verify individual migration scripts.
Therefore, migration testing should be a combination of "white box testing" and "black box testing."
Once the verification related to the migration is completed and the corresponding tests are passed, the team can proceed with post-migration tests.
Phase 3: Post-migration testing
After the application was successfully migrated, the post-migration test began.
Here, end-to-end system testing is performed in a test environment. Testers use old data and a new set of data to execute certain test cases, test scenarios, and use cases.
In addition, the following specific items need to be verified in the migrated environment:
All of these are recorded as test cases and included in the "test specification" document.
Check whether all data from the old version will be migrated to the new application during the planned downtime. To ensure this, compare the number of records between the old application and the new application for each table and view in the database. In addition, report the time required for migration, such as the migration of 10,000 records.
Check whether all schema changes (added or deleted fields and tables) are updated in accordance with the new system.
Data migrated from the old version to the new application should retain its value and format unless it is not specified to do so. To ensure this, compare data values between the old application database and the new application database.
Test the migrated data against the new application. Here is the largest possible scenario. To ensure 100% data migration verification coverage, please use automated testing tools.
Check database security.
Check the data integrity of all possible sample records.
Check to make sure that the earlier supported functions in the old system can work as expected in the new system.
Check the data flow in the application covering most components.
The interface between the components should be extensively tested, because the data should not be modified, lost or damaged as it passes through the components. Integration test cases can be used to verify this.
Check the redundancy of the old data. During the migration, any old data should not be duplicated.
Check for data mismatch, such as data type has changed, storage format has changed, etc.
All field level checks in the old application should also be included in the new application.
Any data additions in the new application should not be reflected on the old version.
It should support updating the data of the old application through the new application. After updating in the new application, it should not be reflected on the old version.
Should support deleting the data of the old application in the new application. Once deleted in the new application, it should not delete the old data.
Verify that the changes made to the old system are supported as part of the new system.
Verify that users of the old system can continue to use the old and new features, especially those that involve changes. Test cases and test results stored during the pre-migration test.
Create a new user on the system and test it to ensure that the functions of the old version and the new application support the newly created user and can work normally.
Use various data samples (different age groups, users from different regions, etc.) to perform function-related tests.
It is also necessary to verify whether the "feature flag" is enabled for the new feature, and turn it on/off to enable or disable the feature.
Performance testing is very important to ensure that migrating to a new system/software will not degrade system performance.
It is also necessary to perform load and stress tests to ensure system stability.
Verify that the software upgrade does not open any security holes, and therefore conduct security testing, especially in areas where changes were made to the system during the migration.
Usability is another aspect that needs to be verified, where if the GUI layout/front-end system has changed or any function has changed, what is the ease of use that the end user feels compared to the old system.
Since the scope of testing after the migration becomes very large, it is best to separate the important tests that need to be performed first to determine whether the migration is successful, and then perform the remaining tests.
It is also recommended to automate end-to-end functional test cases and other possible test cases to reduce test time and get results quickly.
Testers write some suggestions for test cases to be executed after migration:
When an application is migrated, it does not mean that test cases must be written for the entire new application. Test cases that have been designed for the old version should still be applicable to the new application. Therefore, use old test cases as much as possible, and convert old test cases into use cases for new applications when needed.
If there are any functional changes in the new application, the test cases related to that feature should be modified.
If any new function is added to the new application, a new test case should be designed for that specific function.
When any function in the new application is deleted, the test cases of the related old application should not be considered for execution after migration, and they should be marked as invalid and separated.
The designed test cases should always be reliable and consistent in use. The verification of key data should be included in the test case so that it will not be missed during execution.
When the design of the new application is different from the design of the old version (UI), the test cases related to the UI should be modified to adapt to the new design. In this case, the tester can decide whether to update or rewrite based on the amount of change that has occurred.
Backward compatibility test
System migration also requires testers to verify "backward compatibility", where the new system introduced is compatible with the old system (at least two previous versions), and to ensure that it works perfectly with these versions.
Backward compatibility is to ensure:
Does the new system support the previous two versions and the functions supported in the new version?
The system can be successfully migrated from the previous two versions without any trouble.
Therefore, the backward compatibility of the system must be ensured by performing specific tests related to supporting backward compatibility. The tests related to backward compatibility need to be designed and included in the "test specification" document for execution.
Rollback test
If there are any problems while performing the migration, or if the migration fails at any time during the migration process, it should be possible for the system to roll back to the old system and quickly restore its functions without affecting users and previously supported functions.
Therefore, in order to verify this, the migration failure test plan needs to be designed as part of the abnormal test, and the rollback mechanism needs to be tested. The total time required to restore to the old system also needs to be recorded and reported in the test results.
After the rollback, major functions and regression tests (automation) should be run to ensure that the migration does not affect any factors, and the rollback successfully restores the old system to its original position.
Migration test summary report
The test summary report should be generated after the completion of the test, and should cover a summary report of various tests/scenarios executed as part of the various stages of the migration, as well as the result status (pass/fail) and test logs.
The time of recording the following activities should be clearly reported:
Total migration time.
Downtime of the application.
The time it took to migrate 10,000 records.
The time used for rollback.
In addition to the above information, you can also report any comments/suggestions.
Challenges in data migration testing
The main challenge for this test is data. Here are some of them:
1. Data quality
We may find that the data used in the old application is of poor quality in the new/upgraded application. In this case, data quality must be improved to meet business standards.
Factors such as assumptions, data conversion after migration, and data entered in the old version of the application itself are all invalid. Poor data analysis will lead to poor data quality. This can lead to high operating costs, increased data integration risks, and deviations from business goals.
2. Data does not match
The data migrated from the old version to the new/upgraded version of the application may not match in the new version. This may be due to changes in data types and data storage formats, which may redefine the purpose of using data.
This results in a huge effort to modify the necessary changes to correct the mismatched data or accept it and adjust it.
3. Data loss
Data may be lost when migrating from the old version to the new/upgraded application. This can be a required field or an optional field. If the missing data is for non-mandatory fields, the record will still be valid and can be updated again.
However, if the data for the required fields is lost, the record itself becomes invalid and cannot be withdrawn. This will result in huge data loss, if the capture is correct, it must be retrieved from the backup database or audit log.
4. Data volume
Migrating large amounts of data that requires a lot of time within the downtime window of migration activities. For example: scratch cards in the telecommunications industry, users on smart network platforms, etc. The challenge here is that when the time comes, clearing old data will create a large amount of new data and need to be migrated again. Automation is the solution for mass data migration.
5. Real environment simulation (with actual data)
Simulating the real environment in the test laboratory is another real challenge. Testers will encounter different kinds of problems in real data and real systems, and these problems will not be encountered during the test.
Therefore, when performing data migration tests, data sampling, replication of the real environment, and identification of the amount of data related to migration are very important.
6. Simulated data volume
The team needs to study the data in the real system very carefully, and should propose typical data analysis and sampling methods.
For example: users who are under 10 years old and under 10-30 years old should obtain data from the real environment as much as possible. If not, they need to create data in the test environment. Need to use automated tools to create large amounts of data. If capacity cannot be simulated, extrapolation can be used where applicable.
Tips for mitigating data migration risks
Here are some tips to reduce the risk of data migration:
Standardize the data used in the old system so that the standard data will be available in the new system during migration;
Enhance data quality so that qualitative data testing can be carried out during migration, so as to bring a sense of testing to end users;
Clean up the data before migration so that no duplicate data will appear in the new system during migration, which also keeps the entire system clean;
Re-check constraints, stored procedures, and complex queries to produce accurate results so that correct data can be returned in the new system during migration;
Identify the correct automated tools to perform data checks/record checks in the new system compared to the old system.
in conclusion
Therefore, considering the complexity involved in performing data migration testing, remember that any small error in verification during testing will lead to the risk of migration failure in production, so careful and thorough research is very important. System analysis before and after migration, using powerful tools and skilled and well-trained testers to plan and design effective migration strategies.
As we know, migration has a huge impact on the quality of applications. The entire team must invest a lot of effort to verify all aspects of the entire system, such as functionality, performance, security, availability, availability, reliability, compatibility, etc. This ensures a successful "migration test".
At last:
You in the future will definitely thank yourself for your hard work now!
Here I recommend a software testing exchange group I created by myself, QQ: 642830685. The group will share software testing resources, test interview questions, and industry information from time to time. You can actively exchange technology in the group, as well as the bosses. You answer the question.