The ever-increasing amount of data is becoming a major challenge for our companies. The variety of data ranges from employee and customer information to complex scientific data – data that impacts decisions and strategic business movements. The systems that process this data are often very complex and the technical requirements for these systems are evolving at an unprecedented rate. Therefore, these systems need to be adapted in good time to the new requirements. As a result, the data must be moved or integrated into the new system. The process of moving data from one system to another – data migration – is an important process in any organization. For example, companies move their data to start a new application, update databases, or merge data from different sources.
While moving data from a source system to a target system without a clever testing strategy seems straightforward, organizations can easily encounter serious problems. If no data is available in the target system after data migration – this is called data loss – or if the data is not qualitative and integer, the company incurs high costs or important deadlines may be missed.
It is therefore important to have a strategy that guarantees the success of data migration projects. Such projects require systematic planning, the right tools, knowledgeable team members, and intensive testing to be successful. In this article, we focus on the requirements of a successful test strategy.
Planning and strategizing the testing
activities need to get the same attention as planning the appropriate
migration approach, since testing allows you to actually validate the success
of your migration in respect to data completeness, content, or integrity. Therefore,
the following considerations should be included in a data migration testing
Define the test objectives – In a first step, the business requirements
related to data migration need to be identified. It is important that the testing
scope covers these requirements.
Define the testing scope: Validation of the complete data in the source
and the target system is the best approach to identifying data corruption. If you
can only test a subset of your data, ensure that you choose a large enough sample
size so that you have a high enough confidence and you can trust in your
Prepare your source data – Before migrations, the source data needs to
A thorough audit. It is crucial to understand the source data in order
to perform data mapping between your source system and your target application.
In your audit, it is important to include data types, tables, field names, transformations
and business rules. Further, semantic risks need to be identified. For example,
columns in the source and the target system have the same meaning but differ in
their unit of measurement.
Data clean-up – Once you identified any issues with your
source data, unnecessary data must be removed and wrong data must be corrected.
Any issues in your source data can be amplified during your migration.
Risk analysis – In a next step, identify possible errors
within the migration process and try to mitigate these risks. Testing needs to
uncover these scenarios.
Controls – During migration, data gets transformed what
can potentially result in mistakes. Consequently, there must be controls based
on your planning in place to maintain data integrity. Check for data
completeness, data quality, or misplaced data. So include in your testing if
these data transformations are in accordance with your business rules and are
correct according to your mapping.
Governance – Monitoring and reporting the data quality
During the migration process is crucial and it enables a better understanding
of the subsequent steps you need to undertake in order to guarantee a
Regardless of the testing strategy you follow, there are some best practices to remember: Stick to your testing strategy even if the migration run seems to go very smoothly, and test as much as possible. The more information you can collect about your migrated data the more reliable they are.