ETL testing is crucial for data assurance to ensure precision when transforming or migrating data from multiple complex systems. This becomes more challenging with enormous data volumes and multiple structured and unstructured data sources. 

We at Onepoint leveraged our in-house test platform for end-to-end data validation and reporting for a leading Automotive pricing expert.

Challenges

  • The large volume of data from multiple NoSQL data sources
  • Complex data transformations with regex and join columns
  • Time and efforts required for manual data comparison for a large amount of data
  • Absence of an automated solution for end-to-end data validation

Solution options

In-house

Manual

Commercial

Easily Extendable

Smart Multithread to dynamically reduce comparison time 

One-Click regression tests

Customizable reports as per client needs

Time Consuming

Sampling basis -Error-Prone

Regression requires similar effort

Not customizable as per requirements

No support for Aerospike out of the box

Cost per user seat up to $45,000

Solution adopted

  • Used in house Testing Platform for validating data migration
  • Extended the Test Platform to support Aerospike and Mongo data sources
  • Improved performance of test platform by using smart multithread
  • Improved final report for facilitating analysis of results

Results from using an in-house test platform

  • 100% test coverage while validating the data across Aerospike to MongoDB
  • Successful data migration validation from Aerospike to Mongo
  • Reduced effort in Test case preparation using test platform
  • One-Click regression testing