How to speed up the time on the Data Migration activities

During my career, I experienced many challenges in Data Migration activities that support many other professional areas.

Some essential things that need to be considered when discussing Data Migration are the ETL (Extract, Transform & Load) process or preferred ELT (Extract, Load & Transform). This is essential in Data Migration, and setting up a method to perform and analyze this activity should be your project's success or painful history.

In this article, I don′t want to go thru the ETL process and tools for that. I will present a series of articles that support the Data Analyst to explore, analyze, enhance the data and perform some visuals to leverage the accuracy of the transformation peace.

ETL (Extract, Transform & Load) or ELT (Extract, Load & Transform)

One unique reason is that I prefer to use ELT instead of the ETL process. When we started the work, we didn′t know what to expect from the dataset provided to be analyzed.

It means some data that can quickly identify the final transformation rule for some specific fields earlier. But in general, the pain of the data migration is related to the re-work of replacing the regulations every time until getting the final version to load the data to the production system.

During a project, we may have multiple sets of a MOCK process to test the data, transformations, timing, and business activities related to building the Cutover plan. Most of the projects I participated in had 2 or 3 MOCKs before the Go Live, adding a MOCK called Dry Run. The transformation rules often change until the last minute of the load to production.

To reduce the risk of the migration project, extract and load the raw data and identify what necessary transformation needs later, guarantee not to lose any information, and adjust the transformation logic easily along the project′s phases.

Data Analysis for the Data Transformation??

To support the development team that did the transformation logic and applied it to the data set, the Data Analyst needs to go deep into the data and understand each business rule for each field in each migration object.

I have prepared a series of articles to guide the Data Analyst to understand the data set, do the analysis, discover the rules, and discuss the findings with the business.

• Exploring the Data
• Prepare the Dimensions
• Do the Fact table
• Resolving the Issue of Sharing the d Data
• Connect the Dots
• Summarize It

Then, these activities will perform a professional analysis that will handle the transformation piece and provide additional support to the conversations between the Business and Development teams.

To perform these analyses and articles, I used the Power BI Desktop, a free Microsoft tool, easy to use and provides many features to perform the data analysis.

I hope these articles are relevant to you and enjoy the content!

About myself

I′m Rafael Cabrera, a results-oriented Project Manager with 15+ years of hands-on experience in identifying and solving business problems, leading workstreams related to order to cash, integration, and data migration in Pharmaceutical, Electrical, Telecom, and Chemical Industries.