What to do with historical data during migration?

Are you getting scared about migrating historical transactions? How can we successfully untangle all the complexity and inconsistency of the legacy data? Don't panic! With following approach you can achieve successful results in very reasonable time.

Purpose of data migration

If the main purpose of the data migration is to get enough information to get the business up and running on a new system then following approach can help you. First of all, get a clear vision of which information is critical. In most cases, a new ERP system needs only consolidated summary of latest transactions. We don't need to waste time on bringing all the years of data in. There are many approaches how to deal with that kind of information outside of the main ERP database. This simple approach includes only critical transactional data needed for go live.

Quality vs quantity

Quality of data is essential. We all know that. Quantity of records should be minimized to achieve fast and successful data migration. Information should become simple and meaningful. Summarized data can be easily verified and reconciled. Having small quantity of trusted data makes migration faster and less risky. This allows to deliver critical information in a very tight schedule before go live.

Can we just copy those transactions?

Is it possible to copy historical transactions into a new system? The answer is NO. The transactional data from one system cannot be replicated exactly in the other system. All details which make sense in one system (like source of transaction, references to journals, business documents, master data, configuration) are not available in the new system. So we should forget about 1:1 migration.

Consolidated data makes more sense

To understand what transformation is required to prepare legacy data for upload we should get a perfect vision of the expected results. Which data is needed for critical reports? How many details must be there? Will summary be good enough?

ERP systems allow to bring consolidated information through journals. This approach requires grouping of legacy transactions based on certain criteria. Only details which are going to be used in critical reports should be considered. Then summarized information can be reviewed based requirements of the new system. Conversion of summarized data is done with new business rules. After journals are successfully uploaded the new system will have minimum required information needed for reports and operation.

Can we trust migrated data?

Bringing transactional data to the new system is worthless if we cannot trust results. So how shall we check that all information is correct? After all the transformations we need to make sure that results can be trusted. For that reason we need the reconciliation stage.

Business users should understand their data. They should have a bunch of reports where summarized information can be reviewed. Can we compare reports from different systems? Usually reports will not match exactly on the line level. For example, a trial balance in a new system will not match exactly on the line level due to a new structure of accounts. But totals should match. The financial team will need to map new accounts back to legacy ones in order to verify line details. Review and fixing of issues on the line level is easy when data is consolidated. After successful reconciliation of one journal most issues of data conversion will be discovered. The next journals will take fraction of the time to migrate and verify.

Summary

Old transactions are part of data migration requirements. They are not very nice to deal with until we understand what information is valuable for the new system. Consolidating thousands of lines into meaningful summary makes data migration easy. Using journals and business rules of the new system allows to upload transactions which can be used in reports. And finally, make sure that your data is reconciled and verified properly.

If you have any questions about migration of transactional data I'm happy to help. Send me a message.