How do you validate data in ERP implementation?

The common mistake in majority of the ERP programs is to dump all the data as-is from legacy applications to the new ERP system and expect magic to happen. But that's not how it works. 1. Clearly define the volumes of data that is required to migrate into new ERP(for e.g.,last 5 years invoices only) 2. Ensure that the data at source is thoroughly cleansed(for e.g., 10 different part #s might exist in legacy applications for the same part) 3. Get the source data reviewed by business and ensure it is complete 4. Define accurate mappings b/w 2 systems & have data translation rules as per ERP need 5. Generate reconciliation reports to ensure the data migrated is good and in sync b/w 2 systems 6. Run Test scenarios in ERP with migrated data

Begin by creating clear and specific guidelines for the data that must be met. These guidelines will be used to evaluate the quality of the data as it moves from your old systems to the new ERP environment. Make sure these standards match your business goals and can be measured and applied to the tasks that your ERP system will support. Write down these guidelines and share them with all team members and stakeholders involved in the implementation. This will ensure everyone has a common understanding of what qualifies as "good data," reducing the risk of errors or confusion.

Start by making a migration strategy from old to new. The business objectives for why we need "old" data in the new system is the starting point. After that we can analyse what data and in what form will achieve these objectives. If the objectives are to make business processes more efficient by using machine learning algorithms (AI), the data quality needs to be on par to meet those objectives with clear goals and transformation rules of how to transform existing data to the required data quality meeting the objectives.

I'm not into a lot of "bla-bla", but actual doing it. And the result is there: with my team we have been very successfull in datamigration and the quality of data. The reason is that we start fast and migrate many times. The first time data is parsed from the legacy system to the new system without many rules, transformation and cleansing. As the two systems are very different it allows us to have a direct insight into how the old data will work in the new system. Then, steadily we work on improving the quality of the data. And we do this by replicating the companies business processes, because in my eyes it is the only way to validate the data. We repeat this process as many times as needed until we are satisfied with the results.

Define the accepted data picture/format for each condition and field. Be sure to note what is mandatory and what is not. Enforce uniqueness constraints (for example: Customer's ID, which will identify the same record across all data). Avoid unnecessary data migration. Implement audit trails to track the old and new ERP modifications of critical data.

The secret lies in doing the datamigration many times, and then run full end to end business process testing. (See also my remark about data quality). With my team we have done this very sucessfull many times, over and over again. In other words: we do not "theorize" too much, but actually make it happen. This means that one can only find out how the data profile rules will need to be when one first does the actual migration (many times over and over) and run full cycle testing rounds with the business users.

From previous experiences extracting data from legacy systems and profiling data, I've found a big opportunity to use machine learning to define algorithms that use pre defined business rules to assure data quality and completeness before loading in the new system or ERP. Currently I'm studing & researching how to do it for future projects.

A major task after data profiling is data cleansing: This involves correcting, standardising, and enriching the data to improve its quality. This can be done manually or using automated tools with predefined rules, algorithms, and reference data.

This is very important and personally I would recommend to implement strict rules on the data validation. Yes, Its a part of the IT team to extract the correct data. Responsibility lies within the actual business users who validates the extracted data. For Example, Do we really need to Extract Customers who discontinued our agreement 10 years ago.

It is 10 step process, that is applicable to small to large companies.. 1 Create Business Process Analysis and Design documentation. 2 Configure ERP Software Installations and requirements. Technical mapping document 3 Perform Data Migration 4 Perfrom Data Governance, ETL and data cleansing 5 Business process integration 6 Validate Source vs Migrated Data accuracy 7 Perform UAT by Champions/selected SME 8 Train users on ERP and train large audience locally/globally, as needed 9 Review Implementation, perform retrospective, lessons learnt. 10 Let ERP implementation go from Local UAT Users to Global settings.

Well, what more can I see then what I've said in my previous remarks on the other related topics. It's not about "bla-bla". It's about doing it. Systems are different, just as companies and their business processes. There is no golden template, no set of "golden rules". Each ERP implementation, just as its people are different. Therefore you learn by doing it and making mistakes. However, when you start from day one with doing a full migration without too many boundaries and rules, and then immediately start end to end testing with the most crucial business scenarios, you'll learn fast. Faster than those who like to theorize for 6 months and then theorize more another 6 months. Migrate and test many times.

I can see discussing data validation for user entries is one of the major activities that should be considered during collecting business requirements phase

This is just a part of how we implemented a process that really works. Test, and test many times, and combine it often with full datamigration cycles. This way you catch multiple birds in one go. Not only test data by considering it as an isolated process, but combine it to see how the data works with your end to end business processes. And do this from the get go. As a proof: we've helped customers implement their ERP system in under 3 months!

Data testing is a fundamental pillar of data management and analysis, underpinning the assurance of data accuracy and reliability. It encompasses a multifaceted approach involving the meticulous assessment of data quality, validation against predefined criteria, and verification through comparisons with trusted sources. Beyond these core facets, data testing extends its purview to scrutinizing completeness, assessing data transformations, ensuring seamless integration, and maintaining a vigilant stance on data security and privacy. The deployment of automated testing tools further enhances the efficiency of these processes, preserving data integrity and elevating the dependability of data-driven decision-making.

Making quality contributions in Enterprise Resource Planning (ERP) involves several key aspects that can help enhance the effectiveness and value of ERP systems within an organization. Key Contributions in ERP: *Understanding Business Processes and ERP System Knowledge *Data Management and Integration *Process Improvement and Continuous Learning *Change Management and Project Management *Project Management Collaboration and Training *Troubleshooting and Support *Security and Compliance and Documentation *Analytics and Reporting *Cost Management In summary, making quality contributions in ERP involves a combination of technical expertise, business process understanding, communication skills, and a commitment to continuous improvement.

Validating data after an ERP implementation is crucial to ensure the accuracy and functionality of the system. First perform comprehensive data mapping to ensure all data fields from the previous system are correctly mapped to the corresponding fields in the new ERP. This helps identify any discrepancies or missing information. Secondly, conduct thorough data reconciliation by comparing data between the old and new systems. This includes validating critical information such as inventory levels, financial records, and customer data. Additionally, perform end-to-end testing of key business processes using sample data to ensure the ERP functions as intended.

I fully agree with Mike's comments. In addition, where there are significant data mapping changes from the legacy data to the new eg an extension/constriction of the chart of accounts, customer groups, cost centres etc I have found it useful to transform sample legacy data (through a BI tool or even Excel) and provide reports to the business to demonstrate what the remapping results will be. In this way the remapping exercise can be validated in line with their expectations of the outputs. Mapping tables can be quite complex and mistakes/misinterpretations may not be identified until the data has been migrated. So, providing visualisations at an early stage can save time and effort later on.

Benefits of Data validation is like drinking filtered water (which is processed through carbon filters to remove the chlorine (which improves the taste) and sometimes a micron filter as well. After the filtering, it is ozonated and bottled) vs uncleaned from Ocean.

Using the method I described in the two previous articles, the benefits are very rewarding. We've been able to cut implementation time drastically and have go live moments that are so succesfull that the hypercare phase passes with very little issues. The reason is that it is all about repetition and many cycles of end to end testing with the users. Having them involved deeply from the start, not only invokes their attention and acceptance towards the new system, it also improves not only the quality of data but also the understanding of how their business processes work in the new system.

Attack your known data quality issues that create business risk before your replacement project begins. Using these techniques to reduce your risk now will pay off with a more seamless implementation with fewer stops for data emergencies during the project (assuming you fix the processes that caused the bad data at the same time). Most business leaders know where their data is creating risk, but many believe they don't have the time or money to address them now.

Adding others which are of immense value as Data is the flesh and blood of a good system -MDM -Data Governance -DQ -Data Assurance -ETL

Validation of data converted or imported in new system can be compared with legacy / old system at spreadsheets level using pivot tables and formulae, for example, amount and count values at a very high summary level. Similarly, if data is imported in stages, for example, first in temporary tables and then final tables, then such validation or comparison can be done using SQL queries.

Keep this checklist in mind: 1- Align with Business Goals: Ensure data quality supports ERP objectives. 2- Engage Stakeholders: Collaborate for input on expectations. 3- Data Sources: List ERP integration data sources. 4- Define Critical Data Elements: Identify key data elements. 5- Specify Quality Dimensions: Include accuracy, completeness and timeliness. 6- Set Standards: Establish clear, measurable criteria. 7- Align with Processes: Ensure criteria fit existing workflows. 8- Consider Compliance: Consider regulations if applicable. 9- Prioritize Data Elements: IE by operational impact. 10- Communicate Across Teams: Ensure stakeholder awareness. 11- Test and Refine: Validate criteria with sample data and adjust as needed.