You're facing a company merger. How do you guarantee smooth data flow across different systems?

During a company merger, guaranteeing smooth data flow across different systems requires a thorough assessment of both organizations' data architectures. Start by conducting a comprehensive audit of the existing systems, identifying data sources, formats, and integration points. Evaluate the compatibility of databases, data models, and data governance policies to understand potential conflicts and synergies. Develop a unified data integration plan that includes data mapping, transformation rules, and synchronization mechanisms. Prioritize data security and compliance throughout the process. Engage stakeholders from both companies early to gather insights and foster collaboration.

If you don't already have a CMDB in place in both organizations then the first thing to-do is to get a CMDB in place. Now it is time to hop into excel and for each CMDB ID application import as much about it you can. What you want to have is name, description, business owner, data owner, data sensitivity information at a minimum to start. From there you need to come up with a program where you can run workshops with each of the application teams to review their existing setup and their existing ability to import/export data and if so in which format. If not what it would take to support a standard format you come up with that every system will get mapped too.

Once you have the assessment done you should by and large have the requirements you need to build the plan. The critical path for this plan intersects with the application(s) that will take the longest. You want to prioritize work around those first. Next, systems that fit into a standard pattern should get knocked out. Lastly, any system left over which should be some hand holding and little edge cases here and there. Once you develop this plan for all the system have an entire meeting with everyone involved and let them know not only what they are responsible for but what everyone else is responsible for and by when. This should be very high level talking about milestone and not details of the plan. Having burn down charts are helpful.

Conducting a thorough data audit to understand the existing data landscape. Develop a data mapping strategy to align data structures, formats, and semantics between the merging entities. Implement data integration tools to automate the mapping process, reducing manual errors. Establish data governance policies to maintain data quality and consistency. Regularly monitor and validate data flows, addressing discrepancies promptly. Effective communication and collaboration between IT teams are essential to adapt to evolving requirements and ensure seamless data integration.

If possible do this in code and don't have documentation lying around about what maps to what. Create a new translation layer that fits into the entire data ecosystem that lays out a new schema (use something like Apache Avro) and have application mappings to the Apache Avro schemas sitting in code in your git repositories.

You must evalute points like performance, scalability or fault tolerance aspect from the data architecture because you integrate new data source on the data environment! In general; all test like unit, integrate or UAT test support some features like memory or time; complement it with monitoring tool to verify that data architecture can support the new data source in current status, require to add new resources or detect new requirements to suppor it.

There should be at least the following layers around each system of the application. - Unit Tests (inclusive of data mapping tests) - Integration Tests with the import/export system flow. - Synthetic Data Tests with the import/export system flow. - UAT (Production data) Tests with the import/export system flow.

This is a critical phase where all the planning and preparation come into action. Verify all systems are ready and backups are in place Select a time that minimizes disruption to business operations Communicate the schedule clearly to all affected parties. Begin the process of extracting data from source systems, validate them, apply any necessary transformations and load them into target systems. Run predefined tests to verify the migration's success Check data totals, record counts, and sample records for accuracy The execution phase requires close coordination and quick problem-solving. It's crucial to have experienced team members available throughout the process to address any issues that may arise.

To maintain continuity post-merger, establish a centralized data governance committee that oversees ongoing data management, including compliance, security, and quality. This committee should regularly review and update data policies to reflect the needs of the merged organization. Encourage cross-functional collaboration to identify and address any emerging data challenges. Continuous training and user feedback loops will help refine the system and improve user satisfaction. Additionally, plan for periodic system evaluations to ensure that the data architecture remains aligned with business objectives and technological advancements.