Let's End the Infinite Cycle of Copy and Paste

We need to stop copying and pasting data and content all over the place. Let me explain...

When you’re assembling digital experiences within a composable stack, you need data and content sourced from a variety of backends. To simplify the process of building frontends, oftentimes, organizations end up syncing data from many siloed systems into one or two systems, such as a CMS and a Commerce Platform.? For instance, you may pull product information into your CMS since the CMS is the system of record for content required for your experiences. Or, in some cases, you may be copying content from one siloed space within a CMS to another siloed space within a CMS.? Another example is the DAM.??

In the composable landscape, the role of the CMS is slowly evolving and so should the practices that surround it.? The digital experience is composed of data and content that doesn’t all live in a single CMS. There is a certain level of 'humility' that comes with composability - I'll borrow that term from Sam Brace responsible for Customer Education at Cloudinary.

Keeping these systems in sync is no simple task.? Data is being created and updated in each of the systems of record on an ongoing basis.? Companies often use expensive integration platforms or middleware to automate the syncing of data between backend systems. Real-time data synchronization is typically achieved through the use of APIs (Application Programming Interfaces) and webhooks.? Apart from the fact that the data from other systems of record such as product information or digital assets can easily become inconsistent due to this copy/paste process, there are a couple of other reasons why this is a bad idea.

1. It severely increases the complexity of the overall stack.? Now you have to manage integrations between multiple systems and syndicate data between these systems.? This creates tight coupling between systems as the data needs to be transformed and mapped to structures of the target platform.? Anyone who is doing this right now, knows the headaches associated with that.
2. You’re treating the backend systems like a dumb data store and many times, these backend systems have business logic and capabilities that go far beyond that.? For instance, Cloudinary offers real-time transformation and optimization APIs.? If you simply copy and paste the images and videos from Cloudinary into your CMS, you’ve just lost all of that.
3. What about Legacy systems?? They exist and they’re not going away anytime soon.? They hold data that is essential to build an experience, but are not easy to integrate with.? They also don’t need the extra load of reading/writing data as they were never built to scale this way.?

Is there another option?

Now, remember, we’re talking about a composable architecture, which means that all of the backend platforms are accessible via real-time APIs and most modern headless/composable platforms offer APIs to expose the data and content within. I’d like to make the case that 90% of the time, you shouldn’t sync this data into one or two ‘key’ systems. Instead, you can simplify this architecture by orchestrating the data and the content in real-time.? This means that you are asking for the data from the backend system only when it is needed.

DXO, API Gateways, and GraphQL

There are many options available in the market to handle this problem in a more graceful way.? These include building an API Gateway, using GraphQL, and now, the Digital Experience Orchestration.? Although I have reasons to believe (and I’ve argued this in another post) that DXO is the right approach, all three approaches share the following benefits:

This approach provides the following benefits:

• It simplifies the process of accessing data from multiple systems, and reduces the amount of code required to build integrations between systems.
• Provides a central point of access to multiple APIs and allows clients to access data from multiple systems through a single API endpoint.?
• With proper caching in the orchestration layer, you avoid having to put too much strain on the legacy backend systems and can gain speed, scalability and performance.
• You can ensure that data is up-to-date and consistent across all systems because you’re always accessing it in real-time.
• Instead of spending massive amounts of time, resources and money on this initiative, you can get started with real-time orchestration in a fraction of the time.

?Now, this approach is not for everyone and in every use case.? If all your content is sitting in a single CMS and is organized perfectly, you don’t need to concern yourself with this at all.? I’m only providing an option for organizations where data and content is scattered and siloed.

Additional thoughts….

• There is another line of thinking which is to cache all the data from all the backend systems into one content repository.? There are pros and cons to this approach that are very similar to what I’ve discussed above.? However, there is a case for a central knowledge graph, which can act as an operational data store in certain instances.? I won’t go into those here as that’s a different topic.? However, in principle, I would say that any backend system that is API accessible, headless, performant and scalable should be accessed in real-time where possible.
• For years, the CMS has always been seen as the center of the digital stack.? Over the years, the market has been grappling with various movements, the most important in my opinion being, headless, MACH and composable.? This shift in architecture will likely have an impact on the roles of digital experience practitioners as well as the workflows involved in assembling experiences.? A very well thought out article was recently published by John Collins , head architect at Atlassian.? https://www.dhirubhai.net/pulse/2023-content-trends-roles-specializations-john-collins/