The New Era of Integration Architecture Options

This week I had the pleasure of presenting at the danish Innovation Days for Salesforce, and in one of the sessions there was a lot of interesting conversations, with questions a long the lines of "we already have a unified integration and API platform that has a copy of all our data - do we need this?" and "we have our data warehouse, what's the value add here?", and as the conversations were unfolding, it became more and more clear, that we're entering the next era of integration capabilities, which we all need to wrap our heads around.

From my vantage point, there's essentially three aspects that are converging and has lead us to the next era:

• The rise of ZeroCopy as a pattern, first pioneered by Salesforce, it's now a common offering across a lot of the major players. Essentially it's connectivity built between two vendors, which eliminates the need to physically move data. (Use case dependent of course)
• The shattered barriers for Autonomous Agents, sees technology reach a readiness stage for launching agents. There is no looking back. It will by humans + agents moving forward and this will have ripples effects on how we need to architect and the speed we need to move.
• The cost of DIY for AI has elevated, as most of us are not the next ChatGPT etc. - we will need evaluate what we build (incl. all of the compliance, governance, toxicity detection, inspection capabilities etc.) and what we buy, and how we weave these things together.

These 3 movements has hit us one-by-one and all at once at the same time. Before we look into the implications of this, I feel it's prudent to appreciate the history that went before us - we are only looking further, because we're standing on the shoulders of giants, as smart people have said before us.

A look back in through the history of integration capabilities:

1. Point-to-Point Integrations (1960s–1980s)

• Early integration approaches were simple "point-to-point" connections, where systems were directly linked through custom-coded connections. The lack of standardized protocols meant each connection required custom work, which was challenging to scale.

2. Message-Oriented Middleware (1980s–1990s)

• As the number of systems grew, point-to-point integrations became cumbersome, so middleware solutions were introduced. Message-oriented middleware (MOM), such as IBM MQ Series, enabled systems to communicate via messages without direct connections. This asynchronous communication style allowed systems to send and receive messages in a decoupled manner, reducing integration complexity.

3. Enterprise Application Integration (EAI) (1990s–2000s)

• EAI emerged as a framework to centralize integration, often through a "hub-and-spoke" model, where a central system (hub) managed all connections. This era also saw the rise of business process orchestration, where integrations were designed to automate entire workflows across systems.

4. Service-Oriented Architecture (SOA) (2000s)

• SOA introduced a significant shift by encouraging services that could be reused across applications. Web services (e.g., SOAP) and APIs became prominent, enabling more standardized, protocol-based communication between systems.

5. Event-Driven Architectures and APIs (Late 2000s–2010s)

• Real-time data exchange became essential, leading to event-driven architectures (EDAs), where systems could respond to changes instantly. APIs became the primary integration method, especially with RESTful services, allowing lighter, web-friendly communication. Tools like Apache Kafka enabled high-throughput, event-based streaming for real-time data synchronization.

6. Cloud Integration and iPaaS (2010s–Present)

• The shift to cloud computing required new integration methods, leading to the development of integration Platform-as-a-Service (iPaaS) solutions. iPaaS tools, like MuleSoft, Dell Boomi, and Azure Logic Apps, provide pre-built connectors and manage the infrastructure, making it easier to integrate cloud and on-premises systems. This era also saw the rise of microservices, which necessitated even more flexible, scalable integration techniques.

7. API-First and Data-Centric Architectures (2020s)

• Companies increasingly adopt an "API-first" approach, designing APIs before building applications, which enhances integration. Data-centric architectures, such as data lakes and data mesh, treat data as a shared asset, making it easier to integrate and analyze data across platforms. With the rise of artificial intelligence, integrations now also involve streaming large datasets in real-time and implementing AI-driven data pipelines.

The Implications for the Next Era

In order to have the business capabilities needed, and in order to be able to be in the lead of the future, what the key questions we need to answer as we move ahead is the following:

Question 1: What's our strategy for maximising AI output?

• The underlying ask here, is that you need a harmonized data view (i.e. contacts mapped to the same understanding of what a contact is, or same representation of an order etc.), and this harmonized data view needs to be accessible to your chosen agent platform/AI platform, with the appropriate governance, compliance, auditing, inspection capabilities available.

Question 2: What data do we move, and what do we connect?

• In the future, moving data is optional. There might be reasons for it. But carefully inspect these patterns as you move forward. You will be comparing "storage costs" with "compute costs", and the price of middleware, maintenance of integrations etc. - it's a new pattern, which means that the right, most efficient pattern for you as a company, might not be the same as they used to.

The Closing Note

Above is of course biased. I work with Salesforce's Data Cloud every single day. I see the trust layer in action. I see the data harmonization capabilities in action. I see how this is empowering and enabling AI use cases. I see how this becomes the right foundation for agents to be empowered with the right information.

But... the other side of the coin... I also see a lot of confusion or wondering, when it comes to these new capabilities as we're all trying to wrap our head around the next era of technology.

From the inside, my closing remark, I can't wait! It's going to be amazing to be augmented by technology even more!