Salesforce Streaming API and Mulesoft: an alternative solution to keep data synchronized

Large companies that adopt Salesforce as official CRM, usually need to keep other systems synchronized with information inserted on it. Even before the acquisition of Mulesoft by Salesforce, companies used to combine both solutions to keep their systems up to date. Mulesoft has became very famous in combination with Salesforce mainly because of its cloud architecture (Cloudhub) and connectors that make the integration very easy to be done.

Over the years, I could see some approaches to keep data synchronized from Salesforce to other systems using Mulesoft:

• Creation of outbound message on Salesforce sending the record to an API exposed on Mulesoft
• A custom implementation in Apex with "@future" annotation to be called inside a trigger, sending the record to an API exposed on Mulesoft
• A daily scheduled job in Mulesoft querying for all objects that have some changes on the same day

All the above solutions may work in a certain way with pros and cons, however it is very challenging to make it reliable especially in case of errors on the side of Salesforce. A "retry" mechanism need to be thought of, otherwise there is a risk of losing records.

Salesforce Streaming API

Streaming API enables you to expose near real-time stream of data from Force.com platform. It delivers events that are either tied to changes in Salesforce or based on custom payloads.

These events can be received by:

• Pages in the Salesforce application.
• Application servers outside of Salesforce.
• Clients outside the Salesforce application

It's reliable

Streaming API enables consumers to replay message in case of problems during a period of 24 hours. With this durable streaming, messages aren't lost when a consumer is disconnected for example.

To help subscribers to replay messages, by the time they receive a message from Salesforce, it comes with "ReplayId" field. It refers to the position of the event in the event stream.

Example of payload received by subscribers:

{
 "clientId":"a1ps4wpe52qytvcvbsko09tapc",
 "data":{
    "event":{
            "createdDate":"2016-03-29T19:05:28.334Z",
            "replayId":55
        },
    "payload":"This is a message."
    },
 "channel":"/u/TestStreaming"
}

Creating a Topic

Below, is an example (Apex Code) of how to create a topic named "AccountUpdates" that will push Id, Name, Type, BillingStreet, BillingCity, BillingState, BillingCountry, Phone, AccountNumber, Industry when Accounts are created, updated or deleted:

pic pushTopic = new PushTopic();
pushTopic.Name = 'AccountUpdates';
pushTopic.Query = 'SELECT Id, Name, Type, BillingStreet, BillingCity, BillingState, BillingCountry, Phone, AccountNumber, Industry FROM Account';
pushTopic.ApiVersion = 45.0;
pushTopic.NotifyForOperationCreate = true;
pushTopic.NotifyForOperationUpdate = true;
pushTopic.NotifyForOperationDelete = true;
pushTopic.NotifyForFields = 'Referenced';
insert pushTopic;

Subscribing to a Topic in Mule 4

Let's see how "hard" it is to build a Mule flow that subscribe to a Salesforce Streaming API topic. I build the following flow that will perform an operation to a database according to the changes made on accounts.

You only need to drag the operation "Subscribe topic" from Salesforce Connector and drop on "Source" of your flow. Below, the simple configuration of the Salesforce Connector:

After testing your connection successfully, you just need to inform the topic name and voi la! Any operation performed on Accounts, will be pushed to the Topic and your Mule flow will read it almost instantly.

The Mule Payload will contain the information(as a JSON) selected on the query that you defined on the creation of the topic:

SELECT Id, Name, Type, BillingStreet, BillingCity, BillingState, BillingCountry, Phone, AccountNumber, Industry FROM Account

The information like replayId and the operation performed (inserted, updated, deleted...) will be available on "Attributes" of the Mule Message:

In this flow, I used the "type" information on my "Choice" operation to defined what have to be done on the database.

Easy, isn't it?

So, do not discard this approach if you need to send information from Salesforce to legacy systems when Mulesoft is part of architecture. There's almost nothing to be done on Salesforce side and you can take advantage of this reliable pattern.

Yet, don't forget to get a look into the documentation and understand the limits of streaming apis before using it on your solution.