How to Build More Efficient Flows (Part 3): In defence of Apply to each

If you have been following this series, you might think that I am suggesting that you never use Apply to each. However, this is not the case. In this article, we will learn when and how to use it more effectively.

Apply to each is best used with non-data manipulation actions. These actions are mostly used to move data in and out of your cloud flows and cannot be used directly with the Select action. They include connector-based actions like Get items and HTTP-based actions like HTTP.

Often, we need to use these actions for each item in an array. Apply to each (ATE) is perfect to use here because it has its own “scope” so can accommodate these actions. However, when the required array is built up in the same loop, we run into two efficiency problems:

1. Actions that are necessary to build up the array inside of the loop can significantly increase API consumption.
2. The chances of using nested ATE actions are increased.

Effective Apply to each

So, the array should be manipulated and shaped outside of an ATE. Consider the image below.

Every action in the red box is used to prepare the data before being used as input to the ATE.

Let’s consider the ATE in the image below.

The ATE is simplified and focuses on the HTTP request action. Note the use of a Compose action rather than a variable.

Compose, variables & parallelism

Use Compose instead of variables in ATE actions because the former works with parallelism better. Parallelism means that each iteration of an ATE loop does not need to wait for the previous iteration to finish execution before it can start. This greatly improves the loop’s run time.

To take advantage of parallelism, turn on the Concurrency control in the settings of the loop.

When determining the "degree of parallelism", it's important to consider the throttling limit of the actions being used. This is because if more requests are made in a short period, it can overwhelm the service.

Additionally, variables may not work well with parallelism because they are "shared" data stores. This means that if one iteration locks the variable for use, other iterations running in parallel have to wait in a queue to access it, causing a bottleneck. Loops implemented in this way may perform worse than those with concurrency control turned off.

Compose on the other hand seems to multiply itself with a copy for each iteration. Another neat trick of the Compose action is that it can be accessed outside of the loop as an array of the values stored in the loop. It won’t show up, however, under Dynamic Value; so, you would have to write a simple expression to reference it.

outputs('name_of_your_compose_action')