cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Frequent Visitor

merging array safely within parallel loop

Hi,

Within Apply for each action I have another one that retrieves an array of objects. I want the loop to have concurrency enabled and need to merge all the retrieved arrays into a single one.

My solution is a popular one - have an array variable for results, create compose action to merge current results with the actual loop item (the retrieved "partial" array), set results to the output of that compose.

 

Spoiler
concept of flattening arrayconcept of flattening array

It seems to work fine but I have doubts if that is concurrency/thread-safe solution and will it always work as expected. I can easily imagine that multiple loop executions are halted after the compose action. This could result in overwriting the results when setting the variable by "later" loop execution.

 

I know that it can be achieved by adding another loop with Append to array variable action but I don't like the idea of introducing that loop only for concatenating arrays.

Maybe you know another method to flatten array which is concurrency-safe? Or maybe just more readable (those additional actions like the compose or the loop I'm no a fan...)?

 

 

Kind regards,

Bartosz Domżalski

2 ACCEPTED SOLUTIONS

Accepted Solutions
Highlighted
Community Support
Community Support

Re: merging array safely within parallel loop

Hi @bdomzalski,

 

This method I first met in Jonh Liu 's channel, it indeed could work as appending an array into Compose action within an Apply to each and get output of the Compose could get an array, and there is no problem with it.

There is no document to introduce this method, it is a trick.

Generally, when we combine the array, we always use the Append to array action, it is standard and not get an error.

Otherwise, there is no other better way could do it like compose as far as I know.

There are all functions in Flow:

https://docs.microsoft.com/en-us/azure/logic-apps/workflow-definition-language-functions-reference#c...

 

Best Regards,
Community Support Team _ Lin Tu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

Highlighted
Frequent Visitor

Re: merging array safely within parallel loop

Hi @v-litu-msft,

 

I just learn in practice that this approach (merging arrays in two steps: compose a union of a variable gathering final result with a partial result and in the next step set that variable) does not work correctly within loops with concurrency enabled!

Results are unpredictable because of the parallelism and might not contain all the partial results.

Spoiler
For-each loop concurrency enabledFor-each loop concurrency enabled

So I guess we are left "Append to array variable" within an additional loop (on the partial result collection).

 

 

Kind regards,

Bartosz Domżalski

View solution in original post

3 REPLIES 3
Highlighted
Community Support
Community Support

Re: merging array safely within parallel loop

Hi @bdomzalski,

 

This method I first met in Jonh Liu 's channel, it indeed could work as appending an array into Compose action within an Apply to each and get output of the Compose could get an array, and there is no problem with it.

There is no document to introduce this method, it is a trick.

Generally, when we combine the array, we always use the Append to array action, it is standard and not get an error.

Otherwise, there is no other better way could do it like compose as far as I know.

There are all functions in Flow:

https://docs.microsoft.com/en-us/azure/logic-apps/workflow-definition-language-functions-reference#c...

 

Best Regards,
Community Support Team _ Lin Tu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

Highlighted
Frequent Visitor

Re: merging array safely within parallel loop

Hi @v-litu-msft,

 

I just learn in practice that this approach (merging arrays in two steps: compose a union of a variable gathering final result with a partial result and in the next step set that variable) does not work correctly within loops with concurrency enabled!

Results are unpredictable because of the parallelism and might not contain all the partial results.

Spoiler
For-each loop concurrency enabledFor-each loop concurrency enabled

So I guess we are left "Append to array variable" within an additional loop (on the partial result collection).

 

 

Kind regards,

Bartosz Domżalski

View solution in original post

Highlighted
Community Support
Community Support

Re: merging array safely within parallel loop

Hi @bdomzalski,

 

If it is not a very complicated process, for example, there is only one compose, concurrency test failure as you said, I have not encountered it. If it is merging array, using variables is more stable.

 

Best Regards,
Community Support Team _ Lin Tu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
firstImage

Super User Program Update

Three Super User rank tiers have been launched!

firstImage

Power Platform 2020 release wave 2 plan

Features releasing from October 2020 through March 2021

firstImage

Join the new Power Virtual Agents Community!

We are excited to announce the launch of Power Virtual Agents Community. Check it out now!

firstImage

New & Improved Power Automate Community Cookbook

We've updated and improved the layout and uploading format of the Power Automate Cookbook!

thirdimage

Power Automate Community User Group Member Badge

Fill out a quick form to claim your user group badge now!

Top Solution Authors
Top Kudoed Authors
Users online (8,128)