cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Frequent Visitor

merging array safely within parallel loop

Hi,

Within Apply for each action I have another one that retrieves an array of objects. I want the loop to have concurrency enabled and need to merge all the retrieved arrays into a single one.

My solution is a popular one - have an array variable for results, create compose action to merge current results with the actual loop item (the retrieved "partial" array), set results to the output of that compose.

 

Spoiler
concept of flattening arrayconcept of flattening array

It seems to work fine but I have doubts if that is concurrency/thread-safe solution and will it always work as expected. I can easily imagine that multiple loop executions are halted after the compose action. This could result in overwriting the results when setting the variable by "later" loop execution.

 

I know that it can be achieved by adding another loop with Append to array variable action but I don't like the idea of introducing that loop only for concatenating arrays.

Maybe you know another method to flatten array which is concurrency-safe? Or maybe just more readable (those additional actions like the compose or the loop I'm no a fan...)?

 

 

Kind regards,

Bartosz Domżalski

2 ACCEPTED SOLUTIONS

Accepted Solutions
Community Support
Community Support

Hi @bdomzalski,

 

This method I first met in Jonh Liu 's channel, it indeed could work as appending an array into Compose action within an Apply to each and get output of the Compose could get an array, and there is no problem with it.

There is no document to introduce this method, it is a trick.

Generally, when we combine the array, we always use the Append to array action, it is standard and not get an error.

Otherwise, there is no other better way could do it like compose as far as I know.

There are all functions in Flow:

https://docs.microsoft.com/en-us/azure/logic-apps/workflow-definition-language-functions-reference#c...

 

Best Regards,
Community Support Team _ Lin Tu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

Hi @v-litu-msft,

 

I just learn in practice that this approach (merging arrays in two steps: compose a union of a variable gathering final result with a partial result and in the next step set that variable) does not work correctly within loops with concurrency enabled!

Results are unpredictable because of the parallelism and might not contain all the partial results.

Spoiler
For-each loop concurrency enabledFor-each loop concurrency enabled

So I guess we are left "Append to array variable" within an additional loop (on the partial result collection).

 

 

Kind regards,

Bartosz Domżalski

View solution in original post

3 REPLIES 3
Community Support
Community Support

Hi @bdomzalski,

 

This method I first met in Jonh Liu 's channel, it indeed could work as appending an array into Compose action within an Apply to each and get output of the Compose could get an array, and there is no problem with it.

There is no document to introduce this method, it is a trick.

Generally, when we combine the array, we always use the Append to array action, it is standard and not get an error.

Otherwise, there is no other better way could do it like compose as far as I know.

There are all functions in Flow:

https://docs.microsoft.com/en-us/azure/logic-apps/workflow-definition-language-functions-reference#c...

 

Best Regards,
Community Support Team _ Lin Tu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

Hi @v-litu-msft,

 

I just learn in practice that this approach (merging arrays in two steps: compose a union of a variable gathering final result with a partial result and in the next step set that variable) does not work correctly within loops with concurrency enabled!

Results are unpredictable because of the parallelism and might not contain all the partial results.

Spoiler
For-each loop concurrency enabledFor-each loop concurrency enabled

So I guess we are left "Append to array variable" within an additional loop (on the partial result collection).

 

 

Kind regards,

Bartosz Domżalski

View solution in original post

Hi @bdomzalski,

 

If it is not a very complicated process, for example, there is only one compose, concurrency test failure as you said, I have not encountered it. If it is merging array, using variables is more stable.

 

Best Regards,
Community Support Team _ Lin Tu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
PP Bootcamp Carousel

Global Power Platform Bootcamp

Dive into the Power Platform stack with hands-on sessions and labs, virtually delivered to you by experts and community leaders.

secondImage

Power Platform Community Conference On Demand

Watch Nick Doelman's session from the 2020 Power Platform Community Conference on demand!

MPA Community Blog

Power Automate Community Blog

Check out the community blog page where you can find valuable learning material from community and product team members!

Top Solution Authors
Users online (6,671)