cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Frequent Visitor

merging array safely within parallel loop

Hi,

Within Apply for each action I have another one that retrieves an array of objects. I want the loop to have concurrency enabled and need to merge all the retrieved arrays into a single one.

My solution is a popular one - have an array variable for results, create compose action to merge current results with the actual loop item (the retrieved "partial" array), set results to the output of that compose.

 

Spoiler
concept of flattening arrayconcept of flattening array

It seems to work fine but I have doubts if that is concurrency/thread-safe solution and will it always work as expected. I can easily imagine that multiple loop executions are halted after the compose action. This could result in overwriting the results when setting the variable by "later" loop execution.

 

I know that it can be achieved by adding another loop with Append to array variable action but I don't like the idea of introducing that loop only for concatenating arrays.

Maybe you know another method to flatten array which is concurrency-safe? Or maybe just more readable (those additional actions like the compose or the loop I'm no a fan...)?

 

 

Kind regards,

Bartosz Domżalski

2 ACCEPTED SOLUTIONS

Accepted Solutions
Highlighted
Community Support
Community Support

Re: merging array safely within parallel loop

Hi @bdomzalski,

 

This method I first met in Jonh Liu 's channel, it indeed could work as appending an array into Compose action within an Apply to each and get output of the Compose could get an array, and there is no problem with it.

There is no document to introduce this method, it is a trick.

Generally, when we combine the array, we always use the Append to array action, it is standard and not get an error.

Otherwise, there is no other better way could do it like compose as far as I know.

There are all functions in Flow:

https://docs.microsoft.com/en-us/azure/logic-apps/workflow-definition-language-functions-reference#c...

 

Best Regards,
Community Support Team _ Lin Tu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

Highlighted
Frequent Visitor

Re: merging array safely within parallel loop

Hi @v-litu-msft,

 

I just learn in practice that this approach (merging arrays in two steps: compose a union of a variable gathering final result with a partial result and in the next step set that variable) does not work correctly within loops with concurrency enabled!

Results are unpredictable because of the parallelism and might not contain all the partial results.

Spoiler
For-each loop concurrency enabledFor-each loop concurrency enabled

So I guess we are left "Append to array variable" within an additional loop (on the partial result collection).

 

 

Kind regards,

Bartosz Domżalski

View solution in original post

3 REPLIES 3
Highlighted
Community Support
Community Support

Re: merging array safely within parallel loop

Hi @bdomzalski,

 

This method I first met in Jonh Liu 's channel, it indeed could work as appending an array into Compose action within an Apply to each and get output of the Compose could get an array, and there is no problem with it.

There is no document to introduce this method, it is a trick.

Generally, when we combine the array, we always use the Append to array action, it is standard and not get an error.

Otherwise, there is no other better way could do it like compose as far as I know.

There are all functions in Flow:

https://docs.microsoft.com/en-us/azure/logic-apps/workflow-definition-language-functions-reference#c...

 

Best Regards,
Community Support Team _ Lin Tu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

Highlighted
Frequent Visitor

Re: merging array safely within parallel loop

Hi @v-litu-msft,

 

I just learn in practice that this approach (merging arrays in two steps: compose a union of a variable gathering final result with a partial result and in the next step set that variable) does not work correctly within loops with concurrency enabled!

Results are unpredictable because of the parallelism and might not contain all the partial results.

Spoiler
For-each loop concurrency enabledFor-each loop concurrency enabled

So I guess we are left "Append to array variable" within an additional loop (on the partial result collection).

 

 

Kind regards,

Bartosz Domżalski

View solution in original post

Highlighted
Community Support
Community Support

Re: merging array safely within parallel loop

Hi @bdomzalski,

 

If it is not a very complicated process, for example, there is only one compose, concurrency test failure as you said, I have not encountered it. If it is merging array, using variables is more stable.

 

Best Regards,
Community Support Team _ Lin Tu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
Community Conference

Power Platform Community Conference

Check out the on demand sessions that are available now!

Power Platform ISV Studio

Power Platform ISV Studio

ISV Studio is designed to become the go-to Power Platform destination for ISV’s to monitor & manage published applications.

Upcoming Events

Experience what’s next for Power Automate

See the latest Power Automate innovations, updates, and demos from the Microsoft Business Applications Launch Event.

Top Solution Authors
Top Kudoed Authors
Users online (7,730)