Allow "For Each" to execute in sequence, not parallel

[JRitmeijer]

I have to create a Flow that iterates over all files in a folder (DropBox, SharePoint, OneDrive etc) and merge them together into a single PDF file.

I can retrieve the list of files, and the Muhimbi PDF service has the ability to merge files into a single PDF. However, I ran into an issue that took a while to troubleshoot.

My tests indicate that Flow's 'For Each' capability does not execute each iteration in sequence. Instead it executes multiple iterations in paralllel. As a result it is not possible to feed the output of a previous cycle (the merged document so far) into the next cycle.

With the benefits of hindsight it is easy to reproduce. 

1. Create an Array, or something else you can feed into For Each (email attachments, files in a folder etc)
2. Create a 'For Each', and as a first Action add the 'Delay' option and let it wait for one minute
3. As a next action do something that results in a time stamp (send an email, write a file).

For 4 iterations you'd expect a total run time of 4 minutes (due to the 1 minute delay). However, you'll find that all 4 iterations finish roughtly at the same time, after 1 minute in total.

Is this parallel processing a bug or a feature? if it is a feature, can we have a switch to force the iterator to run in sequence rather than in parallel?

[yoshihirok]

"Apply to Each" seems Parallel in my test flow.

Test abstruct:

  3 Loop with deley 10 seconds each, check end time of each

Test result:

  Deference end time of each loop within few seconds instead 10 second.

  This result seems to Parallel instead of Sequence.

Test flow:

  Trigger : Mannaly tigger a flow

  Action: "split" by Compose "@split( '1,2,3', ',' )" // for create a array [ '1', '2', '3' ]

  Apply to Each

   Action: Deley - 10 seconds.

   Action: "item" by Compose "@item()"

   Action: "utcnow" by Compose "@utcnow()"

   Action: "EachTime" by Compose "@formatDateTime( outputs('utcnow'), 'mm:ss')"

 Action: "EndTime" by Compose "@formatDateTime( utcnow(), 'mm:ss')"

Regards,

Yoshihiro Kawabata

[Stephen]

With a recent change all flows run in sequence, not parallel. We will be adding an "opt-in" to the parallel behavior in the future.

[yoshihirok]

Thank you @Stephen for running "Apply to Each" in Sequence, not parallel.

and will be adding "opt-in" parallel.

I will verify the Sequence run of "Apply to Each", 

and I post two ideas "Max of parallel" for limit parallel count, "Exit from Apply to Each" for exit from apply loop.

1. Idea: Max of Parallel in "Apply to Each".

https://powerusers.microsoft.com/t5/Flow-Ideas/Max-of-Parallel-in-quot-Apply-to-Each-quot/idi-p/3075...

2. Idea: Exit from "Apply to Each"

https://powerusers.microsoft.com/t5/Flow-Ideas/Exit-from-quot-Apply-to-Each-quot/idi-p/30760#M1733

Regards,

Yoshihiro Kawabata

[JRitmeijer]

I see this has been marked as completed, but I am still seeing unexpected behaviour. Does that mean it hasn't been rolled out yet?

[JRitmeijer]

I see this has been marked as completed. However, I am still seeing unexpected behaviour, does that mean it has not yet been rolled out?

[yoshihirok]

I share my sample flow for sequence Apply-to-each.

The result get 5 notification mails 1 - 00:00, 2 - 00:10, 3 - 00:20, 4 - 00:30, 5 - 00:40 in sequence.

Regards,

Yoshihiro Kawabata

[JRitmeijer]

Hi @yoshihirok,
 

Are you on the Microsoft Flow team? If so, what is the easiest way to share a test workflow with you as it is too complex to simply describe.

[yoshihirok]

Hi @JRitmeijer

No, I am not on the Microsoft Flow team,

I like Microsoft Flow, and Microsoft Flow team response.

Mee too, I want to share test workflow to simply describe.

I will try "Templates" for share.

Microsoft Flow team do it by "Templates". like "Check out the template here to try this yourself."

In the blog https://flow.microsoft.com/en-us/blog/html-tables-csvs-computer-vision/

Regards,

Yoshihiro Kawabata