Flow timing out on internal processing

[mjburley]

I have a Flow that does a List Rows on a SharePoint list (circa 2000 rows).

It brings in one column from that list as defined by the view, and this action takes 2 seconds or so.

Now I then need to get a distinct list of IDs returned, so I assumed a simple throw the IDs into an array and do stuff would work well enough.

But the throwing the IDs into an array (append to array) always times out after 10 minutes or so.

I thought as this data was all within the Flow scope, it would be a few seconds, which implies to me that it is doing a network fetch on every row again even though it has the data 'locally'?

Am I being more stupid than normal here ? It's the heat.

Simplified Flow:

SharePoint list defined view 'Ids'. Document ID is a look up to another SharePoint List. 

[annajhaveri · Accepted solution]

@mjburley I would suggest to refer to solution as mentioned in this post https://powerusers.microsoft.com/t5/Power-Automate-Community-Blog/Get-Unique-Values-from-SharePoint-... to get unique values from SharePoint list, it is a better way rather than looping through each item using apply to each.

[eliotcole · Accepted solution]

To expand slightly on @annajhaveri completely correct answer, @mjburley ... you essentially need to perform the following (this relies on your DocumentID column still having that name) :

1. (optional) Ensure that you're doing everything you can to filter within the Get items.
   
   
2. Add a Select action after the Get items, which runs on the value array from that.

   (I'm not renaming this Select, but if you do, ensure you adjust accordingly in step #3)

   
   
   
3. In the Map, tap the little 'T' icon to the right, and paste in the below:
   

   @item()?['DocumentID']

   
   

   If it doesn't look like the image in #1, then input an expression with the same text but without the @ symbol at the start, it will do the same thing.
   
   

4. After that, initialize an Array type variable, and in the value field add this expression:
   

   union( body('Select'), body('Select') )

Congratulations, you now have a list of all the different DocumentID values that are relevant to your logic flow!

You can now potentially do things like:

1. Running an Apply to each action on this array, and as the first action there you could filter the Get items list on that particular DocumentID and act on only those results.
2. Or maybe you wish to just send an update to someone with all the currently used IDs.
3. Or you need to update a separate data source with information based upon this.

You have more flexibility with this now. 👍

[mjburley]

Thanks @eliotcole for the detail and @annajhaveri for the idea.

This works perfectly.

I still don't understand why apply to each times out though, unless it's API throttling ?

[eliotcole]

Pleasure, @mjburley !!

It's more than likely actually still going, just back out and go back into the running flow. However if it does timeout that's an awful long time, so it's probably not he Apply to each (ATE) that's timing out, but a function within it.

That said, if you have 1000s of entries, you're going to have a long running flow ... and there's some limitations on different flow accounts, for sure.

You can mitigate loops with Concurrency controls (in the ATE settings), but that's a subject for another day, or for you to read up on in the docs. 😉

PS - I also marked @annajhaveri's answer as correct, as I simply spelled it out, hope you don't mind!