cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Advocate II
Advocate II

Sorting to overcome 5,000 Item Limit

I have a Flow that operates on a SharePoint library with more than 5,000 items (the dreaded 5,000 limit has been exceeded). One of my first actions is scanning the library using a Get Files (Properties Only) action, which needs to search the entire library, then filter based on one column. Another factor is that I need to sort/order the files based on one of the columns, so that they are retrieved, and eventually merged into a single file, in the correct order. This all works well by using the oData query (just filter the items based on matching a certain value in a column), then an order by query (in this case, I need to order ascending by a column called Doc_Type).  The problem is that now the library will not be scanned for all 5,000 + items due to the limit.  I have to set top count at 4,999.  One nice solution would be to reverse the sort order by using ID desc (descending order of Item ID, or more recent first), in which case, I would solve my issues, because I am typically only dealing with the most recent 500 to 1000 files.  But when I switch to sorting by ID descending first, then Doc_Type ascending, my documents will be retrieved out of order, and later passed into a merge array out of order.  Filter arrays and other actions cannot sort.

 

Are there any solutions to either overcome the 5,000 limit, or to do a second sort after the Get Files (properties only) action, to re-sort the filtered list of items by Doc_Type ascending before passing into the merge array?

 

Thanks for any help on this.  Screenshots below:

Flow2.jpg

 

Flow3.jpg

 

Flow1.jpg

 

 

 

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
Dual Super User III
Dual Super User III

Re: Sorting to overcome 5,000 Item Limit

@Runner55552 

Not sure if this workaround from great John Liu can fit your needs.

http://johnliu.net/blog/2018/12/how-to-implement-sort-with-microsoft-flow-in-3-actions-within-a-loop

So first sort is applied on the Get items via ODATA, second sort is applied via this workaround

Hope this helps



Each time you click on any of our inspiring answers 'Thumb up' icon...
...an ewok scapes from the stormtroopers.

Be grateful, Thumbs up! Save the Galaxy for free!


Escribo sobre Power Automate en:
https://medium.com/anyone-can-automate/

Proud to be a Flownaut!



View solution in original post

3 REPLIES 3
Highlighted
Advocate II
Advocate II

Re: Sorting to overcome 5,000 Item Limit

Screenshots were out of order.  The bottom screenshot should be at the top.  And not shown, but there is a trigger at the the beginning that starts up the Flow when an item is modified in a separate SharePoint list. Then data from that Item is stored in a variable and used in the Flow.

Highlighted
Dual Super User III
Dual Super User III

Re: Sorting to overcome 5,000 Item Limit

@Runner55552 

Not sure if this workaround from great John Liu can fit your needs.

http://johnliu.net/blog/2018/12/how-to-implement-sort-with-microsoft-flow-in-3-actions-within-a-loop

So first sort is applied on the Get items via ODATA, second sort is applied via this workaround

Hope this helps



Each time you click on any of our inspiring answers 'Thumb up' icon...
...an ewok scapes from the stormtroopers.

Be grateful, Thumbs up! Save the Galaxy for free!


Escribo sobre Power Automate en:
https://medium.com/anyone-can-automate/

Proud to be a Flownaut!



View solution in original post

Highlighted
Advocate II
Advocate II

Re: Sorting to overcome 5,000 Item Limit

I accepted the proposed solution from John Liu after reviewing it, and also receiving the same recommendation independently from another source.  I have not tested it yet in my sandbox, but plan to shortly. Conceptually, it looks like it should work.  Thanks!

Helpful resources

Announcements
firstImage

Super User Program Update

Three Super User rank tiers have been launched!

firstImage

Power Platform 2020 release wave 2 plan

Features releasing from October 2020 through March 2021

firstImage

New & Improved Power Automate Community Cookbook

We've updated and improved the layout and uploading format of the Power Automate Cookbook!

thirdimage

Power Automate Community User Group Member Badge

Fill out a quick form to claim your user group badge now!

Top Solution Authors
Top Kudoed Authors
Users online (8,509)