cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Runner55552
Level: Powered On

Sorting to overcome 5,000 Item Limit

I have a Flow that operates on a SharePoint library with more than 5,000 items (the dreaded 5,000 limit has been exceeded). One of my first actions is scanning the library using a Get Files (Properties Only) action, which needs to search the entire library, then filter based on one column. Another factor is that I need to sort/order the files based on one of the columns, so that they are retrieved, and eventually merged into a single file, in the correct order. This all works well by using the oData query (just filter the items based on matching a certain value in a column), then an order by query (in this case, I need to order ascending by a column called Doc_Type).  The problem is that now the library will not be scanned for all 5,000 + items due to the limit.  I have to set top count at 4,999.  One nice solution would be to reverse the sort order by using ID desc (descending order of Item ID, or more recent first), in which case, I would solve my issues, because I am typically only dealing with the most recent 500 to 1000 files.  But when I switch to sorting by ID descending first, then Doc_Type ascending, my documents will be retrieved out of order, and later passed into a merge array out of order.  Filter arrays and other actions cannot sort.

 

Are there any solutions to either overcome the 5,000 limit, or to do a second sort after the Get Files (properties only) action, to re-sort the filtered list of items by Doc_Type ascending before passing into the merge array?

 

Thanks for any help on this.  Screenshots below:

Flow2.jpg

 

Flow3.jpg

 

Flow1.jpg

 

 

 

1 ACCEPTED SOLUTION

Accepted Solutions
Dual Super User
Dual Super User

Re: Sorting to overcome 5,000 Item Limit

@Runner55552 

Not sure if this workaround from great John Liu can fit your needs.

http://johnliu.net/blog/2018/12/how-to-implement-sort-with-microsoft-flow-in-3-actions-within-a-loop

So first sort is applied on the Get items via ODATA, second sort is applied via this workaround

Hope this helps





If anybody solved your question, remember to mark its post as a solution!

Proud to be a Flownaut!





View solution in original post

3 REPLIES 3
Runner55552
Level: Powered On

Re: Sorting to overcome 5,000 Item Limit

Screenshots were out of order.  The bottom screenshot should be at the top.  And not shown, but there is a trigger at the the beginning that starts up the Flow when an item is modified in a separate SharePoint list. Then data from that Item is stored in a variable and used in the Flow.

Dual Super User
Dual Super User

Re: Sorting to overcome 5,000 Item Limit

@Runner55552 

Not sure if this workaround from great John Liu can fit your needs.

http://johnliu.net/blog/2018/12/how-to-implement-sort-with-microsoft-flow-in-3-actions-within-a-loop

So first sort is applied on the Get items via ODATA, second sort is applied via this workaround

Hope this helps





If anybody solved your question, remember to mark its post as a solution!

Proud to be a Flownaut!





View solution in original post

Runner55552
Level: Powered On

Re: Sorting to overcome 5,000 Item Limit

I accepted the proposed solution from John Liu after reviewing it, and also receiving the same recommendation independently from another source.  I have not tested it yet in my sandbox, but plan to shortly. Conceptually, it looks like it should work.  Thanks!

Helpful resources

Announcements
thirdimage

Power Automate Community User Group Member Badge

Fill out a quick form to claim your user group badge now!

firstImage

Incoming: New and improved badges!

We've given our badges an overhaul and also added some brand new ones!

fifthimage

Microsoft Learn

Learn how to build the business apps that you need.

sixthImage

Power Platform World Tour

Find out where you can attend!

seventhimage

Webinars & Video Gallery

Watch & learn from the Power Automate Community Video Gallery!

Top Kudoed Authors
Users Online
Currently online: 187 members 5,234 guests
Please welcome our newest community members: