cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
GerryBrimacombe
Level: Powered On

Searching Large Document Library in SharePoint

I am using Flow to update a large SharePoint list (20,000 items) with links to a large document library (30,000 folders and files). I am having some strange behaviour with odata filtering with "startswith". It seems to work inconsistently, or maybe I am missing something.

 

I am using the action "Get Files (properties only" and using the odata filter "startswith(FileLeafRef,'Full-Report-Number')" where Full-Report-Number is a variable set earlier. As mentioned, this works inconsistently. I suspect that the 5,000 view limit may be affecting me, but in the past when I've bumped into that I've got an explicit error message. This time, it just returns a success code "200" and no items. I thought about setting a view or filtering by an indexed column, but haven't come up with a way to do that yet (I can't add new indexes now bevuase the library is too big, and the files I am looking for could be anywhere and modified anytime, so my view options are limited.)

 

Here's the action:

filter2.jpg

And here's what it looks like after running:

run1.jpg

And here's the Report library search, showing that there are files (and in some cases folders) starting with NS102:

filter 3.jpg

 

Really scratching my head on this one. Thanks for your help.

Gerry

 

 

 

4 REPLIES 4
Community Support Team
Community Support Team

Re: Searching Large Document Library in SharePoint

Hi @GerryBrimacombe ,

 

What’s the error message returned before?

 

Please make sure the variable Full-Report-number return matched value.

 

I have made a simple test on my side, which is working successfully.

1.PNG2.PNG

 

Here I have seen a blog, which could be a reference for you:

https://www.chakkaradeep.com/2018/05/01/deep-dive-into-get-items-and-get-files-sharepoint-actions-in...

 

Best regards,

Mabel  

 

Community Support Team _ Mabel Mao
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
GerryBrimacombe
Level: Powered On

Re: Searching Large Document Library in SharePoint

Thank you for testing and commenting, Mabel.

 

Yes, I have read that article - very useful.

 

I am not surprised your example works - as mentioned, I am getting inconsistent results which is driving me mad. 🙂 Yes, the variable "Full-Report-Number" is being set correctly, so I don't think that's the issue.

I am not getting any error message, just a return code of "200" and and an empty result set. Is there a method or place I should check to get more detailed error messages or logging?

 

Thank you,

Gerry

GerryBrimacombe
Level: Powered On

Re: Searching Large Document Library in SharePoint

A bit more testing and debugging...

 

I'm not sure if this means anything more, but here is the returned Outputs section of the Get Files (properties only) action:

out1.jpg

out2.jpg

out3.jpg

out4.jpg

out5.jpg

out6.jpg

out7.jpg

 

Here's the full text of the BODY section above:

{
  "@odata.nextLink": "https://flow-apim-msmanaged-na-eastus2-01.azure-apim.net/apim/sharepointonline/shared-sharepointonl-...",
  "value": []
}

 

Community Support Team
Community Support Team

Re: Searching Large Document Library in SharePoint

Hi @GerryBrimacombe ,

 

Not sure if it is caused by the large amount files contained in the library.

 

Could you please try getting files (properties only) from a library which doesn’t have too many files?

 

Best regards,

Mabel          

 

Community Support Team _ Mabel Mao
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
thirdimage

Power Automate Community User Group Member Badge

Fill out a quick form to claim your user group badge now!

firstImage

Incoming: New and improved badges!

We've given our badges an overhaul and also added some brand new ones!

fifthimage

Microsoft Learn

Learn how to build the business apps that you need.

sixthImage

Power Platform World Tour

Find out where you can attend!

seventhimage

Webinars & Video Gallery

Watch & learn from the Power Automate Community Video Gallery!

Top Kudoed Authors (Last 30 Days)
Users online (6,321)