cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Regular Visitor

Searching Large Document Library in SharePoint

I am using Flow to update a large SharePoint list (20,000 items) with links to a large document library (30,000 folders and files). I am having some strange behaviour with odata filtering with "startswith". It seems to work inconsistently, or maybe I am missing something.

 

I am using the action "Get Files (properties only" and using the odata filter "startswith(FileLeafRef,'Full-Report-Number')" where Full-Report-Number is a variable set earlier. As mentioned, this works inconsistently. I suspect that the 5,000 view limit may be affecting me, but in the past when I've bumped into that I've got an explicit error message. This time, it just returns a success code "200" and no items. I thought about setting a view or filtering by an indexed column, but haven't come up with a way to do that yet (I can't add new indexes now bevuase the library is too big, and the files I am looking for could be anywhere and modified anytime, so my view options are limited.)

 

Here's the action:

filter2.jpg

And here's what it looks like after running:

run1.jpg

And here's the Report library search, showing that there are files (and in some cases folders) starting with NS102:

filter 3.jpg

 

Really scratching my head on this one. Thanks for your help.

Gerry

 

 

 

4 REPLIES 4
Highlighted
Community Support
Community Support

Re: Searching Large Document Library in SharePoint

Hi @GerryBrimacombe ,

 

What’s the error message returned before?

 

Please make sure the variable Full-Report-number return matched value.

 

I have made a simple test on my side, which is working successfully.

1.PNG2.PNG

 

Here I have seen a blog, which could be a reference for you:

https://www.chakkaradeep.com/2018/05/01/deep-dive-into-get-items-and-get-files-sharepoint-actions-in...

 

Best regards,

Mabel  

 

Community Support Team _ Mabel Mao
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Highlighted
Regular Visitor

Re: Searching Large Document Library in SharePoint

Thank you for testing and commenting, Mabel.

 

Yes, I have read that article - very useful.

 

I am not surprised your example works - as mentioned, I am getting inconsistent results which is driving me mad. 🙂 Yes, the variable "Full-Report-Number" is being set correctly, so I don't think that's the issue.

I am not getting any error message, just a return code of "200" and and an empty result set. Is there a method or place I should check to get more detailed error messages or logging?

 

Thank you,

Gerry

Highlighted
Regular Visitor

Re: Searching Large Document Library in SharePoint

A bit more testing and debugging...

 

I'm not sure if this means anything more, but here is the returned Outputs section of the Get Files (properties only) action:

out1.jpg

out2.jpg

out3.jpg

out4.jpg

out5.jpg

out6.jpg

out7.jpg

 

Here's the full text of the BODY section above:

{
  "@odata.nextLink": "https://flow-apim-msmanaged-na-eastus2-01.azure-apim.net/apim/sharepointonline/shared-sharepointonl-...",
  "value": []
}

 

Highlighted
Community Support
Community Support

Re: Searching Large Document Library in SharePoint

Hi @GerryBrimacombe ,

 

Not sure if it is caused by the large amount files contained in the library.

 

Could you please try getting files (properties only) from a library which doesn’t have too many files?

 

Best regards,

Mabel          

 

Community Support Team _ Mabel Mao
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
firstImage

Super User Program Update

Three Super User rank tiers have been launched!

firstImage

Power Platform 2020 release wave 2 plan

Features releasing from October 2020 through March 2021

firstImage

Join the new Power Virtual Agents Community!

We are excited to announce the launch of Power Virtual Agents Community. Check it out now!

firstImage

New & Improved Power Automate Community Cookbook

We've updated and improved the layout and uploading format of the Power Automate Cookbook!

thirdimage

Power Automate Community User Group Member Badge

Fill out a quick form to claim your user group badge now!

Top Solution Authors
Top Kudoed Authors
Users online (6,655)