cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Frequent Visitor

Searching Large Document Library in SharePoint

I am using Flow to update a large SharePoint list (20,000 items) with links to a large document library (30,000 folders and files). I am having some strange behaviour with odata filtering with "startswith". It seems to work inconsistently, or maybe I am missing something.

 

I am using the action "Get Files (properties only" and using the odata filter "startswith(FileLeafRef,'Full-Report-Number')" where Full-Report-Number is a variable set earlier. As mentioned, this works inconsistently. I suspect that the 5,000 view limit may be affecting me, but in the past when I've bumped into that I've got an explicit error message. This time, it just returns a success code "200" and no items. I thought about setting a view or filtering by an indexed column, but haven't come up with a way to do that yet (I can't add new indexes now bevuase the library is too big, and the files I am looking for could be anywhere and modified anytime, so my view options are limited.)

 

Here's the action:

filter2.jpg

And here's what it looks like after running:

run1.jpg

And here's the Report library search, showing that there are files (and in some cases folders) starting with NS102:

filter 3.jpg

 

Really scratching my head on this one. Thanks for your help.

Gerry

 

 

 

4 REPLIES 4
Community Support
Community Support

Hi @GerryBrimacombe ,

 

What’s the error message returned before?

 

Please make sure the variable Full-Report-number return matched value.

 

I have made a simple test on my side, which is working successfully.

1.PNG2.PNG

 

Here I have seen a blog, which could be a reference for you:

https://www.chakkaradeep.com/2018/05/01/deep-dive-into-get-items-and-get-files-sharepoint-actions-in...

 

Best regards,

Mabel  

 

Community Support Team _ Mabel Mao
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Thank you for testing and commenting, Mabel.

 

Yes, I have read that article - very useful.

 

I am not surprised your example works - as mentioned, I am getting inconsistent results which is driving me mad. 🙂 Yes, the variable "Full-Report-Number" is being set correctly, so I don't think that's the issue.

I am not getting any error message, just a return code of "200" and and an empty result set. Is there a method or place I should check to get more detailed error messages or logging?

 

Thank you,

Gerry

A bit more testing and debugging...

 

I'm not sure if this means anything more, but here is the returned Outputs section of the Get Files (properties only) action:

out1.jpg

out2.jpg

out3.jpg

out4.jpg

out5.jpg

out6.jpg

out7.jpg

 

Here's the full text of the BODY section above:

{
  "@odata.nextLink": "https://flow-apim-msmanaged-na-eastus2-01.azure-apim.net/apim/sharepointonline/shared-sharepointonl-...",
  "value": []
}

 

Hi @GerryBrimacombe ,

 

Not sure if it is caused by the large amount files contained in the library.

 

Could you please try getting files (properties only) from a library which doesn’t have too many files?

 

Best regards,

Mabel          

 

Community Support Team _ Mabel Mao
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Helpful resources

Announcements
Microsoft Ignite

Microsoft Ignite

Join digitally, March 2–4, 2021 to explore new tech that's ready to implement. Experience the keynote in mixed reality through AltspaceVR!

New Super Users

Meet the Power Automate Super Users!

Many congratulations to the Season 1 2021 Flownaut Crew!

Microsoft Ignite

Microsoft Power Platform: 2021 Release Wave 1 Plan

Power Platform release plan for the 2021 release wave 1 describes all new features releasing from April through September 2021.

MPA Community Blog

Power Automate Community Blog

Check out the community blog page where you can find valuable learning material from community and product team members!

Users online (36,990)