Searching Large Document Library in SharePoint

[GerryBrimacombe]

I am using Flow to update a large SharePoint list (20,000 items) with links to a large document library (30,000 folders and files). I am having some strange behaviour with odata filtering with "startswith". It seems to work inconsistently, or maybe I am missing something.

I am using the action "Get Files (properties only" and using the odata filter "startswith(FileLeafRef,'Full-Report-Number')" where Full-Report-Number is a variable set earlier. As mentioned, this works inconsistently. I suspect that the 5,000 view limit may be affecting me, but in the past when I've bumped into that I've got an explicit error message. This time, it just returns a success code "200" and no items. I thought about setting a view or filtering by an indexed column, but haven't come up with a way to do that yet (I can't add new indexes now bevuase the library is too big, and the files I am looking for could be anywhere and modified anytime, so my view options are limited.)

Here's the action:

And here's what it looks like after running:

And here's the Report library search, showing that there are files (and in some cases folders) starting with NS102:

Really scratching my head on this one. Thanks for your help.

Gerry

[v-yamao-msft]

Hi @GerryBrimacombe ,

What’s the error message returned before?

Please make sure the variable Full-Report-number return matched value.

I have made a simple test on my side, which is working successfully.

Here I have seen a blog, which could be a reference for you:

https://www.chakkaradeep.com/2018/05/01/deep-dive-into-get-items-and-get-files-sharepoint-actions-in...

Best regards,

Mabel  

[GerryBrimacombe]

Thank you for testing and commenting, Mabel.

Yes, I have read that article - very useful.

I am not surprised your example works - as mentioned, I am getting inconsistent results which is driving me mad. 🙂 Yes, the variable "Full-Report-Number" is being set correctly, so I don't think that's the issue.

I am not getting any error message, just a return code of "200" and and an empty result set. Is there a method or place I should check to get more detailed error messages or logging?

Thank you,

Gerry

[GerryBrimacombe]

A bit more testing and debugging...

I'm not sure if this means anything more, but here is the returned Outputs section of the Get Files (properties only) action:

Here's the full text of the BODY section above:

{
  "@odata.nextLink": "https://flow-apim-msmanaged-na-eastus2-01.azure-apim.net/apim/sharepointonline/shared-sharepointonl-...",
  "value": []
}

[v-yamao-msft]

Hi @GerryBrimacombe ,

Not sure if it is caused by the large amount files contained in the library.

Could you please try getting files (properties only) from a library which doesn’t have too many files?

Best regards,

Mabel