cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Impactful Individual
Impactful Individual

Get files (properties only) limited to 516 results

I'm trying to get files from a SharePoint library, but no matter the top count, the Get files (properties only) connector only returns 516 items. The total number of items (files and folders) in the library is about 155,000.

I always get 516 items if the top count is set to higher than 516.

There is no longer any pagination option in the Get files (properties only) action, only the top count. Is this expected behaviour? Do I now need to use the Graph API to get more items using the Odata.nextlink property?

2 ACCEPTED SOLUTIONS

Accepted Solutions
Highlighted
Community Support
Community Support

Re: Get files (properties only) limited to 516 results

Hi @WillPage

 

I have made a test on my side and as you said, the pagination function is no longer supported in the Get files action currently.

And I'm afraid that the Get files action would always have the file limit such as 516 as you said, you couldn't use the Get files action to get the all 155,000 files currently and maybe the only way is take a try to use the Graph API to achieve the requirement.

 

Best Regards,

Alice

 

Community Support Team _ Alice Zhang
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

Highlighted
Dual Super User III
Dual Super User III

Re: Get files (properties only) limited to 516 results

Hi!

For 'Get items' you can activate pagination. For 'Get files' you need to use Graph API (as @v-alzhan-msft  suggests) or follow a 'Do Until' based strategy.

If you are interested in this second approach, please check 'Using the Do Until method' section on this great article

https://alextofan.com/2019/08/22/how-to-get-more-than-5000-item-from-sharepoint-online-in-flow/

Article focuses on 'Get items' scenario but 'Using the Do Until method' section can be applied to 'Get files' also.

Finally, from the figures you shared, I would expect the 'Do Until method' approach to be quite slow in terms of performance, since you will need more than 300 iterations

Hope this helps



Each time you click on any of our inspiring answers 'Thumb up' icon...
...an ewok scapes from the stormtroopers.

Be grateful, Thumbs up! Save the Galaxy for free!


Escribo sobre Power Automate en:
https://medium.com/anyone-can-automate/

Proud to be a Flownaut!



View solution in original post

3 REPLIES 3
Highlighted
Community Support
Community Support

Re: Get files (properties only) limited to 516 results

Hi @WillPage

 

I have made a test on my side and as you said, the pagination function is no longer supported in the Get files action currently.

And I'm afraid that the Get files action would always have the file limit such as 516 as you said, you couldn't use the Get files action to get the all 155,000 files currently and maybe the only way is take a try to use the Graph API to achieve the requirement.

 

Best Regards,

Alice

 

Community Support Team _ Alice Zhang
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

Highlighted
Dual Super User III
Dual Super User III

Re: Get files (properties only) limited to 516 results

Hi!

For 'Get items' you can activate pagination. For 'Get files' you need to use Graph API (as @v-alzhan-msft  suggests) or follow a 'Do Until' based strategy.

If you are interested in this second approach, please check 'Using the Do Until method' section on this great article

https://alextofan.com/2019/08/22/how-to-get-more-than-5000-item-from-sharepoint-online-in-flow/

Article focuses on 'Get items' scenario but 'Using the Do Until method' section can be applied to 'Get files' also.

Finally, from the figures you shared, I would expect the 'Do Until method' approach to be quite slow in terms of performance, since you will need more than 300 iterations

Hope this helps



Each time you click on any of our inspiring answers 'Thumb up' icon...
...an ewok scapes from the stormtroopers.

Be grateful, Thumbs up! Save the Galaxy for free!


Escribo sobre Power Automate en:
https://medium.com/anyone-can-automate/

Proud to be a Flownaut!



View solution in original post

Highlighted
Impactful Individual
Impactful Individual

Re: Get files (properties only) limited to 516 results

Thanks for that @efialttes I probably would have used that method if I'd seen it soon enough. Instead I used the Graph API lists endpoint to get the items and handled pagination myself using the odata.nextLink property, because the HTTP action in PA only supports a maximum of 100,000 items and I have more than that.

I still hit a hard limit where the array variable I use to store the results reaches 100MB in size, but by then with $top=999 it's on page 140 and collected very nearly all the results. I could squeeze it in by being judicious with the $select query to reduce the size of the response.

 

A word to the wise: Don't let document libraries get this big!

Helpful resources

Announcements
Community Conference

Power Platform Community Conference

Check out the on demand sessions that are available now!

Power Platform ISV Studio

Power Platform ISV Studio

ISV Studio is designed to become the go-to Power Platform destination for ISV’s to monitor & manage published applications.

Upcoming Events

Experience what’s next for Power Automate

See the latest Power Automate innovations, updates, and demos from the Microsoft Business Applications Launch Event.

Users online (6,753)