cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
MS2
Level: Powered On

Design for listing Dynamics tables with many records

Hi,

 

It seems the Flow/Dynamics connector 'List records' only return 500 records. What is the best practice design to make a loop to go through the entire table in 500 record chunks?

 

I note that the return is ended with:

"@odata.nextLink":"*****&%24skiptoken=%3ccookie+pagenumber%3d%222%22+pagingcookie%3d%22%253ccookie%2520page%253d%25221%2522%253e%253*****last%253d%2522%257b*****first%253d%2522%257b******"

 

 

Thanks

2 REPLIES 2
Community Support Team
Community Support Team

Re: Design for listing Dynamics tables with many records

Hi @MS2,

 

Could you please share a screenshot of your flow's configuration?

Do you want to make a loop to go through 500 records within Microsoft Flow?

 

If you want to make a loop to go through 500 records within Microsoft Flow, I think the "Apply to each" action could achieve your needs. Within the "Apply to each" action, you could add other actions to process each record.

 

More details about the "Apply to each" action, please check the following article:

https://flow.microsoft.com/en-us/blog/build-advanced-flows/

 

Please check and see if the following article would help in your scenario:

https://docs.microsoft.com/en-us/flow/apply-to-each

 

Best regards,

Kris

Community Support Team _ Kris Dai
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
MS2
Level: Powered On

Re: Design for listing Dynamics tables with many records

Hi Kris,

 

I already solved this. The issue is not the foreach but rather that Get records (at least before the latest update) only returned 500 records at a time with no simple way to put that in a do until and keep getting the next 500 until all records have been retrieved. Especially since flow doesn't allow me to nest a foreach in a do until.

 

I solved it by sorting the table on an unique integer field and keeping track of the last value received and adding a ODATA filter to get greater than each do until and then call another sub-flow with the 500 records passed as a parameter-array in which there is a foreach. However this is a somewhat ugly and convoluted solution and also dependent on the entity/table having a unique field that can be sorted and compared with greater than or less than. In my case I have those fields but that's just coincidence. However, I saw something about raised limits in the last flow update so maybe this whole construct is now unnecessary, I haven't checked.

 

As a side note it is slightly annoying that I can't find a way to ODATA-chose which columns to select in the Dynamics connector, resulting in a lot of data being retrieved, i.e. I can't do the equivalent of SELECT Col1, Col2 FROM TableXYZ but have to SELECT * FROM TableXYZ.

 

Best regards,

MS

 

Helpful resources

Announcements
firstImage

Microsoft Business Applications Virtual Launch Event

Join us for an in-depth look at the new innovations across Dynamics 365 and the Microsoft Power Platform.

firstImage

Watch Sessions On Demand!

Continue your learning in our online communities.

Power Platform 2019 Release Wave 2 Plan

Power Platform 2019 Release Wave 2 Plan

Features releasing from October 2019 through March 2020

thirdimage

Flow Community User Group Member Badge

Fill out a quick form to claim your user group badge now!

fifthimage

Microsoft Learn

Learn how to build the business apps that you need

sixthImage

Power Platform World Tour

Find out where you can attend!

seventhimage

Webinars & Video Gallery

Watch & learn from the Flow Community Video Gallery!

Top Kudoed Authors
Users Online
Currently online: 43 members 4,920 guests
Please welcome our newest community members: