cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
MS2
Regular Visitor

Design for listing Dynamics tables with many records

Hi,

 

It seems the Flow/Dynamics connector 'List records' only return 500 records. What is the best practice design to make a loop to go through the entire table in 500 record chunks?

 

I note that the return is ended with:

"@odata.nextLink":"*****&%24skiptoken=%3ccookie+pagenumber%3d%222%22+pagingcookie%3d%22%253ccookie%2520page%253d%25221%2522%253e%253*****last%253d%2522%257b*****first%253d%2522%257b******"

 

 

Thanks

2 REPLIES 2
Community Support
Community Support

Re: Design for listing Dynamics tables with many records

Hi @MS2,

 

Could you please share a screenshot of your flow's configuration?

Do you want to make a loop to go through 500 records within Microsoft Flow?

 

If you want to make a loop to go through 500 records within Microsoft Flow, I think the "Apply to each" action could achieve your needs. Within the "Apply to each" action, you could add other actions to process each record.

 

More details about the "Apply to each" action, please check the following article:

https://flow.microsoft.com/en-us/blog/build-advanced-flows/

 

Please check and see if the following article would help in your scenario:

https://docs.microsoft.com/en-us/flow/apply-to-each

 

Best regards,

Kris

Community Support Team _ Kris Dai
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
MS2
Regular Visitor

Re: Design for listing Dynamics tables with many records

Hi Kris,

 

I already solved this. The issue is not the foreach but rather that Get records (at least before the latest update) only returned 500 records at a time with no simple way to put that in a do until and keep getting the next 500 until all records have been retrieved. Especially since flow doesn't allow me to nest a foreach in a do until.

 

I solved it by sorting the table on an unique integer field and keeping track of the last value received and adding a ODATA filter to get greater than each do until and then call another sub-flow with the 500 records passed as a parameter-array in which there is a foreach. However this is a somewhat ugly and convoluted solution and also dependent on the entity/table having a unique field that can be sorted and compared with greater than or less than. In my case I have those fields but that's just coincidence. However, I saw something about raised limits in the last flow update so maybe this whole construct is now unnecessary, I haven't checked.

 

As a side note it is slightly annoying that I can't find a way to ODATA-chose which columns to select in the Dynamics connector, resulting in a lot of data being retrieved, i.e. I can't do the equivalent of SELECT Col1, Col2 FROM TableXYZ but have to SELECT * FROM TableXYZ.

 

Best regards,

MS

 

Helpful resources

Announcements
MBAS Gallery 2020

MBAS Gallery 2020

Watch Microsoft Business Applications Summit sessions on-demand.

firstImage

New Ranks and Rank Icons released on April 21!

The time has come: We are finally able to share more details on the brand-new ranks coming to the Power Automate Community!

firstImage

Now Live: Power Virtual Agents Community!

We are excited to announce the launch of Power Virtual Agents Community. Check it out now!

firstImage

New & Improved Power Automate Community Cookbook

We've updated and improved the layout and uploading format of the Power Automate Cookbook!

thirdimage

Power Automate Community User Group Member Badge

Fill out a quick form to claim your user group badge now!

sixthImage

Community Summit North America

The top training and networking event across the globe for Microsoft Business Applications

Top Solution Authors
Users online (6,851)