cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
otep-kunnn
New Member

Limitation on Variable and Microsoft Dataverse Connector in Handling More Than 100000 Records

Hi!

I am trying to process more than 100000 records in CRM via Power Automate through utilizing the skip token in the 'List rows' Action to bypass the pagination limit of 100000 records and appending to an array variable. Unfortunately, I am reaching the limit on the allowed data within an array variable. I would like to ask your opinion on how can we create a solution to handle this scenario while not affecting the overall runtime performance of the flow, if possible.

 

Thank you in advance! 🙂

4 REPLIES 4
MarconettiMarco
Super User
Super User

Hello @otep-kunnn ,

what's the source? An excel file?

Have you ever considered to use the following XRMToolbox tools?

 

https://www.xrmtoolbox.com/plugins/JosephMerhej.DataImport/

 

https://www.xrmtoolbox.com/plugins/Cinteros.XrmToolBox.BulkDataUpdater/

 

BR,

Marco

Hello @MarconettiMarco,

 

Thank you for reaching out. The source of the data is Dynamics 365 CRM/MS Dataverse. Unfortunately, we cannot use the XRM Toolbox tools as we need to process the large number of records everyday on a certain schedule, in an automated way.

Regards,
Otep

Hello @otep-kunnn ,

for this kind of volume, I suggest to use Azure services like LogicApps, Data Factory.

 

BR,

Marco

Hi @MarconettiMarco,

 

Thank you for your suggestion. I will definitely check the said services.

I will let you know the outcome.

 

Regards,

Otep

Helpful resources

Announcements
Power Automate News & Announcements

Power Automate News & Announcements

Keep up to date with current events and community announcements in the Power Automate community.

Community Calls Conversations

Community Calls Conversations

A great place where you can stay up to date with community calls and interact with the speakers.

Power Automate Community Blog

Power Automate Community Blog

Check out the latest Community Blog from the community!

Users online (3,492)