power automate microsoft flow 5000 items limit . How can I just insert 5k a time? is there a example flow for this ?
the set up is
1. grab records from some data source ( have 6k+ records)
2. put them in an array
3. insert those records into SQL server
How can i have a for loop or do until loop that basically inserts 5k records at a time so i don't hit this ridiculous limit.
I suggest the below:
1. Add a new field to the data source that indicates if the record has been inserted in the SQL Server Database after it is inserted so you can differentiate between inserted records and non - inserted records.
2. The below example assumes your data source is sharepoint
If this post helps you with your problem, please mark your as Accepted solution.
If you like my response, please give it a Thumbs Up.
MG (Naturally Curious)
Hi @Yusha ,
Please check this online doc, it is mentioned here that if you have Office 365 and Free licenses, the maximum number allowed for Apply to each is 5000 times. The maximum allowed number of Plan 1, Plan 2, Per User, and Per Flow licenses is 100,000, and upgrading the license is an option for reference. https://docs.microsoft.com/en-us/power-automate/limits-and-config#looping-and-debatching-limits
Another way worth referring to is to filter records to reduce the number of records that need to be inserted.
Or there are fields in the table that can separate the data into two parts, so you can insert these data separately in Apply to each.
Hope that makes sense.
@Mira_Ghaly - you're confused.
@v-bacao-msft - can you show me an exampe of how I can use a loop to insert 4000 records for each insert? Assume I have more than 4k record total, that is my use case. I need a sample. I am not understanding how I can do that and @Mira_Ghaly did not provide a relevant example
@Mira_Ghaly anyway you can just give me a example of what I'm asking? lol
1 - retreive 10k records
2 - insert 4000k records at a time so i dont hit 5k limit ?
HI @Yusha ,
For example, when I use the SharePoint list as a data source, I use the ID to divide all items into two parts.
Add a parallel branch to Flow to handle the items contained in these two parts separately.
Hope that makes sense.
@v-bacao-msft - no it doesn't make sense at all. You have a parallel branch , what if i have 20k items, you're telling me I need to make 4 parralell branches? I am receiving records from D365 entities (for context). I have no idea how many items I will have I need a way to loop through the items and delay them once I have hit the limit , wait, and insert another 4k items , rinse and repeat
Check out the News & Announcements to learn more.
Check out the new Power Platform Community Connections gallery!
Congratulations, the new Super User Season 2 for 2021 has started!
Power Platform release plan for the 2021 release wave 2 describes all new features releasing from October 2021 through March 2022.