cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
wannabe_dev
Frequent Visitor

Create .CSV >5000 rows from Dataverse table

I'm attempting to build a scheduled flow to periodically extract a snapshot of a Dataverse table to a .CSV file saved on SharePoint, but am running into a limit of 5000 rows when creating the .CSV.

 

  • The flow starts by using the Dataverse List Rows action, with Pagination turned On and Threshold set to 100,000 in the settings.
  • A Compose step with the length function then confirms that all 25,000+ rows present in the table are returned by the List Rows action.
  • The Create CSV action then generates a CSV from the value output of the initial List Rows step.
  • A Create File step then saves the CSV to SharePoint.

The problem is that the CSV output contains only 5000 rows of data, despite the List Rows step apparently collecting all 25,000+ rows from Dataverse.

 

Does anyone know of the 5000 row output is due to a limitation of the Create CSV action, or an issue with the underlying data returned by the Dataverse List Rows action?

 

Could the Parse JSON and Append to Array actions be used in a loop to manually generate the CSV, or would this run into the same limitation?

 

 

 

2 REPLIES 2
grantjenkins
Community Champion
Community Champion

I'm wondering if this is a limit based on the Power Automate license you are using. Do you have Power Automate Free, or Power Automate Per User license/other? See limits below.

 

Limits and configuration - Power Automate | Microsoft Learn

 

grantjenkins_4-1670123266397.png

 

grantjenkins_5-1670123323670.png

 

I have Power Automate Per User and can generate CSV tables with a very large number of items (I believe up to 100,000). The example below outputs a CSV with just over 10,000 rows.

 

See full flow below. I'll go into each of the actions. Note that for this example I'm using a SharePoint List with 10,997 items but should be the same process for a Dataverse Table.

grantjenkins_0-1670122824673.png

 

Get items retrieves my list items. I've set Top Count to 5000 so it will retrieve the results in batches of 5000. Then in Settings > Pagination I've turned it On and set Threshold to 20,000.

grantjenkins_1-1670122954367.png

grantjenkins_2-1670122986947.png

 

So, in my case it will retrieve two lots of 5,000, and one lot of 997 giving me 10,997 items.

 

Then in Create CSV table, I pass in the value property from my Get items. This gives me a CSV table containing all 10,997 rows of data.

grantjenkins_3-1670123086556.png

 

 



----------------------------------------------------------------------

If I've answered your question, please mark the post as Solved.

If you like my response, please consider giving it a Thumbs Up.
wannabe_dev
Frequent Visitor

@grantjenkins thanks for the reply. Apparently I am on a PowerApps per user plan, so should fall under the medium performance profile and have access to the full 100,000 rows. Will just keep experimenting I think!

Helpful resources

Announcements
Power Automate News & Announcements

Power Automate News & Announcements

Keep up to date with current events and community announcements in the Power Automate community.

Community Calls Conversations

Community Calls Conversations

A great place where you can stay up to date with community calls and interact with the speakers.

Power Automate Community Blog

Power Automate Community Blog

Check out the latest Community Blog from the community!

Users online (5,323)