cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
Advocate I
Advocate I

I have an excel sheet with more than 300,000 rows that needs to be updated, is flow a bad choice?

I have an excel file that would need to be updated every morning.


By update, I mean removing every single row and adding new rows from a sql table.

The excel file contains more than 300,000 rows. And the sql table also contains more than 300,000 rows.

 

On initial testing, its taking a lot more than 2 hours just to delete only the first 100,000 rows. And its still going.

 

So I figured this is the time to ask, is Microsoft Power Automate the proper tool to use for this task? 

Thanks 

 

@v-yuazh-msft @SCTdan 

1 ACCEPTED SOLUTION

Accepted Solutions
Super User III
Super User III

Re: I have an excel sheet with more than 300,000 rows that needs to be updated, is flow a bad choice?

If you are using Excel for Business you are likely running up against the limitation of 20 api calls a minute - https://docs.microsoft.com/en-us/connectors/excelonlinebusiness/

I'll probably be proved wrong but I'm not sure that Power Automate will be the right choice for the volume of data you have.

Rob
Los Gallardos
If I've answered your question or solved your problem, please mark this question as answered. This helps others who have the same question find a solution quickly via the forum search. If you liked my response, please consider giving it a thumbs up. Thanks.

 

View solution in original post

4 REPLIES 4
Super User III
Super User III

Re: I have an excel sheet with more than 300,000 rows that needs to be updated, is flow a bad choice?

If you are using Excel for Business you are likely running up against the limitation of 20 api calls a minute - https://docs.microsoft.com/en-us/connectors/excelonlinebusiness/

I'll probably be proved wrong but I'm not sure that Power Automate will be the right choice for the volume of data you have.

Rob
Los Gallardos
If I've answered your question or solved your problem, please mark this question as answered. This helps others who have the same question find a solution quickly via the forum search. If you liked my response, please consider giving it a thumbs up. Thanks.

 

View solution in original post

Highlighted
Advocate I
Advocate I

Re: I have an excel sheet with more than 300,000 rows that needs to be updated, is flow a bad choice?

@RobElliott  Thanks for your input. 

 

I can confirm I am using Excel Business Online and if you are right about the API Calls, then certainly flow won't work.

Will await other opinions and accept your answer. 

 

Thanks anyways

Highlighted
Community Support
Community Support

Re: I have an excel sheet with more than 300,000 rows that needs to be updated, is flow a bad choice?

 

Hi @mykelboachie ,

 

If Apply to each is included in Flow, you could consider making the following settings:

78.PNG

Please take a try and see if it improves the current situation.

 

If it still doesn't work well, as @RobElliott mentioned, Power Automate is not necessarily the best solution. If there is too much data to be processed, Power Automate may not be able to perform well.

 

Best Regards,

Community Support Team _ Barry
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Highlighted
Super User II
Super User II

Re: I have an excel sheet with more than 300,000 rows that needs to be updated, is flow a bad choice?

The Excel Online connector has many Known issues and limitations that make larger workloads all but impossible. For this reason, it is highly recommended for large and lengthy workloads that you instead utilize SharePoint Lists.

With that said, even with SharePoint lists, you will need to add delays to your Flow(s) because the SharePoint REST API has a call limit of 5000 items per request.

Your problem is the sheer volume of the dataset, and the current platform (Excel) which has limitations. With a dataset of that size, you may want to consider more robust data solutions (CDS, SQL, etc.).

👨🏻‍💻 If this reply answers your question or solves your issue, please ACCEPT AS SOLUTION ☑️. If you find this reply helpful, please consider giving it a LIKE 👍.

Helpful resources

Announcements
firstImage

Super User Program Update

Three Super User rank tiers have been launched!

firstImage

Power Platform 2020 release wave 2 plan

Features releasing from October 2020 through March 2021

firstImage

New & Improved Power Automate Community Cookbook

We've updated and improved the layout and uploading format of the Power Automate Cookbook!

thirdimage

Power Automate Community User Group Member Badge

Fill out a quick form to claim your user group badge now!

Top Solution Authors
Top Kudoed Authors
Users online (7,023)