I have a dataflow that is just extracting data from an excel stored in sharepoint and replaces the data in a cds table with it.
Somehow this takes more than an hour to complete which is unacceptable for my usecase.
I also noticed that the organisations database storage is full.
Is it possible that the dataflow takes so long because of that?
In the docs it just says that you cant create new environments if the storage is full.
Hi @Splush ,
It is unclear whether exceeded database capacity would cause your dataflow to be throttled. It is more likely that the dataflow is being throttled by the Service Protection API limits. See Service Protection API Limits (Microsoft Dataverse) - Power Apps | Microsoft Docs for details.
How many rows do you have in your Excel file?
It is also a good idea to also sort out your database capacity issue, just to rule that out and be compliant with Microsoft's licensing terms.
Hi @HSheild ,
thanks a bunch for your answer. I dont think that the API limits would be the limiting factor here. It is just one Sharepoint request every 10 minutes. The excel itself is around 5000 rows by 50 columns in size. A comparable power query would take just a few seconds
Hi @Splush ,
Sounds like you are potentially trying to update 5000 rows in CDS every 10 minutes? If that is the case then you will be coming up against the Entitlement Limits Requests limits and allocations - Power Platform | Microsoft Docs.
Depending on the license you have, you can only do up to 20,000 API requests per 24hrs. Have you noticed that the first few runs go quickly and then overtime the runs get slower and slower?
Not sure what the current situation is with things, but (other than a Trial) it used to be that if you had exceeded your storage capacity you were unable to add new records (you could still read) but save of a new record would fail.
Hey @Fubar ,
right now we can exceed the limit by a few GB and no other processes seem to be affected by that.
Its just that we can't create new environments.
Hey @HSheild ,
no, the first few flows run at the same speed as the other ones later that day.
I also really can't believe that each row entry would be a seperate API request.
I mean that would mean that all data flows above 20k would be affected and potentially rendering data flows as a whole useless.
Hi @HSheild ,
we already contacted our external support. I will forward this to microsoft if we come to no solution though.
Thanks a bunch for your help though.
Check out new user group experience and if you are a leader please create your group
Did you know that you can visit the Power Query Forum in Power BI and now Power Apps
Participate in the Power Virtual Agents Community Challenge
Power Platform release plan for the 2021 release wave 2 describes all new features releasing from October 2021 through March 2022.