cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Splush
Helper I
Helper I

Are Dataflows being slowed down when the CDS database is full?

Hey,

 

I have a dataflow that is just extracting data from an excel stored in sharepoint and replaces the data in a cds table with it.

Somehow this takes more than an hour to complete which is unacceptable for my usecase.

I also noticed that the organisations database storage is full.

Is it possible that the dataflow takes so long because of that?

In the docs it just says that you cant create new environments if the storage is full.

 

8 REPLIES 8
HSheild
Super User
Super User

Hi @Splush ,

 

It is unclear whether exceeded database capacity would cause your dataflow to be throttled.  It is more likely that the dataflow is being throttled by the Service Protection API limits.  See Service Protection API Limits (Microsoft Dataverse) - Power Apps | Microsoft Docs for details.

 

How many rows do you have in your Excel file?

 

It is also a good idea to also sort out your database capacity issue, just to rule that out and be compliant with Microsoft's licensing terms.

Hi @HSheild ,

 

thanks a bunch for your answer. I dont think that the API limits would be the limiting factor here. It is just one Sharepoint request every 10 minutes. The excel itself is around 5000 rows by 50 columns in size. A comparable power query would take just a few seconds

Hi @Splush ,

 

Sounds like you are potentially trying to update 5000 rows in CDS every 10 minutes?  If that is the case then you will be coming up against the Entitlement Limits Requests limits and allocations - Power Platform | Microsoft Docs


Depending on the license you have, you can only do up to 20,000 API requests per 24hrs.  Have you noticed that the first few runs go quickly and then overtime the runs get slower and slower?

Fubar
Solution Sage
Solution Sage

Not sure what the current situation is with things, but (other than a Trial) it used to be that if you had exceeded your storage capacity you were unable to add new records (you could still read) but save of a new record would fail.

Hey @Fubar ,

right now we can exceed the limit by a few GB and no other processes seem to be affected by that.

Its just that we can't create new environments.

Hey @HSheild ,

no, the first few flows run at the same speed as the other ones later that day.

I also really can't believe that each row entry would be a seperate API request.

I mean that would mean that all data flows above 20k would be affected and potentially rendering data flows as a whole useless.Bildschirmfoto 2020-12-17 um 20.39.04.png

 

Hi @Splush,

 

I'm fairly sure it is one API call per row, however, I can't 100% confirm that.  Have you looked at raising a Support case with Microsoft? Power Platform admin center (microsoft.com)  That would be my next step if no-one else can provide a solution on this forum.

Hi @HSheild ,

we already contacted our external support. I will forward this to microsoft if we come to no solution though.

Thanks a bunch for your help though.

Helpful resources

Announcements
PA User Group

Welcome to the User Group Public Preview

Check out new user group experience and if you are a leader please create your group

Power Query PA Forum 768x460.png

Check it out!

Did you know that you can visit the Power Query Forum in Power BI and now Power Apps

V3_PVA CAmpaign Carousel.png

Community Challenge - Giveaways!

Participate in the Power Virtual Agents Community Challenge

Carousel 2021 Release Wave 2 Plan 768x460.jpg

2021 Release Wave 2 Plan

Power Platform release plan for the 2021 release wave 2 describes all new features releasing from October 2021 through March 2022.

Users online (3,029)