cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
LucasFerronato
Level: Powered On

CDS is a data base?

Hello,

 

I've been trying to know and understand CDS.
As far as I understand, it's supposed to be a unique place for company data.

At first, I found only data that is from "Records" (dimensions / parameters - such as Employee Data, Cost Centers, Customers)
OK...

but what about the other data? Transactional Data? data that usually pass 1 million rows, easy.

Like accounting records, for example.
I tried to load a 1.7M accounting base and received the following error:
Failed to load in Common Data Service. Details: Number of rows (1,702,462) exceed the limit (500,000).

 

So, in an application, will I have to use CDS and Azure SQL Server (for example)?

 

Please, I'm trying to understand this for days.
I've already searched on several sites and have not found the answer.

 

Thanks a lot!

4 REPLIES 4
Community Support Team
Community Support Team

Re: CDS is a data base?

Hi @LucasFerronato ,

 

Did you mean that you want to integrate data into Common Data Service? If so, please check on below documentation for more detailed information.

 

If there is any misunderstanding, please feel free to tell.

 

Regards,

Mona

Community Support Team _ Mona Li
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
LucasFerronato
Level: Powered On

Re: CDS is a data base?

Hi Mona,

 

Tks for your reply.

 

Well, i mean somethink like Data Warehouse 

sometimes use integrations to store data, such as Sales data.
But I can also be used as intermediate storage. For example, my company uses an ERP, and monthly I need to extract financial data to do the calculations, analysis, reports, etc. (monthly update).
but in both cases, the data will exceed 500 thousand records ...

so, this is the question, CDS is a database, like SQL Server, for exemple.

kheiman
Level: Powered On

Re: CDS is a data base?

I think the 500,000 limit you are hitting is the limit of a single project execution. I found a reference here https://docs.microsoft.com/en-us/power-platform/admin/data-integrator saying: 

 

  • To optimize performance and not overload the apps, we currently limit project executions to 500k rows per execution per project.

Have you tried pushing the data through in 'chunks' of 500k each time?

 

CDS for Apps (database/platform/framework) should not limit # of rows in an entity.

 

 

LucasFerronato
Level: Powered On

Re: CDS is a data base?

Hello, 

 

maybe you are right! I was able to load in blocks more than 500 thousand lines.

but I came across other problems ... VERY SLOW, it took about 3-4 hours to load 500k records. Are there any limitations from a performance standpoint?

(I used Azure import, connected azure to Power Query inside CDS and asked to import ...)

Helpful resources

Announcements
firstImage

Microsoft Business Applications Virtual Launch Event

Join us for an in-depth look at the new innovations across Dynamics 365 and the Microsoft Power Platform.

firstImage

Watch Sessions On Demand!

Continue your learning in our online communities.

Power Platform 2019 release wave 2 plan

Power Platform 2019 release wave 2 plan

Features releasing from October 2019 through March 2020

FirstImage

Power Platform World Tour

Coming to a city near you

thirdimage

PowerApps Community User Group Member Badge

Fill out a quick form to claim your user group badge now!

FourthImage

Join PowerApps User Group!!

Connect, share, and learn with your peers year-round

Users Online
Currently online: 337 members 3,660 guests
Please welcome our newest community members: