cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
v-shimud
Level: Powered On

Error loading 500000 rows into entity

Hi ,

 

Earlier when i was trying to load data >  3 million was reciving an error limit only allows 500000 rows. 

But when i am trying to load data by reducing the row size limit to 500000 receiving a new error. Please advise.

 

Our scenario is we are currently using MDS for managing our metadata and would like to move to powerapps. 

 

Load to Common Data Service failed. Details: ([Internal Server Error], [{"error":{"code":"InternalServerError","message":"An internal error occurred (tracking Id: 2f7693b8-17fb-4798-a14f-449f7d5576c0)."}}], The remote server returned an error: (500) Internal Server Error.), Failed to retrieve project schedule from https://management.azure.com/subscriptions/6989f3ba-b425-45dc-affb-047d53242a54/resourceGroups/d365-.... (Job ID: e9f03dda-d86e-4a9a-9a7d-192e3adf1828)

 

Thanks,

Shiva M

9 REPLIES 9
Super User
Super User

Re: Error loading 500000 rows into entity

@v-shimud 

So the question would be - how functional would an App be with > 3 million records in it?

What I'm getting to is that the App should be an interface to information and subsets of information and not an "application" that is going to perform complex actions on large datasets.

So, there are certainly limits and some are able to be worked around - but, in many cases, it comes back to "why do you need so many records in a user interface?"

PowerApps is highly focused around lists and galleries of data - so the question would be, would you want your users scrolling through >3 million records in a list?

 

This is seen many times for large data sources where the concept is - need to search it all, but PowerApps does not do well with searching large sources.  

So, the first think to look at is the DataSource...how much can you offset onto the server before it ever comes back to PowerApps.  

For example - a large SQL database.  We could pull down (with some effort) every row in the database in order to then successfuly search the entire database in PowerApps.  But, perhaps the better method is to create a View or perform a function on the database and only have it return what is necessary/desired.

 

So...not sure if this helps in your design process, but it is something to consider.

_____________________________________________________________________________________
Digging it? - Click on the Thumbs Up. Solved your problem? - Click on Accept as Solution. Others seeking the same answers will be happy you did.
v-shimud
Level: Powered On

Re: Error loading 500000 rows into entity

Thanks Randy,

 

I am just replicating one of our MDS model into powerapps, requirement is load distinct productid's (>3 million) into one entity and then create a lookup relationship with other entity. What i am trying to achieve here is when users create a new record of productid column they should not enter a different id other than that available in the lookup list. Basically they wont have to select anything or scroll through the list but just if they could paste the productid it should throw an error if that id is isnt in the list and succeed if it is.

 

The actual metadata table that holds data is very small, however we have to create some lookup columns which could reference large tables.

 

Could you let me know if there is any such limit that we cannot load data more than 500000 into powerapps or if above scenario dosent work with powerapps.

 

 

 

Super User
Super User

Re: Error loading 500000 rows into entity

@v-shimud 

So in this case you're really only trying to load that list in order to check a productid at time of submit for a new item.

If that is the case, can't you just perform a lookup for that productid prior to executing the submit of the new item?  That way you could determine if it is already there.

You could also do it during the entry of the item - and disable the submit button and make an error message visible if it exists.

Just a thought...perhaps it will fuel some thought for you as well.

_____________________________________________________________________________________
Digging it? - Click on the Thumbs Up. Solved your problem? - Click on Accept as Solution. Others seeking the same answers will be happy you did.
v-shimud
Level: Powered On

Re: Error loading 500000 rows into entity

@RandyHayes 

 

Our scenario is we setup the environment and share the app with business users, who are then responsible for loading data into it. So we cannot validate it prior loading.

Super User
Super User

Re: Error loading 500000 rows into entity

@v-shimud 

Yes, but what I am suggesting is that in your App, when you go to create a new record as you said, at that point have the logic in the formulas to do a lookup on the data and validate the productid as valid or not.

_____________________________________________________________________________________
Digging it? - Click on the Thumbs Up. Solved your problem? - Click on Accept as Solution. Others seeking the same answers will be happy you did.
v-shimud
Level: Powered On

Re: Error loading 500000 rows into entity

Thanks Randy

 

Part of my issue is solved by using lookup function. However my initial issue will still exist as it dosent allow more than 500000 rows in the entity. Please suggest any workaround for this as the lookup function has to lookup values in table that has > 3 million rows.

Highlighted
Super User
Super User

Re: Error loading 500000 rows into entity

@v-shimud 

That one I'm not sure on...I yield to anyone else that might have a response.

_____________________________________________________________________________________
Digging it? - Click on the Thumbs Up. Solved your problem? - Click on Accept as Solution. Others seeking the same answers will be happy you did.
v-shimud
Level: Powered On

Re: Error loading 500000 rows into entity

@RandyHayes  i am creating model-driven apps, i cant seem to find a way to write functions in model driven apps. So to get the option for writing lookup functions should i be using canvas apps. 

 

Thanks,

Shiva M

Super User
Super User

Re: Error loading 500000 rows into entity

@v-shimud 

Not entirely sure the differences you might be experiencing.  My world is around canvas apps at the moment.  Model driven is on the agenda for this month.

_____________________________________________________________________________________
Digging it? - Click on the Thumbs Up. Solved your problem? - Click on Accept as Solution. Others seeking the same answers will be happy you did.

Helpful resources

Announcements
thirdimage

Power Automate Community User Group Member Badge

Fill out a quick form to claim your user group badge now!

sixthImage

Power Platform World Tour

Find out where you can attend!

Power Platform 2019 release wave 2 plan

Power Platform 2019 release wave 2 plan

Features releasing from October 2019 through March 2020

fifthimage

Microsoft Learn

Learn how to build the business apps that you need.

Top Kudoed Authors
Users Online
Currently online: 97 members 4,838 guests
Recent signins:
Please welcome our newest community members: