cancel
Showing results for 
Search instead for 
Did you mean: 

Fix 500 Record Limitation

Problem:

  • At most 500 records can be returned at a time, whether filtered or not.
  • OneDrive, Dropbox, and Google Sheets can only read from the first 500 records.
  • CDS can read from all records, but can only return 500 records on a Filter.
  • It is not possible to Collect all records of an unfiltered datasource.

 

Suggestion 1:

Could you create a function to only pre-load and reload entity data to a temporary collection? It would do the same as this, but it could pull in everything without the 500 record limit:

Refresh(datasource);
ClearCollect(temp_datasource,datasource)

It would only import all records or none; so if you wanted to filter, you would have to do it after the temporary collection has been created. Furthermore, to reduce drag on the network, you could restrict this specific function to queue up or a quota like 100 runs per second.

 

 

Rationale: If we can collect the entire contents of a datasource to a temporary collection, we can operate on the temporary collection instead. This would allow for all aggregate functions and other non-delegable functions to work on the temporary collection. Then from time to time, we can reload the datasource to have the most updated data in the temporary collection.

 

 

Suggestion 2:

Could you create a function that would pull in 500 records at a time until all are pulled in? It would work like my solution, but it would be automated with fewer lines of code.

 

 

 

 

I am suggesting these functions as temporary solutions to not having all functions delegable.

 

Please consider this idea.

Status: Completed
Comments
Power Apps

So, I have a couple of questions.  I assume you've seen the post about how rasing it that high causes performance issues.  

 

1. Do you have a subscription/license to CDS?

2. What data source are you connecting to?  Are you working, for instance, with Excel or what?

3. Are these read or read/write scenarios?

 

Thanks,

Lance

 

 

Advocate II

Lance,

 

The app I am writing needs to work 99% of the time offline - it caches alot of meta data when the user is initially online, then collects field data and pictures while offline and is working fairly well overall.  I have one screen where the user has to select an item by working down through 3 levels of dropdowns and the total list had more that 3600 items in it (very narrow data set) - thus the frustration.  I have worked around this limitation now by caching two different lists which logically made sense, so it is not to bastardized.

 

I am working against SQL Azure for data & Azure Blob Storage for photos and signatures - lots of Flow.  The dropdown list information in this case are always readonly.

 

Thanks!

-Eric Niemiec

 

 

Anonymous
Not applicable

 

Both Filter method &  "=" are delegable then why it has a dependency on the Experimental Setting.

Experimental Setting says it is required for those which are non-delegable. (Delegation is not supported)

Am I missing something here.

 

Why I have to find out work arounds.

My list grew to 4000 items in 2 months. And users have stopped using it.

They directly go to the list form and submit.

 

Serious loss of face for my IT departement.

Why cannot we have a filter at the source based on criterias and be able to pull just the right information required?

Power Apps

It depends on your back-end data source.  What is your back-end data source?

Power Apps

@EricNiemiec   We need a better caching story for pure offline scenario like you're describing.  The current technology is really limited around the 2K number.  

Anonymous
Not applicable

SharePoint List

Advocate II

I am going live with an app next month that does a ton of offline caching.  I just wrote up a document with some of the issues i am having, some features that would be nice to have along with a discription of what our app does for our company - can i share this with you and perhaps show you a demo of the application as it stands right now?  I would love to share a real world field application and help the product become more robust.  Contact me at my first.last at outlook dot com.

 

Thanks!

-Eric Niemiec

Regular Visitor

This problem is still relevant. Will Microsoft do something about it? I need to see more than two thousand records in the search view and gallery