There should be way of filtering data directly from CRM instead of filtering on Collection/DataSource.
It will increase the PowerApps performance by avoiding the unwanted data from CRM.
Or there should be a way to create Custom Data Source which can have only data which is really needed to process in PowerApps and that should be based on query or filter criteria.
We are working on a feature to enable CDS Views inside canvas appsYou can test it in your app by enabling the experimental flag called "Improve data sources experience and Common Data Service views"
See this blog post for more details:https://powerapps.microsoft.com/en-us/blog/improved-data-source-selection-and-common-data-service-vi...Is that what you are looking for?