Do you have get a blue-dot suggestions with the formula that you provided?
If you have gotten a blue-dot suggestions with the formula that you provided, I think your thought is correct, you have faced Non-delegable issue.
If the formula could not be delegated within PowerApps, it would be processed locally. By default, you could only process 500 records at most locally. If you want to process much more records, you could consider change "Data row limit for non-delegable queries" within the Advanced settings of App Settings:
Then change the Value to 2000 (maximum value).
More details about the Non-Delegable limits in PowerApps, please check the following article: