Delegation workaround with AddColumns() and Filter() not showing a warning when it should.

[hjaf]

Scenario: There is a list of locations and a list of actions. The "ActionsList" have a single record lookup column ("Location") for the locations list. 

Sometimes I need to find actions for a selected location.
In my example the selected location has the ID of 10:
No warning:

CountRows(Filter(AddColumns(ActionList, "LocationId", Location.Id), LocationId = 10))

Delegation warning: 

CountRows(Filter(ActionList, Location.Id = 10))

I suspect that the warning should be there for AddColumns as well as filter, otherwise this will be a valid workaround for filtering lookup-columns.

There is some mention of delegation and AddColumns here:
https://docs.microsoft.com/en-us/powerapps/maker/canvas-apps/functions/function-table-shaping#descri...
But the examples are used outside and not inside the filter like the example.

AddRows is also described under the Delegable functions section here: 
https://docs.microsoft.com/en-us/powerapps/maker/canvas-apps/delegation-overview#table-shaping-funct...
"
AddColumns, DropColumns, RenameColumns, and ShowColumns partially support delegation. Formulas in their arguments can be delegated. However, the output of these functions are subject to the non-delegation record limit.
"
What does this actually mean? Is it if the content of the column exceeding the delegation record limit? Or the source of the lookup column exceeding the limit? (e.g. 3 records selected in the column but from a list with 100000 records). 

Any insight/clarification is appreciated 🙂

PS:
I also noticed some potential caveats in regards to performance using this workaround. A single request freezed the app in studio for a second. I can imagine this would freeze the app for a good while if used in a ForAll or a gallery.

[mdevaney]

@hjaf 
Great question.  I am happy to answer it for you.

The table of records returned by AddColumns is limited to 2,000 rows (assuming you have increased the delegation limit in advanced settings to the maximum).

Example:
You are using ADDCOLUMNS like this to add a new column to a list of 10,000 rows.  The function will read-in all 10,000 rows but only the 1st 2,000 rows will be returned.  Using ADDCOLUMNS inside FILTER will be subject to the same rules.

AddColumns(ActionList, "LocationId", Location.Id)

Let me know what other questions you have!

---
Please click "Accept as Solution" if my post answered your question so that others may find it more quickly. If you found this post helpful consider giving it a "Thumbs Up."

[hjaf]

Ok thanks for your quick response!

Just to clarify further.
A: If an action in position 5000 match the location, then it will NOT be returned, and only actions within the first 2000 items will be returned.
or
B: The action in position 5000 is returned, however, if all actions from number 2000 - 5000 match, only the first 2000 items will be returned and consequently the action in position 5000 is not returned?

If A should there not be a delegation warning in studio?
If B, then awesome! 🙂

[eka24]

Some points here:

1. With 100000 records, what datasource are you using?

2. The AddColumns is internal therefore not strange it not giving you delegation error.

3. With the CountRows and AddColumns, have you checked if you have the right results from the 100000 rows? For instance if you expect 100 rows, do you get all the 100 as the right output?

4. CountRows according do the Documentations is not Delegable, therefore you will get the delegation errors especially with your number of rows;

------------

If you like this post, give a Thumbs up. Where it solved your request, Mark it as a Solution to enable other users find it.

[eka24]

I will leave you with @mdevaney 

------------

If you like this post, give a Thumbs up. Where it solved your request, Mark it as a Solution to enable other users find it.

[hjaf]

@eka24 
1. Sorry, I forgot to mention the data-source. The example is using SharePoint lists.
2. This is what i'm trying to clarify. If the results are incomplete because of it, then should it not warn about this? *EDIT*
3. I have yet to produce the amount of data in two lists to test it fully, but filtering a shorter list with a lookup from a 20k+ list seems correct so far. 
4. the countRows can be ignored here. replace it with ClearCollect(colTest, *)


Edit 2: (mention)

[mdevaney]

@hjaf 

The answer is A.

PowerApps Studio “should” produce a delegation warning ⚠️ but it is not always reliable.  The best way to ensure your app will work as expected is to go into the advanced settings and change the delegation row limit to “1”.  Then only delegation compliant formulas will return multiple results.

---
Please click "Accept as Solution" if my post answered your question so that others may find it more quickly. If you found this post helpful consider giving it a "Thumbs Up."

[hjaf]

@mdevaney 

Ouch, that the delegation warning is unreliable to this extent may be somewhat serious. I have been naive enough to rely on the delegation warning 😕  This implies there are a lot of issues throughout my app, and judging from a few other posts this might produce a lot of unknown issues for a lot of users.

Could you confirm/clarify that setting the delegation limit to 1 returns multiple rows when formulas are delegation compliant?
I just tried this with 'ClearCollect(colTest, SharePointList)'. If I understood you correctly, it seems that no formula is delegation-compliant as long as powerapps sees any delegation-limitation on the source. 

[gebhardd]

@hjaf / @mdevaney 

I understand that this post is from 2020; however, it appears that delegation has not changed in these past years either so I fear your test and thoughts are still valid.

I too would like to know how reliable is the Data row limit setting?  As a test I created a SharePoint list called "MainSharePointList" and added 15 items to it.  This list also has a lookup column for a separate topic, but for this test I added a simple number column called "Flag" and set odd rows to 2 and even rows to 1.  

**EDIT: My following test, tests the delegation utilizing a Listbox; however, it appears that delegation will work with a Gallery and a DataTable but does not work with the Listbox, Dropdown box, or the Combobox.  I understand the Combobox because the Search function is not a delegate function, but why the same data source performs differently in the Listbox and Dropdown box, I did not see those mentioned specifically in the Microsoft documentation.

I added a Listbox to a blank Canvas PowerApp and set the File > Settings > General > Data row limit to 5

For the Listbox's Items property, I filter the MainSharePoint list on the Flag of 1, expecting to get Main Items 2,4,6,8,10,12, and 14; however, I only get back the first 5 items.

This very basic test seems to show that SharePoint is a 'server-based connection where delegation is not supported' meaning that, no matter what function you use, you will always be limited to the Data row limit you set?  

That would be quite the bummer dude.. Quite the bummer indeed.

[PowerAppsJunkie]

SharePoint does have delegation but you have to use the proper syntax in your formula's that are delegation compliant.

https://learn.microsoft.com/en-us/connectors/sharepointonline/#power-apps-delegable-functions-and-op...

I use a variety of controls in formula's that use delegation, how you use the control output and how you are addressing your SharePoint data source can make that delegable or non-delegable.

In this instance add columns does make formula's non-delegable up to your data limit set point. 500 default, 2000 max.

Setting that setting to 1 works in a way that you can more easily identify if you are going to have an issue as your datasource grows and your formulas are not delegable.

I know this is an older thread but it came up when I was searching for add columns so thought I'd give some feedback based on some of the assumptions made.