cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Helper II
Helper II

Filter Search Gallery with more than 3000 SP List Items

Hello, 

 

I have a SP list with more than 3000 items.  

 

My gallery has a search box with the gallery set as below:

 

Filter(SOURCE, User().Email in 'Users', Year = 'Year Search Select_1'.Selected.Value, TextSearchBox1_1.Text in Description || TextSearchBox1_1.Text in TAGS || TextSearchBox1_1.Text in Atmosphere || TextSearchBox1_1.Text in 'HR' || TextSearchBox1_1.Text in 'Finance' || TextSearchBox1_1.Text in 'MGT ' || TextSearchBox1_1.Text in Location)

 

How can I pull in all data into a collection and then sort?  I've tried adding the below to OnStart but still only able to pull first 2000 records.

 

Concurrent(ClearCollect(CollectionA,Filter('SOURCE',ID<2000)),ClearCollect(CollectionB,Filter('SOURCE',ID >=2000 And ID<4000)));ClearCollect(AllIncidentsSource,CollectionA,CollectionB)

 

thanks!

3 ACCEPTED SOLUTIONS

Accepted Solutions

Hi @LYNCHJL ,

That code will never work as ID is not delegable.

I posted a workaround here a little while back.

 

Please click Accept as solution if my post helped you solve your issue. This will help others find it more readily. It also closes the item. If the content was useful in other ways, please consider giving it Thumbs Up.

 

View solution in original post

@LYNCHJL ,

No matter what you do with the ID, you cannot use > (greater than) on more than the first 2000 records. The only delegable operator you can use with ID is = (equals). I will post the full solution below

I keep a "shadow" ID number numeric field in the list and do this on the OnSuccess of my new record Forms. 

UpdateIf(
    MyListName,
    ID = MyNewForm.LastSubmit.ID,
    {MyShadowID: MyNewForm.LastSubmit.ID}
)

You will have to populate all existing records with the present ID number.

You can then Collect as many records as you want with

ClearCollect(
   colTemp,
   MySPList
);
If(
   CountRows(colTemp) = 2000,
   Set(
      vID,
      Max(colTemp,IDRef)  //My Shadow field
   );
   Collect(
      colTemp,
      Filter(
         MySPList,
         IDRef>vID
      )
   )
);
If(
   CountRows(colTemp) = 4000,
   Set(
      vID,
      Max(colTemp,IDRef)
   );
   Collect(
      colTemp,
      Filter(
         MySPList,
         IDRef>vID
      )
   )
)

Then keep going in batches of 2000 - If you do this to both lists, you will have fully delegable In filters.

 

Please click Accept as solution if my post helped you solve your issue. This will help others find it more readily. It also closes the item. If the content was useful in other ways, please consider giving it Thumbs Up.

 

View solution in original post

Hi @LYNCHJL ,

Strangely that is exactly what I do, but usually when I mention Workflows, I get confused questions . . .

Yes, by far the easiest way to set the number is a SharePoint Designer 2013 Workflow - set on Item Creation with Set Field in Current Item - set ItemID to ID. It works very quickly in the background and required no other attention.
You do however need to create a new numeric field and set it to the ID (easiest way to get a sequential number) I call mine IDRef.

Also do not use Concurrent - you may have deleted records and the Max() in my code picks up the highest number in the 2000 collected previously.

 

Please click Accept as solution if my post helped you solve your issue. This will help others find it more readily. It also closes the item. If the content was useful in other ways, please consider giving it Thumbs Up.

 

 

View solution in original post

8 REPLIES 8
Super User III
Super User III

Have you checked the delegation limit in File....Settings...Advanced Setting if the delegation is increased to 2000 because you are pulling in bataches of two 2000

 

You can then try:

 

Concurrent(

            ClearCollect(CollectionA, Filter(SOURCE,ID >= 1 && ID <= 2000)),

            ClearCollect(CollectionB, Filter(SOURCE,ID >= 2001 && ID <= 4000)));

ClearCollect(AllIncidentSource, CollectionA, CollectionB)

 

Edited

------------

If you like this post, give a Thumbs up. Where it solved your request, Mark it as a Solution to enable other users find it.

 

Hi @LYNCHJL ,

That code will never work as ID is not delegable.

I posted a workaround here a little while back.

 

Please click Accept as solution if my post helped you solve your issue. This will help others find it more readily. It also closes the item. If the content was useful in other ways, please consider giving it Thumbs Up.

 

View solution in original post

Hi @eka24,

 

yes I did check and it is set at 2000. 

I Updated my post please check

------------

If you like this post, give a Thumbs up. Where it solved your request, Mark it as a Solution to enable other users find it.

@LYNCHJL ,

No matter what you do with the ID, you cannot use > (greater than) on more than the first 2000 records. The only delegable operator you can use with ID is = (equals). I will post the full solution below

I keep a "shadow" ID number numeric field in the list and do this on the OnSuccess of my new record Forms. 

UpdateIf(
    MyListName,
    ID = MyNewForm.LastSubmit.ID,
    {MyShadowID: MyNewForm.LastSubmit.ID}
)

You will have to populate all existing records with the present ID number.

You can then Collect as many records as you want with

ClearCollect(
   colTemp,
   MySPList
);
If(
   CountRows(colTemp) = 2000,
   Set(
      vID,
      Max(colTemp,IDRef)  //My Shadow field
   );
   Collect(
      colTemp,
      Filter(
         MySPList,
         IDRef>vID
      )
   )
);
If(
   CountRows(colTemp) = 4000,
   Set(
      vID,
      Max(colTemp,IDRef)
   );
   Collect(
      colTemp,
      Filter(
         MySPList,
         IDRef>vID
      )
   )
)

Then keep going in batches of 2000 - If you do this to both lists, you will have fully delegable In filters.

 

Please click Accept as solution if my post helped you solve your issue. This will help others find it more readily. It also closes the item. If the content was useful in other ways, please consider giving it Thumbs Up.

 

View solution in original post

Thanks @WarrenBelz .

 

Can I replace ID with a new number field (lets say ItemID) that gets assigned by a workflow on creation?  I could set the workflow to set ItemID as ID.  I would then use ItemID in my ClearConnect statement.

 

Would this work or would i run into the same issue?

 

thanks!

The number filed should be an incremental field, so you cant use the ItemID Field.

Concurrent(

            ClearCollect(CollectionA, Filter(SOURCE,ID >= 1 && ID <= 2000)),

            ClearCollect(CollectionB, Filter(SOURCE,ID >= 2001 && ID <= 4000)));

ClearCollect(AllIncidentSource, CollectionA, CollectionB)

------------

If you like this post, give a Thumbs up. Where it solved your request, Mark it as a Solution to enable other users find it.

 

Hi @LYNCHJL ,

Strangely that is exactly what I do, but usually when I mention Workflows, I get confused questions . . .

Yes, by far the easiest way to set the number is a SharePoint Designer 2013 Workflow - set on Item Creation with Set Field in Current Item - set ItemID to ID. It works very quickly in the background and required no other attention.
You do however need to create a new numeric field and set it to the ID (easiest way to get a sequential number) I call mine IDRef.

Also do not use Concurrent - you may have deleted records and the Max() in my code picks up the highest number in the 2000 collected previously.

 

Please click Accept as solution if my post helped you solve your issue. This will help others find it more readily. It also closes the item. If the content was useful in other ways, please consider giving it Thumbs Up.

 

 

View solution in original post

Helpful resources

Announcements
New Badges

New Solution Badges!

Check out our new profile badges recognizing authored solutions!

New Power Super Users

Congratulations!

We are excited to announce the Power Apps Super Users!

Power Apps Community Call

Power Apps Community Call: February

Did you miss the call? Check out the Power Apps Community Call here.

Microsoft Ignite

Microsoft Ignite

Join digitally, March 2–4, 2021 to explore new tech that's ready to implement. Experience the keynote in mixed reality through AltspaceVR!

Top Solution Authors
Top Kudoed Authors
Users online (73,529)