cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
PowerAppsJunkie
Helper II
Helper II

Filter a SharePoint list with more than 2000 records

Is it possible to filter on a SharePoint list with more than 2000 records?

 

What I want to do is do a filter in for a column that has text like "NameA; NameB; NameC".  So there are 3 names in this column that are seperated by a ";"  I need to filter on a name so lets use "NameB" as an example.  So that if "NameB" exists anywhere in that column it will show the list item in a gallery.

I can't use StartsWith because well Name B could exist anywhere in that column...so it could be "NameC;NameA;NameB" on one list item or it might be "NameD;NameB;NameA" in a different list item.  I need to show all SP list items that could have "NameB" anywhere in that column.

The issue I'm running into is how to search anywhere in that column for "NameB" and not run into delegation issues because the list has 3,300 items in it.

So is there any work around or can I use Flow to help fix the problem?

2 ACCEPTED SOLUTIONS

Accepted Solutions
v-xida-msft
Community Support
Community Support

Hi @PowerAppsJunkie , 

Currently, the 'in' operator is not delegable within SP Connector, it means that you could not delegate the data process from your app to your SP List data source, instead, you could only process data locally.

 

In default, you could only process 500 records locally at most, you coud change the "Data row limit for Non-delegable queries" option to maximum value -- 2000 in Advanced settings of App settings, then you could process 2000 records locally in your app.

 

As an alternative solution, you could consider bulk-load your SP List records into your app. Please check and see if the alternative solution mentioned within the following thread:

https://powerusers.microsoft.com/t5/General-Discussion/Pulling-in-large-ish-SQL-tables/m-p/243777#M7...

On your side, please set the OnStart property of the App to following:

 

Concurrent(
          ClearCollect(col1, Filter('YourSPList', ID >= 1 && ID <= 2000)),
          ClearCollect(col2, Filter('YourSPList', ID >= 2001 && ID <= 4000))
);
ClearCollect(MergedCollection, col1, col2)

Then modify your Filter formula as below:

Filter(MergedCollection, "NameB" in NameColumn)

then re-load your app (fire the OnStart proeprty of the App), then check if the issue is solved.

Note: Please make sure you have changed the "Data row limit for Non-delegable queries" option to maximum value -- 2000 in Advanced settings of App settings already.

 

Best regards,

 

Community Support Team _ Kris Dai
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

Well, you'll have to have a job run through all the records that exist now first so you can update the existing records to the new column as well. But you will use that RealID lets call it column, as your filter so RealID >= 2001 && RealID <= 4000 etc.

View solution in original post

21 REPLIES 21
yashag2255
Super User
Super User

Hi @PowerAppsJunkie 

 

Can you try to use either of the below formulas?
 
Expressions:
Filter(SPListName, "NameB" in ColumnName)
Or
Filter(SPListName, "NameB" in Split(ColumnName, ";"))
 
These expression works for Text type column, if this differs please share more info about the column type.
 
Hope this Helps!
 
If this reply has answered your question or solved your issue, please mark this question as answered. Answered questions helps users in the future who may have the same issue or question quickly find a resolution via search. If you liked my response, please consider giving it a thumbs up. THANKS!
v-xida-msft
Community Support
Community Support

Hi @PowerAppsJunkie , 

Currently, the 'in' operator is not delegable within SP Connector, it means that you could not delegate the data process from your app to your SP List data source, instead, you could only process data locally.

 

In default, you could only process 500 records locally at most, you coud change the "Data row limit for Non-delegable queries" option to maximum value -- 2000 in Advanced settings of App settings, then you could process 2000 records locally in your app.

 

As an alternative solution, you could consider bulk-load your SP List records into your app. Please check and see if the alternative solution mentioned within the following thread:

https://powerusers.microsoft.com/t5/General-Discussion/Pulling-in-large-ish-SQL-tables/m-p/243777#M7...

On your side, please set the OnStart property of the App to following:

 

Concurrent(
          ClearCollect(col1, Filter('YourSPList', ID >= 1 && ID <= 2000)),
          ClearCollect(col2, Filter('YourSPList', ID >= 2001 && ID <= 4000))
);
ClearCollect(MergedCollection, col1, col2)

Then modify your Filter formula as below:

Filter(MergedCollection, "NameB" in NameColumn)

then re-load your app (fire the OnStart proeprty of the App), then check if the issue is solved.

Note: Please make sure you have changed the "Data row limit for Non-delegable queries" option to maximum value -- 2000 in Advanced settings of App settings already.

 

Best regards,

 

Community Support Team _ Kris Dai
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

@v-xida-msft 

I used the collection as you put in but it's only grabbing the first collection of 2000 records.  I don't get anything from the 2nd collection and when I preview it via view it shows the collection as blank.

It's acting like it's not pulling in the records.

I created a button to pull them in manually into a collection like this:

 

ClearCollect(col2, Filter('YourSPList', ID >= 2001 && ID <= 4000))

But when I click that it doesn't pull any records in but when I click the button when changing it to this...it works...


ClearCollect(col1, Filter('YourSPList', ID >= 1 && ID <= 2000))

 

Any ideas?

ID is not delegable, so you're only getting the first 2000 id's no matter what you put in there. You have to go in an create a flow that takes your ID's and replicates it to a number column that you use in the list.

So I create a flow when a new item is added to the list to lookup the ID of the item ID and copy it to a new number field? Then in the clearcollect grab that column instead?

Well, you'll have to have a job run through all the records that exist now first so you can update the existing records to the new column as well. But you will use that RealID lets call it column, as your filter so RealID >= 2001 && RealID <= 4000 etc.

Yup I was just writing a flow to do the ID's over to a new column now and then implementing a new flow for any new items added.

Will test as soon as the flow finishes but sounds like it should work as intended.

Now that Lookup columns and choice columns are delegable, I don't know if Calculated columns work, but you could try to create a calculated column first and just try it so that the calculated column called CalcID and use [ID] as it's value. Might work, I'd try that to see if it's updated to use delegation real quick, if not, then you'll unfortunately have to go the other route.

Ok I tried a CalcID field with [ID] and in the formula it says invalid argument type expecting a number value.

I did calculated field as a number value and a a single line of text but both give me the error

Helpful resources

Announcements

Back to Basics Tuesday Tip #10: Community Support

This is the TENTH post in our ongoing series dedicated to helping the amazing members of our community--both new members and seasoned veterans--learn and grow in how to best engage in the community! Each Tuesday, we feature new content that will help you best understand the community--from ranking and badges to profile avatars, from Super Users to blogging in the community. Our hope is that this information will help each of our community members grow in their experience with Power Platform, with the community, and with each other!   This Week: All About Community Support   Whether you're a seasoned community veteran or just getting started, you may need a bit of help from time to time! If you need to share feedback with the Community Engagement team about the community or are looking for ways we can assist you with user groups, events, or something else, Community Support is the place to start.   Community Support is part of every one of our communities, accessible to all our community members.     Power Apps: https://powerusers.microsoft.com/t5/Community-Support/ct-p/pa_community_support Power Automate: https://powerusers.microsoft.com/t5/Community-Support/ct-p/mpa_community_support Power Pages: https://powerusers.microsoft.com/t5/Community-Support/ct-p/mpp_community_support Copilot Studio: https://powerusers.microsoft.com/t5/Community-Support/ct-p/pva_community-support   Within each community's Community Support page, you'll find three distinct areas, each with a different focus to help you when you need support from us most.     Community Accounts & Registration is the go-to source for any and all information related to your account here in the community. It's full of great knowledge base articles that will help you manage your community account and know what steps to take if you wish to close your account.  ●  Power Apps  ●  Power Automate  ●  Power Pages, ●  Copilot Studio      Using the Community is your source for assistance with everything from Community User Groups to FAQ's and more. If you want to know what kudos are, how badges work, how to level up your User Group or something else, you will probably find the answers here. ●  Power Apps   ● Power Automate    ●  Power Pages  ●  Copilot Studio      Community Feedback is where you can share opportunities, concerns, or get information from the Community Engagement team. It's your best place to post a question about an issue you're having in the community, a general question you need answered. Whatever it is, visit Community Feedback to get the answers you need right away. Our team is honored to partner with you and can't wait to help you!   ●  Power Apps  ● Power Automate   ● Power Pages   ● Copilot Studio  

Microsoft Ignite 2023: The Recap

What an amazing event we had this year, as Microsoft showcased the latest advancements in how AI has the potential to reshape how customers, partners and developers strategize the future of work. Check out below some of our handpicked videos and Ignite announcements to see how Microsoft is driving real change for users and businesses across the globe.   Video Highlights Click the image below to check out a selection of Ignite 2023 videos, including the "Microsoft Cloud in the era of AI" keynote from Scott Guthrie, Charles Lamanna, Arun Ulag, Sarah Bird, Rani Borkar, Eric Boyd, Erin Chapple, Ali Ghodsi, and Seth Juarez. There's also a great breakdown of the amazing Microsoft Copilot Studio with Omar Aftab, Gary Pretty, and Kendra Springer, plus exciting sessions from Rajesh Jha, Jared Spataro, Ryan Jones, Zohar Raz, and many more.     Blog Announcements Microsoft Copilot presents an opportunity to reimagine the way we work—turning natural language into the most powerful productivity tool on the planet. With AI, organizations can unearth value in data across productivity tools like business applications and Microsoft 365. Click the link below to find out more.     Check out the latest features in Microsoft Power Apps that will help developers create AI-infused apps faster, give administrators more control over managing thousands of Microsoft Power Platform makers at scale, and deliver better experiences to users around the world. Click the image below to find out more.     Click below to discover new ways to orchestrate business processes across your organization with Copilot in Power Automate. With its user-friendly interface that offers hundreds of prebuilt drag-and-drop actions, more customers have been able to benefit from the power of automation.     Discover how Microsoft Power Platform and Microsoft Dataverse are activating the strength of your enterprise data using AI, the announcement of “plugins for Microsoft Copilot for Microsoft 365”, plus two new Power Apps creator experiences using Excel and natural language.       Click below to find out more about the general availability of Microsoft Fabric and the public preview of Copilot in Microsoft Fabric. With the launch of these next-generation analytics tools, you can empower your data teams to easily scale the demand on your growing business.     And for the rest of all the good stuff, click the link below to visit the Microsoft Ignite 2023 "Book of News", with over ONE HUNDRED announcements across infrastructure, data, security, new tools, AI, and everything else in-between!        

Back to Basics Tuesday Tip #9: All About the Galleries

This is the ninth post in our series dedicated to helping the amazing members of our community--both new members and seasoned veterans--learn and grow in how to best engage in the community! Each Tuesday, we feature new content that will help you best understand the community--from ranking and badges to profile avatars, from Super Users to blogging in the community. Our hope is that this information will help each of our community members grow in their experience with Power Platform, with the community, and with each other!     Today's Tip: All About the Galleries Have you checked out the library of content in our galleries? Whether you're looking for the latest info on an upcoming event, a helpful webinar, or tips and tricks from some of our most experienced community members, our galleries are full of the latest and greatest video content for the Power Platform communities.   There are several different galleries in each community, but we recommend checking these out first:   Community Connections & How-To Videos Hosted by members of the Power Platform Community Engagement  Team and featuring community members from around the world, these helpful videos are a great way to "kick the tires" of Power Platform and find out more about your fellow community members! Check them out in Power Apps, Power Automate, Power Pages, and Copilot Studio!         Webinars & Video Gallery Each community has its own unique webinars and videos highlighting some of the great work being done across the Power Platform. Watch tutorials and demos by Microsoft staff, partners, and community gurus! Check them out: Power Apps Webinars & Video Gallery Power Automate Webinars & Video Gallery Power Pages Webinars & Video Gallery Copilot Studio Webinars & Video Gallery   Events Whether it's the excitement of the Microsoft Power Platform Conference, a local event near you, or one of the many other in-person and virtual connection opportunities around the world, this is the place to find out more about all the Power Platform-centered events. Power Apps Events Power Automate Events Power Pages Events Copilot Studio Events   Unique Galleries to Each Community Because each area of Power Platform has its own unique features and benefits, there are areas of the galleries dedicated specifically to videos about that product. Whether it's Power Apps samples from the community or the Power Automate Cookbook highlighting unique flows, the Bot Sharing Gallery in Copilot Studio or Front-End Code Samples in Power Pages, there's a gallery for you!   Check out each community's gallery today! Power Apps Gallery Power Automate Gallery Power Pages Gallery Copilot Studio Gallery

Unlocking the Power of Community: A Journey with Featued User Group leaders Geetha Sivasailam and Ben McMann

In the bustling world of technology, two dynamic leaders, Geetha Sivasailam and Ben McMann, have been at the forefront, steering the ship of the Dallas Fort Worth Power Platform User Group since its inception in February 2019. As Practice Lead (Power Platform | Fusion Dev) at Lantern, Geetha brings a wealth of consulting experience, while Ben, a key member of the Studio Leadership team at Lantern, specializes in crafting strategies that leverage Microsoft digital technologies to transform business models.   Empowering Through Community Leadership Geetha and Ben's journey as user group leaders began with a simple yet powerful goal: to create a space where individuals across the DFW area could connect, grow their skills, and add value to their businesses through the Power Platform. The platform, known for its versatility, allows users to achieve more with less code and foster creativity.   The Power of Community Impact Reflecting on their experiences, Geetha and Ben emphasize the profound impact that community engagement has had on both their professional and personal lives. The Power Platform community, they note, is a wellspring of resources and opportunities, fostering continuous learning, skill enhancement, and networking with industry experts and peers.   Favorite Moments and Words of Wisdom The duo's favorite aspect of leading the user group lies in witnessing the transformative projects and innovations community members create with the Power Platform. Their advice to aspiring user group leaders? "Encourage diverse perspectives, maintain an open space for idea-sharing, stay curious, and, most importantly, have fun building a vibrant community."   Building Bridges, Breaking Barriers Geetha and Ben encourage others to step into the realm of user group leadership, citing the rewarding experience of creating and nurturing a community of like-minded individuals. They highlight the chance to influence, impact, and positively guide others, fostering connections that extend beyond mere technology discussions.   Joining a User Group: A Gateway to Growth The leaders stress the importance of joining a user group, emphasizing exposure to diverse perspectives, solutions, and career growth opportunities within the Power Platform community. "Being part of such a group provides a supportive environment for seeking advice, sharing experiences, and navigating challenges."   A Year of Milestones Looking back at the past year, Geetha and Ben express pride in the group's growth and global participation. They recount the enriching experience of meeting members in person at the Microsoft Power Platform conference, showcasing the diverse range of perspectives and guest speakers that enriched the community's overall experience.   Continuous Learning on the Leadership Journey As user group leaders, Geetha and Ben recognize the continuous learning curve, blending interpersonal skills, adaptability, and dedication to foster a vibrant community. They highlight the importance of patience, persistence, and flexibility in achieving group goals, noting the significance of listening to the needs and suggestions of group members.They invite all tech enthusiasts to join the Dallas Fort Worth Power Platform User Group, a thriving hub where the power of community propels individuals to new heights in the dynamic realm of technology.

Visit the Community Lounge at Microsoft Ignite!

Are you attending Microsoft Ignite in Seattle this week? If so, we'd love to see you at the Community Lounge! Hosted by members of our Community team, it's a great place to connect, meet some Microsoft executives, and get a sticker or two. And if you're an MVP there are some special opportunities to meet up!     The Community Lounge is more than just a space—it's a hub of activity, collaboration, and camaraderie. So, dive in, explore, and make the most of your Microsoft Ignite experience by immersing yourself in the vibrant and dynamic community that awaits you.Find out the schedule and all the details here: Community Lounge at Ignite! See you at #MSIgnite!    

Back to Basics Tuesday Tip #8: Subscriptions & Notifications

This is the eighth post in our series dedicated to helping the amazing members of our community--both new members and seasoned veterans--learn and grow in how to best engage in the community! Each Tuesday, we feature new content that will help you best understand the community--from ranking and badges to profile avatars, from Super Users to blogging in the community. Our hope is that this information will help each of our community members grow in their experience with Power Platform, with the community, and with each other!   This Week: All About Subscriptions & Notifications Subscribing to a CategorySubscribing to a TopicSubscribing to a LabelBookmarksManaging & Viewing your Subscriptions & BookmarksA Note on Following Friends on Mobile   Subscriptions ensure that you receive automated messages about the most recent posts and replies. There are multiple ways you can subscribe to content and boards in the community! (Please note: if you have created an AAD (Azure Active Directory) account you won't be able to receive e-mail notifications.)   Subscribing to a Category  When you're looking at the entire category, select from the Options drop down and choose Subscribe.   You can then choose to Subscribe to all of the boards or select only the boards you want to receive notifications. When you're satisfied with your choices, click Save.   Subscribing to a Topic You can also subscribe to a single topic by clicking Subscribe from the Options drop down menu, while you are viewing the topic or in the General board overview, respectively.   Subscribing to a Label You can find the labels at the bottom left of a post.From a particular post with a label, click on the label to filter by that label.          This opens a window containing a list of posts with the label you have selected. Click Subscribe.   Note: You can only subscribe to a label at the board level. If you subscribe to a label named 'Copilot' at board #1, it will not automatically subscribe you to an identically named label at board #2. You will have to subscribe twice, once at each board.   Bookmarks Just like you can subscribe to topics and categories, you can also bookmark topics and boards from the same menus! Simply go to the Topic Options drop down menu to bookmark a topic or the Options drop down to bookmark a board.   The difference between subscribing and bookmarking is that subscriptions provide you with notifications, whereas bookmarks provide you a static way of easily accessing your favorite boards from the My subscriptions area.   Managing & Viewing Your Subscriptions & Bookmarks To manage your subscriptions, click on your avatar and select My subscriptions from the drop-down menu. From the Subscriptions & Notifications tab, you can manage your subscriptions, including your e-mail subscription options, your bookmarks, your notification settings, and your email notification format.     You can see a list of all your subscriptions and bookmarks and choose which ones to delete, either individually or in bulk, by checking multiple boxes.     A Note on Following Friends on Mobile Adding someone as a friend or selecting Follow in the mobile view does not allow you to subscribe to their activity feed. You will merely be able to see your friends’ biography, other personal information, or online status, and send messages more quickly by choosing who to send the message to from a list, as opposed to having to search by username.              

Top Solution Authors
Top Kudoed Authors
Users online (4,160)