cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
zylantha2
Advocate II
Advocate II

Get Files (properties only) claiming that lookup columns threshhold exceeded from 7 July 2022 (other actions are fine, this action was fine until now)

We have a workflow that uses "Get Files (properties only)" to search for files in a library, which was working perfectly fine for a long time up until 5:40pm on 6 July 2022 (AEST UTC+10).

 

Now this morning (from 9am on 7 July 2022 AEST UTC+10) this flow is failing on this action, claiming that -

The query cannot be completed because the number of lookup columns it contains exceeds the lookup column threshold.

 

  1. I can confirm there have been NO changes to the library - NO new columns have been added.  I have a dev copy of the entire site that was taken last week, and the column list is identical - I have done a line by line comparison in Excel.
  2. There were no changes to the flow over between when it was working and now.
  3. The workaround of limiting the columns to a specific view will get the workflow up and running again - however this shouldn't be necessary (it was working fine up until yesterday without the columns being limited).
  4. Actions that operate on the same library other than "Get Files (properties only)" - such as "Get File Properties" do not seem to be failing, even without the columns being limited to a view.  

My current working theory is that this is a bug introduced by Microsoft overnight, and only affects this one action of "Get Files (properties only)".  Can anybody else correlate this issue?

1 ACCEPTED SOLUTION

Accepted Solutions
zylantha2
Advocate II
Advocate II

OK I'm going to close out this issue with a permanent, implementable fix for anybody else who lands in the same place we did now or in the future.

 

Root cause:  Confirmed that on 6/7 July 2022, Microsoft changed the default behaviour of the SharePoint API that is used by "Get Files (properties only)" and PowerApps to retrieve a file from SharePoint, so that if you don't specify a view to restrict the columns being returned, it will attempt to include SharePoint 2010 workflow status columns in the result set.  Previously it did not include these columns, and even if you do specify a view, it ignores these columns and doesn't return them anyway - it will only attempt to include these columns when you do not specify a result set.  No other API calls / PowerAutomate actions are affected, it is only this one call, which only applies to document libraries (lists are unaffected).  This won't affect you if you (a) don't have any defunct SP2010 workflow status columns, or (b) your total number of lookup columns + SP2010 workflow status columns is less than 12, so you have to try fairly hard to reproduce the issue (or already be in state #2 as we are / were).

 

Proper fix:  I have asked for Microsoft support to escalate this bug to the product group to fix / revert the change in the API, despite them discouraging me from doing so / telling me it's fruitless.  Somebody there should get a WSYP restitution for introducing this bug without warning / notice.

 

Implementable fix:  Delete the SP2010 workflow status columns from your library.  You can do this with site collection administrator privileges form PnP Powershell (you do not need SharePoint admin privileges), using the following commands:
1. Connect-PnPOnline -Url <your_site_url> -Interactive
2. $ctx =Get-pnpContext
3. $field = get-pnpField -List "<your_library_name>" -Identity "<your_sp2010_wf_status_column_internal_name>"
4. $ctx.load($field)
5. $field.sealed
>>> #check the status of sealed property, if it is already false, skip to step #7
6. $field.sealed = $false

7. $field.ReadOnlyField

>>> # check the status of the readonly property, if it is already false, skip to step #9 (or #11 if it was neither read only or sealed - but if that was the case you should have been able to delete the column from the web)

8. $field.ReadOnlyField = $false

9. $field.update()
10. $ctx.executequery()
11. Remove-PnPField -List "<your_library_name>" -Identity "your_sp2010_wf_status_column_internal_name" -Force

This should also work for lists that also still have SP2010 workflow status columns in them, and so I'm going to run this against those lists just in case Microsoft decide to break the API on lists as well, as could easily happen.

View solution in original post

8 REPLIES 8
Pstork1
Most Valuable Professional
Most Valuable Professional

Was a change made to any of the views in the library?  I know you said you aren't using a specific view to limit columns, but I believe it then defaults to the AllItems view.  If a change was made to that it could have affected the flow.  Or it could be a bug.  I don't have a library with enough lookup columns to test it though.



-------------------------------------------------------------------------
If I have answered your question, please mark your post as Solved.
If you like my response, please give it a Thumbs Up.

I don't believe so.  We don't have an AllItems view as it has been deleted/renamed at some point, and I don't think it does default to any particular view if you don't specify one - otherwise columns that are not in that default view would not be available (if you don't specify a view, you get 'all columns').

 

There are some lookup columns that normally don't count towards the lookup column threshhold - like "Created By", "Modified By", and "Checked Out To" - because these are system-generated and you can't remove them; perhaps there was a backend change that resulted in one of these suddenly being counted / included in this action when they weren't previously, resulting in the error occurring.

Pstork1
Most Valuable Professional
Most Valuable Professional

You can rename the AllItems view, but you can't delete it unless you make some other view the default.  There has to be a default view.  That is the one Power Automate will use if you don't specify a view or choose the option to not limit columns (which is not the default).  AS you can see from the screenshot below there is a specific choice for 'Use All Columns' which is not the default setting.  In my experience it does limit the columns unless you pick that entry.  I assume it limits them to the default view.

image.png

Also, the other columns you mentioned do count, but they are normally subtracted from the actual limit to arrive at the number 7.  As I said it could be a bug, but I would check to see if anything has changed in the default view.  Since you said it works if you specify a view I would think it is somehow defaulting to a view that has changed.



-------------------------------------------------------------------------
If I have answered your question, please mark your post as Solved.
If you like my response, please give it a Thumbs Up.

I have tried removing a lookup column from the default view, but it didn't help (and I don't think this was a new addition to the view, it's been there for some time).

 

There wasn't a screenshot attached to your message?

 

I still don't think it uses the default view if you don't specify one.  Also, if the default view was being used if you didn't specify a view, then it should be being used for all SharePoint actions - not just this one action which stopped working.  The issue is only with this one action - all other actions on the list other than this one are still fine, and none of them use a specific view to filter the columns.

Pstork1
Most Valuable Professional
Most Valuable Professional

The view selection isn't in all the actions, because most of them don't retrieve metadata. But I agree it could just be a bug.  I'm just trying to explore workarounds or possible explanations.  Sorry about forgetting the screenshot.  I edited the reply and added it.



-------------------------------------------------------------------------
If I have answered your question, please mark your post as Solved.
If you like my response, please give it a Thumbs Up.
zylantha2
Advocate II
Advocate II

I have a working theory as to why I'm seeing this issue on the library.

 

I mentioned earlier that I have a copy of the library on a dev site - and that the column list on that dev site is identical to that on the production site.  Except that isn't actually the case - I realised that there are various hidden columns that aren't displayed in the library settings, but are available when editing a view.

 

One of those hidden columns is an old SharePoint 2010 workflow status column - a workflow that hasn't been used for years for obvious reason, however we never deleted it and therefore the workflow status column is still hanging around.

 

That Workflow status column is in the production system (where we get this error), but it wasn't copied over to the dev site (where we are NOT getting the error) - so if you count the workflow status column as a lookup column (which it kinda is), then our prod system where we have this error DOES have one more lookup column than our dev system.

 

That does NOT explain why other Power Automate actions still work fine, and it's only "Get Files (Properties Only)" has an issue, however if we consider a possibility where back on 6 / 7 July there was a change made to this action's default behaviour (where no view was selected), if previously excluded workflow status columns - but now includes workflow status columns, then that would explain why this action started failing.

 

In fact I can CONFIRM that the last successful execution of this action on 6 July did NOT include this old workflow status column in the output - as I can see the raw outputs of the step still.

 

I can also confirm that if I set up a view that has every column in the library in it EXCEPT for the old workflow status column, then this action works successfully on this library with no problem.

 

From this I am concluding that -

  1. A workflow status column counts as a lookup column
  2. Prior to 7 July, these workflow status columns were not being included in the results "Get Files (Properties Only)" actions - and so did not count against your standard limit of 12 lookup columns.
  3. If you use a view to limit the columns being returned by "Get Files (Properties Only)" to exclude workflow status columns (a crude and annoying workaround) then this will successfully resolve the issue for that instance of the action, in that one workflow.

I am therefore strongly suspicious that I am right in that there was a change to the default behaviour of this action that caused this.  There is also no reason for it to behave like this - why would anybody want SharePoint 2010 workflow status columns to be returned any more - and as such it is definitely a bug.

 

Note also that we have seen this same issue ocurring in PowerApps when it queries the library for information about a document / folder, and we have not found a workaround for this.

 

Possibly the final workaround available is for me to forcefully delete these old workflow status columns from the library - which we should probably do, because they serve no ongoing purpose.  Unfortunately this isn't easy - I've already deleted the workflows but this didn't delete the columns, and it looks like I have to jump through some hoops to do that, but it is possible.

 

I'll report back if this fixes the issue, meanwhile this was a very interesting revelation that I had this afternoon about this issue - but it may help somebody else in confirming (or not being able to confirm) the issue in a test environment - since it's probably now impossible to create any new SharePoint 2010 workflows in order to replicate the circumstances that I have where I'm seeing the issue.

 

 

 

zylantha2
Advocate II
Advocate II

One minor update to the above.  I have demoed the issue and my findings and theory about what has caused the issue to Microsoft Support.

 

As part of this, I created a view that includes ALL COLUMNS of the list - including the defunct SharePoint 2010 workflow status column.  This view fails to display in SharePoint due to there being more than 12 lookup columns (it's fine if you exclude the workflow status column).  However, if you restrict the columns being returned by Get Files (properties only) in Power Automate, then it still works - it doesn't return the workflow status column.

 

This seems to reinforce my conclusion that it is only the default behaviour - when the view is blank, or set to "Use All Columns (do not limit)" that there is a problem in the action introduced on 6/7 July, in which it tries to include SP 2010 Workflow status columns in the output, when it never did previously.

 

That this issue also manifests itself in PowerApps (with no workaround) actually suggests this might be a SharePoint problem - in the default behaviour of the API call that both this action and PowerAutomate call to get the information out of SharePoint.

zylantha2
Advocate II
Advocate II

OK I'm going to close out this issue with a permanent, implementable fix for anybody else who lands in the same place we did now or in the future.

 

Root cause:  Confirmed that on 6/7 July 2022, Microsoft changed the default behaviour of the SharePoint API that is used by "Get Files (properties only)" and PowerApps to retrieve a file from SharePoint, so that if you don't specify a view to restrict the columns being returned, it will attempt to include SharePoint 2010 workflow status columns in the result set.  Previously it did not include these columns, and even if you do specify a view, it ignores these columns and doesn't return them anyway - it will only attempt to include these columns when you do not specify a result set.  No other API calls / PowerAutomate actions are affected, it is only this one call, which only applies to document libraries (lists are unaffected).  This won't affect you if you (a) don't have any defunct SP2010 workflow status columns, or (b) your total number of lookup columns + SP2010 workflow status columns is less than 12, so you have to try fairly hard to reproduce the issue (or already be in state #2 as we are / were).

 

Proper fix:  I have asked for Microsoft support to escalate this bug to the product group to fix / revert the change in the API, despite them discouraging me from doing so / telling me it's fruitless.  Somebody there should get a WSYP restitution for introducing this bug without warning / notice.

 

Implementable fix:  Delete the SP2010 workflow status columns from your library.  You can do this with site collection administrator privileges form PnP Powershell (you do not need SharePoint admin privileges), using the following commands:
1. Connect-PnPOnline -Url <your_site_url> -Interactive
2. $ctx =Get-pnpContext
3. $field = get-pnpField -List "<your_library_name>" -Identity "<your_sp2010_wf_status_column_internal_name>"
4. $ctx.load($field)
5. $field.sealed
>>> #check the status of sealed property, if it is already false, skip to step #7
6. $field.sealed = $false

7. $field.ReadOnlyField

>>> # check the status of the readonly property, if it is already false, skip to step #9 (or #11 if it was neither read only or sealed - but if that was the case you should have been able to delete the column from the web)

8. $field.ReadOnlyField = $false

9. $field.update()
10. $ctx.executequery()
11. Remove-PnPField -List "<your_library_name>" -Identity "your_sp2010_wf_status_column_internal_name" -Force

This should also work for lists that also still have SP2010 workflow status columns in them, and so I'm going to run this against those lists just in case Microsoft decide to break the API on lists as well, as could easily happen.

Helpful resources

Announcements

Update! June 13th, Community Ambassador Call for User Group Leaders and Super Users

Calling all Super Users & User Group Leaders   UPDATE:  We just wrapped up June's Community Ambassador monthly calls for Super Users and User Group Leaders. We had a fantastic call with lots of engagement. We are excited to share some highlights with you!    Big THANK YOU to our special guest Thomas Verhasselt, from the Copilot Studio Product Team for sharing how to use Power Platform Templates to achieve next generation growth.     A few key takeaways: Copilot Studio Cookbook Challenge:  Week 1 results are posted, Keep up the great work!Summer of Solutions:  Starting on Monday, June 17th. Just by providing solutions in the community, you can be entered to win tickets to Power Platform Community Conference.Super User Season 2: Coming SoonAll communities moving to the new platform end of July We also honored two different community members during the call, Mohamed Amine Mahmoudi and Markus Franz! We are thankful for both leaders' contributions and engagement with their respective communities. 🎉   Be sure to mark your calendars and register for the meeting on July 11th and stay up to date on all of the changes that are coming. Check out the Super User Forum boards for details.   We're excited to connect with you and continue building a stronger community together.   See you at the call!

Copilot Cookbook Challenge | Week 2 Results | Win Tickets to the Power Platform Conference

We are excited to announce the "The Copilot Cookbook Community Challenge is a great way to showcase your creativity and connect with others. Plus, you could win tickets to the Power Platform Community Conference in Las Vegas in September 2024 as an amazing bonus.   Two ways to enter: 1. Copilot Studio Cookbook Gallery:  https://aka.ms/CS_Copilot_Cookbook_Challenge 2. Power Apps Copilot Cookbook Gallery: https://aka.ms/PA_Copilot_Cookbook_Challenge   There will be 5 chances to qualify for the final drawing: Early Bird Entries: March 1 - June 2Week 1: June 3 - June 9Week 2: June 10 - June 16Week 3: June 17 - June 23Week 4: June 24 - June 30     At the end of each week, we will draw 5 random names from every user who has posted a qualifying Copilot Studio template, sample or demo in the Copilot Studio Cookbook or a qualifying Power Apps Copilot sample or demo in the Power Apps Copilot Cookbook. Users who are not drawn in a given week will be added to the pool for the next week. Users can qualify more than once, but no more than once per week. Four winners will be drawn at random from the total qualifying entrants. If a winner declines, we will draw again at random for the next winner.  A user will only be able to win once. If they are drawn multiple times, another user will be drawn at random. Prizes:  One Pass to the Power Platform Conference in Las Vegas, Sep. 18-20, 2024 ($1800 value, does not include travel, lodging, or any other expenses) Winners are also eligible to do a 10-minute presentation of their demo or solution in a community solutions showcase at the event. To qualify for the drawing, templates, samples or demos must be related to Copilot Studio or a Copilot feature of Power Apps, Power Automate, or Power Pages, and must demonstrate or solve a complete unique and useful business or technical problem. Power Automate and Power Pagers posts should be added to the Power Apps Cookbook. Final determination of qualifying entries is at the sole discretion of Microsoft. Weekly updates and the Final random winners will be posted in the News & Announcements section in the communities on July 29th, 2024. Did you submit entries early?  Early Bird Entries March 1 - June 2:  If you posted something in the "early bird" time frame complete this form: https://aka.ms/Copilot_Challenge_EarlyBirds if you would like to be entered in the challenge.   Week 1 Results:  Congratulations to the Week 1 qualifiers, you are being entered in the random drawing that will take place at the end of the challenge. Copilot Cookbook Gallery:Power Apps Cookbook Gallery:1.  @Mathieu_Paris 1.   @SpongYe 2.  @Dhanush 2.   @Deenuji 3.  n/a3.   @Nived_Nambiar  4.  n/a4.   @ManishSolanki 5.  n/a5.    n/a   Week 2 Results:  Congratulations to the Week 2 qualifiers, you are being entered in the random drawing that will take place at the end of the challenge. Copilot Cookbook Gallery:Power Apps Cookbook Gallery:1. Kasun_Pathirana1. ManishSolanki2. cloudatica2. madlad3. n/a3. SpongYe4. n/a4. n/a5. n/a5. n/a

Win free tickets to the Power Platform Conference | Summer of Solutions

We are excited to announce the Summer of Solutions Challenge!    This challenge is kicking off on Monday, June 17th and will run for (4) weeks.  The challenge is open to all Power Platform (Power Apps, Power Automate, Copilot Studio & Power Pages) community members. We invite you to participate in a quest to provide solutions to as many questions as you can. Answers can be provided in all the communities.    Entry Period: This Challenge will consist of four weekly Entry Periods as follows (each an “Entry Period”)   - 12:00 a.m. PT on June 17, 2024 – 11:59 p.m. PT on June 23, 2024 - 12:00 a.m. PT on June 24, 2024 – 11:59 p.m. PT on June 30, 2024 - 12:00 a.m. PT on July 1, 2024 – 11:59 p.m. PT on July 7, 2024 - 12:00 a.m. PT on July 8, 2024 – 11:59 p.m. PT on July 14, 2024   Entries will be eligible for the Entry Period in which they are received and will not carryover to subsequent weekly entry periods.  You must enter into each weekly Entry Period separately.   How to Enter: We invite you to participate in a quest to provide "Accepted Solutions" to as many questions as you can. Answers can be provided in all the communities. Users must provide a solution which can be an “Accepted Solution” in the Forums in all of the communities and there are no limits to the number of “Accepted Solutions” that a member can provide for entries in this challenge, but each entry must be substantially unique and different.    Winner Selection and Prizes: At the end of each week, we will list the top ten (10) Community users which will consist of: 5 Community Members & 5 Super Users and they will advance to the final drawing. We will post each week in the News & Announcements the top 10 Solution providers.  At the end of the challenge, we will add all of the top 10 weekly names and enter them into a random drawing.  Then we will randomly select ten (10) winners (5 Community Members & 5 Super Users) from among all eligible entrants received across all weekly Entry Periods to receive the prize listed below. If a winner declines, we will draw again at random for the next winner.  A user will only be able to win once overall. If they are drawn multiple times, another user will be drawn at random.  Individuals will be contacted before the announcement with the opportunity to claim or deny the prize.  Once all of the winners have been notified, we will post in the News & Announcements of each community with the list of winners.   Each winner will receive one (1) Pass to the Power Platform Conference in Las Vegas, Sep. 18-20, 2024 ($1800 value). NOTE: Prize is for conference attendance only and any other costs such as airfare, lodging, transportation, and food are the sole responsibility of the winner. Tickets are not transferable to any other party or to next year’s event.   ** PLEASE SEE THE ATTACHED RULES for this CHALLENGE**

Celebrating the June Super User of the Month: Markus Franz

Markus Franz is a phenomenal contributor to the Power Apps Community. Super Users like Markus inspire others through their example, encouragement, and active participation.    The Why: "I do this to help others achieve what they are trying to do. As a total beginner back then without IT background I know how overwhelming things can be, so I decided to jump in and help others. I also do this to keep progressing and learning myself." Thank you, Markus Franz, for your outstanding work! Keep inspiring others and making a difference in the community! 🎉  Keep up the fantastic work! 👏👏 Markus Franz | LinkedIn  Power Apps: mmbr1606  

Your Moment to Shine: 2024 PPCC’s Got Power Awards Show

For the third year, we invite you, our talented community members, to participate in the grand 2024 Power Platform Community Conference's Got Power Awards. This event is your opportunity to showcase solutions that make a significant business impact, highlight extensive use of Power Platform products, demonstrate good governance, or tell an inspirational story. Share your success stories, inspire your peers, and show off some hidden talents.  This is your time to shine and bring your creations into the spotlight!  Make your mark, inspire others and leave a lasting impression. Sign up today for a chance to showcase your solution and win the coveted 2024 PPCC’s Got Power Award. This year we have three categories for you to participate in: Technical Solution Demo, Storytelling, and Hidden Talent.      The Technical solution demo category showcases your applications, automated workflows, copilot agentic experiences, web pages, AI capabilities, dashboards, and/or more. We want to see your most impactful Power Platform solutions!  The Storytelling category is where you can share your inspiring story, and the Hidden Talent category is where your talents (such as singing, dancing, jump roping, etc.) can shine! Submission Details:  Fill out the submission form https://aka.ms/PPCCGotPowerSignup by July 12th with details and a 2–5-minute video showcasing your Solution impact. (Please let us know you're coming to PPCC, too!)After review by a panel of Microsoft judges, the top storytellers will be invited to present a virtual demo presentation to the judges during early August. You’ll be notified soon after if you have been selected as a finalist to share your story live at PPCC’s Got Power!  The live show will feature the solution demos and storytelling talents of the top contestants, winner announcements, and the opportunity to network with your community.  It's not just a showcase for technical talent and storytelling showmanship, show it's a golden opportunity to make connections and celebrate our Community together! Let's make this a memorable event! See you there!   Mark your calendars! Date and Time: Thursday, Sept 19th Location: PPCC24 at the MGM Grand, Las Vegas, NV 

Tuesday Tip | Accepting Solutions

It's time for another TUESDAY TIPS, your weekly connection with the most insightful tips and tricks that empower both newcomers and veterans in the Power Platform Community! Every Tuesday, we bring you a curated selection of the finest advice, distilled from the resources and tools in the Community. Whether you’re a seasoned member or just getting started, Tuesday Tips are the perfect compass guiding you across the dynamic landscape of the Power Platform Community.   To enhance our collaborative environment, it's important to acknowledge when your question has been answered satisfactorily. Here's a quick guide on how to accept a solution to your questions: Find the Helpful Reply: Navigate to the reply that has effectively answered your question.Accept as Solution: Look for the "Accept as Solution" button or link, usually located at the bottom of the reply.Confirm Your Selection: Clicking this button may prompt you for confirmation. Go ahead and confirm that this is indeed the solution.Acknowledgment: Once accepted, the reply will be highlighted, and the original post will be marked as "Solved". This helps other community members find the same solution quickly. By marking a reply as an accepted solution, you not only thank the person who helped you but also make it easier for others with similar questions to find answers. Let's continue to support each other by recognizing helpful contributions. 

Users online (2,544)