cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
mmollet
Community Champion
Community Champion

Add Row to Excel Table issue

I have a flow that is working well... when it works. The issue is I have a step where I take a compose output in JSON format and add a step that adds it into an excel table. This all works but sometimes this step will take 10+ min while other times it takes 1 sec. When I check that same point in the run on other flow runs it completes in 1 sec. Any ideas on how to make this occur less or how to make it retry faster etc? This is killing the flows usability as 2 min is acceptable but 22 min is far from is. 

 

This is the issue I am getting.

Screenshot 2023-03-20 at 2.17.51 PM.png

 

And here is the same point in the flow the previous run where it had no issues. 

Screenshot 2023-03-20 at 2.25.49 PM.png

 

 

TYIA!


If this post solves your issue please mark it as a solution.
1 ACCEPTED SOLUTION

Accepted Solutions

I think I am understanding your flow much better now. I have not done anything exactly like this, but I think the concurrency in your Apply to each may be causing the issue.

 

I thought initially you had some process where you just needed to add rows to an existing spreadsheet. In reality, you are getting data from SharePoint, processing it, creating a separate spreadsheet for a number of people from a template file, and then sending this to them. You said you have concurrency set to 50. I would try lowering that until performance is acceptable. There is probably some throttling going on at this setting.

 

This lists some of the limits of concurrency: Limits and configuration - Power Automate | Microsoft Learn

 

One thing I have done when I have hit some throttling limits is to put the steps you have in your apply to each inside of a do Until. I then limit how many items can be returned from the Get items action (in situations where I have applied this, the throttling limit kicks in at 100, so I limit it to 99 items. I create a variable named Completed with the default value of Pending and set the Do until to run until it equals Completed.

 

Then after the get items action, I have a condition set to where the length of the items from the Get items action is greater than 0. The condition uses this expression:

int(length(outputs('Get_items')?['body/value']))

 If it is greater than 0, it runs down the Yes side and loops again. If it is zero, then it goes down the No side and sets the Completed variable to completed so it does not loop again.

 

In my SharePoint list where I am getting the items, there is a column with the default value of Pending. In my apply to each, it updates this to Processed. The Get items action has a filter query to get the pending items.

getitems.jpg

At the end of the yes, there is a delay of two minutes to allow the throttle limit to reset. The flow looks like this:

David_MA_1-1679498913628.png

 

View solution in original post

12 REPLIES 12
David_MA
Super User
Super User

Have you tried enabling concurrency in the apply to each action?

wskinnermctc
Memorable Member
Memorable Member

I've noticed those issues if the workbook is opened. Even more if the workbook is opened and a cell is selected like a value is about to be typed into the cell.

It is like PA is trying to call the workbook but getting denied because it is opened or being used.

 

You can do it to an open workbook, but there seems to be some delay or timeouts with power automate.

Most likely related to Excel getting locked as explained here: Excel file is locked for editing - Microsoft Support. This probably affects Power Automate, which is...

 

I have encountered this in some of my flows, but since it hasn't failed, I have not looked into it much further. I've tried to duplicate this in different ways and the flow has always added or updated the row. There has to be some certain behavior in Excel that triggers it to lock it from editing.

 

This may be key from the above Microsoft post:
Tip: Sometimes a file may get locked if everyone editing isn't using a version that supports co-authoring. To co-author together, make sure everyone is using the Microsoft 365 subscription version of Excel or Excel for the web.

 

I know in my company we still have a few users on Office 2016 for some legacy applications support.

wskinnermctc
Memorable Member
Memorable Member

@mmollet Are you using the Excel file as a source for something else? If there is an issue about time delays, you could use a SharePoint List instead of Excel.

mmollet
Community Champion
Community Champion

My flow gets a start date, an end date, the user's email, and a json array of users to get rows for from power apps. It creates a single excel file with a single table for each user passed in and is a table of their time entries between the two dates passed in. I will post Images of the flow below.

 

Flow Details

  • The outer loop is set to a concurrency of 50 as I have 20-30 users and why not run them all in parallel. (if i am misunderstanding how this works please let me know)
  • The inner loop has a concurrency of 1. (not sure if this is correct but seems to work fine)
  • Get items filter query makes sure to only pull rows for the current user and only rows that have a non-null end date.
  • Gets the template, creates a copy of that file with username+datetime as file name then loops through the output of above step and inserts rows with dates inside the valid range by using a compose statement to build the object.
  • Delay to allow row addition to finish
  • Get contents of the new excel file/table
  • Send as attachment
  • Once this all works the file will simply be deleted after the email is sent as it is no longer needed. 
  • No other users have access to the file that is created, the template, the SP list, or the folder any of the items are even in.

 

Also while I'm here I am getting a blank line at the top of each table for each user. Has 0 impact but my perfectionist brain is really not enjoying it all so any ideas on fixing that would be appreciated as well. Thank you so much for all the help and suggestions.

Screenshot 2023-03-22 at 8.53.54 AM.png

 

Screenshot 2023-03-22 at 8.54.15 AM.png

 

Screenshot 2023-03-22 at 8.54.23 AM.png

 

Screenshot 2023-03-22 at 8.53.36 AM.png

 


If this post solves your issue please mark it as a solution.

Grabbing data from a single SP list based on UserName column and EndTime column via get items call from flow. Output is an excel table/file that also has formulas for calculating line item totals as well as overall totals for two different types of entries, "Travel" entries and anything non-travel. This file is then attached to an email and sent to the accountant as that users time sheet for the week. The file is created using a template word doc like this: 

 

Screenshot 2023-03-22 at 9.42.30 AM.png

 


If this post solves your issue please mark it as a solution.

I think I am understanding your flow much better now. I have not done anything exactly like this, but I think the concurrency in your Apply to each may be causing the issue.

 

I thought initially you had some process where you just needed to add rows to an existing spreadsheet. In reality, you are getting data from SharePoint, processing it, creating a separate spreadsheet for a number of people from a template file, and then sending this to them. You said you have concurrency set to 50. I would try lowering that until performance is acceptable. There is probably some throttling going on at this setting.

 

This lists some of the limits of concurrency: Limits and configuration - Power Automate | Microsoft Learn

 

One thing I have done when I have hit some throttling limits is to put the steps you have in your apply to each inside of a do Until. I then limit how many items can be returned from the Get items action (in situations where I have applied this, the throttling limit kicks in at 100, so I limit it to 99 items. I create a variable named Completed with the default value of Pending and set the Do until to run until it equals Completed.

 

Then after the get items action, I have a condition set to where the length of the items from the Get items action is greater than 0. The condition uses this expression:

int(length(outputs('Get_items')?['body/value']))

 If it is greater than 0, it runs down the Yes side and loops again. If it is zero, then it goes down the No side and sets the Completed variable to completed so it does not loop again.

 

In my SharePoint list where I am getting the items, there is a column with the default value of Pending. In my apply to each, it updates this to Processed. The Get items action has a filter query to get the pending items.

getitems.jpg

At the end of the yes, there is a delay of two minutes to allow the throttle limit to reset. The flow looks like this:

David_MA_1-1679498913628.png

 

mmollet
Community Champion
Community Champion

I am going to check out the article you posted. Thank you btw! My thought was that if I have 20-30 employees that I would want my concurrency to be at least at that level so that each iteration of the loop could get assigned its own thread on the server side as the flow ran. This would mean that my 1 min delay would be a 1 min total rather than 1min X number of users total if they ran one after the other for example. I am going to try to lower this to 30 and see what happens. Ty again!


If this post solves your issue please mark it as a solution.
wskinnermctc
Memorable Member
Memorable Member

Half the time I see issues here is a result of trying to use Excel as form of middle calculator within the flow.

 

Users are familiar with Excel and set up formulas or lookups within a workbook. Then they try to plug in items from power automate and let Excel do the math/lookups.

This is not a good route to go for multiple reasons. Delays, Calculation Doesn't work on closed books, not transparent, rarely supports rapid flow runs. 

 

It is better to find a process or method that has Power Automate doing the calculations or lookups. This might require setting up tables differently or utilizing different lists. It might take some extra work to figure out how to put the correct expression or format in Power Automate to get the result you want, but it will be a much more transparent and consistent process.

The formulas in the excel sheet are:

=SUM(UserHoursTable[MinutesOnTask])-$L$5 to get total normal hours

=SUMIF(UserHoursTable[TaskCategory],"Travel",UserHoursTable[MinutesOnTask]) to get Travel hours 

=SUM(UserHoursTable[MinutesOnTask]) to get total of both types of hours combined

=($L$4*$L$3)+($L$6*$L$5) to get total pay owed (Normal hours * Normal Rate) + (Travel Hours * Travel Rate)

 

I am hoping that none of these are an issue. This being in the sheet saves the accountant from having to use paper and calculate by hand for each one of 20+ employees that are hourly. There are no lookups or anything beyond what is posted in red above. I am a strong advocate of the "Excel isn't a **bleep** database so stop using as such" argument! 😂


If this post solves your issue please mark it as a solution.
mmollet
Community Champion
Community Champion

Changed the value from 50 to 30 and this flow has now run with no issues at all and only took 1:54 to complete. Seems that this has done the trick. I assumed that it would only use the needed number of threads up to a maximum of 50 but it seems there is more going on than I thought. I am going to be testing more this week so I will post any updates here good or otherwise. Thank you again for the solution!


If this post solves your issue please mark it as a solution.

If I'm looking at this correctly, the Excel workbook is made so the accountant can manually input the Green boxes? All other values exist in SharePoint List, and the unknown value is the Normal Rate and Travel Rate?

 

If you could find a way to get the Normal Rate and Travel Rate, then the special Excel sheet isn't needed. Maybe you can't and that's why this exists in the first place.

 

However, I after reading the other post and Solution, I see that the separate Excel sheets being generated and sent is helping skip over a lot of issues. The problems usually come from like you said, using Excel as a database. This process isn't really using Excel as a middle calculator, it's more like the end output is an Excel calculator, so it works.

 

Glad you got it working,

 

Helpful resources

Announcements

Power Platform Connections Ep 15 | L. Baybutt | Thursday, 1 June 2023

Episode Fifteen of Power Platform Connections sees David Warner and Hugo Bernier talk to Microsoft MVP Lewis Baybutt aka Low Code Lewis, alongside the latest news and community blogs.   Use the hashtag #PowerPlatformConnects on social media for a chance to have your work featured on the show.        Action requested: Feel free to provide feedback on how we can make our community more inclusive and diverse.  This episode premiers live on our YouTube at 12pm PST on Thursday 1st June 2023.  Video series available at Power Platform Community YouTube channel.    Upcoming events:  European Power Platform conference – Jun. 20-22nd - Dublin Microsoft Power Platform Conference – Oct. 3-5th - Las Vegas  Join our Communities:  Power Apps Community Power Automate Community Power Virtual Agents Community Power Pages Community  If you’d like to hear from a specific community member in an upcoming recording and/or have specific questions for the Power Platform Connections team, please let us know. We will do our best to address all your requests or questions.  Action requested: Feel free to provide feedback on how we can make our community more inclusive and diverse.  This episode premiers live on our YouTube at 12pm PST on Thursday 1st June 2023.  Video series available at Power Platform Community YouTube channel.    Upcoming events:  European Power Platform conference – Jun. 20-22nd - Dublin Microsoft Power Platform Conference – Oct. 3-5th - Las Vegas  Join our Communities:  Power Apps Community Power Automate Community Power Virtual Agents Community Power Pages Community  If you’d like to hear from a specific community member in an upcoming recording and/or have specific questions for the Power Platform Connections team, please let us know. We will do our best to address all your requests or questions. 

May 2023 Community Newsletter and Upcoming Events

Welcome to our May 2023 Community Newsletter, where we'll be highlighting the latest news, releases, upcoming events, and the great work of our members inside the Biz Apps communities. If you're new to this LinkedIn group, be sure to subscribe here in the News & Announcements to stay up to date with the latest news from our ever-growing membership network who "changed the way they thought about code".         LATEST NEWS "Mondays at Microsoft" LIVE on LinkedIn - 8am PST - Monday 15th May  - Grab your Monday morning coffee and come join Principal Program Managers Heather Cook and Karuana Gatimu for the premiere episode of "Mondays at Microsoft"! This show will kick off the launch of the new Microsoft Community LinkedIn channel and cover a whole host of hot topics from across the #PowerPlatform, #ModernWork, #Dynamics365, #AI, and everything in-between. Just click the image below to register and come join the team LIVE on Monday 15th May 2023 at 8am PST. Hope to see you there!     Executive Keynote | Microsoft Customer Success Day CVP for Business Applications & Platform, Charles Lamanna, shares the latest #BusinessApplications product enhancements and updates to help customers achieve their business outcomes.         S01E13 Power Platform Connections - 12pm PST - Thursday 11th May Episode Thirteen of Power Platform Connections sees Hugo Bernier take a deep dive into the mind of co-host David Warner II, alongside the reviewing the great work of Dennis Goedegebuure, Keith Atherton, Michael Megel, Cat Schneider, and more. Click below to subscribe and get notified, with David and Hugo LIVE in the YouTube chat from 12pm PST. And use the hashtag #PowerPlatformConnects on social media for a chance to have your work featured on the show.         UPCOMING EVENTS   European Power Platform Conference - early bird ticket sale ends! The European Power Platform Conference early bird ticket sale ends on Friday 12th May 2023! #EPPC23 brings together the Microsoft Power Platform Communities for three days of unrivaled days in-person learning, connections and inspiration, featuring three inspirational keynotes, six expert full-day tutorials, and over eighty-five specialist sessions, with guest speakers including April Dunnam, Dona Sarkar, Ilya Fainberg, Janet Robb, Daniel Laskewitz, Rui Santos, Jens Christian Schrøder, Marco Rocca, and many more. Deep dive into the latest product advancements as you hear from some of the brightest minds in the #PowerApps space. Click here to book your ticket today and save!      DynamicMinds Conference - Slovenia - 22-24th May 2023 It's not long now until the DynamicsMinds Conference, which takes place in Slovenia on 22nd - 24th May, 2023 - where brilliant minds meet, mingle & share! This great Power Platform and Dynamics 365 Conference features a whole host of amazing speakers, including the likes of Georg Glantschnig, Dona Sarkar, Tommy Skaue, Monique Hayward, Aleksandar Totovic, Rachel Profitt, Aurélien CLERE, Ana Inés Urrutia de Souza, Luca Pellegrini, Bostjan Golob, Shannon Mullins, Elena Baeva, Ivan Ficko, Guro Faller, Vivian Voss, Andrew Bibby, Tricia Sinclair, Roger Gilchrist, Sara Lagerquist, Steve Mordue, and many more. Click here: DynamicsMinds Conference for more info on what is sure an amazing community conference covering all aspects of Power Platform and beyond.    Days of Knowledge Conference in Denmark - 1-2nd June 2023 Check out 'Days of Knowledge', a Directions 4 Partners conference on 1st-2nd June in Odense, Denmark, which focuses on educating employees, sharing knowledge and upgrading Business Central professionals. This fantastic two-day conference offers a combination of training sessions and workshops - all with Business Central and related products as the main topic. There's a great list of industry experts sharing their knowledge, including Iona V., Bert Verbeek, Liza Juhlin, Douglas Romão, Carolina Edvinsson, Kim Dalsgaard Christensen, Inga Sartauskaite, Peik Bech-Andersen, Shannon Mullins, James Crowter, Mona Borksted Nielsen, Renato Fajdiga, Vivian Voss, Sven Noomen, Paulien Buskens, Andri Már Helgason, Kayleen Hannigan, Freddy Kristiansen, Signe Agerbo, Luc van Vugt, and many more. If you want to meet industry experts, gain an advantage in the SMB-market, and acquire new knowledge about Microsoft Dynamics Business Central, click here Days of Knowledge Conference in Denmark to buy your ticket today!   COMMUNITY HIGHLIGHTS Check out our top Super and Community Users reaching new levels! These hardworking members are posting, answering questions, kudos, and providing top solutions in their communities.   Power Apps:  Super Users: @WarrenBelz, @LaurensM  @BCBuizer  Community Users:  @Amik@ @mmollet, @Cr1t    Power Automate:  Super Users: @Expiscornovus , @grantjenkins, @abm  Community Users: @Nived_Nambiar, @ManishSolanki    Power Virtual Agents:  Super Users: @Pstork1, @Expiscornovus  Community Users: @JoseA, @fernandosilva, @angerfire1213    Power Pages: Super Users: @ragavanrajan  Community Users: @Fubar, @Madhankumar_L,@gospa  LATEST COMMUNITY BLOG ARTICLES  Power Apps Community Blog  Power Automate Community Blog  Power Virtual Agents Community Blog  Power Pages Community Blog  Check out 'Using the Community' for more helpful tips and information:  Power Apps , Power Automate, Power Virtual Agents, Power Pages 

Announcing | Super Users - 2023 Season 1

Super Users – 2023 Season 1    We are excited to kick off the Power Users Super User Program for 2023 - Season 1.  The Power Platform Super Users have done an amazing job in keeping the Power Platform communities helpful, accurate and responsive. We would like to send these amazing folks a big THANK YOU for their efforts.      Super User Season 1 | Contributions July 1, 2022 – December 31, 2022  Super User Season 2 | Contributions January 1, 2023 – June 30, 2023    Curious what a Super User is? Super Users are especially active community members who are eager to help others with their community questions. There are 2 Super User seasons in a year, and we monitor the community for new potential Super Users at the end of each season. Super Users are recognized in the community with both a rank name and icon next to their username, and a seasonal badge on their profile.  Power Apps  Power Automate  Power Virtual Agents  Power Pages  Pstork1*  Pstork1*  Pstork1*  OliverRodrigues  BCBuizer  Expiscornovus*  Expiscornovus*  ragavanrajan  AhmedSalih  grantjenkins  renatoromao    Mira_Ghaly*  Mira_Ghaly*      Sundeep_Malik*  Sundeep_Malik*      SudeepGhatakNZ*  SudeepGhatakNZ*      StretchFredrik*  StretchFredrik*      365-Assist*  365-Assist*      cha_cha  ekarim2020      timl  Hardesh15      iAm_ManCat  annajhaveri      SebS  Rhiassuring      LaurensM  abm      TheRobRush  Ankesh_49      WiZey  lbendlin      Nogueira1306  Kaif_Siddique      victorcp  RobElliott      dpoggemann  srduval      SBax  CFernandes      Roverandom  schwibach      Akser  CraigStewart      PowerRanger  MichaelAnnis      subsguts  David_MA      EricRegnier  edgonzales      zmansuri  GeorgiosG      ChrisPiasecki  ryule      AmDev  fchopo      phipps0218  tom_riha      theapurva  takolota     Akash17  momlo     BCLS776  Shuvam-rpa     rampprakash  ScottShearer     Rusk  ChristianAbata     cchannon  Koen5     a33ik  Heartholme     AaronKnox  okeks      Matren  David_MA     Alex_10        Jeff_Thorpe        poweractivate        Ramole        DianaBirkelbach        DavidZoon        AJ_Z        PriyankaGeethik        BrianS        StalinPonnusamy        HamidBee        CNT        Anonymous_Hippo        Anchov        KeithAtherton        alaabitar        Tolu_Victor        KRider        sperry1625        IPC_ahaas      zuurg    rubin_boer   cwebb365   Dorrinda   G1124   Gabibalaban   Manan-Malhotra   jcfDaniel   WarrenBelz   Waegemma   drrickryp   GuidoPreite   metsshan    If an * is at the end of a user's name this means they are a Multi Super User, in more than one community. Please note this is not the final list, as we are pending a few acceptances.  Once they are received the list will be updated. 

Check out the new Power Platform Communities Front Door Experience!

We are excited to share the ‘Power Platform Communities Front Door’ experience with you!   Front Door brings together content from all the Power Platform communities into a single place for our community members, customers and low-code, no-code enthusiasts to learn, share and engage with peers, advocates, community program managers and our product team members. There are a host of features and new capabilities now available on Power Platform Communities Front Door to make content more discoverable for all power product community users which includes ForumsUser GroupsEventsCommunity highlightsCommunity by numbersLinks to all communities Users can see top discussions from across all the Power Platform communities and easily navigate to the latest or trending posts for further interaction. Additionally, they can filter to individual products as well.   Users can filter and browse the user group events from all power platform products with feature parity to existing community user group experience and added filtering capabilities.     Users can now explore user groups on the Power Platform Front Door landing page with capability to view all products in Power Platform.      Explore Power Platform Communities Front Door today. Visit Power Platform Community Front door to easily navigate to the different product communities, view a roll up of user groups, events and forums.

Microsoft Power Platform Conference | Registration Open | Oct. 3-5 2023

We are so excited to see you for the Microsoft Power Platform Conference in Las Vegas October 3-5 2023! But first, let's take a look back at some fun moments and the best community in tech from MPPC 2022 in Orlando, Florida.   Featuring guest speakers such as Charles Lamanna, Heather Cook, Julie Strauss, Nirav Shah, Ryan Cunningham, Sangya Singh, Stephen Siciliano, Hugo Bernier and many more.   Register today: https://www.powerplatformconf.com/   

Users online (2,836)