cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
AZArena
Regular Visitor

Processing past emails is VERY slow

I am tasked to process past emails by copying all attachments to a SharePoint Online Library along with a few pieces of data (sender, subject, received date/time stamp). The process works except that it is excruciatingly slow. The source mailboxes have over 10k emails to process so I can't rely on Flow if it is going to move this slowly.

 

Here are some rough metrics from my executions:
3 emails with 12 attachments total took roughly 15 minutes
20 days of emails with roughly 135 attachments processed in over 2 hours when I eventually had to cancel

 

The Flow is constructed as follows:

I manually start the flow with 2 date pickers for a starting date and ending date. I use an integer counter with addDays function to set a CurrentDate variable which is then used as a search query by Get Emails (V3) to restrict emails to a received date. Ex: CurrentDate = addDays(variables('StartDate'),variables('intCount'),'yyyy-MM-dd')

 

So basically, inside a Do Until, the Get Emails will process each calendar date from Start Date to End Date. An (Apply to Each) will handle emails with attachments. A second (Apply to Each) will copy each attachment to SharePoint and update the metadata fields with the source email's sender, subject, and received date. After the first cancelled run, I went back and enabled the Concurrency option on both of the (Apply to Each) controls. That didn't seem to help.

 

Here are screenshots of the Flow.

Flow_1.png

 

Flow_2.png

 

Flow_3.png

 

Flow_4.png

 

The run that succeeded shows 14 minutes occurring at the second (Apply to Each), but the actual Create File and Update File operations then only show a few seconds each. So the time doesn't equate to those operations.

Flow_5_Create_Time.png

 

Any thoughts are appreciated. This is basically for a one-time "get all attachments" operation. If I can't speed this up I may just cut my losses and go to C#.

 

Thanks,

Anthony

1 ACCEPTED SOLUTION

Accepted Solutions
FCGALLEG
Community Support
Community Support

H Anthony, 

 

Which plan are you using? Maybe the performance issue may be in the type of license you have.

 

Your plan determines how often your flows run. For example, your flows may run every 15 minutes if you're on the free plan. If a flow is triggered less than 15 minutes after its last run, it's queued until 15 minutes have elapsed.

https://docs.microsoft.com/en-us/power-automate/billing-questions

 

Regards, 

Javier

View solution in original post

2 REPLIES 2
FCGALLEG
Community Support
Community Support

H Anthony, 

 

Which plan are you using? Maybe the performance issue may be in the type of license you have.

 

Your plan determines how often your flows run. For example, your flows may run every 15 minutes if you're on the free plan. If a flow is triggered less than 15 minutes after its last run, it's queued until 15 minutes have elapsed.

https://docs.microsoft.com/en-us/power-automate/billing-questions

 

Regards, 

Javier

Javier,

 

I'm certain you are correct. Thanks for the link with info on confirming the subscription plan for the executing account.

 

For any others looking for an alternative, I created a simple C# console app using Exchange Web Service (EWS) api and SharePoint Client api. For uploading to SPO, I followed guidance from this link:

https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/upload-large-files-sample-app-for-... 

 

I also borrowed ideas from this link on downloading email attachments:

https://blog.sqltreeo.com/read-emails-from-exchange-online-mailbox-office-365-into-sql-server/ 

 

With this approach (for each email - download local then upload then update metadata then delete local), I was able to process emails as hoped (thousands of emails in a few hours).

 

Thanks,

Anthony

 

Helpful resources

Announcements
Power Platform Conf 2022 768x460.jpg

Join us for Microsoft Power Platform Conference

The first Microsoft-sponsored Power Platform Conference is coming in September. 100+ speakers, 150+ sessions, and what's new and next for Power Platform.

MPA Virtual Workshop Carousel 768x460.png

Register for a Free Workshop

Learn to digitize and optimize business processes and connect all your applications to share data in real time.

Power Automate Designer Feedback_carousel.jpg

Help make Flow Design easier

Are you new to designing flows? What is your biggest struggle with Power Automate Designer? Help us make it more user friendly!

Users online (2,237)