cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
AZArena
Level: Powered On

Processing past emails is VERY slow

I am tasked to process past emails by copying all attachments to a SharePoint Online Library along with a few pieces of data (sender, subject, received date/time stamp). The process works except that it is excruciatingly slow. The source mailboxes have over 10k emails to process so I can't rely on Flow if it is going to move this slowly.

 

Here are some rough metrics from my executions:
3 emails with 12 attachments total took roughly 15 minutes
20 days of emails with roughly 135 attachments processed in over 2 hours when I eventually had to cancel

 

The Flow is constructed as follows:

I manually start the flow with 2 date pickers for a starting date and ending date. I use an integer counter with addDays function to set a CurrentDate variable which is then used as a search query by Get Emails (V3) to restrict emails to a received date. Ex: CurrentDate = addDays(variables('StartDate'),variables('intCount'),'yyyy-MM-dd')

 

So basically, inside a Do Until, the Get Emails will process each calendar date from Start Date to End Date. An (Apply to Each) will handle emails with attachments. A second (Apply to Each) will copy each attachment to SharePoint and update the metadata fields with the source email's sender, subject, and received date. After the first cancelled run, I went back and enabled the Concurrency option on both of the (Apply to Each) controls. That didn't seem to help.

 

Here are screenshots of the Flow.

Flow_1.png

 

Flow_2.png

 

Flow_3.png

 

Flow_4.png

 

The run that succeeded shows 14 minutes occurring at the second (Apply to Each), but the actual Create File and Update File operations then only show a few seconds each. So the time doesn't equate to those operations.

Flow_5_Create_Time.png

 

Any thoughts are appreciated. This is basically for a one-time "get all attachments" operation. If I can't speed this up I may just cut my losses and go to C#.

 

Thanks,

Anthony

1 ACCEPTED SOLUTION

Accepted Solutions
Microsoft FCGALLEG
Microsoft

Re: Processing past emails is VERY slow

H Anthony, 

 

Which plan are you using? Maybe the performance issue may be in the type of license you have.

 

Your plan determines how often your flows run. For example, your flows may run every 15 minutes if you're on the free plan. If a flow is triggered less than 15 minutes after its last run, it's queued until 15 minutes have elapsed.

https://docs.microsoft.com/en-us/power-automate/billing-questions

 

Regards, 

Javier

View solution in original post

2 REPLIES 2
Microsoft FCGALLEG
Microsoft

Re: Processing past emails is VERY slow

H Anthony, 

 

Which plan are you using? Maybe the performance issue may be in the type of license you have.

 

Your plan determines how often your flows run. For example, your flows may run every 15 minutes if you're on the free plan. If a flow is triggered less than 15 minutes after its last run, it's queued until 15 minutes have elapsed.

https://docs.microsoft.com/en-us/power-automate/billing-questions

 

Regards, 

Javier

View solution in original post

AZArena
Level: Powered On

Re: Processing past emails is VERY slow

Javier,

 

I'm certain you are correct. Thanks for the link with info on confirming the subscription plan for the executing account.

 

For any others looking for an alternative, I created a simple C# console app using Exchange Web Service (EWS) api and SharePoint Client api. For uploading to SPO, I followed guidance from this link:

https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/upload-large-files-sample-app-for-... 

 

I also borrowed ideas from this link on downloading email attachments:

https://blog.sqltreeo.com/read-emails-from-exchange-online-mailbox-office-365-into-sql-server/ 

 

With this approach (for each email - download local then upload then update metadata then delete local), I was able to process emails as hoped (thousands of emails in a few hours).

 

Thanks,

Anthony

 

Helpful resources

Announcements
firstImage

New Ranks and Rank Icons in April

Read the announcement for more information!

firstImage

Better Together Contest Finalists Announced!

Congrats to the finalists of our ‘Better Together’-themed T-shirt design contest! Click for the top entries.

firstImage

New & Improved Power Automate Community Cookbook

We've updated and improved the layout and uploading format of the Power Automate Cookbook!

thirdimage

Power Automate Community User Group Member Badge

Fill out a quick form to claim your user group badge now!

sixthImage

Community Summit North America

The top training and networking event across the globe for Microsoft Business Applications

Top Kudoed Authors
Users online (10,017)