cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
New Member

File Copy From SFTP to SharePoint Online Is Slow

I am load-testing Flow for a job that will involve copying a million 55 KB PDFs from an SFTP server to a SharePoint Online Document Center library.  I uploaded a test bach of 6,000 files to SFTP server from my local machine -- this took ten minutes.  Now, my Flow is running and moving only eight files per minute.  I am using the free trial Flow Plan 2, and have 45 days left.  Looking at my tenant Flow stats, I do not think this job qualifies for throttling -- this counts as only one run -- but maybe it is being throttled at SharePoint, not Flow? 

 

For each file, Flow creates the copy at SharePoint, then deletes the original from the SFTP server.  After four hours, it is still happily chugging along.  Is this simply the way it works, or are there tricks I am missing to optimize?  Thanks,  Nick

2 REPLIES 2
Highlighted
Community Support
Community Support

Re: File Copy From SFTP to SharePoint Online Is Slow

Hi @npettijohn,

 

Could you please share a screenshot of your flow's configuration?

Why not upload your local files to your SharePoint library directly?

Further, do you want to copy 6,000 files from your SFTP server to your SharePoint library?

 

I could not reproduce your issue on my side currently, please check if your Network is in a good condition. I think the amount of the files would also take effect on the execution speed of your flow.

 

Flow would have its trigger interval, there are some limits for a single flow run, the max Run duration of a single flow run is 30 days. More details about Run duration and retention limits for a single flow run, please check the following article:

https://docs.microsoft.com/en-us/flow/limits-and-config#run-duration-and-retention

 

In addition, I think you add a "Apply to each" action within your flow, is it true? There are some Looping and debatching limits for a single flow run, the limitation of Foreach items is 5000. More details about the Looping and debatching limits for a single flow run, please check the following article:

https://docs.microsoft.com/en-us/flow/limits-and-config#looping-and-debatching-limits

 

You could consider take a try to increase the degree of parallelism in the Apply to each to increase the processing speed of the "Apply to each" action. More details about increasing the degree of  degree of parallelism in the "Apply to each" action, please check the following article:

https://flow.microsoft.com/en-us/blog/gateway-ha-increased-apply-to-each/

 

Best regards,

Kris

Community Support Team _ Kris Dai
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Highlighted
Frequent Visitor

Re: File Copy From SFTP to SharePoint Online Is Slow

Can you send instructions to copy files from an sftp server to sharepoint online document library.

Helpful resources

Announcements
FirstImage

Microsoft Ignite 2020

Check out the announcement of Power Platform content at Microsoft Ignite!

thirdImage

Experience what's new for Power Automate

Join us for an in-depth look at the new Power Automate features and capabilities at the free Microsoft Business Applications Launch Event.

firstImage

Power Platform 2020 release wave 2 plan

Features releasing from October 2020 through March 2021

Users online (6,374)