cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Highlighted
npettijohn
Level: Powered On

File Copy From SFTP to SharePoint Online Is Slow

I am load-testing Flow for a job that will involve copying a million 55 KB PDFs from an SFTP server to a SharePoint Online Document Center library.  I uploaded a test bach of 6,000 files to SFTP server from my local machine -- this took ten minutes.  Now, my Flow is running and moving only eight files per minute.  I am using the free trial Flow Plan 2, and have 45 days left.  Looking at my tenant Flow stats, I do not think this job qualifies for throttling -- this counts as only one run -- but maybe it is being throttled at SharePoint, not Flow? 

 

For each file, Flow creates the copy at SharePoint, then deletes the original from the SFTP server.  After four hours, it is still happily chugging along.  Is this simply the way it works, or are there tricks I am missing to optimize?  Thanks,  Nick

2 REPLIES 2
Community Support Team
Community Support Team

Re: File Copy From SFTP to SharePoint Online Is Slow

Hi @npettijohn,

 

Could you please share a screenshot of your flow's configuration?

Why not upload your local files to your SharePoint library directly?

Further, do you want to copy 6,000 files from your SFTP server to your SharePoint library?

 

I could not reproduce your issue on my side currently, please check if your Network is in a good condition. I think the amount of the files would also take effect on the execution speed of your flow.

 

Flow would have its trigger interval, there are some limits for a single flow run, the max Run duration of a single flow run is 30 days. More details about Run duration and retention limits for a single flow run, please check the following article:

https://docs.microsoft.com/en-us/flow/limits-and-config#run-duration-and-retention

 

In addition, I think you add a "Apply to each" action within your flow, is it true? There are some Looping and debatching limits for a single flow run, the limitation of Foreach items is 5000. More details about the Looping and debatching limits for a single flow run, please check the following article:

https://docs.microsoft.com/en-us/flow/limits-and-config#looping-and-debatching-limits

 

You could consider take a try to increase the degree of parallelism in the Apply to each to increase the processing speed of the "Apply to each" action. More details about increasing the degree of  degree of parallelism in the "Apply to each" action, please check the following article:

https://flow.microsoft.com/en-us/blog/gateway-ha-increased-apply-to-each/

 

Best regards,

Kris

Community Support Team _ Kris Dai
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
aribaudo
Level: Powered On

Re: File Copy From SFTP to SharePoint Online Is Slow

Can you send instructions to copy files from an sftp server to sharepoint online document library.

Helpful resources

Announcements
thirdimage

Power Automate Community User Group Member Badge

Fill out a quick form to claim your user group badge now!

firstImage

Incoming: New and improved badges!

We've given our badges an overhaul and also added some brand new ones!

fifthimage

Microsoft Learn

Learn how to build the business apps that you need.

sixthImage

Power Platform World Tour

Find out where you can attend!

seventhimage

Webinars & Video Gallery

Watch & learn from the Power Automate Community Video Gallery!

Top Kudoed Authors (Last 30 Days)
Users online (5,242)