cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
MPoe
Level: Powered On

Insert multiple rows with SQL connector

Hi,

One of our vendors delivers a pipe delimited file for my company every evening ~6pm. I have a flow that successfully parses and inserts the records, however it takes about an hour to do so. This is because the SQL connector throttles to 100 calls per 10 seconds and the file is usually around 32,000 records. 

 

Is there a way I can insert multiple rows at a time? 

Info: I'm using a gateway and on-premise SQL Server, so writing the SQL directly is not available to me.

2 REPLIES 2
Community Support Team
Community Support Team

Re: Insert multiple rows with SQL connector

Hi @MPoe ,

 

SQL Server connector does have such limitations.

If you need to insert 32,000 records, it seems that it will take an hour or so.

Could you provide a screenshot of the Flow configuration?

Perhaps we can optimize the configuration to improve the efficiency of Flow execution.

 

Best Regards,

Community Support Team _ Barry
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
MPoe
Level: Powered On

Re: Insert multiple rows with SQL connector

Hi Barry, thanks for the offer. Do you know of any other connectors that allow you to insert multiple rows at the same time, if the SQL Connector does not?

 

Here's the basic structure:

  • trigger: 6pm each day M-F
  • initialize several variables
    • CRLF: is a string of the enter key. most effective way I've found to handle new line splits
    • Fetch date: pulls today's date. is a variable so I can set other days easily for testing, back loading
    • Curve list: not used
  • HTTP: fetch the file contents. Returns a pipe( | ) delimited string
  • Split the string into an array on CRLF
  • Begin loop (tried 50x parrallel, too many failures. Now sequential, still testing to find right fit)
    • split first line into columns on |
    • check if line is valid. if not, ignore
    • reformat date strings into yyyy-MM-dd
    • still getting a /r line feed in last column, so remove it
    • insert row

 

Here are screenshots

ice flow 01.PNG

 

Here is what's in the loop:

ice flow 02.PNG

Helpful resources

Announcements
firstImage

Microsoft Business Applications Virtual Launch Event

Watch the event on demand for an in-depth look at the new innovations across Dynamics 365 and the Microsoft Power Platform.

firstImage

Watch Sessions On Demand!

Continue your learning in our online communities.

Power Platform 2019 Release Wave 2 Plan

Power Platform 2019 Release Wave 2 Plan

Features releasing from October 2019 through March 2020.

thirdimage

Flow Community User Group Member Badge

Fill out a quick form to claim your user group badge now!

fifthimage

Microsoft Learn

Learn how to build the business apps that you need.

sixthImage

Power Platform World Tour

Find out where you can attend!

seventhimage

Webinars & Video Gallery

Watch & learn from the Flow Community Video Gallery!

Users Online
Currently online: 388 members 4,406 guests
Recent signins:
Please welcome our newest community members: