cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
chriscanavan58
Frequent Visitor

SQL Get Rows is producing duplicates

Hi Everyone I hope someone would be able to help. I have a flow which connects to a SQL database to extract the data from a table/view which has just short of 15000 records using Get Rows (V2) with pagination set to 20000 rows. It then creates a CSV table and then saves it to a SharePoint. 

 

The issue I am having is that it is returning the correct amount of rows which are in the table but it is duplicating a number of the rows sometimes up to 3 times. 

1 ACCEPTED SOLUTION

Accepted Solutions

@v-litu-msft 

 

I found a solution. The tables required indexing. once this was completed it resolved the issue 

View solution in original post

10 REPLIES 10
v-litu-msft
Community Support
Community Support

Hi @chriscanavan58,

 

Could you please share screenshots of your Flow configuration?

Please make sure there is no additional Apply to each action.

 

"It is returning the correct amount of rows which are in the table but it is duplicating a number of the rows sometimes up to 3 times." Is it means some rows will be replaced by duplicated rows if the amount of rows is correct, so there will be some rows loss? 

 

Best Regards,
Community Support Team _ Lin Tu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Hi @v-litu-msft  thanks for the response

 

here is the screen grab, as you can see there are no apply to each actions in there

 

FTSG.png

 

 

Hi @chriscanavan58,

 

There seems to be nothing wrong with Flow.

Is it possible that these duplicate rows exist in the SQL table before getting via Flow?

 

Best Regards,
Community Support Team _ Lin Tu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Hi @v-litu-msft 

 

I have checked the source data and there are definitely no duplicates as they all have a primary key assigned to them.

@v-litu-msft  I have noticed on another branch where it is extracting 650 rows of the data the issue isnt present. it only seems to happen when it is pulling in a large number of rows. 

Hi @chriscanavan58,

 

OK, this issue has never met before. You can temporarily set the number of Get rows to a more secure range. If you need to deal with these records, you can use filter query in Get rows to minimize rows number. If the same situation still happens to other users in the future, and we can reproduce it on our side, we will report this issue, in time and let you know.

 

Best Regards,
Community Support Team _ Lin Tu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

@v-litu-msft  limiting the rows isnt going to be possible as I require all the rows to be extracted unless you can think of another way of getting all this data?

Hi @chriscanavan58,

 

Is it an on-premise SQL instance? If it is, there is no way to do it.

If not, you could use Execute a SQL query action with SQL query to get all rows.

Annotation 2020-06-02 180425.jpg

 

Best Regards,
Community Support Team _ Lin Tu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

@v-litu-msft 

 

yes it is an on premise SQL and we are not allowed to use the execute SQL query 😞

@v-litu-msft 

 

I found a solution. The tables required indexing. once this was completed it resolved the issue 

Helpful resources

Announcements
Power Automate News & Announcements

Power Automate News & Announcements

Keep up to date with current events and community announcements in the Power Automate community.

Community Calls Conversations

Community Calls Conversations

A great place where you can stay up to date with community calls and interact with the speakers.

Power Automate Community Blog

Power Automate Community Blog

Check out the latest Community Blog from the community!

Top Solution Authors
Top Kudoed Authors
Users online (4,387)