cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
GanCW
Resolver II
Resolver II

Removing duplicate records in datatable

I have a datatable loaded from Excel table with the following records

 

  Col-1  Col-2 Col-3
1 Apple  USA   A
2 Orange China A
3 Apple  USA   B
4 Apple  China A
5 Pear   Japan A
6 Pear   Japan B

 

and i would like to create a new table (or list of lists) with unique value of Col-1 and Col-2

  Col-1   Col 2

1 Apple   USA
2 Orange  China
3 Apple   China
4 Pear    Japan

 

My initial thought is to load this as list of lists and use the 'Remove Duplicate Item from List' action to remove the duplicate records but this doesn't work.

 

Does anyone has suggestion on how best to accomplish this other than the brute force iterative method?

 

 

1 ACCEPTED SOLUTION

Accepted Solutions
miketran13
Impactful Individual
Impactful Individual

Hello,

 

I would do these steps, so that you can have a list of data you want, then you can just convert it to a datatable or whatever you want.

 

listdata.png

Thanks and hope it can help you. 

Mike

---------------------------------

Did I answer your question? Please consider to Mark my post as a solution! to guide others

View solution in original post

10 REPLIES 10
miketran13
Impactful Individual
Impactful Individual

Hello,

 

I would do these steps, so that you can have a list of data you want, then you can just convert it to a datatable or whatever you want.

 

listdata.png

Thanks and hope it can help you. 

Mike

---------------------------------

Did I answer your question? Please consider to Mark my post as a solution! to guide others

Hi Mike,

Thanks for the tips. Converting the list to text string does the trick. 

I need to write the table back into to Excel so I will just use Split text to extract the two values and write to separate cells.

 

Gan

 

 

 

Anonymous
Not applicable

Hello @miketran13 ,

 

 I have  a list in power automate desktop , after I performed actions like 'Remove duplicate items in the list'. I have to change the list values(generic value type) to data table (with data rows), the data coming from the action 'remove duplicate items in the list'. As I need to write the data back to a excel file. 

 

My list contains dynamic values , as every time it will be changing. not specific one's . So kindly suggest what format to use for converting list values to data table, as I need to write back again in to the excel file in the same flow

 

Kindly look into this .Thanks in advance.

MichaelAnnis
Super User
Super User

What if instead, you just originally read column 1 and column 2 only...then, Remove Duplicates would work.  It would shorten this work around down immensely.  If you still need all 3 columns as another list, that's fine, but you should be able to take just column 1 and column 2 from Excel, into a DataTable, remove duplicates, and write back to excel.

 

Best of Luck!

----
If my post has answered your question, please thumbs up and mark this post as a solution.

I also offer paid consulting services. If you would like to discuss this option, please feel free to DM me and we can set up a time to join a Zoom call and fix any issues you are having.
Anonymous
Not applicable

Thank you for the response @MichaelAnnis , but to remove duplicates from excel data(as it will be as data table). Need to make use of these actions (create list, add items to list, remove duplicates in list), like need to convert data table into list & after performing above actions ,then convert back to data table as I need to get in excel file , back again.

 

or is there any alternative method other than this , please brief it as I don't know other methods than the above mentioned

Anonymous
Not applicable

@MichaelAnnis , awaiting for your response. Kindly look into the above scenario I mentioned. Basically over writing the excel data , but based on above SQL table condition

I revert you back to what @miketran13 said; so you end up here:

MichaelAnnis_0-1631636343617.png

 

Then, write to Excel in A1 (new workbook or new worksheet).

Select Column A

Data -> Text to Columns (you can do all this through recorder)

MichaelAnnis_1-1631636457176.png

Delimited -> Next

MichaelAnnis_2-1631636494208.png

Check "Comma" only (it is usually tab by default, so uncheck tab and check comma)

Then click Finish

MichaelAnnis_3-1631636569098.png

 

This will leave the fruit in Column A and move the Country to Column B.

----
If my post has answered your question, please thumbs up and mark this post as a solution.

I also offer paid consulting services. If you would like to discuss this option, please feel free to DM me and we can set up a time to join a Zoom call and fix any issues you are having.
Anonymous
Not applicable

Thanks a lot @MichaelAnnis for following up on this. But to stop confusion & to be more clear I mentioned my scenario in a different thread , which was posted by me . As the scenario here is little different.

 

Sorry to create a confusion between here & there , I understand that. Kindly continue through this thread & even I'll follow up on the same -> Re: format to use for converting list values to da... - Power Platform Community (microsoft.com)

 

kindly look into the scenario I mentioned there on using SQL, please look into that , expecting quick response. Thanks in advance.

pmascal
Regular Visitor

For anybody who finds this post in the future, I have a solution if you want a more efficient way to do this without having to

1. Loop through each DataRow within the DataTable and adding to a list delimited by a comma

2. manipulate the data afterwards in excel

 

Assume your DataTable is in a variable called "Data"

pmascal_1-1696256170720.png

Step 1: Create a Temporary File

pmascal_0-1696023670248.png

 

Step 2: Write Data Table to CSV File. Include column names

pmascal_0-1696255927316.png

 

 

Step 3: Read text from file (Not CSV File), and store contents as a list

pmascal_2-1696023670250.png

 

pmascal_2-1696256230835.png

 

Step 4: Remove Duplicates from List

pmascal_7-1696256374576.png

 

pmascal_3-1696256248578.png

 

Step 5: Join Text

pmascal_4-1696023670251.png

 

pmascal_4-1696256266363.png

 

Step 7: Write text to file

 

pmascal_6-1696023670252.png

 

 

 

 

Step 8: Read from CSV File, First line contains column names is true

 

pmascal_6-1696256329800.png

 

pmascal_5-1696256307028.png

 

 

Agnius
Most Valuable Professional
Most Valuable Professional

You could also write %DataTable.ColumnHeadersRow% to the text file before writing the data itself. This will help you retain the headers.

-------------------------------------------------------------------------------------------------------------------------
If I have answered your question, please mark it as the preferred solution. If you like my response, please give it a Thumbs Up.
Regards, Agnius Bartninkas

Helpful resources

Announcements

Check out the Copilot Studio Cookbook today!

We are excited to announce our new Copilot Cookbook Gallery in the Copilot Studio Community. We can't wait for you to share your expertise and your experience!    Join us for an amazing opportunity where you'll be one of the first to contribute to the Copilot Cookbook—your ultimate guide to mastering Microsoft Copilot. Whether you're seeking inspiration or grappling with a challenge while crafting apps, you probably already know that Copilot Cookbook is your reliable assistant, offering a wealth of tips and tricks at your fingertips--and we want you to add your expertise. What can you "cook" up?   Click this link to get started: https://aka.ms/CS_Copilot_Cookbook_Gallery   Don't miss out on this exclusive opportunity to be one of the first in the Community to share your app creation journey with Copilot. We'll be announcing a Cookbook Challenge very soon and want to make sure you one of the first "cooks" in the kitchen.   Don't miss your moment--start submitting in the Copilot Cookbook Gallery today!     Thank you,  Engagement Team

Announcing Power Apps Copilot Cookbook Gallery

We are excited to share that the all-new Copilot Cookbook Gallery for Power Apps is now available in the Power Apps Community, full of tips and tricks on how to best use Microsoft Copilot as you develop and create in Power Apps. The new Copilot Cookbook is your go-to resource when you need inspiration--or when you're stuck--and aren't sure how to best partner with Copilot while creating apps.   Whether you're looking for the best prompts or just want to know about responsible AI use, visit Copilot Cookbook for regular updates you can rely on--while also serving up some of your greatest tips and tricks for the Community. Check Out the new Copilot Cookbook for Power Apps today: Copilot Cookbook - Power Platform Community.  We can't wait to see what you "cook" up!    

Welcome to the Power Automate Community

You are now a part of a fast-growing vibrant group of peers and industry experts who are here to network, share knowledge, and even have a little fun.   Now that you are a member, you can enjoy the following resources:   Welcome to the Community   News & Announcements: The is your place to get all the latest news around community events and announcements. This is where we share with the community what is going on and how to participate.  Be sure to subscribe to this board and not miss an announcement.   Get Help with Power Automate Forums: If you're looking for support with any part of Power Automate, our forums are the place to go. From General Power Automate forums to Using Connectors, Building Flows and Using Flows.  You will find thousands of technical professionals, and Super Users with years of experience who are ready and eager to answer your questions. You now have the ability to post, reply and give "kudos" on the Power Automate community forums. Make sure you conduct a quick search before creating a new post because your question may have already been asked and answered. Galleries: The galleries are full of content and can assist you with information on creating a flow in our Webinars and Video Gallery, and the ability to share the flows you have created in the Power Automate Cookbook.  Stay connected with the Community Connections & How-To Videos from the Microsoft Community Team. Check out the awesome content being shared there today.   Power Automate Community Blog: Over the years, more than 700 Power Automate Community Blog articles have been written and published by our thriving community. Our community members have learned some excellent tips and have keen insights on the future of process automation. In the Power Automate Community Blog, you can read the latest Power Automate-related posts from our community blog authors around the world. Let us know if you'd like to become an author and contribute your own writing — everything Power Automate-related is welcome.   Community Support: Check out and learn more about Using the Community for tips & tricks. Let us know in the Community Feedback  board if you have any questions or comments about your community experience. Again, we are so excited to welcome you to the Microsoft Power Automate community family. Whether you are brand new to the world of process automation or you are a seasoned Power Automate veteran - our goal is to shape the community to be your 'go to' for support, networking, education, inspiration and encouragement as we enjoy this adventure together.     Power Automate Community Team

Hear what's next for the Power Up Program

Hear from Principal Program Manager, Dimpi Gandhi, to discover the latest enhancements to the Microsoft #PowerUpProgram, including a new accelerated video-based curriculum crafted with the expertise of Microsoft MVPs, Rory Neary and Charlie Phipps-Bennett. If you’d like to hear what’s coming next, click the link below to sign up today! https://aka.ms/PowerUp  

Tuesday Tip | How to Report Spam in Our Community

It's time for another TUESDAY TIPS, your weekly connection with the most insightful tips and tricks that empower both newcomers and veterans in the Power Platform Community! Every Tuesday, we bring you a curated selection of the finest advice, distilled from the resources and tools in the Community. Whether you’re a seasoned member or just getting started, Tuesday Tips are the perfect compass guiding you across the dynamic landscape of the Power Platform Community.   As our community family expands each week, we revisit our essential tools, tips, and tricks to ensure you’re well-versed in the community’s pulse. Keep an eye on the News & Announcements for your weekly Tuesday Tips—you never know what you may learn!   Today's Tip: How to Report Spam in Our Community We strive to maintain a professional and helpful community, and part of that effort involves keeping our platform free of spam. If you encounter a post that you believe is spam, please follow these steps to report it: Locate the Post: Find the post in question within the community.Kebab Menu: Click on the "Kebab" menu | 3 Dots, on the top right of the post.Report Inappropriate Content: Select "Report Inappropriate Content" from the menu.Submit Report: Fill out any necessary details on the form and submit your report.   Our community team will review the report and take appropriate action to ensure our community remains a valuable resource for everyone.   Thank you for helping us keep the community clean and useful!

Users online (4,077)