cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
GaryH
Frequent Visitor

Find unique rows, update fields, then delete duplicates

Hello folks,

 

Complete PA newbie here and I spent longer than I care to mention trying to work this out on my own, so I am coming to you feeling a little defeated 😅. I'm close, but there's one part that I'm stumped on and I suspect it is very straightforward to someone that has a basic understanding of what these objects are doing.

 

Scenario:

I have a SharePoint list that is populated regularly with a complete list of stories that will be pushed in the next fix version. This means there is a lot of duplicate handling, but it's a little more complex.

 

After they are imported I need to:

  1. Check the list for unique keys (i.e. things there were included in a fix version but have been removed)
  2. Update the fix version field for these unique rows to prompt checking
  3. After the fix versions have been updated, delete the most recent duplicates

 

My starting point was the very helpful Find and Remove Duplicates thread by @takolota, but I have been having trouble iterating on it.

 

Test data:

Fix VersionProjectKey
2301AA1
2301AA1
2301BB1
2301BB1
2301BB2

 

Expected output:

Fix VersionProjectKey
2301AA1
2301BB1
CHECKBB2

 

Actual Output:

Fix VersionProjectKey
CHECKAA1
CHECKBB1
CHECKBB2

 

Flow:

GaryH_1-1688722959747.png

Here's where I suspect the trouble occurs:

GaryH_4-1688723047560.png

ID: items('Apply_to_each')?['ID']

Key: items('Apply_to_each')?[title] (Sharepoint's default "title" column was renamed "key", I've tried both.)

 

If I use Dynamic Values for ID and Key, I get the same output but an additional Apply to each loop is added to my flow within the one above.

 

GaryH_3-1688723018461.png

 

Thanks for your time!

/Gary

 

14 REPLIES 14
Matthy79
Super User
Super User

Hello @GaryH 

 

If you really only have an array like your sample array, “union” will do the work. Just add a compose action with this formular: “union(yourArray, yourArray)”

 

Instead of yourArray you have to put in the output of your array.

 

I didn’t read the other topic, but I’m pretty sure, they are doing something more complicated than you want to do.

Thanks for responding - I think Union could be used to simplify this flow somewhat, but my need is to identify and highlight items that are not duplicate entries before the removal of duplicates so I need to get this Update Item component working somehow.

Matthy79
Super User
Super User

Hello @GaryH ,

 

sorry for the confusion. Here is a sample code how this could be done:

 

{"id":"6e684ed2-27bf-4327-aa96-60b0279d69b3","brandColor":"#8C3900","connectionReferences":{"shared_sharepointonline":{"connection":{"id":"/providers/Microsoft.PowerApps/apis/shared_sharepointonline/connections/shared-sharepointonl-b959cb53-5ecb-4b41-805f-06e7795a0bd2"}},"shared_office365":{"connection":{"id":"/providers/Microsoft.PowerApps/apis/shared_office365/connections/1597557c9920416e9a930edfee990646"}},"shared_commondataserviceforapps":{"connection":{"id":"/providers/Microsoft.PowerApps/apis/shared_commondataserviceforapps/connections/62947705b1a746319ca1e59aae985afc"}}},"connectorDisplayName":"Control","icon":"data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMzIiIGhlaWdodD0iMzIiIHZlcnNpb249IjEuMSIgdmlld0JveD0iMCAwIDMyIDMyIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPg0KIDxwYXRoIGQ9Im0wIDBoMzJ2MzJoLTMyeiIgZmlsbD0iIzhDMzkwMCIvPg0KIDxwYXRoIGQ9Im04IDEwaDE2djEyaC0xNnptMTUgMTF2LTEwaC0xNHYxMHptLTItOHY2aC0xMHYtNnptLTEgNXYtNGgtOHY0eiIgZmlsbD0iI2ZmZiIvPg0KPC9zdmc+DQo=","isTrigger":false,"operationName":"Scope_mark_fix_version","operationDefinition":{"type":"Scope","actions":{"Compose_fix_version":{"type":"Compose","inputs":[{"FixVersion":"2301","Project":"A","Key":"A1"},{"FixVersion":"2301","Project":"A","Key":"A1"},{"FixVersion":"2301","Project":"B","Key":"B1"},{"FixVersion":"2301","Project":"B","Key":"B1"},{"FixVersion":"2301","Project":"B","Key":"B2"}],"runAfter":{}},"Compose_xml_fix_version":{"type":"Compose","inputs":"@xml(json(concat('{ \"root\": { \"data\": ', outputs('Compose_fix_version'), '}}')))","runAfter":{"Compose_fix_version":["Succeeded"]}},"Select_fix_version":{"type":"Select","inputs":{"from":"@union(outputs('Compose_fix_version'),outputs('Compose_fix_version'))","select":{"FixVersion":"@if(equals(length(xpath(outputs('Compose_xml_fix_version'), concat('//root/data[FixVersion=\"', item()['FixVersion'], '\" and Project=\"', item()['Project'], '\" and Key=\"', item()['Key'], '\"]'))), 1), 'CHECK', item()['FixVersion'])","Project":"@item()['Project']","Key":"@item()['Key']"}},"runAfter":{"Compose_xml_fix_version":["Succeeded"]}}},"runAfter":{}}}

Matthy79_0-1689085186647.png

 

Thanks for coming back on this.

 

I think I have my head wrapped around this, and - correct me if I'm wrong - it looks like it would work really well if the fix version, project and key fields are clearly defined values. However, looking at this part,

Compose_fix_version":{"type":"Compose","inputs":[{"FixVersion":"2301","Project":"A","Key":"A1"},{"FixVersion":"2301","Project":"A","Key":"A1"},{"FixVersion":"2301","Project":"B","Key":"B1"},{"FixVersion":"2301","Project":"B","Key":"B1"},{"FixVersion":"2301","Project":"B","Key":"B2"

it will not really work unless everything is predefined in this manner?

 

My real dataset will have rolling fix versions, 20+ projects, and generated keys when a new story is created (numbers in the hundreds), which is why I'm trying to use the dynamic values from the SharePoint list we're keeping.

 

I've built a PowerApp for the team to handle the checking prompts, so we don't need to work from a large SharePoint list but streamlining this duplicate handling is the sticking point - I'm very new to Power Automate concepts, so very much appreciate your patience and help 🙂

Matthy79
Super User
Super User

Hello @GaryH 

 

Just try it, it works 🙂

To the best of my ability, I recreated the flow from what I could see in your code and the notes from your screenshots:

 

GaryH_1-1689162724519.png

 

 

but, alas:

 

GaryH_2-1689162800279.png

 

Matthy79
Super User
Super User

You don't have to recreate anything. I already posted a sample flow. You just need to copy the JSON provided and paste it using the clipboard.

 

Matthy79_1-1689164986591.png

 

Press "Ctrl+v"

 

Matthy79_2-1689165049949.png

 

 

After that you can click on it and it will be added to your sample flow.

 

Sorry for the confusion. I thought this was already clear.

 

For your error. Your JSON is not valid. After the closing of the array there shouldn't be anything more. Also the xml is wrong because it has the escape slashes. Don't mind about it. Just copy the code provided and paste it into a dummy flow to see what it does.

I can see from the outputs that your code returns "check" where expected

GaryH_0-1689169313698.png

But I'm afraid I still have the same difficulty I started the thread with - writing the results to SharePoint 🙈

Matthy79
Super User
Super User

Hello @GaryH 

 

If you want to update an item using this array, it won't work. You will need the unique id (integer value) in this array too. Since union won't work for the whole array (because the id is unique) you will have to remove id first and add it again during the select statement using xpath again. That is why Power Automate creates a second apply to each, because you picked the ID from another array.

 

Maybe you should start with something simpler for the first project?

Hi Matthy,

 

I'm afraid this is a need-driven project rather than an interest-driven one, so I need to solve this. It sounds like I was closer to what I needed with my original solution, but thank you for trying to help 🤝

takolota
Multi Super User
Multi Super User

@GaryH 

 

Based on this…

IMG_3549.jpeg

 

It looks like you want to remove any exact duplicates 1st (where the key matches, then find any records with the same/duplicate VersionProject values & mark those with a CHECK

 

I think for that you may have to use the set-up twice. So run a standard version of the template flow using only the Key column in the initial Select, then create a copy of the same set-up below that, only you will use VersionProject in the Select instead of the Key & you will use an Update action instead of a Delete action.

Hi Takolota,

 

Sorry if I've been unclear in my original post but I'll try to rephrase what my challenge is.

 

My columns are:

Fix Version

Project

Key

 

I have been able to successfully delete duplicates using your package but I am trying to modify it so that I can

 

Detect unique "key" values, label them with "CHECK" under Fix Version, then delete duplicates.

 

I believe it occurred because I misunderstood how the Apply to Each step is working. I thought if I wrote "CHECK" in the appropriate Update Item field then it would only apply to filtered unique keys but, as you can see, my out put is nothing but "CHECK" all the way down the line.

 

I then tried to make a condition to test if the output is a 1 or 0 and 

 

GaryH_0-1690445446488.png

 

It also hit me with 

 

GaryH_1-1690445500766.png

 

The expected output was that only A1 should have "CHECK" beside it.

 

For me, I'm not understanding why the fields I refer to is good enough to have the duplicates deleted, but when using the Update Item function, it all falls apart.

 

Any help or insight on what to read into would be welcome here.

 

Thanks,

Gary

takolota
Multi Super User
Multi Super User

@GaryH 
That doesn’t really make sense as all your Key values are unique.

 

If you mean you want to get distinct rows for Project & Fix version columns, then I have this similar set up for getting distinct records based on set columns…

https://powerusers.microsoft.com/t5/Power-Automate-Cookbook/Get-Distinct-Records/td-p/2191533

Hi Takolota,

 

Yes, that screenshot is the output. The list was populated with

 

A1, A2, B1, B2

then

A2, B1, B2, A3

were added

 

meaning my expected output would have been deleted duplicates (which worked) and as A1 was missing from the second set imported to the list (meaning A1 is a unique value) only  A1 and A3 should have the Fix Version as "Check".

 

A couple of weeks ago I was able to adjust your 'delete duplicate' template to return unique values instead of duplicates (the screenshots in the first post), but I'm having real trouble getting Update Items to target only those rows.

 

Ideally, I would be able to differentiate between A1 (was on the original list) and A3 (completely new) but after the journey this has been, I'll just be happy to ditch that idea. 😂

Helpful resources

Announcements

Win free tickets to the Power Platform Conference | Summer of Solutions

We are excited to announce the Summer of Solutions Challenge!    This challenge is kicking off on Monday, June 17th and will run for (4) weeks.  The challenge is open to all Power Platform (Power Apps, Power Automate, Copilot Studio & Power Pages) community members. We invite you to participate in a quest to provide solutions to as many questions as you can. Answers can be provided in all the communities.    Entry Period: This Challenge will consist of four weekly Entry Periods as follows (each an “Entry Period”)   - 12:00 a.m. PT on June 17, 2024 – 11:59 p.m. PT on June 23, 2024 - 12:00 a.m. PT on June 24, 2024 – 11:59 p.m. PT on June 30, 2024 - 12:00 a.m. PT on July 1, 2024 – 11:59 p.m. PT on July 7, 2024 - 12:00 a.m. PT on July 8, 2024 – 11:59 p.m. PT on July 14, 2024   Entries will be eligible for the Entry Period in which they are received and will not carryover to subsequent weekly entry periods.  You must enter into each weekly Entry Period separately.   How to Enter: We invite you to participate in a quest to provide "Accepted Solutions" to as many questions as you can. Answers can be provided in all the communities. Users must provide a solution which can be an “Accepted Solution” in the Forums in all of the communities and there are no limits to the number of “Accepted Solutions” that a member can provide for entries in this challenge, but each entry must be substantially unique and different.    Winner Selection and Prizes: At the end of each week, we will list the top ten (10) Community users which will consist of: 5 Community Members & 5 Super Users and they will advance to the final drawing. We will post each week in the News & Announcements the top 10 Solution providers.  At the end of the challenge, we will add all of the top 10 weekly names and enter them into a random drawing.  Then we will randomly select ten (10) winners (5 Community Members & 5 Super Users) from among all eligible entrants received across all weekly Entry Periods to receive the prize listed below. If a winner declines, we will draw again at random for the next winner.  A user will only be able to win once overall. If they are drawn multiple times, another user will be drawn at random.  Individuals will be contacted before the announcement with the opportunity to claim or deny the prize.  Once all of the winners have been notified, we will post in the News & Announcements of each community with the list of winners.   Each winner will receive one (1) Pass to the Power Platform Conference in Las Vegas, Sep. 18-20, 2024 ($1800 value). NOTE: Prize is for conference attendance only and any other costs such as airfare, lodging, transportation, and food are the sole responsibility of the winner. Tickets are not transferable to any other party or to next year’s event.   ** PLEASE SEE THE ATTACHED RULES for this CHALLENGE**

Celebrating the June Super User of the Month: Markus Franz

Markus Franz is a phenomenal contributor to the Power Apps Community. Super Users like Markus inspire others through their example, encouragement, and active participation.    The Why: "I do this to help others achieve what they are trying to do. As a total beginner back then without IT background I know how overwhelming things can be, so I decided to jump in and help others. I also do this to keep progressing and learning myself." Thank you, Markus Franz, for your outstanding work! Keep inspiring others and making a difference in the community! 🎉  Keep up the fantastic work! 👏👏 Markus Franz | LinkedIn  Power Apps: mmbr1606  

Copilot Cookbook Challenge | Week 1 Results | Win Tickets to the Power Platform Conference

We are excited to announce the "The Copilot Cookbook Community Challenge is a great way to showcase your creativity and connect with others. Plus, you could win tickets to the Power Platform Community Conference in Las Vegas in September 2024 as an amazing bonus.   Two ways to enter: 1. Copilot Studio Cookbook Gallery:  https://aka.ms/CS_Copilot_Cookbook_Challenge 2. Power Apps Copilot Cookbook Gallery: https://aka.ms/PA_Copilot_Cookbook_Challenge   There will be 5 chances to qualify for the final drawing: Early Bird Entries: March 1 - June 2Week 1: June 3 - June 9Week 2: June 10 - June 16Week 3: June 17 - June 23Week 4: June 24 - June 30     At the end of each week, we will draw 5 random names from every user who has posted a qualifying Copilot Studio template, sample or demo in the Copilot Studio Cookbook or a qualifying Power Apps Copilot sample or demo in the Power Apps Copilot Cookbook. Users who are not drawn in a given week will be added to the pool for the next week. Users can qualify more than once, but no more than once per week. Four winners will be drawn at random from the total qualifying entrants. If a winner declines, we will draw again at random for the next winner.  A user will only be able to win once. If they are drawn multiple times, another user will be drawn at random. Prizes:  One Pass to the Power Platform Conference in Las Vegas, Sep. 18-20, 2024 ($1800 value, does not include travel, lodging, or any other expenses) Winners are also eligible to do a 10-minute presentation of their demo or solution in a community solutions showcase at the event. To qualify for the drawing, templates, samples or demos must be related to Copilot Studio or a Copilot feature of Power Apps, Power Automate, or Power Pages, and must demonstrate or solve a complete unique and useful business or technical problem. Power Automate and Power Pagers posts should be added to the Power Apps Cookbook. Final determination of qualifying entries is at the sole discretion of Microsoft. Weekly updates and the Final random winners will be posted in the News & Announcements section in the communities on July 29th, 2024. Did you submit entries early?  Early Bird Entries March 1 - June 2:  If you posted something in the "early bird" time frame complete this form: https://aka.ms/Copilot_Challenge_EarlyBirds if you would like to be entered in the challenge.   Week 1 Results:  Congratulations to the Week 1 qualifiers, you are being entered in the random drawing that will take place at the end of the challenge. Copilot Cookbook Gallery:Power Apps Cookbook Gallery:1.  @Mathieu_Paris 1.   @SpongYe 2.  @Dhanush 2.   @Deenuji 3.  n/a3.   @Nived_Nambiar  4.  n/a4.   @ManishSolanki 5.  n/a5.    n/a

Your Moment to Shine: 2024 PPCC’s Got Power Awards Show

For the third year, we invite you, our talented community members, to participate in the grand 2024 Power Platform Community Conference's Got Power Awards. This event is your opportunity to showcase solutions that make a significant business impact, highlight extensive use of Power Platform products, demonstrate good governance, or tell an inspirational story. Share your success stories, inspire your peers, and show off some hidden talents.  This is your time to shine and bring your creations into the spotlight!  Make your mark, inspire others and leave a lasting impression. Sign up today for a chance to showcase your solution and win the coveted 2024 PPCC’s Got Power Award. This year we have three categories for you to participate in: Technical Solution Demo, Storytelling, and Hidden Talent.      The Technical solution demo category showcases your applications, automated workflows, copilot agentic experiences, web pages, AI capabilities, dashboards, and/or more. We want to see your most impactful Power Platform solutions!  The Storytelling category is where you can share your inspiring story, and the Hidden Talent category is where your talents (such as singing, dancing, jump roping, etc.) can shine! Submission Details:  Fill out the submission form https://aka.ms/PPCCGotPowerSignup by July 12th with details and a 2–5-minute video showcasing your Solution impact. (Please let us know you're coming to PPCC, too!)After review by a panel of Microsoft judges, the top storytellers will be invited to present a virtual demo presentation to the judges during early August. You’ll be notified soon after if you have been selected as a finalist to share your story live at PPCC’s Got Power!  The live show will feature the solution demos and storytelling talents of the top contestants, winner announcements, and the opportunity to network with your community.  It's not just a showcase for technical talent and storytelling showmanship, show it's a golden opportunity to make connections and celebrate our Community together! Let's make this a memorable event! See you there!   Mark your calendars! Date and Time: Thursday, Sept 19th Location: PPCC24 at the MGM Grand, Las Vegas, NV 

Tuesday Tip | Accepting Solutions

It's time for another TUESDAY TIPS, your weekly connection with the most insightful tips and tricks that empower both newcomers and veterans in the Power Platform Community! Every Tuesday, we bring you a curated selection of the finest advice, distilled from the resources and tools in the Community. Whether you’re a seasoned member or just getting started, Tuesday Tips are the perfect compass guiding you across the dynamic landscape of the Power Platform Community.   To enhance our collaborative environment, it's important to acknowledge when your question has been answered satisfactorily. Here's a quick guide on how to accept a solution to your questions: Find the Helpful Reply: Navigate to the reply that has effectively answered your question.Accept as Solution: Look for the "Accept as Solution" button or link, usually located at the bottom of the reply.Confirm Your Selection: Clicking this button may prompt you for confirmation. Go ahead and confirm that this is indeed the solution.Acknowledgment: Once accepted, the reply will be highlighted, and the original post will be marked as "Solved". This helps other community members find the same solution quickly. By marking a reply as an accepted solution, you not only thank the person who helped you but also make it easier for others with similar questions to find answers. Let's continue to support each other by recognizing helpful contributions. 

Reminder: To register for the Community Ambassador Call on June 13th

Calling all Super Users & User Group Leaders   Reminder: To register for the Community Ambassador Call on June 13th—for an exclusive event for User Group Leaders and Super Users! This month is packed with exciting updates and activities within our community.   What's Happening: Community Updates: We'll share the latest developments and what's new in our vibrant community.Special Guest Speaker: Get ready for an insightful talk and live demo of Microsoft Copilot Studio templates by our special guest.Regular Updates: Stay informed with our routine updates for User Groups and Super Users.Community Insights: We'll provide general information about ongoing and upcoming community initiatives. Don't Miss Out: Register Now: Choose the session that fits your schedule best.Check your private messages or Super User Forum for registration links. We're excited to connect with you and continue building a stronger community together.   See you at the call!  

Users online (3,346)