Showing results for 
Search instead for 
Did you mean: 

Delete duplicate records/ items from Dynamics 365 CRM/ CDS/ SharePoint

In this post, I will show you how to detect duplicate records/ items in Dynamics 365 CRM/ CDS/ SharePoint, delete the record/ item and send a list of deleted records/ items as an email attachment. We will look at finding duplicate entries based on specific fields/ columns (specific values on these fields/ columns), deleting those and sending an email with the list of deleted records/ items.

Let’s get started with the Flow

Trigger: Manual Button

Action: Initialize an array variable: This will accommodate the unique identifier to look for in the duplicate records.

Action: Initialize an array variable 2: This will accommodate all the deleted items to be added to the List that will be sent as an email attachment.

Action: List Records (For Dynamics and CDS); Get Items (For SharePoint)DF1.PNG

Note: in the List Records action, we are trying to fetch as many as 10000 records and iterate over them. To get duplicates of a particular value on a field/ column, the ODATA filter query can be modified to get filtered records/ items only. DF4.PNGThis will only fetch records where the 'crceb_name' field equals "Yash". In the ODATA filter query, to filter on a particular field, the logical name of the attribute needs to be used for the filter query. 

Control: Apply to each (to iterate over each record/item retrieved in the previous step).

//Loop starts

Control: Condition: To check if Array 1 contains the value for the field/column on which a duplicate entry is to be identified.

If Array 1 contains the value,

Action: Append to Array Variable 2: append the current item of the loop to the Array 2

Action: Delete a record (for Dynamics and CDS); Delete Item (for SharePoint): To delete the current item.

If Array 1 does not contain the value,

Action: Append to Array Variable 1: append the unique identifier field to Array 1 (this will be used to populate the unique values for the particular field/ column on which the duplicate entries are to be checked)

//Loop endsDF2.PNG

Action: Create an HTML table: to create a table of the deleted records/ items.

Action: Send an Email: to send an email with the list of deleted items as an attachment.DF3.PNGIn this post, I have shown you how to identify and delete duplicate records/ items from Dynamics 365 CRM/ CDS/ SharePoint based on a particular field/ column. The same logic flow can be applied for any other data sources with minor tweaks. This flow is a manually triggered one for data that is already present in the respective data-sources. A schedule trigger can be used in this flow to run it on a daily basis and the ODATA query in the List Records/ Get Items action can be modified to get records created during the past 24 hours only and check if any duplicate entries have been made in the last 24 hours (however, to check the duplicate entries with data older than the last 24 hours, the original flow can be used). 

I hope you found this interesting and it helped you. Thank you for reading!

Meet Our Blog Authors
  • Experienced Consultant with a demonstrated history of working in the information technology and services industry. Skilled in Office 365, Azure, SharePoint Online, PowerShell, Nintex, K2, SharePoint Designer workflow automation, PowerApps, Microsoft Flow, PowerShell, Active Directory, Operating Systems, Networking, and JavaScript. Strong consulting professional with a Bachelor of Engineering (B.E.) focused in Information Technology from Mumbai University.
  • I am a Microsoft Business Applications MVP and a Senior Manager at EY. I am a technology enthusiast and problem solver. I work/speak/blog/Vlog on Microsoft technology, including Office 365, Power Apps, Power Automate, SharePoint, and Teams Etc. I am helping global clients on Power Platform adoption and empowering them with Power Platform possibilities, capabilities, and easiness. I am a leader of the Houston Power Platform User Group and Power Automate community superuser. I love traveling , exploring new places, and meeting people from different cultures.
  • Blog site: MCT | SharePoint, Microsoft 365 and Power Platform Consultant | Contributor on SharePoint StackExchange, Techcommunity
  • Encodian Owner / Founder - Ex Microsoft Consulting Services - Architect / Developer - 20 years in SharePoint - PowerPlatform Fan
  • Founder of SKILLFUL SARDINE, a company focused on productivity and the Power Platform. You can find me on LinkedIn: and twitter I also write at, so if you want some Power Automate, SharePoint or Power Apps content I'm your guy 🙂
  • I am the Owner/Principal Architect at Don't Pa..Panic Consulting. I've been working in the information technology industry for over 30 years, and have played key roles in several enterprise SharePoint architectural design review, Intranet deployment, application development, and migration projects. I've been a Microsoft Most Valuable Professional (MVP) 12 consecutive years and am also a Microsoft Certified SharePoint Masters (MCSM) since 2013.
  • Big fan of Power Platform technologies and implemented many solutions.
  • Passionate #Programmer #SharePoint #SPFx #M365 #Power Platform| Microsoft MVP | SharePoint StackOverflow, Github, PnP contributor
  • Web site – Youtube channel -