Hello!
I am wondering how to format my filter array step to get the following:
I got 2 arrays that are the files from 2 different libraries that are supposed to be "synced" (The same files in both libraries) where one of the libraries are considered the "source" and the other library are just copies.
Now i dont want to loop every item just to check if its already "synced" or not, which is why im trying to do it in a filter array step.
Basically i want to filter out any files that have the same URL and where the modified date is less in the source than in the "mirrored library" (This means that it's already the correct version in the mirrored library).
Any help is very much appreciated. 😃
Maybe you can guide me in the right direction? @Expiscornovus
Solved! Go to Solution.
@StretchFredrik Hopefully this works as expected. It should copy over update files and newly added files. I combined what @Chriddle did with the solution as a lot nicer and easier to handle the new files added.
See full flow below. I'll go into each of the actions.
Get files Library A and Get files Library B are the same as the original solution.
Select B extracts just the Modified date and the Full Path (excluding the Library Name) from Get files Library B. The expression used is:
join(skip(split(item()?['{FullPath}'], '/'), 1), '/')
Select A extracts out the Identifier, Modified date and Path from Get files Library A, plus the matching Modified date from Library B. The expression used to get the Modified date from Library B is:
xpath(
xml(json(concat('{"root": { value:', body('Select_B'), '}}'))),
concat('string(//root/value[FullPath="', join(skip(split(item()?['{FullPath}'], '/'), 1), '/'), '"]/Modified/text())')
)
Filter array uses the output from Select A with the following filter.
//ModifiedB is empty (new file added) or ModifiedA is greater than ModifiedB (file updated in Library A)
@or(
equals(item()?['ModifiedB'], ''),
greater(item()?['ModifiedA'], item()?['ModifiedB'])
)
Apply to each iterates over each of the items in our Filter array.
Copy file uses the following expressions to copy the new/updated files from Library A to Library B.
//File to Copy
item()?['Identifier']
//Destination Folder - NOTE that you would need to put your Library names here
slice(replace(item()?['Path'], 'LibraryA', 'LibraryB'), 0, lastIndexOf(replace(item()?['Path'], 'LibraryA', 'LibraryB'), '/'))
----------------------------------------------------------------------
If I've answered your question, please mark the post as Solved.
If you like my response, please consider giving it a Thumbs Up.
I've got an idea on how to do this within a single Filter array, but just off to sleep now so won't be able to get you something until later (12:30AM for me at the moment). If someone else can provide a solution prior to that then even better 🙂
Are they both Document Libraries that would have identical folder structures/files?
Assuming this would be a scheduled flow that ran daily/weekly?
Hi @StretchFredrik,
Normally I would say, have a look at the Except method described in this blog:
https://pnp.github.io/blog/post/comparing-two-arrays-without-an-apply-to-each/
However, you want to check two things (Url and Modified date time).
Only workaround I can think of at the moment is getting like the max modified date as the latest sync time. Not a great workaround because it doesn't compare it with the modified of the target item itself, but just with the max modified date time of the whole collection of items.
But because it is a sync process, that might be ok? 😁
@and(contains(body('Select_-_Target_Paths'), item()['File']), greater(ticks(item()['Modified']), max(body('Select_-_Target_Modified'))))
Thank you for your reply @Expiscornovus , i would need to compare each items modified date since the sync might take different documents each time since they will be approved or worked on at different times. So a file might not be touched for a year meanwhile another document is worked on daily. The library in question has around 50 000 documents.
Thank you for your reply @grantjenkins
Are they both Document Libraries that would have identical folder structures/files?
Yes they have the same folder structure and files.
Assuming this would be a scheduled flow that ran daily/weekly?
It will run every 15 or 30 minutes, which is why i want it to only spend time on the documents that are out of sync.
Ok, thanks for clarifying. In that case what I provided isn't sufficient.
Let's wait for the response of @grantjenkins. I am sure he can come up with a great solution 😀
Have you considered using Sharepoint (and maybe OneDrive) sync capabilities instead of a Power Automate flow?
Yes, that does not work since the main goal of this is to have one library with ONLY major versions of files and only read permissions @Chriddle.
Yes, that does not work since the main goal of this is to have one library with ONLY major versions of files and only read permissions @Chriddle
Maybe that helps:
I created an array "source" with 500 objects (I hope this is a resonable number of changed files between two flow runs)
and an array "destination" with 50000 objects.
Maybe the amount of destination objects can be reduced by a clever odata filter.
For each object in "source", the Select action "combined" does a lookup in the destinations for an entry with same name and get its "created" with the help of xpath.
You can filter this output with date comparison,
With this amount of values this flow runs in round about 2 minutes.
Of course this is only a POC and you would have to add times, check what happens if the objects are bigger (because of longer names) and probably more 😉
source (Select):
"inputs": {
"from": "@range(0,500)",
"select": {
"name": "@concat('file-',string(item()))",
"created": "@concat('2023-02-', rand(1, 28))"
}
}
destination (Select):
"inputs": {
"from": "@range(0, 50000)",
"select": {
"name": "@concat('file-', string(item()))",
"created": "@concat('2023-02-', rand(1, 28))"
}
}
destinationXML (Compose):
"inputs": "@xml(json(concat('{\"root\":{\"item\":', body('destination'),'}}')))"
combined (Select):
"inputs": {
"from": "@body('source')",
"select": {
"name": "@item()['name']",
"created": "@item()['created']",
"created_destination": "@first(xpath(outputs('destinationXML'), concat('//item[name=\"',item()['name'],'\"]/created/text()')))"
}
}
Filter array:
"inputs": {
"from": "@body('combined')",
"where": "@greater(item()['created'], item()['created_destination'])"
}
I think I might have something that will work.
One concern is the number of files in each library (50,000). We can't just apply a Filter Query within our Get files, so would need to return all files from both libraries (100,000+ files) then apply filtering. The filtering will be quick - just the initial retrieval of files.
The other concern is if there are a lot of files out of sync it will take some time to copy them across which could take quite a while depending on number of files and size of those files. One thing I haven't done here is copied the actual metadata (properties) across. Can easily do that, but not sure if your requirement is just the file sync, or properties too.
If you go with this approach, you may need to run the flow manually a few times to see how long it takes to complete, then schedule the flow accordingly.
See full flow below. I'll go into each of the actions.
Get files Library A and Get files Library B are both using Get files (properties only) actions. They both have the filter FSObjType eq 0 which means only get files (not folders). I've also set the Top Count to 5000 for each of them.
I've also gone into the Settings for Get files Library A and Get files Library B, turned on Pagination, and set the Threshold to 60000 (needs to be a number larger than the number of files you will have over the next couple of years at least). This will take a while to retrieve all your files.
Select extracts out a couple of properties from Get files Library B that we will convert to XML so we can apply XPath within the filter later. The expressions used are:
//FullPath - removes the library name from the full path
join(skip(split(item()?['{FullPath}'], '/'), 1), '/')
//Modified - replaces characters so we are left with a number (required for XPath comparison)
replace(replace(replace(replace(item()?['Modified'], '-', ''), 'T', ''), ':', ''), 'Z', '')
Filter array uses the output from our Select and the following expression to filter our items that have been updated in Library A since being copied to Library B (in need of updating). It uses an XPath expression to compare both the FullPath (including Filename) and the Modified Date. And if the length of items returned is greater than 0 then we need to update the item.
@greater(
length(
xpath(
xml(json(concat('{"root": { value:', body('Select'), '}}'))),
concat('//root/value[FullPath = "', join(skip(split(item()?['{FullPath}'], '/'), 1), '/'), '" and Modified < "', replace(replace(replace(replace(item()?['Modified'], '-', ''), 'T', ''), ':', ''), 'Z', ''), '"]')
)
),
0
)
Apply to each iterates over each of the items in our Filter array (files that need to be updated).
Copy file uses the following expressions for File to Copy and Destination folder.
//File to Copy
item()?['{Identifier}']
//Destination Folder - NOTE that you would need to put your Library names here
slice(replace(item()?['{Path}'], 'LibraryA', 'LibraryB'), 0, lastIndexOf(replace(item()?['{Path}'], 'LibraryA', 'LibraryB'), '/'))
----------------------------------------------------------------------
If I've answered your question, please mark the post as Solved.
If you like my response, please consider giving it a Thumbs Up.
I think i got it close to working, will test a bit more tomorrow. But i would need the the filter array to also include any items that don't already exist in destination. @grantjenkins
Thank you for your help this far 😃
@StretchFredrik Yea I was thinking about that and already started a bit of a redesign, but off to sleep now, so won't be able to get anything to you until a bit later. I know how to achieve it, but just need to rebuild a part of it.
Also, what if you delete an item from the source - would you also need to delete that from the destination?
Sounds awesome take your time and sleep well. the delete part ive already done by comparing urls with the filter array step. It then loops the ones present in destination and not in source and tries to find the file by document-id to see if its moved in source but not published yet. If it doesnt find it by document-id, it gets deleted.
Thank you for your time @grantjenkins
@StretchFredrik Hopefully this works as expected. It should copy over update files and newly added files. I combined what @Chriddle did with the solution as a lot nicer and easier to handle the new files added.
See full flow below. I'll go into each of the actions.
Get files Library A and Get files Library B are the same as the original solution.
Select B extracts just the Modified date and the Full Path (excluding the Library Name) from Get files Library B. The expression used is:
join(skip(split(item()?['{FullPath}'], '/'), 1), '/')
Select A extracts out the Identifier, Modified date and Path from Get files Library A, plus the matching Modified date from Library B. The expression used to get the Modified date from Library B is:
xpath(
xml(json(concat('{"root": { value:', body('Select_B'), '}}'))),
concat('string(//root/value[FullPath="', join(skip(split(item()?['{FullPath}'], '/'), 1), '/'), '"]/Modified/text())')
)
Filter array uses the output from Select A with the following filter.
//ModifiedB is empty (new file added) or ModifiedA is greater than ModifiedB (file updated in Library A)
@or(
equals(item()?['ModifiedB'], ''),
greater(item()?['ModifiedA'], item()?['ModifiedB'])
)
Apply to each iterates over each of the items in our Filter array.
Copy file uses the following expressions to copy the new/updated files from Library A to Library B.
//File to Copy
item()?['Identifier']
//Destination Folder - NOTE that you would need to put your Library names here
slice(replace(item()?['Path'], 'LibraryA', 'LibraryB'), 0, lastIndexOf(replace(item()?['Path'], 'LibraryA', 'LibraryB'), '/'))
----------------------------------------------------------------------
If I've answered your question, please mark the post as Solved.
If you like my response, please consider giving it a Thumbs Up.
Thank you @grantjenkins , the filter array step fails on "Greater expects all of its paramteres to be either integer or decimal numbers, Found invalid parameter type "Null". So guessing the filter array query fails when the modifiedB is null(empty).
Did you put the filters in the same order that I had? If ModifiedB is empty, then it wouldn't try to evaluate the second filter.
This is what i have, should be the same:
Nevermind, im stupid, i forgot to change the input of filter array @grantjenkins . Will try again! 😃
Episode Fifteen of Power Platform Connections sees David Warner and Hugo Bernier talk to Microsoft MVP Lewis Baybutt aka Low Code Lewis, alongside the latest news and community blogs. Use the hashtag #PowerPlatformConnects on social media for a chance to have your work featured on the show. Action requested: Feel free to provide feedback on how we can make our community more inclusive and diverse. This episode premiers live on our YouTube at 12pm PST on Thursday 1st June 2023. Video series available at Power Platform Community YouTube channel. Upcoming events: European Power Platform conference – Jun. 20-22nd - Dublin Microsoft Power Platform Conference – Oct. 3-5th - Las Vegas Join our Communities: Power Apps Community Power Automate Community Power Virtual Agents Community Power Pages Community If you’d like to hear from a specific community member in an upcoming recording and/or have specific questions for the Power Platform Connections team, please let us know. We will do our best to address all your requests or questions. Action requested: Feel free to provide feedback on how we can make our community more inclusive and diverse. This episode premiers live on our YouTube at 12pm PST on Thursday 1st June 2023. Video series available at Power Platform Community YouTube channel. Upcoming events: European Power Platform conference – Jun. 20-22nd - Dublin Microsoft Power Platform Conference – Oct. 3-5th - Las Vegas Join our Communities: Power Apps Community Power Automate Community Power Virtual Agents Community Power Pages Community If you’d like to hear from a specific community member in an upcoming recording and/or have specific questions for the Power Platform Connections team, please let us know. We will do our best to address all your requests or questions.
Welcome to our May 2023 Community Newsletter, where we'll be highlighting the latest news, releases, upcoming events, and the great work of our members inside the Biz Apps communities. If you're new to this LinkedIn group, be sure to subscribe here in the News & Announcements to stay up to date with the latest news from our ever-growing membership network who "changed the way they thought about code". LATEST NEWS "Mondays at Microsoft" LIVE on LinkedIn - 8am PST - Monday 15th May - Grab your Monday morning coffee and come join Principal Program Managers Heather Cook and Karuana Gatimu for the premiere episode of "Mondays at Microsoft"! This show will kick off the launch of the new Microsoft Community LinkedIn channel and cover a whole host of hot topics from across the #PowerPlatform, #ModernWork, #Dynamics365, #AI, and everything in-between. Just click the image below to register and come join the team LIVE on Monday 15th May 2023 at 8am PST. Hope to see you there! Executive Keynote | Microsoft Customer Success Day CVP for Business Applications & Platform, Charles Lamanna, shares the latest #BusinessApplications product enhancements and updates to help customers achieve their business outcomes. S01E13 Power Platform Connections - 12pm PST - Thursday 11th May Episode Thirteen of Power Platform Connections sees Hugo Bernier take a deep dive into the mind of co-host David Warner II, alongside the reviewing the great work of Dennis Goedegebuure, Keith Atherton, Michael Megel, Cat Schneider, and more. Click below to subscribe and get notified, with David and Hugo LIVE in the YouTube chat from 12pm PST. And use the hashtag #PowerPlatformConnects on social media for a chance to have your work featured on the show. UPCOMING EVENTS European Power Platform Conference - early bird ticket sale ends! The European Power Platform Conference early bird ticket sale ends on Friday 12th May 2023! #EPPC23 brings together the Microsoft Power Platform Communities for three days of unrivaled days in-person learning, connections and inspiration, featuring three inspirational keynotes, six expert full-day tutorials, and over eighty-five specialist sessions, with guest speakers including April Dunnam, Dona Sarkar, Ilya Fainberg, Janet Robb, Daniel Laskewitz, Rui Santos, Jens Christian Schrøder, Marco Rocca, and many more. Deep dive into the latest product advancements as you hear from some of the brightest minds in the #PowerApps space. Click here to book your ticket today and save! DynamicMinds Conference - Slovenia - 22-24th May 2023 It's not long now until the DynamicsMinds Conference, which takes place in Slovenia on 22nd - 24th May, 2023 - where brilliant minds meet, mingle & share! This great Power Platform and Dynamics 365 Conference features a whole host of amazing speakers, including the likes of Georg Glantschnig, Dona Sarkar, Tommy Skaue, Monique Hayward, Aleksandar Totovic, Rachel Profitt, Aurélien CLERE, Ana Inés Urrutia de Souza, Luca Pellegrini, Bostjan Golob, Shannon Mullins, Elena Baeva, Ivan Ficko, Guro Faller, Vivian Voss, Andrew Bibby, Tricia Sinclair, Roger Gilchrist, Sara Lagerquist, Steve Mordue, and many more. Click here: DynamicsMinds Conference for more info on what is sure an amazing community conference covering all aspects of Power Platform and beyond. Days of Knowledge Conference in Denmark - 1-2nd June 2023 Check out 'Days of Knowledge', a Directions 4 Partners conference on 1st-2nd June in Odense, Denmark, which focuses on educating employees, sharing knowledge and upgrading Business Central professionals. This fantastic two-day conference offers a combination of training sessions and workshops - all with Business Central and related products as the main topic. There's a great list of industry experts sharing their knowledge, including Iona V., Bert Verbeek, Liza Juhlin, Douglas Romão, Carolina Edvinsson, Kim Dalsgaard Christensen, Inga Sartauskaite, Peik Bech-Andersen, Shannon Mullins, James Crowter, Mona Borksted Nielsen, Renato Fajdiga, Vivian Voss, Sven Noomen, Paulien Buskens, Andri Már Helgason, Kayleen Hannigan, Freddy Kristiansen, Signe Agerbo, Luc van Vugt, and many more. If you want to meet industry experts, gain an advantage in the SMB-market, and acquire new knowledge about Microsoft Dynamics Business Central, click here Days of Knowledge Conference in Denmark to buy your ticket today! COMMUNITY HIGHLIGHTS Check out our top Super and Community Users reaching new levels! These hardworking members are posting, answering questions, kudos, and providing top solutions in their communities. Power Apps: Super Users: @WarrenBelz, @LaurensM @BCBuizer Community Users: @Amik@ @mmollet, @Cr1t Power Automate: Super Users: @Expiscornovus , @grantjenkins, @abm Community Users: @Nived_Nambiar, @ManishSolanki Power Virtual Agents: Super Users: @Pstork1, @Expiscornovus Community Users: @JoseA, @fernandosilva, @angerfire1213 Power Pages: Super Users: @ragavanrajan Community Users: @Fubar, @Madhankumar_L,@gospa LATEST COMMUNITY BLOG ARTICLES Power Apps Community Blog Power Automate Community Blog Power Virtual Agents Community Blog Power Pages Community Blog Check out 'Using the Community' for more helpful tips and information: Power Apps , Power Automate, Power Virtual Agents, Power Pages
Super Users – 2023 Season 1 We are excited to kick off the Power Users Super User Program for 2023 - Season 1. The Power Platform Super Users have done an amazing job in keeping the Power Platform communities helpful, accurate and responsive. We would like to send these amazing folks a big THANK YOU for their efforts. Super User Season 1 | Contributions July 1, 2022 – December 31, 2022 Super User Season 2 | Contributions January 1, 2023 – June 30, 2023 Curious what a Super User is? Super Users are especially active community members who are eager to help others with their community questions. There are 2 Super User seasons in a year, and we monitor the community for new potential Super Users at the end of each season. Super Users are recognized in the community with both a rank name and icon next to their username, and a seasonal badge on their profile. Power Apps Power Automate Power Virtual Agents Power Pages Pstork1* Pstork1* Pstork1* OliverRodrigues BCBuizer Expiscornovus* Expiscornovus* ragavanrajan AhmedSalih grantjenkins renatoromao Mira_Ghaly* Mira_Ghaly* Sundeep_Malik* Sundeep_Malik* SudeepGhatakNZ* SudeepGhatakNZ* StretchFredrik* StretchFredrik* 365-Assist* 365-Assist* cha_cha ekarim2020 timl Hardesh15 iAm_ManCat annajhaveri SebS Rhiassuring LaurensM abm TheRobRush Ankesh_49 WiZey lbendlin Nogueira1306 Kaif_Siddique victorcp RobElliott dpoggemann srduval SBax CFernandes Roverandom schwibach Akser CraigStewart PowerRanger MichaelAnnis subsguts David_MA EricRegnier edgonzales zmansuri GeorgiosG ChrisPiasecki ryule AmDev fchopo phipps0218 tom_riha theapurva takolota Akash17 momlo BCLS776 Shuvam-rpa rampprakash ScottShearer Rusk ChristianAbata cchannon Koen5 a33ik Heartholme AaronKnox okeks Matren David_MA Alex_10 Jeff_Thorpe poweractivate Ramole DianaBirkelbach DavidZoon AJ_Z PriyankaGeethik BrianS StalinPonnusamy HamidBee CNT Anonymous_Hippo Anchov KeithAtherton alaabitar Tolu_Victor KRider sperry1625 IPC_ahaas zuurg rubin_boer cwebb365 Dorrinda G1124 Gabibalaban Manan-Malhotra jcfDaniel WarrenBelz Waegemma drrickryp GuidoPreite metsshan If an * is at the end of a user's name this means they are a Multi Super User, in more than one community. Please note this is not the final list, as we are pending a few acceptances. Once they are received the list will be updated.
We are excited to share the ‘Power Platform Communities Front Door’ experience with you! Front Door brings together content from all the Power Platform communities into a single place for our community members, customers and low-code, no-code enthusiasts to learn, share and engage with peers, advocates, community program managers and our product team members. There are a host of features and new capabilities now available on Power Platform Communities Front Door to make content more discoverable for all power product community users which includes ForumsUser GroupsEventsCommunity highlightsCommunity by numbersLinks to all communities Users can see top discussions from across all the Power Platform communities and easily navigate to the latest or trending posts for further interaction. Additionally, they can filter to individual products as well. Users can filter and browse the user group events from all power platform products with feature parity to existing community user group experience and added filtering capabilities. Users can now explore user groups on the Power Platform Front Door landing page with capability to view all products in Power Platform. Explore Power Platform Communities Front Door today. Visit Power Platform Community Front door to easily navigate to the different product communities, view a roll up of user groups, events and forums.
We are so excited to see you for the Microsoft Power Platform Conference in Las Vegas October 3-5 2023! But first, let's take a look back at some fun moments and the best community in tech from MPPC 2022 in Orlando, Florida. Featuring guest speakers such as Charles Lamanna, Heather Cook, Julie Strauss, Nirav Shah, Ryan Cunningham, Sangya Singh, Stephen Siciliano, Hugo Bernier and many more. Register today: https://www.powerplatformconf.com/
User | Count |
---|---|
44 | |
29 | |
23 | |
20 | |
14 |
User | Count |
---|---|
88 | |
59 | |
42 | |
25 | |
21 |