11-27-2021 16:55 PM - last edited 05-24-2023 15:45 PM
Batch update with external data template flow.
Explanation Video: https://youtu.be/l0NuYtXdcrQ
Compared to the using the basic Apply to Each, Get items, & Update item approach on large amounts of data, this method requires a small fraction of the API calls towards the daily action limit and drastically reduces flow run-times.
It's currently set to take data from any Excel sheet and update records in SharePoint with corresponding data. It works with any potential Power Automate data-source, including HTTP API GET requests, but Excel provides a simple example.
Part of it works like a Vlookup function where it identifies if a row of updated data in Excel has some set values in it and matches it to the SharePoint record with those set values through the SharePoint ID. Then it uses the batch update method to update those SharePoint records.
The flow can easily expand to any size of SharePoint list by adding more Get items actions and batch update parallel branches. If speed is a concern for anyone, there are ways to make the Get items actions all load in parallel too (up to 50 branches). It's really only limited by how much effort people want to put into their flows & lists.
Version 1.5
This version makes it easier to handle cases where the list name may have changed since its creation and moves a few of the primary key column matching inputs to the 'settings' compose action so users don't have to look through & edit the more complicated expressions to set up the flow.
Google Drive Link to Flow Zip Files: https://drive.google.com/file/d/10p7EB730xsEj-azVYuTIuu8dS0w-AflR/view?usp=sharing
Google Drive Link to Text File to a Scope Action Containing The Flow: https://drive.google.com/file/d/1BVGoeM5mykYlMobAyFkhuLRh3r7jMSLz/view?usp=sharing
Version 1
Download The Template Batch Update Flow
Google Drive Link to Flow Zip Files: https://drive.google.com/file/d/10gFkycdx6zpRfrI-s_jCDwIK6dpyyDqk/view?usp=sharing
Google Drive Link to Text File to a Scope Action Containing The Flow: https://drive.google.com/file/d/1e6-INUykIT22ppVh5m4kxz8us_7qXy7q/view?usp=sharing
***LISTS WITH UNDERSCORES _ IN THE TITLE MAY CAUSE ERRORS WITH EARLIER VERSIONS OF BATCH ACTIONS.***
If you have a list with underscores in the name, then in TachyTelic.Net's batch create you will want to go to the __metadata line on the GenerateSPData select action and substitute the expression
for
json(concat('{"type":"SP.Data.', replace(replace(outputs('settings')['listName'], '_', '_x005f_'), ' ', '_x0020_'), 'ListItem"}'))
TachyTelic.Net Blog & Videos
Batch Create Flow
Blog: https://www.tachytelic.net/2021/06/power-automate-flow-batch-create-sharepoint-list-items/
Video: https://youtu.be/2dV7fI4GUYU
Batch Delete Flow
Blog: https://www.tachytelic.net/2021/06/power-automate-delete-sharepoint-items/
Video: https://www.youtube.com/watch?v=2ImkuGpEeoo
See @DamoBird365 go through a join / merge example similar to the Reformat JSON pieces of this template here: https://youtu.be/R_bD8nAoJ8Y
Formulas For Random Number Columns
SharePoint Rand1To50 Column Calculated Default Value Formula:
=INT(INT(RIGHT(NOW(),2))/2)
Excel Random Column Formula:
=ROUNDDOWN(((Rand()*100)+1)/2, 0)
If the batch call succeeds, but individual rows fail to load to SharePoint, then the flow will not throw an error. I personally change out the Results Compose actions with Append to variable actions and use an expression that returns the results text if it identifies an error. Then at the end of the flow I have a condition checking the length of that variable. If it has some text/length to it, then something failed and it sends me an email with a link to the flow run.
Expression to conditionally return results on failure:
if(
or(
or(
or(
or(
or(
or(
or(
or(
contains(base64ToString(body('sendBatch')['$content']), '400 Bad Request')),
contains(base64ToString(body('sendBatch')['$content']), '401 Unauthorized')),
contains(base64ToString(body('sendBatch')['$content']), '403 Forbidden')),
contains(base64ToString(body('sendBatch')['$content']), '404 Not Found')),
contains(base64ToString(body('sendBatch')['$content']), '408 Request Timeout')),
contains(base64ToString(body('sendBatch')['$content']), '409 Conflict')),
contains(base64ToString(body('sendBatch')['$content']), '500 Internal Server Error')),
contains(base64ToString(body('sendBatch')['$content']), '502 Bad Gateway')),
base64ToString(body('sendBatch')['$content']),
'')
watch?v=l0NuYtXdcrQ
Sorry, more complex expressions in Select statements often get messed up with the copy & paste.
You’ll want to import the flow the normal way. See the start of this video for importing a similar flow:
This update template is not set up well for batch create, I definitely suggest checking @Paulie78’s material at tachy telic for the batch create template:
https://www.tachytelic.net/2021/06/power-automate-flow-batch-create-sharepoint-list-items/?amp
I looked at your flow, and can't tell what exactly is the issue, but I would check your siteAddress and listName. They are used differently in the batches and in some places the things like underscores _ or spaces ' ' need to be replaced. However, sometimes it depends on if your list name has changed. You can use the updated Batch Update that separates sharepoint list webaddress vs sharepoint list view name. Then change the items where it is used.
For a quick try solution, go to the expression in the GenerateSPData step in the _metadata field.
Original Expression Below:
json(concat('{"type":"SP.Data.', replace(outputs('settings')['listName'], ' ', '_x0020_'), 'ListItem"}'))
Change the expression to:
json(concat('{"type":"SP.Data.', replace(replace(replace(outputs('settings')['listname'], '_', '_x005f_'), ' ', '_x0020_'), '-', '_x002d_'), 'ListItem"}'))
This should help accommodate the underscores in your sharepoint list name.
In the SendBatch step make sure you select your Site Address: in the dropdown instead of trying to use a custom value reference from the settings. I found that never worked for me.
Hope this helps,
yes, sorry for confusion. I am download his template and literally just replace my site address and list name
I tried both, it still not work, and the whole flow run through so fast in couple secs. it supposed not that fast. Isn't it?
So @Yi66 it seems there will be more trouble shooting to find your problem. To keep from blowing up the comments on this post for an unrelated issue you should make your own post stating your issue and provide a little explanation and screenshots of the results. Then tag myself we can look at your flow on your own post.
Thanks for jumping in skinner as I’m only able to intermittently check this while working on other projects.
And good point, @Yi66 should probably start a forum support thread in the Building Flows section.
https://powerusers.microsoft.com/t5/Building-Flows/bd-p/BuildingFlows
I think we should get a picture of @Yi66’s Send batch or Results action output in a flow run, then work backwards from there.
In a later update, I may need to include the better error handling set-up I use to force the flow to fail when the Send batch action contains failed parts of a call.
I did just recreate another thread. please chip in there. Thankx
I added the expression I use to check for sendBatch failures & return the results if there is a failure. That should help anyone who wants to implement better error handling for that action.
if(
or(
or(
or(
or(
or(
or(
or(
or(
contains(base64ToString(body('sendBatch')['$content']), '400 Bad Request')),
contains(base64ToString(body('sendBatch')['$content']), '401 Unauthorized')),
contains(base64ToString(body('sendBatch')['$content']), '403 Forbidden')),
contains(base64ToString(body('sendBatch')['$content']), '404 Not Found')),
contains(base64ToString(body('sendBatch')['$content']), '408 Request Timeout')),
contains(base64ToString(body('sendBatch')['$content']), '409 Conflict')),
contains(base64ToString(body('sendBatch')['$content']), '500 Internal Server Error')),
contains(base64ToString(body('sendBatch')['$content']), '502 Bad Gateway')),
base64ToString(body('sendBatch')['$content']),
'')