Power Automate Data Transformation From JSON, Triple Nested Apply to Each Loop

[nmasmo]

I have fictionalized and simplified this problem for discussion purposes.

Suppose I own a toy factory. Let’s keep it simple: This toy factory has a 2 day workweek (Monday and Tuesday) and a 2 hour day (Hour 1 and Hour 2). Every week, the toy factory produces several types of toys (the quantity of different types of toys that are produced can vary).

Before the start of the week, the toy factory manager creates a Toy Production Plan. The most important thing about the Toy Production Plan is it tells us how many units of a particular toy the factory is planning to make that week. The Toy Production Plan might look something like this:

Or, it might look something like this…:

… the point I am trying to make with these examples is that..

1. There is a variety of different toy types that can be manufactured each week
2. The Toy Production Plan starts on the Monday of that week
3. The Toy Production Plan has 4 columns: [Toy], [Workers per Toy], [Monday Target], [Tuesday Target].

Every week, this data will be passed to a Power Automate flow as a JSON object from a Power App.

I essentially need to do a data transformation with a Power Automate flow. The flow needs to transform the factory manager’s production plan into a structure that works with the data table that the data needs to be stored in (dataverse table) Below, I show what the transformation needs to look like:

In the dataverse data table, each row corresponds to a particular date – hour of day – toy combination. Each row of the dataverse table will contain an “Hour Target”, which is the target number of units that must be produced for that particular hour/date/toy. This value is equal to the target of units for the day, divided by 2. I don’t need to make this column in Power Automate, as I can just have a calculated column in the dataverse table that calculates this value simply by dividing the [Day Target] column’s value by 2.

With my current approach to solving this data transformation problem with Power Automate, I am attempting to make a triple nested “Apply to each” loop. The structure of the triple nested “Apply to each” loop may look something like this:

Loop through dates d of the week

               Loop through hours h of the day

                              Loop through the toys t that will be produced that week

                                             Write d, h, t, Day Target, and Workers per Toy to new row in dataverse table

However, when I try to build this flow, I end up running into tons of problems. If someone was able to figure out this flow in Power Automate, I’d worship you like the Power Automate god/goddess that you are.

Thank you!

[grantjenkins]

Challenge accepted 🙂

A couple of questions:

1. Will the Week Commencing Date also be passed into Power Automate from the Power App?
2. Does it need to work for any number of days and hours? And if so, how would you know the total number of hours.
3. Would it be a different number of days and/or hours each time you pass it into Power Automate from Power Apps?

I have it working for the simple set of data you provided in your example, but not flexible with regards to number of days/hours.

[nmasmo]

@grantjenkins Yes, the week commencing date will be passed into Power Automate from the Power App.

While I am interested in seeing what a solution would look like for any number of days and hours, for now I think its best to keep it simple (because my Power Automate skills are not advanced yet) and assume the number of work days and work hours stays constant. In the real-world situation where I will have to apply this solution, I will need to make it work for 5 business days in a work week, and 9 work hours in a work-day.

[grantjenkins · Accepted solution]

Hopefully this is what you're looking for. Note that I've built it to cater for X number of hours (would need to be passed into the flow, as well as the WC date). It will work out the number of days dynamically based on the Target properties (Monday Target, Tuesday Target, etc.).

For this example, I've used a single JSON object that contains the WC Date, Number of hours, and actual data. Assume this would all be passed into the flow via Power Apps in your solution. Also, note that I haven't added the action to add to your Dataverse table - just building up the objects that you can then easily use.

See full flow below. I'll go into each of the actions.

Body is a Compose that contains the data that would be passed in via Power Apps.

{
    "WC": "2023-03-20",
    "Hours": 3,
    "Values": [
        {
            "Toy": "Ping poing ball",
            "Workers per Toy": 20,
            "Monday Target": 300,
            "Tuesday Target": 200
        },
        {
            "Toy": "Racecar",
            "Workers per Toy": 18,
            "Monday Target": 200,
            "Tuesday Target": 250
        },
        {
            "Toy": "Basketball",
            "Workers per Toy": 25,
            "Monday Target": 180,
            "Tuesday Target": 200
        }
    ]
}

XML is a Compose that converts our Values array (our data) to XML so we can use XPath expressions later in the flow. The expression used is.

xml(json(concat('{"root": { value:', outputs('Body')?['Values'], '}}')))

Select extracts out the Target property names from our data. This will allow us to retrieve the correct Targets for each day, and to dynamically calculate the number of days. The expressions used are:

//From
xpath(outputs('XML'), '//root/value[1]/*[contains(name(), "Target")]')

//Map
replace(xpath(item(), 'name(/*)'), '_x0020_', ' ')

Initialize variable Items creates a new variable called Items of type Array. This will eventually contain all of our objects.

Apply to each Day iterates over the number of days (starting with the value 0). If there were 3 days, we would iterate over 0, 1, and 2. It uses the following expression to generate the range we want to iterate over.

range(0, length(xpath(outputs('XML'), '//root/value[1]/*[contains(name(), "Target")]')))

Apply to each Hour iterates over each hour (starting with the value 1). If there were 4 hours, we would iterate over 1, 2, 3, and 4. It uses the following expression to generate the range we want to iterate over.

range(1, outputs('Body')?['Hours'])

Apply to each Day iterates over each of our data objects within the Values array. It uses the following expression.

outputs('Body')?['Values']

Append to array variable Items appends a new object for the current iteration. Below is the full expression for the object. Beneath that are each of the individual expressions used.

{
    "Date": @{addDays(outputs('Body')?['WC'], items('Apply_to_each_Day'), 'yyyy-MM-dd')},
    "Hour of Day": @{items('Apply_to_each_Hour')},
    "Toy": @{items('Apply_to_each_Toy')?['Toy']},
    "Workers per Toy": @{items('Apply_to_each_Toy')?['Workers per Toy']},
    "Day Target": @{items('Apply_to_each_Toy')[body('Select')[items('Apply_to_each_Day')]]},
    "Hour Target": @{div(items('Apply_to_each_Toy')[body('Select')[items('Apply_to_each_Day')]], outputs('Body')?['Hours'])}
}

//Date
addDays(outputs('Body')?['WC'], items('Apply_to_each_Day'), 'yyyy-MM-dd')

//Hour of Day
items('Apply_to_each_Hour')

//Toy
items('Apply_to_each_Toy')?['Toy']

//Workers per Toy
items('Apply_to_each_Toy')?['Workers per Toy']

//Day Target
items('Apply_to_each_Toy')[body('Select')[items('Apply_to_each_Day')]]

//Hour Target (Day Target divided by number of days)
div(items('Apply_to_each_Toy')[body('Select')[items('Apply_to_each_Day')]], outputs('Body')?['Hours'])

For your example, you could remove the Items variable completely, and replace it with your Add a new row action and add the data directly into your table using the expressions above.

If we ran the flow, we would see the following output that's stored in our Array variable called Items.

[
  {
    "Date": "2023-03-20",
    "Hour of Day": 1,
    "Toy": "Ping poing ball",
    "Workers per Toy": 20,
    "Day Target": 300,
    "Hour Target": 100
  },
  {
    "Date": "2023-03-20",
    "Hour of Day": 1,
    "Toy": "Racecar",
    "Workers per Toy": 18,
    "Day Target": 200,
    "Hour Target": 66
  },
  {
    "Date": "2023-03-20",
    "Hour of Day": 1,
    "Toy": "Basketball",
    "Workers per Toy": 25,
    "Day Target": 180,
    "Hour Target": 60
  },
  {
    "Date": "2023-03-20",
    "Hour of Day": 2,
    "Toy": "Ping poing ball",
    "Workers per Toy": 20,
    "Day Target": 300,
    "Hour Target": 100
  },
  {
    "Date": "2023-03-20",
    "Hour of Day": 2,
    "Toy": "Racecar",
    "Workers per Toy": 18,
    "Day Target": 200,
    "Hour Target": 66
  },
  {
    "Date": "2023-03-20",
    "Hour of Day": 2,
    "Toy": "Basketball",
    "Workers per Toy": 25,
    "Day Target": 180,
    "Hour Target": 60
  },
  {
    "Date": "2023-03-20",
    "Hour of Day": 3,
    "Toy": "Ping poing ball",
    "Workers per Toy": 20,
    "Day Target": 300,
    "Hour Target": 100
  },
  {
    "Date": "2023-03-20",
    "Hour of Day": 3,
    "Toy": "Racecar",
    "Workers per Toy": 18,
    "Day Target": 200,
    "Hour Target": 66
  },
  {
    "Date": "2023-03-20",
    "Hour of Day": 3,
    "Toy": "Basketball",
    "Workers per Toy": 25,
    "Day Target": 180,
    "Hour Target": 60
  },
  {
    "Date": "2023-03-21",
    "Hour of Day": 1,
    "Toy": "Ping poing ball",
    "Workers per Toy": 20,
    "Day Target": 200,
    "Hour Target": 66
  },
  {
    "Date": "2023-03-21",
    "Hour of Day": 1,
    "Toy": "Racecar",
    "Workers per Toy": 18,
    "Day Target": 250,
    "Hour Target": 83
  },
  {
    "Date": "2023-03-21",
    "Hour of Day": 1,
    "Toy": "Basketball",
    "Workers per Toy": 25,
    "Day Target": 200,
    "Hour Target": 66
  },
  {
    "Date": "2023-03-21",
    "Hour of Day": 2,
    "Toy": "Ping poing ball",
    "Workers per Toy": 20,
    "Day Target": 200,
    "Hour Target": 66
  },
  {
    "Date": "2023-03-21",
    "Hour of Day": 2,
    "Toy": "Racecar",
    "Workers per Toy": 18,
    "Day Target": 250,
    "Hour Target": 83
  },
  {
    "Date": "2023-03-21",
    "Hour of Day": 2,
    "Toy": "Basketball",
    "Workers per Toy": 25,
    "Day Target": 200,
    "Hour Target": 66
  },
  {
    "Date": "2023-03-21",
    "Hour of Day": 3,
    "Toy": "Ping poing ball",
    "Workers per Toy": 20,
    "Day Target": 200,
    "Hour Target": 66
  },
  {
    "Date": "2023-03-21",
    "Hour of Day": 3,
    "Toy": "Racecar",
    "Workers per Toy": 18,
    "Day Target": 250,
    "Hour Target": 83
  },
  {
    "Date": "2023-03-21",
    "Hour of Day": 3,
    "Toy": "Basketball",
    "Workers per Toy": 25,
    "Day Target": 200,
    "Hour Target": 66
  }
]

[nmasmo]

@grantjenkins 
FYI: I'm still working on your solution idea, I will update you as soon as I've got it worked out (I've got some new concepts to learn here like XML and Xpath)

[nmasmo]

@grantjenkins 
Thank you for this. Very impressive solution. I just mimicked your solution on my environment. I understand it for the most part.
Next I am going to try to implement this proof of concept into the real-world business problem I'm facing. I'll keep you posted.

[nmasmo]

@grantjenkins  Just did a bigger, more realistic test of this solution. It worked! Just "Accepted as Solution".

This solution will be good for helping people do some advanced data transformations in Power Automate.