cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
ArchieGoodwin
Advocate I
Advocate I

Extracting a substring from parsed JSON data in a Loop

I'm stumped on this one, folks.

I am taking JSON data from an online form, parsing it, and converting it into a .csv for import. I have built and tested a flow that works exactly as needed - almost. 

The online form gives users the option to enter information for anywhere from 0 to 10 children (family law intake). So, the JSON can contain data for 0 to 10 children. Incoming data for the fields contains either a six-or-eight-character Code, and then an explanation (it's in a drop-down on the online form). I need to export only a substring of that data (either the left 6 or 8 characters) into the .csv for import.

 

Problem is, I can't use Initialize Variable to do this within a loop (that will run x times, where x = 0 to 10).

Is there a way to substring just the raw parsed JSON data so I only grab the characters I need for inclusion in the variable table I have created for the data (prior to export to .csv)? I have tried to use the 'substring' expression, but it won't even let me select the variable I need from within the sub-data for "CHILDINV" - I'm stuck.

Child Data Loop.JPG

1 ACCEPTED SOLUTION

Accepted Solutions
eliotcole
Power Participant
Power Participant

Absolutely, @ArchieGoodwin, here's a solution which produces all the rows, all the headers, and the full 10 kids.

 

I'll admit that were this me I'd space out the logic a bit more so that others won't be blinded by the long expressions on show. One of the main things I try to show folks in a work environment that Power Automate makes an office less reliable on Jon/Janey and their wizard excel formulas. As soon as you start sprinkling long 'formula' everywhere here, anyone taking over the management of it might still be a bit taken aback.

 

If I were to do that here, I might make the two sides of that union in the CSV separate object variables then reference them.


Solution

This solution is transferrable if you want to apply it to more than one array in the original data.

finally.jpg

@Paulie78 thanks for the help, and for the range fiddle, too! I had originally done ten fields that long way, and thought that there *had* to be a way to get it done with less typing. 😅


Code

kidsVAR

This just counts the number of kids, and subtracts it by one. This saves overcomplicating the multiple if() statements in the Select action.

 

 

sub(length(variables('jsonInputVAR')?['body']?['CHILDINV']), 1)

 

 

SelectKidsData

In the 'From' field this uses a trick that Paulie taught me elsewhere, the range() function. So it iterates through 10 items, regardless.

 

 

range(0, 10)

 

 

In the 'Map' 'key' fields (left), this uses some basis logic looking at the current item() number in the range, and adds 1. This provides your child number.

 

 

add(item(), 1)

 

 

In the 'Map' 'value' fields (right), this uses a simple condition. If the current item() number is greater than the number returned in the kidsVAR variable, then it places nothing ('') in the field, otherwise it picks the child data that is correspondent to the current item() number. It looks like more than it is. 👍

 

 

if(greater(item(), variables('kidsVAR')), '', variables('jsonInputVAR')?['body']?['CHILDINV'][item()]['Id'])

if(greater(item(), variables('kidsVAR')), '', variables('jsonInputVAR')?['body']?['CHILDINV'][item()]['C1NAME'])

if(greater(item(), variables('kidsVAR')), '', variables('jsonInputVAR')?['body']?['CHILDINV'][item()]['C1BD'])

if(greater(item(), variables('kidsVAR')), '', variables('jsonInputVAR')?['body']?['CHILDINV'][item()]['Gender'])

if(greater(item(), variables('kidsVAR')), '', variables('jsonInputVAR')?['body']?['CHILDINV'][item()]['C1LEGAL'])

if(greater(item(), variables('kidsVAR')), '', variables('jsonInputVAR')?['body']?['CHILDINV'][item()]['C1PHYS'])

if(greater(item(), variables('kidsVAR')), '', variables('jsonInputVAR')?['body']?['CHILDINV'][item()]['C1VISIT'])

if(greater(item(), variables('kidsVAR')), '', variables('jsonInputVAR')?['body']?['CHILDINV'][item()]['FIELD8'])

if(greater(item(), variables('kidsVAR')), '', variables('jsonInputVAR')?['body']?['CHILDINV'][item()]['ItemNumber'])

 

 

Create CSV Table

This one is basically constructing an array for the Create CSV Table action to make its data from. Don't forget to place the square brackets around it in the 'From' field:

 

 

union(removeProperty(variables('jsonInputVAR')?['body'], 'CHILDINV') , json(replace(join(body('SelectKidsData'), ', '), '}, {', ',')))

 

 

The union() function is usually used to join two arrays together, but it works with objects, too, so the main thing here is ensuring that the child data is represented as an object instead of an array, after the previous Select action has done its thing.

 

So, in the second section of that union function this is done by:

  1. Using a join() function to join the separate children items together in a string value.
  2. Once they're in the string, they're still actually separated by curly brackets, so the replace() function removes the ones separating the children '}, {', and puts a normal comma in their place.
  3. Then finally the json() function makes that all readable by the union otherwise it would error thinking it's just a string.

 

The first part is getting all the non-child data by simply using the removeProperty() function to strip the CHILDINV data out of the original object.

 

Additional Arrays

To apply this to other fields in the original JSON just requires a few tweaks. Let's look at the potential for the DOCUMENTS array field to need it.

  1. Add a branch after the input.
  2. Initialise a docsVAR integer variable in the new branch to count the items in DOCUMENTS.
  3. For the Select copy the original and edit it to save time to SelectDocsData.
  4. Change the letters either side of the add(item(),1) expressions in the key fields to make your document headers.
  5. Edit each value to point to DOCUMENTS instead of CHILDINV, the new docsVAR for the counts, and change the referenced field names.
  6. Finally, bridge the branches with the CSV:

 

union(removeProperty(removeProperty(variables('jsonInputVAR')?['body'], 'CHILDINV'), 'DOCUMENTS') , json(replace(join(body('SelectKidsData'), ', '), '}, {', ',')) , json(replace(join(body('SelectDocsData'), ', '), '}, {', ',')))

 

If there is no set limit to the documents, then take the docsVAR and add 1 to it for the other side of the range() expression.

 

EDIT - I've tested this on ten children ... and they're all still alive to tell the tale!

View solution in original post

27 REPLIES 27
eliotcole
Power Participant
Power Participant

If I'm reading you right (I might not be) I don't see any reason why not, but I think if anyone is going to help here we might need to know a bit more.

 

Could you put up an example of the received data from a form in its JSON format, please?

 

It can be fake data, but the data how it looks before the Parse JSON action (what you'd have used to create the parse) would be wonderful.

 

 

Additionally, if you're appending that data to a CSV, you'll need to create rows if you intend to capture more than one item (with headers, etc). So I would suggest creating a String variable that contains a single carriage return (I tend to call them 'carriageVAR') and placing that at the start of each of these Append to string variable actions.

 

Placing new lines at the start avoids spurious additional lines in documents.

ArchieGoodwin
Advocate I
Advocate I

No matter what I do, I cannot post the JSON. I have stripped out the headers, everything. It keeps getting marked as SPAM and my replies deleted. 

 

Very frustrating.

ArchieGoodwin
Advocate I
Advocate I

OK, I have had to resort to taking screen snips of the JSON.

 

JSON 1.JPGJSON 2.JPGJSON 3.JPGJSON 4.JPG

eliotcole
Power Participant
Power Participant

Ouf, if that's real data, you might wanna scrub it again, quicksharp, mate!

 

((( did you try the code button? </> )))

ArchieGoodwin
Advocate I
Advocate I

NOT real data. All fake. 

eliotcole
Power Participant
Power Participant

Phew! 😅

Alright, for starters, with regards to ...

 

Variables

You initialise any variables (using the Initialize variable action) that you want at the start of your flow, have them sitting waiting for the data.

 

Then, when the time comes, you use the Set variable action to set the variable.

 

Calling JSON Data Directly
Now, it's entirely possible that you're not calling the data correctly in the Apply to each action that you're running there which you have named "Child DATA from JSON Entries" up above.

 

If you are not running a Parse JSON action, and working off of that (by fair the easiest way to pick values), then for each fieldname that you want to call, you need to use this syntax:

 

items('NAME_OF_APPLY_TO_ALL_ACTION_THIS_IS_IN')?['NAME_OF_FIELD']

 

So, in your case, a typical field would be:

 

items('Child_DATA_from_JSON_Entries')?['C1NAME']

 


Now, if you want to work that into a variable, so that it's useful within the same Apply to each loop run, then you place it in an expression, there, then just pick the variable up wherever needed.

 

Running the Whole Thing Without An Apply to each action

This is unwise, but it can be done.

 

It's unwise because:

  • It's Wasteful - You will need to create 10 data entry points, one for each possible and in each handle the potential for null / not present or empty fields ... on all fields.
  • It's Intensive - The extra processing will likely (over time) mean more average flow actions, due to the wastefulness.
  • Error Adjustments - There's more potential to have small bits of expressions slightly off.

Anyway, if you wish to reference a particular item from that asdasd array in the JSON, then you would just use something like (name 'JSON_Input' right according to your flow):

 

body('JSON_Input')?['CHILDINV'][0]?['C1NAME']

 

Now some might raise an eyebrow about the question marks and that [0] placement, but I'm pretty sure that flow is intelligent enough to follow that (it has for me, before).

 

Here's an example that's not quite analogous of me using that to grab the first result from a Filter array action that I know will get 1 response every time:

body('Filter_array')[0]?['Id']

taking the first entry from a filter array.jpg

 

ArchieGoodwin
Advocate I
Advocate I

Thanks for your help so far. To clarify, the Append to String Variable above is working. Basically, this flow doubles as a schema to map the JSON over to the formats (date from US to normal, removing extra characters, etc.) and creating a header row that matches the names of the data fields in the Advantage database we will be importing the results to:

headers matter data.JPG

 

So, as you can see, I'm able to initialize variables and create expressions to convert the data to the format I need, grab the substring, etc. - for all the data EXCEPT the data about children, because once I call CHILDINV, it goes into a loop. What I need is to be able to <formatDateTime> the child's birthday (C1BD), and <substring> custody status (C1PHYS, C1LEGAL) and visitation (C1VISIT), for 1 to 10 sets of data.

I can initialize variables and create my conversion expressions for the top-level data, but not for the data in the loop.

 

I suppose could create another flow that grabs the final CSV from Sharepoint, converts the data, and then saves it to a new CSV, but I was hoping to figure out how to do this before I created the matters.csv file.

 

So, in essence, how do I do this:

conversion date.JPG

...for the data inside the CHILDINV part of the JSON / parsed data.

 

Hope this helps, and thanks again.

eliotcole
Power Participant
Power Participant

Got ya, gimme a second! 🙂

eliotcole
Power Participant
Power Participant

I was going to put this in the solution post, but I have a feeling that it would've just made it stupidly long ... which is something I do, anyway. 😏

 

Here's the JSON data that I was working from:

{
  "body": {
    "FIRST": "Harvey",
    "MIDDLE": "Weinstein",
    "LAST": "Wallbanger",
    "MAIDEN": "Smith",
    "DOB": "2001-01-01",
    "CLASS": "CLI0LGMR - Legally Married",
    "ADDl": "123 Sesame Street",
    "LABEL2": "Apartment 7",
    "LABEL4": "R.R. #2",
    "CITY": "Trenton",
    "ZIP": "K8V 1R6",
    "Province": "Ontarlo",
    "JURISDICr": "4 years",
    "PRIVEMAIL": true,
    "EMAIL": "glenn@xasdasd.ca",
    "PREFEMAIL": true,
    "PHONE4": "343-263-4643",
    "PRIVPHONEC": true,
    "DAYPHONE": "613-966-777'",
    "PRIVPHONE": true,
    "COMPANY": "KDA Law",
    "REFERREDBY": "Jeff van de K2e-t",
    "FIRST2": "Karen",
    "MIDDLE2": "Snarf",
    "LAST2": "Wallbanger",
    "MAIDEN2": "deKaren",
    "DOB2": "1998-01-24",
    "CLASS2": "OPP-LG1 - Legally Married",
    "ADD12": "123 Complaint Bouleva~d",
    "LABEL22": "Apartment 2",
    "LABEL42": "R.R. 17",
    "City2": "Picton",
    "Province2": "Ontario",
    "ZIP2": "K7L 2E4",
    "JURISDICT2": "3 months",
    "OpposingPartysEmailAddress": "ka~e~Sw'~ine~.ccr~",
    "PHONE42": "613-966-7771",
    "DAYPHONE2": "647-265-8597",
    "COMPANY2": "Nestle Quinte West",
    "COUNTY": "Hastings",
    "COHABITATE": true,
    "DATEMARRY": "2020-01-16",
    "STATEMARRY": "Ontario",
    "DATESEP": "2021-08-02",
    "STATESEP": "Ontario",
    "FIELDl": "Belleville",
    "PRENUPTUAL": false,
    "ISFATHER": true,
    "ISMOTHER": true,
    "DOM": "2000-01-27",
    "DIVDATE": "2004-07-29",
    "CHILDREN": true,
    "ISRELATED": true,
    "CHILDINV": [
      {
        "Id": "3klAhq",
        "C1NAME": "aaaaa aaaaa",
        "C1BD": "2020-10-01",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 1
      },
      {
        "Id": "3UM0Xn",
        "C1NAME": "bbbbb bbbbb",
        "C1BD": "2020-10-02",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 2
      },
      {
        "Id": "2s3wBA",
        "C1NAME": "ccccc ccccc",
        "C1BD": "2020-10-03",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 3
      },
      {
        "Id": "lhgNwf",
        "C1NAME": "ddddd ddddd",
        "C1BD": "2020-10-04",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 4
      }
    ],
    "CHILDINV_Minimum": 0,
    "INCMECY": 35000,
    "INCMESY": 35000,
    "OWNHOME": true,
    "JTRE": 350000,
    "HOUSEOWNED": 200000,
    "HOUSETITITLEE": "TICOMM - Tenancy in Common",
    "SELLRES": true,
    "JTCASH": 15000,
    "JTINS": 15000,
    "JTPTY": 65000,
    "JTRET": 47000,
    "BIRTHEXP": 28500,
    "DOCUMENTS": [],
    "AGREFEE": true,
    "AGREFEE_Amount": 335,
    "AGREFEE_QuantitySelected": 1,
    "AGREFEE_Price": 365,
    "WILL": false,
    "SELFPROVED": false,
    "PUBLICASST": false,
    "Id": "1-3",
    "INCMECY_IncrementBy": 1,
    "INCMESY_IncrementBy": 1,
    "JTRE_IncrementBy": 1,
    "HOUSEOWNED_IncrementBy": 1,
    "JTCASH_IncrementBy": 1,
    "JTINS_IncrementBy": 1,
    "JTPTY_IncrementBy": 1,
    "JTRET_IncrementBy": 1,
    "BIRTHEXP_IncrementBy": 1
  }
}
eliotcole
Power Participant
Power Participant

Solution

OK, so this might seem daunting if you haven't been exposed to some of this, but I'll be honest, I only *barely* understand how XML works, and all I do know is that these bits and bobs are pure magic.

 

This has heavy thanks to @JohnLiu (hope that's the right userID!) whom I pilfered this a while back for some other stuff (on here, and elsewhere).

 

XML & XPATH

So, this uses XML to parse the JSON data, and convert it into XML data. As far as I can fathom it, the XML data can be either an array or an object, but it still finds a way to get what you need.

 

You'll see two expressions used more than once below, the columnName if() statement, and the columnValue xpath() statement. I'm utilising these generic statements to grab the header information from the various fields, and then equally grab the values from those fields.

 

The key part in both of them is item(), which in a Select or Filter action will always be referring to a generic individual item in the array that they are processing.

 

Field Name if() Expression (columnName)

This really just ensures that when the field name is extracted from the XML it is formatted correctly, accounting for potential issues with code in the name, or strange alphaNumeric strings.

 

 

 

if(startsWith(string(xpath(item(), 'name(/*)')), '_x003'), if(equals(length(string(xpath(item(), 'name(/*)'))), 7), substring(string(xpath(item(), 'name(/*)')), 5, 1), concat(substring(string(xpath(item(), 'name(/*)')), 5, 1), substring(string(xpath(item(), 'name(/*)')), 7))), string(xpath(item(), 'name(/*)')))

 

 

 

 

Field Value xpath() Expression (columnValue)

This is much simpler, and just grabs the value of a given field.

 

 

 

xpath(item(), 'string(//*)')

 

 

 

 

SELECT

You'll also note that in my initial flow, here, I'm using some vanilla Select actions, too. I'm doing this to separate out the logic because I know I'm not a genius(!), and to make the layout a little more obvious as to what goes where.

 

For the Kids' stuff, it's actually helpful that I've gone focused on this pass, when I post the more generic version, you'll be able to understand where the changes are, and why.

 

The Flow

As you can see, there's two branches working simultaneously, on the left you're gathering the generic data, and on the right the kids data.

00 - Base Flow.jpg

 

Here's a deeper drill down into the flow:

 

00 - Flow.jpg

 

OK, now on to the specifics.

 

 

General Data Side

This side took a while, as I was uhming and ah'ing about the most expedient way to put everything togther, mostly in the csvVAR at the end, but because of that, I had to keep reconsidering this part and re-testing.

01 - Main Data.jpg

 

The xpath() expression in that first select is:

 

 

 

 

xpath(xml(json(concat('{ "root": { "columns": ', string(variables('jsonInputVAR')?['body']), ' } }'))), '/root/columns/*')

 

 

 

Essentially, this is;

  1. the expression concatenating data together with the concat() function,
  2. then ensuring that it is presented as valid JSON with the json() function,
  3. which is then converted by the xml() function,
  4. to finally be parsed by the xpath() function.

All that magic is then readable by the Select action, and the Map expressions can get to work on pulling what you really want from it.

 

From this data, it produces data like this:

 

 

 

[
  {
    "columnName": "FIRST",
    "columnValue": "Harvey"
  },
  {
    "columnName": "MIDDLE",
    "columnValue": "Weinstein"
  },
  {
    "columnName": "LAST",
    "columnValue": "Wallbanger"
  },
  {
    "columnName": "MAIDEN",
    "columnValue": "Smith"
  } ...

 

 

 

 

Then the rest is quite simply a filter to remove that CHILDINV part, and then producing the arrays that I'll generate the general data headers (SelectHeaders) and data (SelectValues) from.

 

The Kids Side

That initial variables() call is referring directly to the CHILDINV field in the original data.

 

variables('jsonInputVAR')?['body']?['CHILDINV']

 

02 - Kids Data.jpg

 

Here you can see the if() statement spelled out, and similar xpath() expressions that were used on the other side, initially. However the big deal on this side is the part that you had issues with, iterating the CHILDINV data.

 

 

Looking at that first SelectKidsData action, you'll see field/column names defined and values also. Every single piece of dynamic data in that set is simply done with:

 

 

 

item()?['FIELD_NAME']

 

 

 

Where item()?['FIELD_NAME'] is one of the names presented in the CHILDINV array, like  item()?['Gender'].

 

So, you can see, your ItemNumber field is SUPER useful here, by looking at example data from what it spouts out:

 

 

 

[
  {
    "C1ID": "3klAhq",
    "C1NAME": "aaaaa aaaaa",
    "C1BD": "2020-10-01",
    "C1SEX": "Male",
    "C1LEGAL": "CLIENT - You",
    "C1PHYS": "CLIENT - You",
    "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
    "C1NOTES": "Lots of comments because I like to counent on stuff.",
    "C1NUM": 1
  },
  {
    "C2ID": "3UM0Xn",
    "C2NAME": "bbbbb bbbbb",
    "C2BD": "2020-10-02",
    "C2SEX": "Male",
    "C2LEGAL": "CLIENT - You",
    "C2PHYS": "CLIENT - You",
    "C2VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
    "C2NOTES": "Lots of comments because I like to counent on stuff.",
    "C2NUM": 2
  }...

 

 

 

 

CSV Creation

Finally, this feels like the most straightforward part, the string variable just pulls all the data together into one place, and formats it for CSV with each CSV entry encased in speech marks just in case there are commas in the data that's sent through.

03 - CSV Builder.jpg

 

 

 

 

HEADERS
join(split(first(split(last(split(string(body('SelectHeaders')), '[{"columnName":"')), '"}]')), '"},{"columnName":"'), '","')

join(split(first(split(last(split(string(body('SelectKidsHeaders')), '[{"columnName":"')), '"}]')), '"},{"columnName":"'), '","')

DATA
join(split(first(split(last(split(string(body('SelectValues')), '[{"columnValue":"')), '"}]')), '"},{"columnValue":"'), '","')

join(split(first(split(last(split(string(body('SelectKidsValues')), '[{"columnValue":"')), '"}]')), '"},{"columnValue":"'), '","')

 

 

 

Each of the four parts in the csvVAR action are basically the same, just referring to different sources, you could do it all at separate points if you'd rather, and pool it here. Essentially each expression works thusly:

 

  1. It takes a given array of data, and presents it as a string().
  2. It uses last() and split() on that text string (to make an array 🤪);
    1. splitting it on the initial characters ([{"columnValue":" for the general data),
    2. then taking the last item in the two part array it makes.
  3. Then it splits it again, this time on the final characters ("}]) in the string, here you want the first() of the two produced array items.
  4. The final split is on the data that is left, and you want to split on the JSON data that separating the meat that we want, "},{"columnValue":".
  5. Once all of that has been done, you're left with an array of all the items that you want, so you just need to join() them together with "," so that they are readable in CSV format.

The rest of the csvVAR action is just framing that data with a couple more speech marks and commas!

 

That should be it!

 

EDIT - I'm going to leave optimising the Kids' side for now as it's both late, and I need to understand how to pull a value out of that XML!

Paulie78
Super User
Super User

You can do it easily with a select action. Take a look at this screenshot:

https://ibb.co/F7MJcdJ

To make it simple for you to recreate, I copied my actions into a Scope. If you would like a working demonstration at your end, simply do the following:

  • Copy the code below.
  • Create a new flow.
  • Add an action.
  • Go to "My clipboard"
  • Press CTRL-V
  • Click the scope action that appears.

Here is the code for you to copy:

{"id":"936a697a-e46e-41e6-b201-2d05-01ede363","brandColor":"#8C3900","connectionReferences":{},"connectorDisplayName":"Control","icon":"data&colon;image/svg+xml;base64,PHN2ZyB3aWR0aD0iMzIiIGhlaWdodD0iMzIiIHZlcnNpb249IjEuMSIgdmlld0JveD0iMCAwIDMyIDMyIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPg0KIDxwYXRoIGQ9Im0wIDBoMzJ2MzJoLTMyeiIgZmlsbD0iIzhDMzkwMCIvPg0KIDxwYXRoIGQ9Im04IDEwaDE2djEyaC0xNnptMTUgMTF2LTEwaC0xNHYxMHptLTItOHY2aC0xMHYtNnptLTEgNXYtNGgtOHY0eiIgZmlsbD0iI2ZmZiIvPg0KPC9zdmc+DQo=","isTrigger":false,"operationName":"Scope","operationDefinition":{"type":"Scope","actions":{"Compose":{"type":"Compose","inputs":{"body":{"FIRST":"Harvey","MIDDLE":"Weinstein","LAST":"Wallbanger","MAIDEN":"Smith","DOB":"2001-01-01","CLASS":"CLI0LGMR - Legally Married","ADDl":"123 Sesame Street","LABEL2":"Apartment 7","LABEL4":"R.R. #2","CITY":"Trenton","ZIP":"K8V 1R6","Province":"Ontarlo","JURISDICr":"4 years","PRIVEMAIL":true,"EMAIL":"glenn@xasdasd.ca","PREFEMAIL":true,"PHONE4":"343-263-4643","PRIVPHONEC":true,"DAYPHONE":"613-966-777'","PRIVPHONE":true,"COMPANY":"KDA Law","REFERREDBY":"Jeff van de K2e-t","FIRST2":"Karen","MIDDLE2":"Snarf","LAST2":"Wallbanger","MAIDEN2":"deKaren","DOB2":"1998-01-24","CLASS2":"OPP-LG1 - Legally Married","ADD12":"123 Complaint Bouleva~d","LABEL22":"Apartment 2","LABEL42":"R.R. 17","City2":"Picton","Province2":"Ontario","ZIP2":"K7L 2E4","JURISDICT2":"3 months","OpposingPartysEmailAddress":"ka~e~Sw'~ine~.ccr~","PHONE42":"613-966-7771","DAYPHONE2":"647-265-8597","COMPANY2":"Nestle Quinte West","COUNTY":"Hastings","COHABITATE":true,"DATEMARRY":"2020-01-16","STATEMARRY":"Ontario","DATESEP":"2021-08-02","STATESEP":"Ontario","FIELDl":"Belleville","PRENUPTUAL":false,"ISFATHER":true,"ISMOTHER":true,"DOM":"2000-01-27","DIVDATE":"2004-07-29","CHILDREN":true,"ISRELATED":true,"CHILDINV":[{"Id":"3klAhq","C1NAME":"aaaaa aaaaa","C1BD":"2020-10-01","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":1},{"Id":"3UM0Xn","C1NAME":"bbbbb bbbbb","C1BD":"2020-10-02","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":2},{"Id":"2s3wBA","C1NAME":"ccccc ccccc","C1BD":"2020-10-03","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":3},{"Id":"lhgNwf","C1NAME":"ddddd ddddd","C1BD":"2020-10-04","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":4}],"CHILDINV_Minimum":0,"INCMECY":35000,"INCMESY":35000,"OWNHOME":true,"JTRE":350000,"HOUSEOWNED":200000,"HOUSETITITLEE":"TICOMM - Tenancy in Common","SELLRES":true,"JTCASH":15000,"JTINS":15000,"JTPTY":65000,"JTRET":47000,"BIRTHEXP":28500,"DOCUMENTS":[],"AGREFEE":true,"AGREFEE_Amount":335,"AGREFEE_QuantitySelected":1,"AGREFEE_Price":365,"WILL":false,"SELFPROVED":false,"PUBLICASST":false,"Id":"1-3","INCMECY_IncrementBy":1,"INCMESY_IncrementBy":1,"JTRE_IncrementBy":1,"HOUSEOWNED_IncrementBy":1,"JTCASH_IncrementBy":1,"JTINS_IncrementBy":1,"JTPTY_IncrementBy":1,"JTRET_IncrementBy":1,"BIRTHEXP_IncrementBy":1}},"runAfter":{}},"Select":{"type":"Select","inputs":{"from":"@outputs('Compose')['BODY']['CHILDINV']","select":{"Id":"@item()['Id']","C1NAME":"@item()['C1NAME']","C1BD":"@formatDateTime(item()['C1BD'], 'dd-MM-yyyy')","Gender":"@item()['Gender']","C1LEGAL":"@substring(item()['C1LEGAL'],0, 6)","C1PHYS":"@substring(item()['C1PHYS'],0, 6)","C1VISIT":"@substring(item()['C1VISIT'],0, 8)","FIELD8":"@item()['FIELD8']","ItemNumber":"@item()['ItemNumber']"}},"runAfter":{"Compose":["Succeeded"]},"description":"outputs('Compose')['BODY']['CHILDINV']"},"Create_CSV_table":{"type":"Table","inputs":{"from":"@body('Select')","format":"CSV"},"runAfter":{"Select":["Succeeded"]},"description":"body('Select')"}},"runAfter":{}}}

See how you get on and let me know if you need further guidance.

 

Blog: tachytelic.net

YouTube: https://www.youtube.com/c/PaulieM/videos

If I answered your question, please accept it as a solution 😘

eliotcole
Power Participant
Power Participant

He needs to iterate the children column names, though, @Paulie78 ... but my solution can be slimmed down a bit still (for example I'd put a removeProperty() in the xpath() on the main Select action and remove the filter action) but for now it shows the logic. I was wondering if the range trick would help there, and it would, but only if he hadn't already defined the child number.

 

Anyway, yeah, on the iterative headings: So if there are four children (like the example data I put above) then there will be four additional sets of nine columns in the CSV. Nine columns for each child.

 

"C1ID","C1NAME","C1BD","C1SEX","C1LEGAL","C1PHYS","C1VISIT","C1NOTES","C1NUM","C2ID","C2NAME","C2BD","C2SEX","C2LEGAL","C2PHYS","C2VISIT","C2NOTES","C2NUM","C3ID","C3NAME","C3BD","C3SEX","C3LEGAL","C3PHYS","C3VISIT","C3NOTES","C3NUM","C4ID","C4NAME","C4BD","C4SEX","C4LEGAL","C4PHYS","C4VISIT","C4NOTES","C4NUM"

 

 

Wow...Lots to take in there, but I'm up for the challenge! I will take a look at it in more detail later today, and try to get it working tonight - I'll keep you posted!

That's the real trick...It doesn't really matter what the data is called in the schema or the flow, all that matters is that it comes out in the CSV with the right header row above it, and the right data field name.

eliotcole
Power Participant
Power Participant

Cheers, @ArchieGoodwin, I've definitely got it working with the sample data above, I'm just working on slimming it down some.

 

If you want to make it less cumbersome in your head, just look at the parts where I say what to do, rather than the explanations. If, like me, you're a 'learn by doing' person (kineathetic learner!) then by implementing it you'll probably get to the understanding (or maybe fiddling around with a copy).

With regards to the slimming ... Like I say, removing the property negates the need for the filter action, which is good, and I think I'm on to removing the need for the final action and changing that to a generic CSV, just playing with that child data, and some of Paulie's tricks from elsewhere!

Paulie78
Super User
Super User

I slightly misunderstood, but you can still do it easily in one action:

2021-08-23_19-36-10.png

 

I haven't done every field, but enough to get you started off. Output produced from this is:

FIRST,MIDDLE,LAST,CHILD1_ID,CHILD1_NAME,CHILD2_ID,CHILD2_NAME
Harvey,Weinstein,Wallbanger,3klAhq,aaaaa aaaaa,3UM0Xn,bbbbb bbbbb

This was my input JSON:

{
    "FIRST": "Harvey",
    "MIDDLE": "Weinstein",
    "LAST": "Wallbanger",
    "MAIDEN": "Smith",
    "DOB": "2001-01-01",
    "CLASS": "CLI0LGMR - Legally Married",
    "ADDl": "123 Sesame Street",
    "LABEL2": "Apartment 7",
    "LABEL4": "R.R. #2",
    "CITY": "Trenton",
    "ZIP": "K8V 1R6",
    "Province": "Ontarlo",
    "JURISDICr": "4 years",
    "PRIVEMAIL": true,
    "EMAIL": "glenn@xasdasd.ca",
    "PREFEMAIL": true,
    "PHONE4": "343-263-4643",
    "PRIVPHONEC": true,
    "DAYPHONE": "613-966-777'",
    "PRIVPHONE": true,
    "COMPANY": "KDA Law",
    "REFERREDBY": "Jeff van de K2e-t",
    "FIRST2": "Karen",
    "MIDDLE2": "Snarf",
    "LAST2": "Wallbanger",
    "MAIDEN2": "deKaren",
    "DOB2": "1998-01-24",
    "CLASS2": "OPP-LG1 - Legally Married",
    "ADD12": "123 Complaint Bouleva~d",
    "LABEL22": "Apartment 2",
    "LABEL42": "R.R. 17",
    "City2": "Picton",
    "Province2": "Ontario",
    "ZIP2": "K7L 2E4",
    "JURISDICT2": "3 months",
    "OpposingPartysEmailAddress": "ka~e~Sw'~ine~.ccr~",
    "PHONE42": "613-966-7771",
    "DAYPHONE2": "647-265-8597",
    "COMPANY2": "Nestle Quinte West",
    "COUNTY": "Hastings",
    "COHABITATE": true,
    "DATEMARRY": "2020-01-16",
    "STATEMARRY": "Ontario",
    "DATESEP": "2021-08-02",
    "STATESEP": "Ontario",
    "FIELDl": "Belleville",
    "PRENUPTUAL": false,
    "ISFATHER": true,
    "ISMOTHER": true,
    "DOM": "2000-01-27",
    "DIVDATE": "2004-07-29",
    "CHILDREN": true,
    "ISRELATED": true,
    "CHILDINV": [
      {
        "Id": "3klAhq",
        "C1NAME": "aaaaa aaaaa",
        "C1BD": "2020-10-01",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 1
      },
      {
        "Id": "3UM0Xn",
        "C1NAME": "bbbbb bbbbb",
        "C1BD": "2020-10-02",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 2
      },
      {
        "Id": "2s3wBA",
        "C1NAME": "ccccc ccccc",
        "C1BD": "2020-10-03",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 3
      },
      {
        "Id": "lhgNwf",
        "C1NAME": "ddddd ddddd",
        "C1BD": "2020-10-04",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 4
      }
    ],
    "CHILDINV_Minimum": 0,
    "INCMECY": 35000,
    "INCMESY": 35000,
    "OWNHOME": true,
    "JTRE": 350000,
    "HOUSEOWNED": 200000,
    "HOUSETITITLEE": "TICOMM - Tenancy in Common",
    "SELLRES": true,
    "JTCASH": 15000,
    "JTINS": 15000,
    "JTPTY": 65000,
    "JTRET": 47000,
    "BIRTHEXP": 28500,
    "DOCUMENTS": [],
    "AGREFEE": true,
    "AGREFEE_Amount": 335,
    "AGREFEE_QuantitySelected": 1,
    "AGREFEE_Price": 365,
    "WILL": false,
    "SELFPROVED": false,
    "PUBLICASST": false,
    "Id": "1-3",
    "INCMECY_IncrementBy": 1,
    "INCMESY_IncrementBy": 1,
    "JTRE_IncrementBy": 1,
    "HOUSEOWNED_IncrementBy": 1,
    "JTCASH_IncrementBy": 1,
    "JTINS_IncrementBy": 1,
    "JTPTY_IncrementBy": 1,
    "JTRET_IncrementBy": 1,
    "BIRTHEXP_IncrementBy": 1
  }

This is how you do it:

Create a select action. For the from, just literally type [0]

For the Header records, just access them directly, examples:

outputs('Compose')['FIRST']
outputs('Compose')['MIDDLE']
outputs('Compose')['LAST']

For the array elements, as you know there will never be more than ten, you can just build columns for each one. Examples:

outputs('Compose')['CHILDINV']?[0]['Id']
outputs('Compose')['CHILDINV']?[0]['C1NAME']
outputs('Compose')['CHILDINV']?[1]['Id']
outputs('Compose')['CHILDINV']?[1]['C1NAME']

The above give the ID and name for child one and two, you would just continue to populate the select and then you'd have the whole thing on one line of a CSV.

 

As before, I have copied it into a scope for you so you can see how it works:

{"id":"8a8a92db-c996-4c42-bd8f-4367-c314d244","brandColor":"#8C3900","connectionReferences":{},"connectorDisplayName":"Control","icon":"data&colon;image/svg+xml;base64,PHN2ZyB3aWR0aD0iMzIiIGhlaWdodD0iMzIiIHZlcnNpb249IjEuMSIgdmlld0JveD0iMCAwIDMyIDMyIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPg0KIDxwYXRoIGQ9Im0wIDBoMzJ2MzJoLTMyeiIgZmlsbD0iIzhDMzkwMCIvPg0KIDxwYXRoIGQ9Im04IDEwaDE2djEyaC0xNnptMTUgMTF2LTEwaC0xNHYxMHptLTItOHY2aC0xMHYtNnptLTEgNXYtNGgtOHY0eiIgZmlsbD0iI2ZmZiIvPg0KPC9zdmc+DQo=","isTrigger":false,"operationName":"Scope","operationDefinition":{"type":"Scope","actions":{"Compose":{"type":"Compose","inputs":{"FIRST":"Harvey","MIDDLE":"Weinstein","LAST":"Wallbanger","MAIDEN":"Smith","DOB":"2001-01-01","CLASS":"CLI0LGMR - Legally Married","ADDl":"123 Sesame Street","LABEL2":"Apartment 7","LABEL4":"R.R. #2","CITY":"Trenton","ZIP":"K8V 1R6","Province":"Ontarlo","JURISDICr":"4 years","PRIVEMAIL":true,"EMAIL":"glenn@xasdasd.ca","PREFEMAIL":true,"PHONE4":"343-263-4643","PRIVPHONEC":true,"DAYPHONE":"613-966-777'","PRIVPHONE":true,"COMPANY":"KDA Law","REFERREDBY":"Jeff van de K2e-t","FIRST2":"Karen","MIDDLE2":"Snarf","LAST2":"Wallbanger","MAIDEN2":"deKaren","DOB2":"1998-01-24","CLASS2":"OPP-LG1 - Legally Married","ADD12":"123 Complaint Bouleva~d","LABEL22":"Apartment 2","LABEL42":"R.R. 17","City2":"Picton","Province2":"Ontario","ZIP2":"K7L 2E4","JURISDICT2":"3 months","OpposingPartysEmailAddress":"ka~e~Sw'~ine~.ccr~","PHONE42":"613-966-7771","DAYPHONE2":"647-265-8597","COMPANY2":"Nestle Quinte West","COUNTY":"Hastings","COHABITATE":true,"DATEMARRY":"2020-01-16","STATEMARRY":"Ontario","DATESEP":"2021-08-02","STATESEP":"Ontario","FIELDl":"Belleville","PRENUPTUAL":false,"ISFATHER":true,"ISMOTHER":true,"DOM":"2000-01-27","DIVDATE":"2004-07-29","CHILDREN":true,"ISRELATED":true,"CHILDINV":[{"Id":"3klAhq","C1NAME":"aaaaa aaaaa","C1BD":"2020-10-01","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":1},{"Id":"3UM0Xn","C1NAME":"bbbbb bbbbb","C1BD":"2020-10-02","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":2},{"Id":"2s3wBA","C1NAME":"ccccc ccccc","C1BD":"2020-10-03","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":3},{"Id":"lhgNwf","C1NAME":"ddddd ddddd","C1BD":"2020-10-04","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":4}],"CHILDINV_Minimum":0,"INCMECY":35000,"INCMESY":35000,"OWNHOME":true,"JTRE":350000,"HOUSEOWNED":200000,"HOUSETITITLEE":"TICOMM - Tenancy in Common","SELLRES":true,"JTCASH":15000,"JTINS":15000,"JTPTY":65000,"JTRET":47000,"BIRTHEXP":28500,"DOCUMENTS":[],"AGREFEE":true,"AGREFEE_Amount":335,"AGREFEE_QuantitySelected":1,"AGREFEE_Price":365,"WILL":false,"SELFPROVED":false,"PUBLICASST":false,"Id":"1-3","INCMECY_IncrementBy":1,"INCMESY_IncrementBy":1,"JTRE_IncrementBy":1,"HOUSEOWNED_IncrementBy":1,"JTCASH_IncrementBy":1,"JTINS_IncrementBy":1,"JTPTY_IncrementBy":1,"JTRET_IncrementBy":1,"BIRTHEXP_IncrementBy":1},"runAfter":{}},"Hope__you":{"type":"Compose","inputs":"Enjoy :D","runAfter":{"Create_CSV_table":["Succeeded"]}},"Select":{"type":"Select","inputs":{"from":[0],"select":{"FIRST":"@outputs('Compose')['FIRST']","MIDDLE":"@outputs('Compose')['MIDDLE']","LAST":"@outputs('Compose')['LAST']","CHILD1_ID":"@outputs('Compose')['CHILDINV']?[0]['Id']","CHILD1_NAME":"@outputs('Compose')['CHILDINV']?[0]['C1NAME']","CHILD2_ID":"@outputs('Compose')['CHILDINV']?[1]['Id']","CHILD2_NAME":"@outputs('Compose')['CHILDINV']?[1]['C1NAME']"}},"runAfter":{"Compose":["Succeeded"]}},"Create_CSV_table":{"type":"Table","inputs":{"from":"@body('Select')","format":"CSV"},"runAfter":{"Select":["Succeeded"]}}},"runAfter":{}}}

 

eliotcole
Power Participant
Power Participant

Don't you still have to manually create those keys in the select, mate? When there's less than ten that's going to create pointless columns, no?

Paulie78
Super User
Super User

Yeah, but it is a one off operation. You'd want the column names to be static, so that your import was always consistent. You would still want a column for Child10_Name, even if there was no child 10. It's the whole operation wrapped in a single API action, so it will finish instantly and it can't get any more efficient than that. 

 

What's the issue with manually creating the keys? Did I miss something?

So, this is closer - but I took a look at the scope, ran it, and the output appeared to be the same as your earlier answer above.

I need the output to actually be two rows, with the first row being the "header" row, and the second one being the data row.

It makes sense to me, how to compose the header row and then pull the data (not sure how you separate the two rows)?

Helpful resources

Announcements
UG GA Amplification 768x460.png

Launching new user group features

Learn how to create your own user groups today!

Community Connections 768x460.jpg

Community & How To Videos

Check out the new Power Platform Community Connections gallery!

M365 768x460.jpg

Microsoft 365 Collaboration Conference | December 7–9, 2021

Join us, in-person, December 7–9 in Las Vegas, for the largest gathering of the Microsoft community in the world.

Users online (2,429)