Extracting a substring from parsed JSON data in a Loop

[ArchieGoodwin]

I'm stumped on this one, folks.

I am taking JSON data from an online form, parsing it, and converting it into a .csv for import. I have built and tested a flow that works exactly as needed - almost. 

The online form gives users the option to enter information for anywhere from 0 to 10 children (family law intake). So, the JSON can contain data for 0 to 10 children. Incoming data for the fields contains either a six-or-eight-character Code, and then an explanation (it's in a drop-down on the online form). I need to export only a substring of that data (either the left 6 or 8 characters) into the .csv for import.

Problem is, I can't use Initialize Variable to do this within a loop (that will run x times, where x = 0 to 10).

Is there a way to substring just the raw parsed JSON data so I only grab the characters I need for inclusion in the variable table I have created for the data (prior to export to .csv)? I have tried to use the 'substring' expression, but it won't even let me select the variable I need from within the sub-data for "CHILDINV" - I'm stuck.

[eliotcole]

If I'm reading you right (I might not be) I don't see any reason why not, but I think if anyone is going to help here we might need to know a bit more.

Could you put up an example of the received data from a form in its JSON format, please?

It can be fake data, but the data how it looks before the Parse JSON action (what you'd have used to create the parse) would be wonderful.

Additionally, if you're appending that data to a CSV, you'll need to create rows if you intend to capture more than one item (with headers, etc). So I would suggest creating a String variable that contains a single carriage return (I tend to call them 'carriageVAR') and placing that at the start of each of these Append to string variable actions.

Placing new lines at the start avoids spurious additional lines in documents.

[ArchieGoodwin]

No matter what I do, I cannot post the JSON. I have stripped out the headers, everything. It keeps getting marked as SPAM and my replies deleted. 

Very frustrating.

[ArchieGoodwin]

OK, I have had to resort to taking screen snips of the JSON.

[eliotcole]

Ouf, if that's real data, you might wanna scrub it again, quicksharp, mate!

((( did you try the code button? </> )))

[ArchieGoodwin]

NOT real data. All fake. 

[eliotcole]

Phew! 😅

Alright, for starters, with regards to ...

Variables

You initialise any variables (using the Initialize variable action) that you want at the start of your flow, have them sitting waiting for the data.

Then, when the time comes, you use the Set variable action to set the variable.

Calling JSON Data Directly
Now, it's entirely possible that you're not calling the data correctly in the Apply to each action that you're running there which you have named "Child DATA from JSON Entries" up above.

If you are not running a Parse JSON action, and working off of that (by fair the easiest way to pick values), then for each fieldname that you want to call, you need to use this syntax:

items('NAME_OF_APPLY_TO_ALL_ACTION_THIS_IS_IN')?['NAME_OF_FIELD']

So, in your case, a typical field would be:

items('Child_DATA_from_JSON_Entries')?['C1NAME']

Now, if you want to work that into a variable, so that it's useful within the same Apply to each loop run, then you place it in an expression, there, then just pick the variable up wherever needed.

Running the Whole Thing Without An Apply to each action

This is unwise, but it can be done.

It's unwise because:

• It's Wasteful - You will need to create 10 data entry points, one for each possible and in each handle the potential for null / not present or empty fields ... on all fields.
• It's Intensive - The extra processing will likely (over time) mean more average flow actions, due to the wastefulness.
• Error Adjustments - There's more potential to have small bits of expressions slightly off.

Anyway, if you wish to reference a particular item from that asdasd array in the JSON, then you would just use something like (name 'JSON_Input' right according to your flow):

body('JSON_Input')?['CHILDINV'][0]?['C1NAME']

Now some might raise an eyebrow about the question marks and that [0] placement, but I'm pretty sure that flow is intelligent enough to follow that (it has for me, before).

Here's an example that's not quite analogous of me using that to grab the first result from a Filter array action that I know will get 1 response every time:

body('Filter_array')[0]?['Id']

[ArchieGoodwin]

Thanks for your help so far. To clarify, the Append to String Variable above is working. Basically, this flow doubles as a schema to map the JSON over to the formats (date from US to normal, removing extra characters, etc.) and creating a header row that matches the names of the data fields in the Advantage database we will be importing the results to:

So, as you can see, I'm able to initialize variables and create expressions to convert the data to the format I need, grab the substring, etc. - for all the data EXCEPT the data about children, because once I call CHILDINV, it goes into a loop. What I need is to be able to <formatDateTime> the child's birthday (C1BD), and <substring> custody status (C1PHYS, C1LEGAL) and visitation (C1VISIT), for 1 to 10 sets of data.

I can initialize variables and create my conversion expressions for the top-level data, but not for the data in the loop.

I suppose could create another flow that grabs the final CSV from Sharepoint, converts the data, and then saves it to a new CSV, but I was hoping to figure out how to do this before I created the matters.csv file.

So, in essence, how do I do this:

...for the data inside the CHILDINV part of the JSON / parsed data.

Hope this helps, and thanks again.

[eliotcole]

Got ya, gimme a second! 🙂

[eliotcole]

I was going to put this in the solution post, but I have a feeling that it would've just made it stupidly long ... which is something I do, anyway. 😏

Here's the JSON data that I was working from:

{
  "body": {
    "FIRST": "Harvey",
    "MIDDLE": "Weinstein",
    "LAST": "Wallbanger",
    "MAIDEN": "Smith",
    "DOB": "2001-01-01",
    "CLASS": "CLI0LGMR - Legally Married",
    "ADDl": "123 Sesame Street",
    "LABEL2": "Apartment 7",
    "LABEL4": "R.R. #2",
    "CITY": "Trenton",
    "ZIP": "K8V 1R6",
    "Province": "Ontarlo",
    "JURISDICr": "4 years",
    "PRIVEMAIL": true,
    "EMAIL": "glenn@xasdasd.ca",
    "PREFEMAIL": true,
    "PHONE4": "343-263-4643",
    "PRIVPHONEC": true,
    "DAYPHONE": "613-966-777'",
    "PRIVPHONE": true,
    "COMPANY": "KDA Law",
    "REFERREDBY": "Jeff van de K2e-t",
    "FIRST2": "Karen",
    "MIDDLE2": "Snarf",
    "LAST2": "Wallbanger",
    "MAIDEN2": "deKaren",
    "DOB2": "1998-01-24",
    "CLASS2": "OPP-LG1 - Legally Married",
    "ADD12": "123 Complaint Bouleva~d",
    "LABEL22": "Apartment 2",
    "LABEL42": "R.R. 17",
    "City2": "Picton",
    "Province2": "Ontario",
    "ZIP2": "K7L 2E4",
    "JURISDICT2": "3 months",
    "OpposingPartysEmailAddress": "ka~e~Sw'~ine~.ccr~",
    "PHONE42": "613-966-7771",
    "DAYPHONE2": "647-265-8597",
    "COMPANY2": "Nestle Quinte West",
    "COUNTY": "Hastings",
    "COHABITATE": true,
    "DATEMARRY": "2020-01-16",
    "STATEMARRY": "Ontario",
    "DATESEP": "2021-08-02",
    "STATESEP": "Ontario",
    "FIELDl": "Belleville",
    "PRENUPTUAL": false,
    "ISFATHER": true,
    "ISMOTHER": true,
    "DOM": "2000-01-27",
    "DIVDATE": "2004-07-29",
    "CHILDREN": true,
    "ISRELATED": true,
    "CHILDINV": [
      {
        "Id": "3klAhq",
        "C1NAME": "aaaaa aaaaa",
        "C1BD": "2020-10-01",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 1
      },
      {
        "Id": "3UM0Xn",
        "C1NAME": "bbbbb bbbbb",
        "C1BD": "2020-10-02",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 2
      },
      {
        "Id": "2s3wBA",
        "C1NAME": "ccccc ccccc",
        "C1BD": "2020-10-03",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 3
      },
      {
        "Id": "lhgNwf",
        "C1NAME": "ddddd ddddd",
        "C1BD": "2020-10-04",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 4
      }
    ],
    "CHILDINV_Minimum": 0,
    "INCMECY": 35000,
    "INCMESY": 35000,
    "OWNHOME": true,
    "JTRE": 350000,
    "HOUSEOWNED": 200000,
    "HOUSETITITLEE": "TICOMM - Tenancy in Common",
    "SELLRES": true,
    "JTCASH": 15000,
    "JTINS": 15000,
    "JTPTY": 65000,
    "JTRET": 47000,
    "BIRTHEXP": 28500,
    "DOCUMENTS": [],
    "AGREFEE": true,
    "AGREFEE_Amount": 335,
    "AGREFEE_QuantitySelected": 1,
    "AGREFEE_Price": 365,
    "WILL": false,
    "SELFPROVED": false,
    "PUBLICASST": false,
    "Id": "1-3",
    "INCMECY_IncrementBy": 1,
    "INCMESY_IncrementBy": 1,
    "JTRE_IncrementBy": 1,
    "HOUSEOWNED_IncrementBy": 1,
    "JTCASH_IncrementBy": 1,
    "JTINS_IncrementBy": 1,
    "JTPTY_IncrementBy": 1,
    "JTRET_IncrementBy": 1,
    "BIRTHEXP_IncrementBy": 1
  }
}

[eliotcole]

Solution

OK, so this might seem daunting if you haven't been exposed to some of this, but I'll be honest, I only *barely* understand how XML works, and all I do know is that these bits and bobs are pure magic.

This has heavy thanks to @JohnLiu (hope that's the right userID!) whom I pilfered this a while back for some other stuff (on here, and elsewhere).

XML & XPATH

So, this uses XML to parse the JSON data, and convert it into XML data. As far as I can fathom it, the XML data can be either an array or an object, but it still finds a way to get what you need.

You'll see two expressions used more than once below, the columnName if() statement, and the columnValue xpath() statement. I'm utilising these generic statements to grab the header information from the various fields, and then equally grab the values from those fields.

The key part in both of them is item(), which in a Select or Filter action will always be referring to a generic individual item in the array that they are processing.

Field Name if() Expression (columnName)

This really just ensures that when the field name is extracted from the XML it is formatted correctly, accounting for potential issues with code in the name, or strange alphaNumeric strings.

if(startsWith(string(xpath(item(), 'name(/*)')), '_x003'), if(equals(length(string(xpath(item(), 'name(/*)'))), 7), substring(string(xpath(item(), 'name(/*)')), 5, 1), concat(substring(string(xpath(item(), 'name(/*)')), 5, 1), substring(string(xpath(item(), 'name(/*)')), 7))), string(xpath(item(), 'name(/*)')))

Field Value xpath() Expression (columnValue)

This is much simpler, and just grabs the value of a given field.

xpath(item(), 'string(//*)')

SELECT

You'll also note that in my initial flow, here, I'm using some vanilla Select actions, too. I'm doing this to separate out the logic because I know I'm not a genius(!), and to make the layout a little more obvious as to what goes where.

For the Kids' stuff, it's actually helpful that I've gone focused on this pass, when I post the more generic version, you'll be able to understand where the changes are, and why.

The Flow

As you can see, there's two branches working simultaneously, on the left you're gathering the generic data, and on the right the kids data.

Here's a deeper drill down into the flow:

OK, now on to the specifics.

General Data Side

This side took a while, as I was uhming and ah'ing about the most expedient way to put everything togther, mostly in the csvVAR at the end, but because of that, I had to keep reconsidering this part and re-testing.

The xpath() expression in that first select is:

xpath(xml(json(concat('{ "root": { "columns": ', string(variables('jsonInputVAR')?['body']), ' } }'))), '/root/columns/*')

Essentially, this is;

1. the expression concatenating data together with the concat() function,
2. then ensuring that it is presented as valid JSON with the json() function,
3. which is then converted by the xml() function,
4. to finally be parsed by the xpath() function.

All that magic is then readable by the Select action, and the Map expressions can get to work on pulling what you really want from it.

From this data, it produces data like this:

[
  {
    "columnName": "FIRST",
    "columnValue": "Harvey"
  },
  {
    "columnName": "MIDDLE",
    "columnValue": "Weinstein"
  },
  {
    "columnName": "LAST",
    "columnValue": "Wallbanger"
  },
  {
    "columnName": "MAIDEN",
    "columnValue": "Smith"
  } ...

Then the rest is quite simply a filter to remove that CHILDINV part, and then producing the arrays that I'll generate the general data headers (SelectHeaders) and data (SelectValues) from.

The Kids Side

That initial variables() call is referring directly to the CHILDINV field in the original data.

variables('jsonInputVAR')?['body']?['CHILDINV']

Here you can see the if() statement spelled out, and similar xpath() expressions that were used on the other side, initially. However the big deal on this side is the part that you had issues with, iterating the CHILDINV data.

Looking at that first SelectKidsData action, you'll see field/column names defined and values also. Every single piece of dynamic data in that set is simply done with:

item()?['FIELD_NAME']

Where item()?['FIELD_NAME'] is one of the names presented in the CHILDINV array, like  item()?['Gender'].

So, you can see, your ItemNumber field is SUPER useful here, by looking at example data from what it spouts out:

[
  {
    "C1ID": "3klAhq",
    "C1NAME": "aaaaa aaaaa",
    "C1BD": "2020-10-01",
    "C1SEX": "Male",
    "C1LEGAL": "CLIENT - You",
    "C1PHYS": "CLIENT - You",
    "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
    "C1NOTES": "Lots of comments because I like to counent on stuff.",
    "C1NUM": 1
  },
  {
    "C2ID": "3UM0Xn",
    "C2NAME": "bbbbb bbbbb",
    "C2BD": "2020-10-02",
    "C2SEX": "Male",
    "C2LEGAL": "CLIENT - You",
    "C2PHYS": "CLIENT - You",
    "C2VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
    "C2NOTES": "Lots of comments because I like to counent on stuff.",
    "C2NUM": 2
  }...

CSV Creation

Finally, this feels like the most straightforward part, the string variable just pulls all the data together into one place, and formats it for CSV with each CSV entry encased in speech marks just in case there are commas in the data that's sent through.

HEADERS
join(split(first(split(last(split(string(body('SelectHeaders')), '[{"columnName":"')), '"}]')), '"},{"columnName":"'), '","')

join(split(first(split(last(split(string(body('SelectKidsHeaders')), '[{"columnName":"')), '"}]')), '"},{"columnName":"'), '","')

DATA
join(split(first(split(last(split(string(body('SelectValues')), '[{"columnValue":"')), '"}]')), '"},{"columnValue":"'), '","')

join(split(first(split(last(split(string(body('SelectKidsValues')), '[{"columnValue":"')), '"}]')), '"},{"columnValue":"'), '","')

Each of the four parts in the csvVAR action are basically the same, just referring to different sources, you could do it all at separate points if you'd rather, and pool it here. Essentially each expression works thusly:

1. It takes a given array of data, and presents it as a string().
2. It uses last() and split() on that text string (to make an array 🤪);
   1. splitting it on the initial characters ([{"columnValue":" for the general data),
   2. then taking the last item in the two part array it makes.
3. Then it splits it again, this time on the final characters ("}]) in the string, here you want the first() of the two produced array items.
4. The final split is on the data that is left, and you want to split on the JSON data that separating the meat that we want, "},{"columnValue":".
5. Once all of that has been done, you're left with an array of all the items that you want, so you just need to join() them together with "," so that they are readable in CSV format.

The rest of the csvVAR action is just framing that data with a couple more speech marks and commas!

That should be it!

EDIT - I'm going to leave optimising the Kids' side for now as it's both late, and I need to understand how to pull a value out of that XML!

[Paulie78]

You can do it easily with a select action. Take a look at this screenshot:

https://ibb.co/F7MJcdJ

To make it simple for you to recreate, I copied my actions into a Scope. If you would like a working demonstration at your end, simply do the following:

• Copy the code below.
• Create a new flow.
• Add an action.
• Go to "My clipboard"
• Press CTRL-V
• Click the scope action that appears.

Here is the code for you to copy:

{"id":"936a697a-e46e-41e6-b201-2d05-01ede363","brandColor":"#8C3900","connectionReferences":{},"connectorDisplayName":"Control","icon":"data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMzIiIGhlaWdodD0iMzIiIHZlcnNpb249IjEuMSIgdmlld0JveD0iMCAwIDMyIDMyIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPg0KIDxwYXRoIGQ9Im0wIDBoMzJ2MzJoLTMyeiIgZmlsbD0iIzhDMzkwMCIvPg0KIDxwYXRoIGQ9Im04IDEwaDE2djEyaC0xNnptMTUgMTF2LTEwaC0xNHYxMHptLTItOHY2aC0xMHYtNnptLTEgNXYtNGgtOHY0eiIgZmlsbD0iI2ZmZiIvPg0KPC9zdmc+DQo=","isTrigger":false,"operationName":"Scope","operationDefinition":{"type":"Scope","actions":{"Compose":{"type":"Compose","inputs":{"body":{"FIRST":"Harvey","MIDDLE":"Weinstein","LAST":"Wallbanger","MAIDEN":"Smith","DOB":"2001-01-01","CLASS":"CLI0LGMR - Legally Married","ADDl":"123 Sesame Street","LABEL2":"Apartment 7","LABEL4":"R.R. #2","CITY":"Trenton","ZIP":"K8V 1R6","Province":"Ontarlo","JURISDICr":"4 years","PRIVEMAIL":true,"EMAIL":"glenn@xasdasd.ca","PREFEMAIL":true,"PHONE4":"343-263-4643","PRIVPHONEC":true,"DAYPHONE":"613-966-777'","PRIVPHONE":true,"COMPANY":"KDA Law","REFERREDBY":"Jeff van de K2e-t","FIRST2":"Karen","MIDDLE2":"Snarf","LAST2":"Wallbanger","MAIDEN2":"deKaren","DOB2":"1998-01-24","CLASS2":"OPP-LG1 - Legally Married","ADD12":"123 Complaint Bouleva~d","LABEL22":"Apartment 2","LABEL42":"R.R. 17","City2":"Picton","Province2":"Ontario","ZIP2":"K7L 2E4","JURISDICT2":"3 months","OpposingPartysEmailAddress":"ka~e~Sw'~ine~.ccr~","PHONE42":"613-966-7771","DAYPHONE2":"647-265-8597","COMPANY2":"Nestle Quinte West","COUNTY":"Hastings","COHABITATE":true,"DATEMARRY":"2020-01-16","STATEMARRY":"Ontario","DATESEP":"2021-08-02","STATESEP":"Ontario","FIELDl":"Belleville","PRENUPTUAL":false,"ISFATHER":true,"ISMOTHER":true,"DOM":"2000-01-27","DIVDATE":"2004-07-29","CHILDREN":true,"ISRELATED":true,"CHILDINV":[{"Id":"3klAhq","C1NAME":"aaaaa aaaaa","C1BD":"2020-10-01","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":1},{"Id":"3UM0Xn","C1NAME":"bbbbb bbbbb","C1BD":"2020-10-02","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":2},{"Id":"2s3wBA","C1NAME":"ccccc ccccc","C1BD":"2020-10-03","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":3},{"Id":"lhgNwf","C1NAME":"ddddd ddddd","C1BD":"2020-10-04","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":4}],"CHILDINV_Minimum":0,"INCMECY":35000,"INCMESY":35000,"OWNHOME":true,"JTRE":350000,"HOUSEOWNED":200000,"HOUSETITITLEE":"TICOMM - Tenancy in Common","SELLRES":true,"JTCASH":15000,"JTINS":15000,"JTPTY":65000,"JTRET":47000,"BIRTHEXP":28500,"DOCUMENTS":[],"AGREFEE":true,"AGREFEE_Amount":335,"AGREFEE_QuantitySelected":1,"AGREFEE_Price":365,"WILL":false,"SELFPROVED":false,"PUBLICASST":false,"Id":"1-3","INCMECY_IncrementBy":1,"INCMESY_IncrementBy":1,"JTRE_IncrementBy":1,"HOUSEOWNED_IncrementBy":1,"JTCASH_IncrementBy":1,"JTINS_IncrementBy":1,"JTPTY_IncrementBy":1,"JTRET_IncrementBy":1,"BIRTHEXP_IncrementBy":1}},"runAfter":{}},"Select":{"type":"Select","inputs":{"from":"@outputs('Compose')['BODY']['CHILDINV']","select":{"Id":"@item()['Id']","C1NAME":"@item()['C1NAME']","C1BD":"@formatDateTime(item()['C1BD'], 'dd-MM-yyyy')","Gender":"@item()['Gender']","C1LEGAL":"@substring(item()['C1LEGAL'],0, 6)","C1PHYS":"@substring(item()['C1PHYS'],0, 6)","C1VISIT":"@substring(item()['C1VISIT'],0, 8)","FIELD8":"@item()['FIELD8']","ItemNumber":"@item()['ItemNumber']"}},"runAfter":{"Compose":["Succeeded"]},"description":"outputs('Compose')['BODY']['CHILDINV']"},"Create_CSV_table":{"type":"Table","inputs":{"from":"@body('Select')","format":"CSV"},"runAfter":{"Select":["Succeeded"]},"description":"body('Select')"}},"runAfter":{}}}

See how you get on and let me know if you need further guidance.

Blog: tachytelic.net

YouTube: https://www.youtube.com/c/PaulieM/videos

If I answered your question, please accept it as a solution 😘

[eliotcole]

He needs to iterate the children column names, though, @Paulie78 ... but my solution can be slimmed down a bit still (for example I'd put a removeProperty() in the xpath() on the main Select action and remove the filter action) but for now it shows the logic. I was wondering if the range trick would help there, and it would, but only if he hadn't already defined the child number.

Anyway, yeah, on the iterative headings: So if there are four children (like the example data I put above) then there will be four additional sets of nine columns in the CSV. Nine columns for each child.

"C1ID","C1NAME","C1BD","C1SEX","C1LEGAL","C1PHYS","C1VISIT","C1NOTES","C1NUM","C2ID","C2NAME","C2BD","C2SEX","C2LEGAL","C2PHYS","C2VISIT","C2NOTES","C2NUM","C3ID","C3NAME","C3BD","C3SEX","C3LEGAL","C3PHYS","C3VISIT","C3NOTES","C3NUM","C4ID","C4NAME","C4BD","C4SEX","C4LEGAL","C4PHYS","C4VISIT","C4NOTES","C4NUM"

[ArchieGoodwin]

Wow...Lots to take in there, but I'm up for the challenge! I will take a look at it in more detail later today, and try to get it working tonight - I'll keep you posted!

[ArchieGoodwin]

That's the real trick...It doesn't really matter what the data is called in the schema or the flow, all that matters is that it comes out in the CSV with the right header row above it, and the right data field name.

[eliotcole]

Cheers, @ArchieGoodwin, I've definitely got it working with the sample data above, I'm just working on slimming it down some.

If you want to make it less cumbersome in your head, just look at the parts where I say what to do, rather than the explanations. If, like me, you're a 'learn by doing' person (kineathetic learner!) then by implementing it you'll probably get to the understanding (or maybe fiddling around with a copy).

With regards to the slimming ... Like I say, removing the property negates the need for the filter action, which is good, and I think I'm on to removing the need for the final action and changing that to a generic CSV, just playing with that child data, and some of Paulie's tricks from elsewhere!

[Paulie78]

I slightly misunderstood, but you can still do it easily in one action:

I haven't done every field, but enough to get you started off. Output produced from this is:

FIRST,MIDDLE,LAST,CHILD1_ID,CHILD1_NAME,CHILD2_ID,CHILD2_NAME
Harvey,Weinstein,Wallbanger,3klAhq,aaaaa aaaaa,3UM0Xn,bbbbb bbbbb

This was my input JSON:

{
    "FIRST": "Harvey",
    "MIDDLE": "Weinstein",
    "LAST": "Wallbanger",
    "MAIDEN": "Smith",
    "DOB": "2001-01-01",
    "CLASS": "CLI0LGMR - Legally Married",
    "ADDl": "123 Sesame Street",
    "LABEL2": "Apartment 7",
    "LABEL4": "R.R. #2",
    "CITY": "Trenton",
    "ZIP": "K8V 1R6",
    "Province": "Ontarlo",
    "JURISDICr": "4 years",
    "PRIVEMAIL": true,
    "EMAIL": "glenn@xasdasd.ca",
    "PREFEMAIL": true,
    "PHONE4": "343-263-4643",
    "PRIVPHONEC": true,
    "DAYPHONE": "613-966-777'",
    "PRIVPHONE": true,
    "COMPANY": "KDA Law",
    "REFERREDBY": "Jeff van de K2e-t",
    "FIRST2": "Karen",
    "MIDDLE2": "Snarf",
    "LAST2": "Wallbanger",
    "MAIDEN2": "deKaren",
    "DOB2": "1998-01-24",
    "CLASS2": "OPP-LG1 - Legally Married",
    "ADD12": "123 Complaint Bouleva~d",
    "LABEL22": "Apartment 2",
    "LABEL42": "R.R. 17",
    "City2": "Picton",
    "Province2": "Ontario",
    "ZIP2": "K7L 2E4",
    "JURISDICT2": "3 months",
    "OpposingPartysEmailAddress": "ka~e~Sw'~ine~.ccr~",
    "PHONE42": "613-966-7771",
    "DAYPHONE2": "647-265-8597",
    "COMPANY2": "Nestle Quinte West",
    "COUNTY": "Hastings",
    "COHABITATE": true,
    "DATEMARRY": "2020-01-16",
    "STATEMARRY": "Ontario",
    "DATESEP": "2021-08-02",
    "STATESEP": "Ontario",
    "FIELDl": "Belleville",
    "PRENUPTUAL": false,
    "ISFATHER": true,
    "ISMOTHER": true,
    "DOM": "2000-01-27",
    "DIVDATE": "2004-07-29",
    "CHILDREN": true,
    "ISRELATED": true,
    "CHILDINV": [
      {
        "Id": "3klAhq",
        "C1NAME": "aaaaa aaaaa",
        "C1BD": "2020-10-01",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 1
      },
      {
        "Id": "3UM0Xn",
        "C1NAME": "bbbbb bbbbb",
        "C1BD": "2020-10-02",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 2
      },
      {
        "Id": "2s3wBA",
        "C1NAME": "ccccc ccccc",
        "C1BD": "2020-10-03",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 3
      },
      {
        "Id": "lhgNwf",
        "C1NAME": "ddddd ddddd",
        "C1BD": "2020-10-04",
        "Gender": "Male",
        "C1LEGAL": "CLIENT - You",
        "C1PHYS": "CLIENT - You",
        "C1VISIT": "CL-DISCR - Visitation at t~e Discretion of Client",
        "FIELD8": "Lots of comments because I like to counent on stuff.",
        "ItemNumber": 4
      }
    ],
    "CHILDINV_Minimum": 0,
    "INCMECY": 35000,
    "INCMESY": 35000,
    "OWNHOME": true,
    "JTRE": 350000,
    "HOUSEOWNED": 200000,
    "HOUSETITITLEE": "TICOMM - Tenancy in Common",
    "SELLRES": true,
    "JTCASH": 15000,
    "JTINS": 15000,
    "JTPTY": 65000,
    "JTRET": 47000,
    "BIRTHEXP": 28500,
    "DOCUMENTS": [],
    "AGREFEE": true,
    "AGREFEE_Amount": 335,
    "AGREFEE_QuantitySelected": 1,
    "AGREFEE_Price": 365,
    "WILL": false,
    "SELFPROVED": false,
    "PUBLICASST": false,
    "Id": "1-3",
    "INCMECY_IncrementBy": 1,
    "INCMESY_IncrementBy": 1,
    "JTRE_IncrementBy": 1,
    "HOUSEOWNED_IncrementBy": 1,
    "JTCASH_IncrementBy": 1,
    "JTINS_IncrementBy": 1,
    "JTPTY_IncrementBy": 1,
    "JTRET_IncrementBy": 1,
    "BIRTHEXP_IncrementBy": 1
  }

This is how you do it:

Create a select action. For the from, just literally type [0]

For the Header records, just access them directly, examples:

outputs('Compose')['FIRST']
outputs('Compose')['MIDDLE']
outputs('Compose')['LAST']

For the array elements, as you know there will never be more than ten, you can just build columns for each one. Examples:

outputs('Compose')['CHILDINV']?[0]['Id']
outputs('Compose')['CHILDINV']?[0]['C1NAME']
outputs('Compose')['CHILDINV']?[1]['Id']
outputs('Compose')['CHILDINV']?[1]['C1NAME']

The above give the ID and name for child one and two, you would just continue to populate the select and then you'd have the whole thing on one line of a CSV.

As before, I have copied it into a scope for you so you can see how it works:

{"id":"8a8a92db-c996-4c42-bd8f-4367-c314d244","brandColor":"#8C3900","connectionReferences":{},"connectorDisplayName":"Control","icon":"data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMzIiIGhlaWdodD0iMzIiIHZlcnNpb249IjEuMSIgdmlld0JveD0iMCAwIDMyIDMyIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPg0KIDxwYXRoIGQ9Im0wIDBoMzJ2MzJoLTMyeiIgZmlsbD0iIzhDMzkwMCIvPg0KIDxwYXRoIGQ9Im04IDEwaDE2djEyaC0xNnptMTUgMTF2LTEwaC0xNHYxMHptLTItOHY2aC0xMHYtNnptLTEgNXYtNGgtOHY0eiIgZmlsbD0iI2ZmZiIvPg0KPC9zdmc+DQo=","isTrigger":false,"operationName":"Scope","operationDefinition":{"type":"Scope","actions":{"Compose":{"type":"Compose","inputs":{"FIRST":"Harvey","MIDDLE":"Weinstein","LAST":"Wallbanger","MAIDEN":"Smith","DOB":"2001-01-01","CLASS":"CLI0LGMR - Legally Married","ADDl":"123 Sesame Street","LABEL2":"Apartment 7","LABEL4":"R.R. #2","CITY":"Trenton","ZIP":"K8V 1R6","Province":"Ontarlo","JURISDICr":"4 years","PRIVEMAIL":true,"EMAIL":"glenn@xasdasd.ca","PREFEMAIL":true,"PHONE4":"343-263-4643","PRIVPHONEC":true,"DAYPHONE":"613-966-777'","PRIVPHONE":true,"COMPANY":"KDA Law","REFERREDBY":"Jeff van de K2e-t","FIRST2":"Karen","MIDDLE2":"Snarf","LAST2":"Wallbanger","MAIDEN2":"deKaren","DOB2":"1998-01-24","CLASS2":"OPP-LG1 - Legally Married","ADD12":"123 Complaint Bouleva~d","LABEL22":"Apartment 2","LABEL42":"R.R. 17","City2":"Picton","Province2":"Ontario","ZIP2":"K7L 2E4","JURISDICT2":"3 months","OpposingPartysEmailAddress":"ka~e~Sw'~ine~.ccr~","PHONE42":"613-966-7771","DAYPHONE2":"647-265-8597","COMPANY2":"Nestle Quinte West","COUNTY":"Hastings","COHABITATE":true,"DATEMARRY":"2020-01-16","STATEMARRY":"Ontario","DATESEP":"2021-08-02","STATESEP":"Ontario","FIELDl":"Belleville","PRENUPTUAL":false,"ISFATHER":true,"ISMOTHER":true,"DOM":"2000-01-27","DIVDATE":"2004-07-29","CHILDREN":true,"ISRELATED":true,"CHILDINV":[{"Id":"3klAhq","C1NAME":"aaaaa aaaaa","C1BD":"2020-10-01","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":1},{"Id":"3UM0Xn","C1NAME":"bbbbb bbbbb","C1BD":"2020-10-02","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":2},{"Id":"2s3wBA","C1NAME":"ccccc ccccc","C1BD":"2020-10-03","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":3},{"Id":"lhgNwf","C1NAME":"ddddd ddddd","C1BD":"2020-10-04","Gender":"Male","C1LEGAL":"CLIENT - You","C1PHYS":"CLIENT - You","C1VISIT":"CL-DISCR - Visitation at t~e Discretion of Client","FIELD8":"Lots of comments because I like to counent on stuff.","ItemNumber":4}],"CHILDINV_Minimum":0,"INCMECY":35000,"INCMESY":35000,"OWNHOME":true,"JTRE":350000,"HOUSEOWNED":200000,"HOUSETITITLEE":"TICOMM - Tenancy in Common","SELLRES":true,"JTCASH":15000,"JTINS":15000,"JTPTY":65000,"JTRET":47000,"BIRTHEXP":28500,"DOCUMENTS":[],"AGREFEE":true,"AGREFEE_Amount":335,"AGREFEE_QuantitySelected":1,"AGREFEE_Price":365,"WILL":false,"SELFPROVED":false,"PUBLICASST":false,"Id":"1-3","INCMECY_IncrementBy":1,"INCMESY_IncrementBy":1,"JTRE_IncrementBy":1,"HOUSEOWNED_IncrementBy":1,"JTCASH_IncrementBy":1,"JTINS_IncrementBy":1,"JTPTY_IncrementBy":1,"JTRET_IncrementBy":1,"BIRTHEXP_IncrementBy":1},"runAfter":{}},"Hope__you":{"type":"Compose","inputs":"Enjoy :D","runAfter":{"Create_CSV_table":["Succeeded"]}},"Select":{"type":"Select","inputs":{"from":[0],"select":{"FIRST":"@outputs('Compose')['FIRST']","MIDDLE":"@outputs('Compose')['MIDDLE']","LAST":"@outputs('Compose')['LAST']","CHILD1_ID":"@outputs('Compose')['CHILDINV']?[0]['Id']","CHILD1_NAME":"@outputs('Compose')['CHILDINV']?[0]['C1NAME']","CHILD2_ID":"@outputs('Compose')['CHILDINV']?[1]['Id']","CHILD2_NAME":"@outputs('Compose')['CHILDINV']?[1]['C1NAME']"}},"runAfter":{"Compose":["Succeeded"]}},"Create_CSV_table":{"type":"Table","inputs":{"from":"@body('Select')","format":"CSV"},"runAfter":{"Select":["Succeeded"]}}},"runAfter":{}}}

[eliotcole]

Don't you still have to manually create those keys in the select, mate? When there's less than ten that's going to create pointless columns, no?

[Paulie78]

Yeah, but it is a one off operation. You'd want the column names to be static, so that your import was always consistent. You would still want a column for Child10_Name, even if there was no child 10. It's the whole operation wrapped in a single API action, so it will finish instantly and it can't get any more efficient than that. 

What's the issue with manually creating the keys? Did I miss something?

[ArchieGoodwin]

So, this is closer - but I took a look at the scope, ran it, and the output appeared to be the same as your earlier answer above.

I need the output to actually be two rows, with the first row being the "header" row, and the second one being the data row.

It makes sense to me, how to compose the header row and then pull the data (not sure how you separate the two rows)?