Get Power BI Dataflow csv from Azure Blob Storage

[JeffN]

Greetings experts!  I'm trying to retrieve Power BI Dataflow connected to Gen2 Blob storage location.  I can retrieve the list of blobs (Lists  blob (V2)) that represent the entities from each Dataflow refresh from the 'model.json.snapshots/ folder.  Now I need to select the name of the blob with the most recent data in it, e.g. the third blob in this set:  

model.json@snapshot=2021-12-29T13:43:11.8221557Z
model.json@snapshot=2021-12-29T15:42:47.6820754Z
model.json@snapshot=2022-01-02T13:57:44.4572616Z 

Then I plan to set that file name to a variable and use as a parameter in a Get Blob Content action. 

I should also note that the output from the Lists Blob V2 action is a JSON object, which I then pass to a Create CSV table that generates the lines pasted above.  I need to get the JSON member for the most recent datetime (in red below), so maybe it is not necessary to make the csv first.  How can I extract/query the member with the recent datetime?  Here is the JSON output from the List blobs action:  

[

  {

    "Id": "xxxxx==",

    "Name": "model.json@snapshot=2021-12-29T13:43:11.8221557Z",

    "DisplayName": "model.json@snapshot=2021-12-29T13:43:11.8221557Z",

    "Path": "/powerbi/Azure test/Excel Datasource via Gateway/model.json.snapshots/model.json@snapshot=2021-12-29T13:43:11.8221557Z",

    "LastModified": "2021-12-29T13:43:11Z",

    "Size": 1267,

    "MediaType": "application/octet-stream",

    "IsFolder": false,

    "ETag": "\"xxxxx\"",

    "FileLocator": "xxxxx=="

  },

  {

    "Id": "xxxxx==",

    "Name": "model.json@snapshot=2021-12-29T15:42:47.6820754Z",

    "DisplayName": "model.json@snapshot=2021-12-29T15:42:47.6820754Z",

    "Path": "/powerbi/Azure test/Excel Datasource via Gateway/model.json.snapshots/model.json@snapshot=2021-12-29T15:42:47.6820754Z",

    "LastModified": "2021-12-29T15:42:47Z",

    "Size": 1587,

    "MediaType": "application/octet-stream",

    "IsFolder": false,

    "ETag": "\"xxxxx\"",

    "FileLocator": "xxxxx=="

  },

  {

    "Id": "xxxxx==",

    "Name": "model.json@snapshot=2022-01-02T13:57:44.4572616Z",

    "DisplayName": "model.json@snapshot=2022-01-02T13:57:44.4572616Z",

    "Path": "/powerbi/Azure test/Excel Datasource via Gateway/model.json.snapshots/model.json@snapshot=2022-01-02T13:57:44.4572616Z",

    "LastModified": "2022-01-02T13:57:44Z",

    "Size": 1587,

    "MediaType": "application/octet-stream",

    "IsFolder": false,

    "ETag": "\"xxxxx\"",

    "FileLocator": "xxxxx=="

  }

]

[JeffN]

model.json@snapshot=2021-12-29T13:43:11.8221557Z
model.json@snapshot=2021-12-29T15:42:47.6820754Z
model.json@snapshot=2022-01-02T13:57:44.4572616Z 

Then I plan to set that file name to a variable and use as a parameter in a Get Blob Content action. 

I should also note that the output from the Lists Blob V2 action is a JSON object, which I then pass to a Create CSV table that generates the lines pasted above.  I need to get the JSON member for the most recent datetime (in red below), so maybe it is not necessary to make the csv first.  How can I extract/query the member with the recent datetime?  Here is the JSON output from the List blobs action:  

[

  {

    "Id": "xxxxx==",

    "Name": "model.json@snapshot=2021-12-29T13:43:11.8221557Z",

    "DisplayName": "model.json@snapshot=2021-12-29T13:43:11.8221557Z",

    "Path": "/powerbi/Azure test/Excel Datasource via Gateway/model.json.snapshots/model.json@snapshot=2021-12-29T13:43:11.8221557Z",

    "LastModified": "2021-12-29T13:43:11Z",

    "Size": 1267,

    "MediaType": "application/octet-stream",

    "IsFolder": false,

    "ETag": "\"xxxxx\"",

    "FileLocator": "xxxxx=="

  },

  {

    "Id": "xxxxx==",

    "Name": "model.json@snapshot=2021-12-29T15:42:47.6820754Z",

    "DisplayName": "model.json@snapshot=2021-12-29T15:42:47.6820754Z",

    "Path": "/powerbi/Azure test/Excel Datasource via Gateway/model.json.snapshots/model.json@snapshot=2021-12-29T15:42:47.6820754Z",

    "LastModified": "2021-12-29T15:42:47Z",

    "Size": 1587,

    "MediaType": "application/octet-stream",

    "IsFolder": false,

    "ETag": "\"xxxxx\"",

    "FileLocator": "xxxxx=="

  },

  {

    "Id": "xxxxx==",

    "Name": "model.json@snapshot=2022-01-02T13:57:44.4572616Z",

    "DisplayName": "model.json@snapshot=2022-01-02T13:57:44.4572616Z",

    "Path": "/powerbi/Azure test/Excel Datasource via Gateway/model.json.snapshots/model.json@snapshot=2022-01-02T13:57:44.4572616Z",

    "LastModified": "2022-01-02T13:57:44Z",

    "Size": 1587,

    "MediaType": "application/octet-stream",

    "IsFolder": false,

    "ETag": "\"xxxxx\"",

    "FileLocator": "xxxxx=="

  }

]

[jinivthakkar]

@JeffN in compose I have added the Json array you gave

Instead of compose just pass the action output you are using in earlier step

outputs('Compose')[2]

--------------------------------------------------------------------------------

If this post helps answer your question, please click on “Accept as Solution” to help other members find it more quickly. If you thought this post was helpful, please give it a Thumbs Up.

[JeffN]

That's good , using the [x] record reference.  The problem is that when I refreshed the dataflow again, the set now has four members:  

model.json@snapshot=2021-12-29T13:43:11.8221557Z
model.json@snapshot=2021-12-29T15:42:47.6820754Z
model.json@snapshot=2022-01-02T13:57:44.4572616Z
model.json@snapshot=2022-01-05T15:46:51.3687507Z

So now I need to refer to [3].

My problem is how can I 'select' the most recent (or last in a sort.asc, first in a sort.desc) item and assign that text (or object) to a variable?  Perhaps there is a function that can count the items and can be used in place of [x]?  Or a way to query/filter the list?

[v-xiaochen-msft]

Hi @JeffN ,

I provided a sample in your another post.

https://powerusers.microsoft.com/t5/Building-Flows/Get-Power-BI-Dataflow-csv-from-Azure-Blob-Storage... 

Best Regards,

Wearsky

[jinivthakkar · Accepted solution]

@JeffN please see below

In compose 4, I check the length of compose which has your json array(your results) and I use subtract method to subtract it by 1, which gives me the last index

and then in compose 2, I pass the value from compose 4 which was obtained after doing subtract operation

--------------------------------------------------------------------------------

If this post helps answer your question, please click on “Accept as Solution” to help other members find it more quickly. If you thought this post was helpful, please give it a Thumbs Up.