Turning on Paging option for HTTP rest calls to flow not working

[Mick282]

Hi all,

I've got a issue regarding the paging function Microsoft Flow has included in their HTTP action/trigger using a continuation token. I've setup a simple get request to the EventBrite REST API to retrieve a complete list of event attendees for a particular event periodically. I've set the HTTP URL to the below:

You can see in the JSON value returned that it includes a continuation token which can be used to navigate to the next page.

I then turned on paging.

So I then turned on the paging option on the HTTP action but when I tested the flow it's giving me the following error:

I just can't figure out how to get this working on the one flow. I checked the EventBrite API documentation and they noted that to retrieve all attendees for an event. EventBrite returns an array of Attendee objects as a paginated response.

How do I get this working with the out of the box paging option available in Flow?

https://www.eventbrite.com/platform/docs/attendees

[rjhale]

Did you ever get this worked out?  I'm running into the same limitations, and I could really use some help finding a solution. 

Thanks. 

[Anonymous]

I'm getting the same error.....I've got a list of 8,000+ URL's in an Excel file, trying to download each and create a blob in Azure blob storage for each resulting text file. When I ignore pagination and just run it as is, I get the expected 256 results.......when I enable pagination and set the threshold to 8,200, I get the same "invalid paginated response" message. Here is my flow:

[LeeJBS]

I also have the same issue. Did anyone get around it?

[Anonymous]

 Hi All, I have this possible solution:

Initilise a Page, PageCount, dataarray and ContinueToken variable, you can call these whatever you choose. Set the Page to 1 and leave the other two blank

1. run the first call to get the number of pages and then parse to JSON, then store the Page_count value into a variable.

2. Create a DO until loop and set to [Page] is greather than [PageCount]

3. Do your HTTP call again and where you would put &continuation=[continuation value] put in the variable for Continue Token, so it should look like: https://www.eventbriteapi.com/v3/organizations/456876091330/events/?token=[token]@{variables('ContinueToken'...

as the variable is empty to start it will get the first page fine. It will grab a continue code as necessary, if there is only 1 page, it will only iterate once anyway.
Then parse again to JSON, and append to the data array variable the fields as you like. You may need to use select and an apply to each.

4. inside the do until and after you have got the pages data, you are going to set the ContinueToken to the current continuation field from the Parse JSON, but you also need to include the property string, so your set variable looks like:

&continuation=@{body('Parse_JSON_2')?['pagination']?['continuation']}. Then increment the Page variable by one. 

This will run a loop that gets the data on the page we have, gets the continuation uri bit and code and then sets the page number to the next one.

I can't add images to this post, but if you need, I can send some screen shots out.

[Venkat_Aus]

@Anonymous Hi, Can you please post the screenshots of the workaround that you mentioned? Currently, I have built a flow which which only first page of the eventbrite attendee result. It would be helpful if you can post the screenshot.

Thanks

Venkat

[Anonymous]

Hi @Venkat_Aus 
Sure, so you start by getting the first page:

Then you do a do until, counting the pages as you go. The first thing we do is then get the page and parse it

Then in my case, I was checking the event date to make sure it was after today and iterate through each event on the page

And adding the row to an array

Then finally I get the new continuation token and "turn" the page by incrementing the page variable, so page 1 becomes page 2 etc.

At the end of both loops, I will have an array with all the event IDs and their start times. You can get more fields than I did.

[Venkat_Aus]

Thanks @Anonymous  for the screenshots. I cannot set variable for the continuation token. I can see that token number in the output of the https though. Can you please paste the parse json that you used?

[Anonymous]

Hello! Please see below Parse JSON schema that I used 

{
    "type": "object",
    "properties": {
        "pagination": {
            "type": "object",
            "properties": {
                "object_count": {
                    "type": "integer"
                },
                "page_number": {
                    "type": "integer"
                },
                "page_size": {
                    "type": "integer"
                },
                "page_count": {
                    "type": "integer"
                },
                "continuation": {
                    "type": "string"
                },
                "has_more_items": {
                    "type": "boolean"
                }
            }
        },
        "events": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "name": {
                        "type": "object",
                        "properties": {
                            "text": {
                                "type": "string"
                            },
                            "html": {
                                "type": "string"
                            }
                        }
                    },
                    "id": {
                        "type": "string"
                    },
                    "url": {
                        "type": "string"
                    },
                    "start": {
                        "type": "object",
                        "properties": {
                            "timezone": {
                                "type": "string"
                            },
                            "local": {
                                "type": "string"
                            },
                            "utc": {
                                "type": "string"
                            }
                        }
                    },
                    "organization_id": {
                        "type": "string"
                    },
                    "created": {
                        "type": "string"
                    },
                    "changed": {
                        "type": "string"
                    },
                    "resource_uri": {
                        "type": "string"
                    },
                    "capacity": {
                        "type": "integer"
                    },
                    "status": {
                        "type": "string"
                    }
                },
                "required": [
                    "name",
                    "id",
                    "url",
                    "start",
                    "organization_id",
                    "created",
                    "changed",
                    "resource_uri"
                ]
            }
        }
    }
}

you should see the continuation field in the pagination section.

[Venkat_Aus]

Thanks @Anonymous . 

I used the following approach to make it work. Got the page count, then looped http based on page number and parsed the data. Your approach on this one helped me to achieve the desired result. Thanks 🙂

I used https://www.eventbriteapi.com/v3/events/XXXXXXXXX/attendees/?token=XXXXXXXXXXX&page=0

[Automate123]

Hey @Mick282 @Venkat_Aus @Anonymous @LeeJBS 

Could somebody help me understand how I can get this communication token in the response of the HTTP connector?

Also what Token are you adding into the URI? Is it the Authentication Token?

I have been trying to research this but keep coming up short.

Any help would be much appreciated.

Thanks

[Venkat_Aus]

@Automate123 

1. Log in to your eventbrite account.

2.  Then access https://www.eventbrite.com.au/platform/

3. You will get a private token for your account. This token is specific to your eventbrite account. 

Replace the Entertoken using the token that you obtained in the previous step.

https://www.eventbriteapi.com/v3/events/EntereventID/attendees/?token=Entertoken

Then replace the EntereventID with the eventbrite event id. 

I hope this helps.

[Automate123]

Hi @Venkat_Aus 

Thank you for the response!

Ok it makes a lot more sense now. So I assume this token is specific to event bride.

I am trying to get more than 100 results using the JIRA REST API, and through reading through this thread thought maybe it would be possible through pagination.

Do you have any ideas on how I can achieve this? or how to get a Token specific to Jira?

Thank you

[Venkat_Aus]

@Automate123  There are a few ways to pull multiple records. My research initially suggested continuation token. I tried and failed because I might not be using it correctly. So I took an alternative approach. When you look at the attendee information on the API end and when there are more record numbers, then they will be displayed on the next page.  E.g. "https://www.eventbriteapi.com/v3/events/XXXXXXXXX/attendees/?token=XXXXXXXXXXX&page=2"

So I initially got the number of pages and performed a Do until on the http until the page numbers matches the maximum. I hope this helps. 

Cheers

Venkat

[Automate123]

Hey @Venkat_Aus 

Thank you very much for the reply. It defiantly helps in understanding the logic used to get all issues. The flow I created is  utilizing the Jira REST API and so my URI is something like:

https://<My_Org>.atlassian.net/rest/api/3/search?jql=<MYJQL>&maxResults=500

However I notice in your URI used, you have a token embedded. Would the logic you use still work for my flow do you think?

In terms of the JSON response from the call I get the following:

{
"expand": "schema,names",
"startAt": 0,
"maxResults": 100,
"total": 142,
"issues": [{ 

The Issues are then filtered using Select, placed in a CSV file and placed onto a mailing list. However I am only ever getting back 100 Jira Issues.

Could I ask you to post your full flow with each connector configuration, so I could try and apply the logic you have used to my flow?

It really would be appreciated. I am very new to using Power Automate and Iv been trying to figure this out for weeks now. I haven't been able to find much on it, apart from this thread.

For reference the flow that I am trying to paginate can be seen below. It is extracting 100 Jira issue and successfully emailing these issues, however I need to extract all 142 issues.

Cheers