cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
rconety
New Member

Extract data from web page - paging

Good Afternoon,

 

I am trying to extract data from a table on a web page with pagination.  If I ignore pagination and just pull data from the first page, it works fine.  When I try and turn on pagination and use the live web helper, it does generate the css selector for next page of search results but it does not work.  I'm wondering if this is an issue with the paging being a dropdown?


Advanced Settings.PNG

Base CSS Selector
html > body > form > table > tbody > tr > td:eq(2) > iframe > html > body > form > table:eq(1) > tbody > tr:eq(1) > td > div > div:eq(1) > div > div > table > tbody > tr:eq(1) > td > table > tbody > tr:eq(1) > td > div > div > table > tbody > tr

 

CSS selector
td:eq(1) > a > span

CSS selector for next page of search results
iframe[id="contentIFrame"] select[id="pgrUsersTop_ddlPageList"], html > body > form > table > tbody > tr > td:eq(2) > iframe > html > body > form > table:eq(1) > tbody > tr:eq(1) > td > div > div:eq(1) > div > div > table > tbody > tr:eq(1) > td > table > tbody > tr:eq(0) > td > table > tbody > tr > td:eq(1) > select

 

Here is a screen shot of the snipping tools
internet scripting tools.PNG

 

Here is a screen shot of part of my flow

Extract data from web page.PNG

 

And here is the error
error.PNG

Can you please assist?

 

 

 

12 REPLIES 12
UK_Mike
Post Prodigy
Post Prodigy

Me personally, I think the pager is broken !

Try this, works for me every time.

Extract just a single page, save it.

Edit the parameters again but now select all pages, it will give an error "no pager selected " etc.

Select the pager.

 

Ive come across this so many times...

rconety
New Member

By Extracting a single page, do you mean turning off the slider for 'Use Paging' and then clicking save?

 

 

 

Yes, im pretty sure its off by default.

As I said, it works for me.  At least we could rule it out rather than thinking its the site.

I turned the using pager off and clicked save.  I then toggled the use paging back on and selected extract data from single page and max web pages to process 1.  Clicked save. 

 

I then get the error.  Parameter 'Pager CSS selector': Can't be empty.

I then manually went into the advanced settings and added the css selector
iframe[id="contentIFrame"] select[id="pgrUsersTop_ddlPageList"], html > body > form > table > tbody > tr > td:eq(2) > iframe > html > body > form > table:eq(1) > tbody > tr:eq(1) > td > div > div:eq(1) > div > div > table > tbody > tr:eq(1) > td > table > tbody > tr:eq(0) > td > table > tbody > tr > td:eq(1) > select

It would not let me select it via right click add as pager.
I then changed extract data to All Available
I then saved and ran again.  Same error. "Failed to extract data (web page error wile extracting data)


9/10 it get an error like yours, pager not found etc.

I will just refresh my way of doing it.

Bring in a new extract data from webpage, select the data you require but just a single page, save it.

Reopen the parameters, edit to say you want all pages, you will get that error.

Select the pager.

 

If this doesn't work its probably a selector issue.

See if you can interact with the pager, as in forget the data, just keep pressing next page etc.

If you can accomplish this then we can move on to the next step.

Although you say the pager is in a dropdown, its actually the arrow to the right so technically its not in the dropdown so shouldn't matter...

 

Just rereading your shot, try it outside of all the other actions, just 1 extraction to a variable, no loops etc.

I re-created a simple flow to navigate to the web site, and extract data from web page without any loops.  I first tried to select data from a single page and then by manually entering the selector for paging.  Still does not work.  The only way that I can set the pager and get it to set automatically is if I from the beginning toggle paging on, selecting my fields and then setting the pager.

FYI.  I did not even realize their were next buttons... lol.  I tried repeating the process with the arrow (next) buttons and it still does not work.  It may be a selector issue.  I also confirmed that I can next through the paging on the screen.

next selector.PNG

And here is the css selector for pagination...
This is only generated if I toggle paging on from the beginning before selecting my fields on the screen and then setting the pager...

iframe[id="contentIFrame"] input[id="pgrUsersTop_ibNext"], html > body > form > table > tbody > tr > td:eq(2) > iframe > html > body > form > table:eq(1) > tbody > tr:eq(1) > td > div > div:eq(1) > div > div > table > tbody > tr:eq(1) > td > table > tbody > tr:eq(0) > td > table > tbody > tr > td:eq(2) > input:eq(2)

FYI.  I did not even realize their were next buttons... lol. 😂

 

Im sure someone else will chime in here eventually.

What happens when you press the "end button" next to next page, go to last page ?

If so this could be your loop index.

What happens when you manually type 2 into the text box, does it go to page 2 ?

If so we could type into using an increase variable / loop index etc.

There are many ways to get what you want but I would endeavour to get that next page selector working correctly.

If I press the end button (got to last page), it does indeed go to the last page.

If I click on the drop-down, type '2', and then hit enter, it does navigate to page 2.

Manually select the last page yourself.

Use action "Get details of a window" on this number, if it returns the page number correctly then this the end of your loop index.

So in a loop index starting at 1 and ending at the number you got from the above action, get the current page data into a variable, write to Excel ? im presuming. Beware every loop will overwrite the previous, to counter this increase your Excel row using the loop index as the new row number.

Please Please Please try everything to get that bloody next page element !!!

Helpful resources

Announcements
Power Platform Conf 2022 768x460.jpg

Join us for Microsoft Power Platform Conference

The first Microsoft-sponsored Power Platform Conference is coming in September. 100+ speakers, 150+ sessions, and what's new and next for Power Platform.

MPA Virtual Workshop Carousel 768x460.png

Register for a Free Workshop

Learn to digitize and optimize business processes and connect all your applications to share data in real time.

Power Automate Designer Feedback_carousel.jpg

Help make Flow Design easier

Are you new to designing flows? What is your biggest struggle with Power Automate Designer? Help us make it more user friendly!

Top Solution Authors
Top Kudoed Authors
Users online (1,567)