cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
awadheshyona
Frequent Visitor

Loop through multiple links on web page and get details

Hi All,

 

I am new to PowerAutomate Desktop and I want to know how can I achieve this.

 

1. Open web browser(chrome) and paste a URL.

2. This URL has multiple links which open in a different page if clicked.

3. Click link and extract some information from each clicked link.

 

I am able to do all of the above process with suggestions provided by @Highboy , however only for one of the first link(or any link) on that URL. I want to repeat this process for all links available on that URL and grab all information. I tried Web recorder also but it does not go back to the URL and opens second link and so on. I tried adding it one by one and it works however number of links on the URL(web page) will keep on changing daily.

 

Please suggest. Let me know if you need further details.

 

@Highboy  Can you please help.

 

Regards,

Awadhesh

1 ACCEPTED SOLUTION

Accepted Solutions
Highboy
Super User
Super User

I am not sure i understand what you are trying to do.
Could you attach a screenshot of the webpage you are working with?

But this could work

Highboy_0-1635421958676.png

Use "Extract data from web page" to extract all the links, and loop throuh all of them - no need to go back to first page.

Highboy_1-1635422171295.png

 

View solution in original post

19 REPLIES 19
Highboy
Super User
Super User

I am not sure i understand what you are trying to do.
Could you attach a screenshot of the webpage you are working with?

But this could work

Highboy_0-1635421958676.png

Use "Extract data from web page" to extract all the links, and loop throuh all of them - no need to go back to first page.

Highboy_1-1635422171295.png

 

That is really helpful @Highboy  This is what I was trying to achieve and it worked. Thanks once again.

BzshBzsh
Regular Visitor

Hi, @awadheshyona @Highboy 

 

I am trying to do the same thing or at least something very similar. I have a web page with 116 links on it. Each link takes me to a contact page where I need the Name, Phone, Fax and Address. I would like all the results to show up in 1 excel sheet with 116 rows. Using the example, I was able to save the links, have the webpage load and have the first link spit out the right data, then it loops, opens the next link but spits out a blank excel document. What am I missing? Also, I'm not sure if this matters but some of the links don't have a fax number.

BzshBzsh
Regular Visitor

I got it figured out!

I needed to save my extracted data as a variable and then use that variable to write it to an excel sheet.

Here is what my loop looks like.

RCMP Loop Flow.png

 

mhenderson95
New Member

Would you be able to post a screenshot of the entire flow you put together? I am trying to do something very similar and keep hitting a wall!

You may not need all the excel steps and you could simply launch excel before your loop starts. I have mine set up to copy the current data from sheet 1, paste it into sheet 2, erase the data from sheet 1 and then paste the new information into sheet 1. I then have conditional formatting set in my excel to check for differences between the 2 sheets. This may not be the best way to have it set up but it seems to work for me. Hopefully you have luck too.

RCMP Flow.PNG

Powerlearnerd
Frequent Visitor

Hi All,

 

I am new to PowerAutomate Desktop and I want to know how can I achieve this.

 

1. Open web browser(chrome) and paste a URL.

2. This URL has multiple links which open in a different page if clicked.

3. Click link and extract some information from each clicked link.

4. Then store all information in one excel file

 

I am able to do all of the three above processes, however with the fourth step I struggle as it is then opening all the results in different new excel files. I want the information to be stored in only one excel file. Does somebody know the answer to this problem.

 

Please suggest. Let me know if you need further details.

Hi Highboy,

Could you help me out with a similar sturggle cause I'm desperately looking for an answer. I also would like to loop thourgh multiple links on a web page and get the details. I want to enter a product name in www.exporthub.com and then webscrape all information of indivual webpages from suppliers. Hope you could help me out.

BzshBzsh
Regular Visitor

I had a similar problem and solved it above. When you are extracting data in your loop you need to save it as a variable not excel. Then use "get first free row" then "Write to excel worksheet". At this step you will pick your variable that you saved. Then finally "Increase Variable" by 1. Check my screenshot and let me know if it helps you.Loop Screenshot.jpg

Powerlearnerd
Frequent Visitor

 

Hi @BzshBzsh

I very much appreciate your reply. 

 

I noticed your prior contributions before but still I can't manage to get the right results. I included a screenshot of the whole flow I made so you can see what is lacking. 

 

Powerlearnerd_1-1661958856787.png

I want to scrape from these engineered wood suppliers on Export Hub 

for each of them their - Business nature, - Country region etc. when you click on their link.

 

Could you indicate where in my flow there is a mistake. Much appreciation in advance.

 

BzshBzsh
Regular Visitor

Try changing your last step to your extracted links variable and add row1 to the end.

Try changing your last step to your extracted links variable and add row1 to the end.

Powerlearnerd
Frequent Visitor

Thank you very much @bszetey  this helped me a lot. 

This is how I finally got the data in Excel. 

Powerlearnerd_0-1662020806844.png

Instead of having the information summed up in one column, I'd like to transpose each item with information into a row. So you will have all manufactuers in one column and number of people in the next column etc. Is there an easy way to do this with PowerAutomate. Thanks again in advance. 

May i know when you say extracting data & save as variable. I assumed under your screenshot will be "DataFromWebPage" right? But when you Increase variable, I see the variable name is "ExtractedLinksrow1" which doesn't seems to be the same one?

May I know where does the ExtractedLinksrow 1 came from? 

 

I'm trying to do something similiar with dynamic link, but it keep failed to loop the following link & it's only get the first link. 

Each of the view po below is dynamic, and each time the amount of the link shown won't be the same amount. 

 

 

view po.png

 

If you view my full flow screenshot from 02-11-2022 you will see that I extract data twice. The first time is outside of the loop where I pull all the links from the website and save it as extractedlinks. Then inside the loop I am pulling the data I want from each of those links. So it goes to the first link, takes the data, writes to excel then increasing the variable by one goes to the next link and repeats the process. I hope this helps!

Sorry m8, have been busy, and haven't been here for a while.

But I am glad that you found a solution .

Hi All,

Is there any possibility to achieve this requirement which is discussed here in power automate cloud flow??
Any suggestion would be helpful at least until what point we can achieve.

Hi @Highboy 

I want to download the files by clicking from the multiple links on webpage. 

I am hoping it is the same thing as mentioned in the post. But i am stuck after doing the step of extracting links from the webpage. how do i proceed further. from downloading the files from these links. 

Would you be able to give a proper instruction for this. I am trying to do something similar to this, but i am struggling to do it. 

My problem statement is defined in steps below:

1. I am logging into the website.

2. Go to the proper section in website from where i have to download multiple excels. 

3. Click on links for downloading the excels. 

 

Implemented Solution:

1. I was able to login to the website. 

2. Was able to go the appropriate section in the website. 

3. Stored all the links on the website in the excel. 

 

Problem: When i apply the loop in the extracted links, it is downloading the excel from the same link again and again. 

I want to download different excels from the extracted links. I am attaching my sequence of the flow. Please let me know where i am going wrong.  

 

shivani2_0-1678365591343.png

 

Helpful resources

Announcements

Power Platform Connections Ep 15 | L. Baybutt | Thursday, 1 June 2023

Episode Fifteen of Power Platform Connections sees David Warner and Hugo Bernier talk to Microsoft MVP Lewis Baybutt aka Low Code Lewis, alongside the latest news and community blogs.   Use the hashtag #PowerPlatformConnects on social media for a chance to have your work featured on the show.        Action requested: Feel free to provide feedback on how we can make our community more inclusive and diverse.  This episode premiers live on our YouTube at 12pm PST on Thursday 1st June 2023.  Video series available at Power Platform Community YouTube channel.    Upcoming events:  European Power Platform conference – Jun. 20-22nd - Dublin Microsoft Power Platform Conference – Oct. 3-5th - Las Vegas  Join our Communities:  Power Apps Community Power Automate Community Power Virtual Agents Community Power Pages Community  If you’d like to hear from a specific community member in an upcoming recording and/or have specific questions for the Power Platform Connections team, please let us know. We will do our best to address all your requests or questions.  Action requested: Feel free to provide feedback on how we can make our community more inclusive and diverse.  This episode premiers live on our YouTube at 12pm PST on Thursday 1st June 2023.  Video series available at Power Platform Community YouTube channel.    Upcoming events:  European Power Platform conference – Jun. 20-22nd - Dublin Microsoft Power Platform Conference – Oct. 3-5th - Las Vegas  Join our Communities:  Power Apps Community Power Automate Community Power Virtual Agents Community Power Pages Community  If you’d like to hear from a specific community member in an upcoming recording and/or have specific questions for the Power Platform Connections team, please let us know. We will do our best to address all your requests or questions. 

May 2023 Community Newsletter and Upcoming Events

Welcome to our May 2023 Community Newsletter, where we'll be highlighting the latest news, releases, upcoming events, and the great work of our members inside the Biz Apps communities. If you're new to this LinkedIn group, be sure to subscribe here in the News & Announcements to stay up to date with the latest news from our ever-growing membership network who "changed the way they thought about code".         LATEST NEWS "Mondays at Microsoft" LIVE on LinkedIn - 8am PST - Monday 15th May  - Grab your Monday morning coffee and come join Principal Program Managers Heather Cook and Karuana Gatimu for the premiere episode of "Mondays at Microsoft"! This show will kick off the launch of the new Microsoft Community LinkedIn channel and cover a whole host of hot topics from across the #PowerPlatform, #ModernWork, #Dynamics365, #AI, and everything in-between. Just click the image below to register and come join the team LIVE on Monday 15th May 2023 at 8am PST. Hope to see you there!     Executive Keynote | Microsoft Customer Success Day CVP for Business Applications & Platform, Charles Lamanna, shares the latest #BusinessApplications product enhancements and updates to help customers achieve their business outcomes.         S01E13 Power Platform Connections - 12pm PST - Thursday 11th May Episode Thirteen of Power Platform Connections sees Hugo Bernier take a deep dive into the mind of co-host David Warner II, alongside the reviewing the great work of Dennis Goedegebuure, Keith Atherton, Michael Megel, Cat Schneider, and more. Click below to subscribe and get notified, with David and Hugo LIVE in the YouTube chat from 12pm PST. And use the hashtag #PowerPlatformConnects on social media for a chance to have your work featured on the show.         UPCOMING EVENTS   European Power Platform Conference - early bird ticket sale ends! The European Power Platform Conference early bird ticket sale ends on Friday 12th May 2023! #EPPC23 brings together the Microsoft Power Platform Communities for three days of unrivaled days in-person learning, connections and inspiration, featuring three inspirational keynotes, six expert full-day tutorials, and over eighty-five specialist sessions, with guest speakers including April Dunnam, Dona Sarkar, Ilya Fainberg, Janet Robb, Daniel Laskewitz, Rui Santos, Jens Christian Schrøder, Marco Rocca, and many more. Deep dive into the latest product advancements as you hear from some of the brightest minds in the #PowerApps space. Click here to book your ticket today and save!      DynamicMinds Conference - Slovenia - 22-24th May 2023 It's not long now until the DynamicsMinds Conference, which takes place in Slovenia on 22nd - 24th May, 2023 - where brilliant minds meet, mingle & share! This great Power Platform and Dynamics 365 Conference features a whole host of amazing speakers, including the likes of Georg Glantschnig, Dona Sarkar, Tommy Skaue, Monique Hayward, Aleksandar Totovic, Rachel Profitt, Aurélien CLERE, Ana Inés Urrutia de Souza, Luca Pellegrini, Bostjan Golob, Shannon Mullins, Elena Baeva, Ivan Ficko, Guro Faller, Vivian Voss, Andrew Bibby, Tricia Sinclair, Roger Gilchrist, Sara Lagerquist, Steve Mordue, and many more. Click here: DynamicsMinds Conference for more info on what is sure an amazing community conference covering all aspects of Power Platform and beyond.    Days of Knowledge Conference in Denmark - 1-2nd June 2023 Check out 'Days of Knowledge', a Directions 4 Partners conference on 1st-2nd June in Odense, Denmark, which focuses on educating employees, sharing knowledge and upgrading Business Central professionals. This fantastic two-day conference offers a combination of training sessions and workshops - all with Business Central and related products as the main topic. There's a great list of industry experts sharing their knowledge, including Iona V., Bert Verbeek, Liza Juhlin, Douglas Romão, Carolina Edvinsson, Kim Dalsgaard Christensen, Inga Sartauskaite, Peik Bech-Andersen, Shannon Mullins, James Crowter, Mona Borksted Nielsen, Renato Fajdiga, Vivian Voss, Sven Noomen, Paulien Buskens, Andri Már Helgason, Kayleen Hannigan, Freddy Kristiansen, Signe Agerbo, Luc van Vugt, and many more. If you want to meet industry experts, gain an advantage in the SMB-market, and acquire new knowledge about Microsoft Dynamics Business Central, click here Days of Knowledge Conference in Denmark to buy your ticket today!   COMMUNITY HIGHLIGHTS Check out our top Super and Community Users reaching new levels! These hardworking members are posting, answering questions, kudos, and providing top solutions in their communities.   Power Apps:  Super Users: @WarrenBelz, @LaurensM  @BCBuizer  Community Users:  @Amik@ @mmollet, @Cr1t    Power Automate:  Super Users: @Expiscornovus , @grantjenkins, @abm  Community Users: @Nived_Nambiar, @ManishSolanki    Power Virtual Agents:  Super Users: @Pstork1, @Expiscornovus  Community Users: @JoseA, @fernandosilva, @angerfire1213    Power Pages: Super Users: @ragavanrajan  Community Users: @Fubar, @Madhankumar_L,@gospa  LATEST COMMUNITY BLOG ARTICLES  Power Apps Community Blog  Power Automate Community Blog  Power Virtual Agents Community Blog  Power Pages Community Blog  Check out 'Using the Community' for more helpful tips and information:  Power Apps , Power Automate, Power Virtual Agents, Power Pages 

Announcing | Super Users - 2023 Season 1

Super Users – 2023 Season 1    We are excited to kick off the Power Users Super User Program for 2023 - Season 1.  The Power Platform Super Users have done an amazing job in keeping the Power Platform communities helpful, accurate and responsive. We would like to send these amazing folks a big THANK YOU for their efforts.      Super User Season 1 | Contributions July 1, 2022 – December 31, 2022  Super User Season 2 | Contributions January 1, 2023 – June 30, 2023    Curious what a Super User is? Super Users are especially active community members who are eager to help others with their community questions. There are 2 Super User seasons in a year, and we monitor the community for new potential Super Users at the end of each season. Super Users are recognized in the community with both a rank name and icon next to their username, and a seasonal badge on their profile.  Power Apps  Power Automate  Power Virtual Agents  Power Pages  Pstork1*  Pstork1*  Pstork1*  OliverRodrigues  BCBuizer  Expiscornovus*  Expiscornovus*  ragavanrajan  AhmedSalih  grantjenkins  renatoromao    Mira_Ghaly*  Mira_Ghaly*      Sundeep_Malik*  Sundeep_Malik*      SudeepGhatakNZ*  SudeepGhatakNZ*      StretchFredrik*  StretchFredrik*      365-Assist*  365-Assist*      cha_cha  ekarim2020      timl  Hardesh15      iAm_ManCat  annajhaveri      SebS  Rhiassuring      LaurensM  abm      TheRobRush  Ankesh_49      WiZey  lbendlin      Nogueira1306  Kaif_Siddique      victorcp  RobElliott      dpoggemann  srduval      SBax  CFernandes      Roverandom  schwibach      Akser  CraigStewart      PowerRanger  MichaelAnnis      subsguts  David_MA      EricRegnier  edgonzales      zmansuri  GeorgiosG      ChrisPiasecki  ryule      AmDev  fchopo      phipps0218  tom_riha      theapurva  takolota     Akash17  momlo     BCLS776  Shuvam-rpa     rampprakash  ScottShearer     Rusk  ChristianAbata     cchannon  Koen5     a33ik  Heartholme     AaronKnox  okeks      Matren  David_MA     Alex_10        Jeff_Thorpe        poweractivate        Ramole        DianaBirkelbach        DavidZoon        AJ_Z        PriyankaGeethik        BrianS        StalinPonnusamy        HamidBee        CNT        Anonymous_Hippo        Anchov        KeithAtherton        alaabitar        Tolu_Victor        KRider        sperry1625        IPC_ahaas      zuurg    rubin_boer   cwebb365   Dorrinda   G1124   Gabibalaban   Manan-Malhotra   jcfDaniel   WarrenBelz   Waegemma   drrickryp   GuidoPreite   metsshan    If an * is at the end of a user's name this means they are a Multi Super User, in more than one community. Please note this is not the final list, as we are pending a few acceptances.  Once they are received the list will be updated. 

Check out the new Power Platform Communities Front Door Experience!

We are excited to share the ‘Power Platform Communities Front Door’ experience with you!   Front Door brings together content from all the Power Platform communities into a single place for our community members, customers and low-code, no-code enthusiasts to learn, share and engage with peers, advocates, community program managers and our product team members. There are a host of features and new capabilities now available on Power Platform Communities Front Door to make content more discoverable for all power product community users which includes ForumsUser GroupsEventsCommunity highlightsCommunity by numbersLinks to all communities Users can see top discussions from across all the Power Platform communities and easily navigate to the latest or trending posts for further interaction. Additionally, they can filter to individual products as well.   Users can filter and browse the user group events from all power platform products with feature parity to existing community user group experience and added filtering capabilities.     Users can now explore user groups on the Power Platform Front Door landing page with capability to view all products in Power Platform.      Explore Power Platform Communities Front Door today. Visit Power Platform Community Front door to easily navigate to the different product communities, view a roll up of user groups, events and forums.

Microsoft Power Platform Conference | Registration Open | Oct. 3-5 2023

We are so excited to see you for the Microsoft Power Platform Conference in Las Vegas October 3-5 2023! But first, let's take a look back at some fun moments and the best community in tech from MPPC 2022 in Orlando, Florida.   Featuring guest speakers such as Charles Lamanna, Heather Cook, Julie Strauss, Nirav Shah, Ryan Cunningham, Sangya Singh, Stephen Siciliano, Hugo Bernier and many more.   Register today: https://www.powerplatformconf.com/   

Users online (5,257)