cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Anonymous
Not applicable

Not able to detect buttons inside the pane

Hi everyone,
I just started using Power automate last week and im trying to click a button inside a UI application but the AI doesnt seem to recognize anything inside that big pane window. Is power automate limited or is there a workaround this. please and thanks

7 REPLIES 7
fraenK
Memorable Member
Memorable Member

There is no AI behind that function 😉

 

You can check with Inspect.exe wich elements you can access directly:

https://docs.microsoft.com/en-us/windows/win32/winauto/inspect-objects

 

Otherwise you'll need to use keyboard/send keys, text (OCR) or image based actions:

https://docs.microsoft.com/en-us/power-automate/desktop-flows/actions-reference/mouseandkeyboard

Anonymous
Not applicable

HI fraenk,
Thanks for your time. Instead of Inspect.exe I installed accessibility insight but im not sure how I can use that in combination with power automate to create a flow. Can you please guide me to any links that has similar topics related. Please and Thanks

kmur
Frequent Visitor

I'm still seeing only the pane on accessibility insight and not the buttons. Which is frustrating because literally yesterday I could access the buttons for UI automation, and now for some reason with no updates or even a restart I can only access the panes.

 

Any idea why this could be happening?

I'm having this same problem now and I wonder if you found a reason?

 

My flow had reference to a specific field as my screenshot shows, and today when I ran it, the step wasn't able to find that Gui Password field... and you can see below it that when I try to select the field again, I only can get a reference to the greater Pane. Very frustrating... I just reinstalled PAD too. In my case it's with SAP client.

 

Thanks for any information.

 

kelway_0-1634161899635.png

 

johnlouie08
Microsoft
Microsoft

I was wondering if this problem has been solved because right now, I am encountering the same issue on automating SAP. I've captured some objects on SAP while developing an RPA but now that it is done, the captured objects are showing an error because the huge Pane is the only thing that is being recognize in SAP UI.

I don't recall exactly, but it definitely just worked again at some later point. As I consider it now after another problem with SAP I'll explain in a bit, I wonder if it's related to SAP Server Scripting being enabled for the user or the SAP GUI client. I think I may have been working with an RPA account that already had server side scripting and I may have been using my own account at the point it failed.

 

WIth SAP, I've found clear behavioral differences when clicking and typing manually vs letting the bot do it with my personal SAP account. I can press Shift F9 to export a report and the dialogue is completely different if the bot does it or if I do the whole process manually using the same SAP account. If, however, I use an account created specifically for RPA job (and assume they set SAP Server Scripting for it), then it works manually OR with the same RPA job. This one caused me some extreme confusion and frustration!

 

Anyway, good luck. Maybe reboot a few times 😉

Helpful resources

Announcements
Power Platform Conf 2022 768x460.jpg

Join us for Microsoft Power Platform Conference

The first Microsoft-sponsored Power Platform Conference is coming in September. 100+ speakers, 150+ sessions, and what's new and next for Power Platform.

New Ideas Forum MPA.jpg

A new place to submit your Ideas for Power Automate

Announcing a new way to share your feedback with the Power Automate Team.

MPA Virtual Workshop Carousel 768x460.png

Register for a Free Workshop

Learn to digitize and optimize business processes and connect all your applications to share data in real time.

MPA Licensing.jpg

Ask your licensing questions at the Power Automate AMA!

Join Priya Kodukula and the licensing team, super users and MVPs to find answers to your questions on Power Automate licensing.

Top Solution Authors
Top Kudoed Authors
Users online (3,803)