Find text on screen (an OCR conundrum)

[yevreeva]

I am a little puzzled by the issue I came across but before I get to it, what is the best way to search for a plain text on a webpage? I have some success using "if text on screen (OCR)" function and had very inconsistent result with "Move mouse to text on screen (OCR)" function.

I have following flow: after opening an Excel file, and getting necessary data from it, I split the text with a comma character, and I end up creating a variable that contains a unique list of values (containing alpha-numerical characters) that I am going to search for on screen (internal company's website) one-by-one (the website screen also has a list of strictly unique values). If one is found, it will click on a checkmark next to the text. 

The issue: while testing, some text consistently is being skipped as if OCR function did not locate it, when I clearly see it on the screen (and can confirm it is present using Ctrl+F function).  For example, the first two values are found, but the third one is skipped, the last two are also found. There's no error message, so nothing to troubleshoot.

0 ABC123
1 XYZ456
2 QWE789
3 TYU987
4 YUI654

Where can I go from here troubleshooting wise?

• All of the text is visible during the flow
• I tried to move the 'offender' value to a different row just to see if that would make any difference. Of course it didn't.
• I tried creating "subflow" for testing purposes, and declared a variable manually to test the same steps. The results were inconsistent (I found the value a couple of times, but majority was a miss)

As a side note: the OCR functionality sometime appear to work on a subflow but not the main flow. Very odd.

I haven't been using Power Automate Desktop for long but everything seems logical until issues like that....

[VJR]

Hi @yevreeva 

If all of them are checkboxes then you can capture an UI element for one the checkboxes, then use a dynamic selector for that UI element by adding a variable in the selector. This variable will contain the value of the text to be searched.

[yevreeva]

It's unfortunately not that simple, not all of the checkboxes on the screen will be checked, which is why I need to find the position/location of the text next to it.

[yevreeva]

After some excessive testing (changing font type in browser; changing brightness/contrast settings of my monitor) I've determined that OCR:

1. Does not see about 50 pixels from the bottom of the screen, even if it's visible and the scroller is all the way down) thus cut off one of my tables therefore I was not able to find one of the search strings.
2. Is not able to read a certain combo of alpha-numerical characters (i.e. if a text started with "D0U", or "S2").

I added following into my workflow that helped me to resolve the issue:

1.  Instead of searching for the text on the full screen I specified what region should be searched by declaring specific coordinates
2. For the text that started with specific alpha-numeric characters, "Get subtext" to extract the majority of text and used that to search the specified region.

Edit: I spoke too soon. I am testing the next row and set of variables from the Excel file and I am getting inconsistent results with searching full or a partial text on the screen.

[ryule]

I agree with your comment on there are certain character combinations that it just can not recognize. Luckily it was in a situation where I had control over the names of the stuff, and I modified them to be all caps. I think I even just opted to use images for a couple of them, which seemed to work out (I used the Advanced algorithm with 15 tolerance)