cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous

How to Manage Handwritten Feedback with Microsoft Flow

Nowadays everything is digital. Almost. Still, many times when I am asked to give a feedback, I get a pen and paper. And I have only one thing on my mind.

Is there someone who reads all of these handwritten feedbacks?

And even worse, does she/he type all feedbacks into some feedback management system?

Hope not. But I feel so sorry that I build simple Flow for managing handwritten feedbacks.

Handling course feedbacks

Let's think about trainers giving different courses. After session participants fulfill feedback forms (in a paper). But after that, the trainer takes a photo from each of the feedback forms and sends them to handle.feedbacks@mytenant.onmicrosoft.com.

There is a Flow waiting for those emails. With every email, it reads texts in images (using Azure Computer Vision) and makes sentiment analysis for them. Results are stored in a SharePoint document library.

Sounds easy. But is it?

SharePoint document library for storing feedbacks

First, we need a place where to store all the feedbacks.

SharePoint document library is the perfect solution for this. Create a new one and add the following columns to it.

  • Course (Name of the course feedback is related to)
  • Content (Handwritten text recognized from paper form)
  • Sentiment (Tone of the feedback)
  • Trainer (Who was the trainer on course feedback is related to)

nc3a4yttc3b6kuva-2018-10-3-kello-17-01-12.png

Azure services

Our solution utilizes Azure services.  Let's create them. If you don't have an Azure account, you can always use trial account.

Click Create a resourcesearch for the Computer Vision API and create a one.

nc3a4yttc3b6kuva-2018-3-17-kello-17-21-55.png

 

After that create a Text Analytics API.

nc3a4yttc3b6kuva-2018-3-17-kello-17-21-55.png

 

Then pick a key from both of the created services. You will need them later.

nc3a4yttc3b6kuva-2018-3-17-kello-17-21-55.png

 

Flow

Our Flow starts every time handle.feedbacks@mytenant.onmicrosoft.com account gets an email.

In the beginning, we initialize variable for parsed text.

flow1.png

 

Then we get all email attachments.

flow2.png

 

Flow doesn't have a connector for the Computer Vision API. Therefore we make an HTTP call to a Computer Vision API service we just created. Use the service key you picked up earlier as a value of Ocp-Apim-Subscription-Key.

flow3.png

 

You get a return message from Computer Vision API in JSON format. It looks like this.

nc3a4yttc3b6kuva-2018-3-18-kello-11-34-02.png

 

If you want to understand in more detail how to use handwritten text recognition service, read API description. Shortly, calling the service doesn't return recognized text but the URL (Operation-Location). You can use the URL for asking actual results. The URL is valid for 24 hours.

So, we need to parse out that Operation-Location value. Fortunately, Flow has the parse JSON action.  You can use sample payload and Parse JSON action generates a schema for you. Simple.

flow45png.png

After that, we can easily use output values of parse JSON action in our Flow.

Since image analysis may take a while, we add 2 minutes delay before trying to get the parsed text.

flow6.png

After waiting, the results are retrieved with another HTTP request. Use the Get operation with Operation Location URL.  Ocp-Admin-Subctiption-Key is the same as in the previous action.

 

And return value is parsed by using parse JSON action.
flow7.png

 

After parsing JSON the written text does not exist in a single variable (which would be way too easy). Words are divided into lines.

That’s not a problem. Just add words to the variable line by line.

flow8.png

Now we have handwritten text in our variable.

The last thing we do is a sentiment analysis for the given feedback.  This can be done with the Sentiment Analyse action.

flow9.png

 

Almost ready! Now, we know the content and sentiment of the handwritten feedback. This information should be saved in our document library.

First, we create a file.

flow.png

 

After that, we update file properties (= our document library columns).

flow10.png

 

At the end remember to initialize handwritten text variable for next attachment. 

flow11-2.png

 

When we are ready our flow looks like this.

flow-overview.png 

 

Does it work?

Surprisingly well. Let's make a few feedbacks and see what happens.

Take a photo of each feedback form. After that, send them to the email address you used in "When a new email arrives" -action. You can use the course name as a title.

send-fedbacks-with-phone2.jpg

 

After waiting for a few minutes, items appear in the course feedback document library!

document-library-with-parsed-texts.png

 

Comments

Great post! Just a note : Flow does have Computer Vision API connector. That could help in getting some of the above steps done more quickly.

2018-10-09 14_45_45-Create your flow _ Microsoft Flow.png

Anonymous

Yes, that's true! Unfortunately, there is no action for analyzing handwritten text. 

Oh ok! got it! Thanks for the clarification.

Anonymous

But this was an excellent point. You should always check out what you get from existing connectors. And if there wasn't connector/action you need a month ago, it doesn't mean that it isn't there today.

 Very impressive.  I have a client that would like to OCR content of a digital photo taken from a mobile camera  (like an equipment tag, or a serial number in box). Could this be used to read QR Code as well?

 

Thanks for writing this up in the community.

Anonymous

Hi!

 

Yes and no... It depends very much about images you have. There is OCR service on Azure computer vision service (https://docs.microsoft.com/en-us/azure/cognitive-services/computer-vision/).

 

You can use it to recognize text from the image. But if there are several different texts in the image, then you have to find out by yourself which text is the right one. Which is a serial number and which is product name for example.

 

Also, there isn't (currently) service in Azure that recognizes QR code from the image.

PowerApps has a Barcode/QR-code reader reader that is pretty good. you can read more about it here - https://powerusers.microsoft.com/t5/PowerApps-Community-Blog/The-current-state-of-working-with-barco...

Anonymous

That's true!

 

If the requirement is (I don't know if it is) to handle existing images on the background (with Flow for example), then PowerApps cannot be used. 

 

 

 

About the Author
  • Experienced Consultant with a demonstrated history of working in the information technology and services industry. Skilled in Office 365, Azure, SharePoint Online, PowerShell, Nintex, K2, SharePoint Designer workflow automation, PowerApps, Microsoft Flow, PowerShell, Active Directory, Operating Systems, Networking, and JavaScript. Strong consulting professional with a Bachelor of Engineering (B.E.) focused in Information Technology from Mumbai University.
  • I am a Microsoft Business Applications MVP and a Senior Manager at EY. I am a technology enthusiast and problem solver. I work/speak/blog/Vlog on Microsoft technology, including Office 365, Power Apps, Power Automate, SharePoint, and Teams Etc. I am helping global clients on Power Platform adoption and empowering them with Power Platform possibilities, capabilities, and easiness. I am a leader of the Houston Power Platform User Group and Power Automate community superuser. I love traveling , exploring new places, and meeting people from different cultures.
  • Read more about me and my achievements at: https://ganeshsanapblogs.wordpress.com/about MCT | SharePoint, Microsoft 365 and Power Platform Consultant | Contributor on SharePoint StackExchange, MSFT Techcommunity
  • Encodian Owner / Founder - Ex Microsoft Consulting Services - Architect / Developer - 20 years in SharePoint - PowerPlatform Fan
  • Founder of SKILLFUL SARDINE, a company focused on productivity and the Power Platform. You can find me on LinkedIn: https://linkedin.com/in/manueltgomes and twitter http://twitter.com/manueltgomes. I also write at https://www.manueltgomes.com, so if you want some Power Automate, SharePoint or Power Apps content I'm your guy 🙂
  • I am the Owner/Principal Architect at Don't Pa..Panic Consulting. I've been working in the information technology industry for over 30 years, and have played key roles in several enterprise SharePoint architectural design review, Intranet deployment, application development, and migration projects. I've been a Microsoft Most Valuable Professional (MVP) 15 consecutive years and am also a Microsoft Certified SharePoint Masters (MCSM) since 2013.
  • Big fan of Power Platform technologies and implemented many solutions.
  • Passionate #Programmer #SharePoint #SPFx #M365 #Power Platform| Microsoft MVP | SharePoint StackOverflow, Github, PnP contributor
  • Web site – https://kamdaryash.wordpress.com Youtube channel - https://www.youtube.com/channel/UCM149rFkLNgerSvgDVeYTZQ/