cancel
Showing results for 
Search instead for 
Did you mean: 

AI Builder Forms Processing

 

Form processing

Form processing identifies the structure of your documents based on examples you provide to extract text from any matching form. Examples might include tax forms or invoices.

In this lab we will build and train a model for recognizing invoices. Then we will build a tablet app to show the detection in action and digitize the content.

Note: If you are building the first model in an environment, click on Explore Templates to get started.

 

Exercise 1

  1. From the left navigation expand AI Builder and select Build. Select Form Processing.

  1. Name your model. Because you are working in a shared environment make sure to include your name as part of the model name. This will make it easier to find later. Click create.

  1. Your screen should look like the following image. Select Add documents.

 

  1. Add the documents from the Train folder. You must have at least five documents to train the model. You can find these documents here 
  2. Confirm the selection and click Upload.

  3. Once your uploads are complete, select Analyze.

 

  1. Select the fields.

  2. Hover over the highlighted fields and confirm the fields that should be returned by the form when processing from our trained model.

  3. Once you have confirmed the fields, click Done.

  1. Train your model.

  1. Locate and open your saved model. If you need help finding it, type your name into the search box.

  1. Review the results of the trained model.

  1. Perform a test with the test invoice.
  2. Perform a test with another image or document.

  1. Publish the model.

 

 

Exercise 2

 

  1. Navigate to Apps and create a new Canvas App. Select Blank app with a tablet layout.

  1. Insert the Form Processor control from the AI Builder.

  1. Map it to your saved model.

  1. Drag and resize the control like the image below.

  1. Play your app.

  2. Click Analyze and add your test file.

  3. Your uploaded form will be analyzed

 

  1. You can see the mapped fields are recognized.

  1. Close the app player.

  2. Let’s take some of the data fields and place them on the screen for the user to review. Add three labels to the screen. Drag them to the right side of the screen and line them up like in the image below. Edit the text to “Invoice Number” , “Due Date” , and “Total”.

  3. Add Text input fields for each row and place them as below.

  4. Now we will map data from the analyzed document. Edit the default values for each field as follows:

     

Invoice Number

FormProcessor1.FormContent.Fields.INVOICE

Due Date

FormProcessor1.FormContent.Fields.’Due Date’

Total

FormProcessor1.FormContent.Fields.Total

 

  1. Play the app and add an invoice to be analyzed.


 

Comments

The article says it can work with tax forms, I tried with Tax Form 5471 and 1120 for the US and the result was not that great, the tool did not identify most of the items in the tax return. and the pdf was very clear. I hope the developers can improve that or show an article about tax forms because that can  truly transform how we review 100+ tax returns throughout the year for so many CPA firms and Coporate Tax departments in each single company

Meet Our Blog Authors
  • PowerApps Community Mentor 2018, Practicing surgeon and former Professor of Surgery, University of Illinois, Chicago.
  • I am the Owner/Principal Architect at Don't Pa..Panic Consulting. I've been working in the information technology industry for over 30 years, and have played key roles in several enterprise SharePoint architectural design review, Intranet deployment, application development, and migration projects. I've been a Microsoft Most Valuable Professional (MVP) 12 consecutive years and am also a Microsoft Certified SharePoint Masters (MCSM) since 2013.
  • My name is Timothy Shaw and I create digital solutions using the Power Platform, Office 365, and Azure SQL and handle the IT for a small company in the energy sector. When not building the future :), I enjoy playing guitar, good (or really bad!) sci-fi, Xbox therapy, and hanging with my wife and son. Twitter: @ShortForTim