cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
GeoffP
New Member

Form recognizer for tally sheet - how to count items?

Hi all. I have a process I've been asked to improve and was hoping to use Form Recognizer or SharePoint Syntex until I can digitise the input. They have operators filling in paper forms that record the number of items for each product type they process.

 

The form has a table where they record details about the product type in the left 3 columns. The 4th/final column is about 75% of the page width and contains a couple of rows of small boxes where they record a single-character code (letter or number) based on the grade of each item. At the end of shift they record the total number of each product type, as well as the total items for each grade. 

 

About 70% of the sheets will have only a single product type recorded and most of the rest would have two, maybe three. Each of the product types would typically have 20-50 items graded. 

 

It would be easy to build a Power App to record the details instead of paper, but there are a couple of hundred operators across several shifts and none are licenced for M365. Licencing them all with M365 F3 might not be affordable, in which case I need to come up with a better way to manage the paper forms (or have a custom app developed). Currently, office staff take the totals from the sheets manually each morning for the previous day/night shift - laborious and delays reporting on shift output.

 

Is there any way with Form Recognizer or Syntex to process the documents (presumably with Power Automate) to count the number of items of each grade for each product type/row?

 

thanks

Geoff

1 ACCEPTED SOLUTION

Accepted Solutions
JoeF-MSFT
Power Apps
Power Apps

Hi @GeoffP,

 

This looks indeed like a great opportunity for automation! 

 

With AI Builder Form Processing you can easily build an AI model to extract the data you need from your tally sheets: Process custom forms with AI Builder - Learn | Microsoft Docs

 

For example, for a document that looks like this:

 

JoeFMSFT_0-1637962466850.png

 

You can:

 

1. Create a new Form Processing model in AI Builder by login to www.powerautomate.com and then selecting AI Builder on the left > Build > Form Processing.

 

2. Define the table like this, where each small box is defined as a column (more info on handling extracting items in tables here: https://docs.microsoft.com/en-us/ai-builder/create-form-processing-model#nested-items-in-tables)

 

JoeFMSFT_1-1637962524968.png

 

3. Upload at least five sample documents of tally sheets. If they have different layouts or templates, you will need to create a collection for each layout.

 

JoeFMSFT_2-1637962698745.png

 

4. Tag the table like this. You will probably need to tag it using the "Advanced mode"

 

JoeFMSFT_3-1637962750896.png

 

5. Once you have trained the model, you can use it in a Power Automate cloud flow to build a workflow to count each line.

 

JoeFMSFT_4-1637962803922.png

 

I hope this helps get you started in this automation journey! 🙂 Let us know if we can help clarify anything. 

View solution in original post

2 REPLIES 2
JoeF-MSFT
Power Apps
Power Apps

Hi @GeoffP,

 

This looks indeed like a great opportunity for automation! 

 

With AI Builder Form Processing you can easily build an AI model to extract the data you need from your tally sheets: Process custom forms with AI Builder - Learn | Microsoft Docs

 

For example, for a document that looks like this:

 

JoeFMSFT_0-1637962466850.png

 

You can:

 

1. Create a new Form Processing model in AI Builder by login to www.powerautomate.com and then selecting AI Builder on the left > Build > Form Processing.

 

2. Define the table like this, where each small box is defined as a column (more info on handling extracting items in tables here: https://docs.microsoft.com/en-us/ai-builder/create-form-processing-model#nested-items-in-tables)

 

JoeFMSFT_1-1637962524968.png

 

3. Upload at least five sample documents of tally sheets. If they have different layouts or templates, you will need to create a collection for each layout.

 

JoeFMSFT_2-1637962698745.png

 

4. Tag the table like this. You will probably need to tag it using the "Advanced mode"

 

JoeFMSFT_3-1637962750896.png

 

5. Once you have trained the model, you can use it in a Power Automate cloud flow to build a workflow to count each line.

 

JoeFMSFT_4-1637962803922.png

 

I hope this helps get you started in this automation journey! 🙂 Let us know if we can help clarify anything. 

View solution in original post

GeoffP
New Member

Thanks JoeF for the detailed reply. I'll need to add a heap of table columns to handle the individual items, but I think the advanced tagging mode might give us what we need - I didn't realise it could take nested data in form fields as separate table columns.


One problem I found when attempting the tagging was that some operators, when working with a basic product that doesn't have any grade distinctions, simply add a tick on the form for each item processed. I couldn't get the form tagging function to recognise their ticks for the field selection - so we might just need to ask them to use a valid code for every item, rather than ticks.

 

thanks again.

Helpful resources

Announcements
2022 Release Wave 1 760x460.png

2022 Release Wave 1 Plan

Power Platform release plan for the 2022 release wave 1 describes all new features releasing from April 2022 through September 2022.

Community Connections 768x460.jpg

Community & How To Videos

Check out the new Power Platform Community Connections gallery!

User Group Leader Meeting January 768x460.png

Calling all User Group Leaders!

Don't miss the User Group Leader meetings on January, 24th & 25th, 2022.

Users online (2,855)