AMA: Question for everyone: how are you using AI Builder? 🙋‍♂️

[JoeF-MSFT]

Hi everyone - welcome to the AMA! Have you tried AI Builder already? What AI model have you used? For what use case?

We'd love to hear your AI Builder story! 🙂

[ZePowerDiver]

To be frank,

I've been working around AI Builder ever since its inception and I love the product and the evolution the product team brings to it.

As for actual projects where we've been able to establish value for the cost, it's for document processing (invoices) for an accounting firm.

The whole invoice processing solution was used (with the apps & flows for processing, review, and approval).

The customer is 100% self-sufficient, so the proof is in the pudding: citizen dev 100%

[AnnaChu]

Amazing Eric! We've heard lots of good things about invoice processing in the forums - I think the team is going to print your reply and frame it 😍

[Tjan]

Just started using it since last week and I'm absolutely amazed by it's opportunities. 

For now, I'm doing basic invoice processing. Planning to do much more in the future! (Great when infused in RPA)!!

[Anonymous]

Processed a few invoices in english and japanese with the AI builder earlier this week! Training A.I. was a breeze, although it did feel like we were both training each other at times,,, lol. it's fun.. love the low code, no code for us 

[Runner55552]

In the environmental world, we do a lot of "field" work. I took some hand-written groundwater sampling forms, and used AI builder document processing to push the raw data into a SharePoint list. Despite the hand-writing, results were good.

Also working on converting 1930s vintage oil well application/drilling/plugging forms, done originally on typewriters. Started with the structured document option, switched to unstructured, and got better results. The character recognition of words on the type-written scanned forms has some difficulties, especially since the answers to the forms were often typed on top of dotted ledger lines (I'm old enough to know how typewriters worked, some of you are not).

[dhaberkorn]

My department supports ~100 retail locations across various categories and we have roughly ~500 different vendors sending invoices to us monthly. If the property-level didn't receive an invoice in a shipment, the old SOP was an email to the home office where a person would then search through the corporate Email Account that vendors sent invoices too, search for the invoice manually and send back to the property-level.

Now when invoices arrive via email, a Flow saves the attachments, AI then extracts the Purchase Order Number and delivery zip code, renames the file to a set cadence based on the extraction. The end-user now submits a Forms Response that asks for the data we extract, and instead of a home office user manually searching, Power Automate returns a shareable URL to the end-user with the document they need. We estimate saving about 720 hours of labor a year. 

[JoeF-MSFT]

Thanks a lot @Anonymous, @Runner55552 , @dhaberkorn for sharing your experience with AI Builder. They're really inspiring! 🤩

[jaybags]

In my current company, we are trying to enable this. I am kind a exploring the Ethical and responsible AI.  Are the pre-built components follow the global AI principles? 🙂

[JoeF-MSFT]

Hi @jaybags - welcome to AI Builder! We apply our responsible AI principles with guidance from committees that advise our engineering team that builds AI Builder. 

As you build solutions that leverage the pre-built components, here is some recommended guidance to set you up for success:

• Understand what it can do: Fully assess the capabilities of any AI model you are using to understand its capabilities and limitations. Understand how it will perform in your particular scenario and context. For example, if you are using the pre-built invoice model, test with real-world invoices from your business processes to analyze and benchmark the results against your existing process metrics.

• Respect an individual's right to privacy: Only collect data and information from individuals for lawful and justifiable purposes. Only use data and information that you have consent to use for this purpose.

• Legal review: Obtain appropriate legal advice to review your solution, particularly if you will use it in sensitive or high-risk applications. Understand what restrictions you might need to work within, and your responsibility to resolve any issues that might come up in the future. Do not provide any legal advice or guidance.

• Human-in-the-loop: Keep a human-in-the-loop, and include human oversight as a consistent pattern area to explore. This means ensuring constant human oversight of the AI-powered product or feature, and maintaining the role of humans in decision-making. Ensure that you can have real-time human intervention in the solution to prevent harm. This enables you to manage situations when the AI model does not perform as required.

• Security: Ensure your solution is secure, and that it has adequate controls to preserve the integrity of your content and prevent unauthorized access.

[LimeLeaf]

Hi everyone,

we are currently doing our first steps with the MS AI builder.
We are working with the "Classify text into custom categories" model - trying to categorize customer mails from a service mail address into business case scenarios . Then extract data from these mails with the "Extract custom entities from your text" model to process these mails with power automate and desktop flows.

The big plan is to relieve customer service from basic customer requests.

I will come back to share how far we made it currently we are in training with the AIs.
So far the results look promising.

[Anonymous]

I have a massive folder of pdf files and need to extract key/value pairs for payroll.

However, 5000  character limit leaves me with no choice but to use Google's DocAI which has no issue extracting from much larger files (which we also have).  

My agency would love to stick with Microsoft if they could compare but it appears that Microsoft is still playing catch up.

[JoeF-MSFT]

Hi @Anonymous - thank you for sharing this. What AI Builder model are you using? Can you share where are you seeing this error? AI Builder document processing doesn't have a 5,000 character limit. Looking forward solving this mystery together. 🙂

[hmperez]

Hi team. I've started using the solution a days ago. I am very excited to using this solution to clients but I have a lot of doubts about the implementation when the documents aren't structured. I am checking a lot of PDF documents with different sheets and fields in the documents could be more or less than others. What advices can you give me to improve my models and get an success extraction?

In other hand the fields confident isn't always sure and the client wants a full process extraction and not human intervention using a rpa but I believe the customer should check the info before in a Excel or using a Power Apps before to continue the process, what do you tell me about that?

Thanks a lot,

Best

[JoeF-MSFT]

Hello @hmperez! Welcome to AI Builder! 🙂

Here's something you can try: create a new Document Processing model and select Unstructured Document as document type. Automate unstructured documents like contracts with AI Builder | Power Automate Blog (microsoft.com) You quickly can start with 5-10 documents, train it and test. Once you do, let us know, are you getting good results?

On human review, each field is extracted with a confidence score. You can then configure in your process that only when a document has a low confidence score, then a human needs to review it.

The more documents you add to the training set, the better the model will be, and less human review will be required. But it's always good to have a human review exception process in case for example an invalid document, or very low-quality documents is processed, to avoid bad data to the next steps. 

[_minnies_]

I am just at my beginnings playing arround with the functionality. I have tried invoice processing and text recognition. I like the functionality but am a little bit unsecure about the lisencing. The latest question I try to answer is how to build my flow the best way to save AI capacity 🙂

[JoeF-MSFT]

Hi @_minnies_ - welcome to AI Builder!! 

Licensing is a good topic to ask about. Here is a high-level overview:

The links on the images are:

• Power Platform Licensing Guide
• Link calculator

Another useful thing to know is about the consumption report, that will tell you how many credits you're consuming. 

Feel free to ask any questions you might have on this topic of licensing or any other AI Builder topic! 🙂

[JHORNA]

Saludos,existe un cronograma o agenda de adiccionar modelos preformateados ? o upgrade en general de AIBuilder ? 

[JoeF-MSFT]

Hola @JHORNA! Puedes encontrar las últimas novedades en este post: Automate Document Processing end-to-end with AI Builder | Power Automate Blog (microsoft.com)

[AVTS]

Team i have seen many extractor but Microsoft AI builder is the best .