cancel
Showing results for 
Search instead for 
Did you mean: 
darogael

Build your very own LEGO Inventory app using PowerApps, Flow and Microsoft Cognitive Vision Services

Recently, I’ve been intrigued by Microsoft Cognitive Vision services. This Machine-Learning functionality can extract rich information

from images to categorize and process visual data – and machine-assisted moderation of images to help curate your services. Now, as

some of you may already know, I’m a huge LEGO fan and I come from a family whose been playing with LEGOs for three generations.

One of the struggles each LEGO fan faces is identifying a part number. I took on this struggle as a challenge and have come up with a

Proof Of Concept using Microsoft PowerApps and Flow’s integration with Microsoft Cognitive Vision Services’ API. I introduce to you

my LEGO Inventory App.

 

Before I started building the app I had to find an Application Programming Interface (API) which would let me access a LEGO inventory

to gather information I need. Fortunately one of my personal favorite websites, Rebrickable.com already provided this API.

Rebrickable.com provides a number of API/Web Services to assist developers build their own websites or apps which use the

Rebrickable database. To be able to use these, you must first Generate an API Key which is free but does require registration

 

 

Rebrickable.png

 

My first iteration of this app involved me using Cognitive Vision's Optical character recognition (OCR) which would detect the LEGO

part numbers extract the recognized numbers into a machine-readable character stream. This gave me the flexibility of taking photos

of text instead of copying to save time and effort. However, I did run into a road-block. You see the part numbers in the LEGO blocks

are so tiny that it was challenging to take a zoomed-in picture. Hence I added an option to type in a part number as well but I did create

some modified blocks to test the OCR feature.

 

This video demonstrates how the first iteration of this app works

 

LEGO Inventory App using PowerApps and Flow

 

 

The first iteration works well, however, there were already two apps available which could do the exact same. Rebrickable Shopper

by Rebrickable is a mobile app where you simply scan the bar code on a LEGO® box and see a wealth of information to help you decide

if you should buy it. The Brick Data by Jean-Michel Agresti is an iPhone app perfect to browse LEGO Sets on your mobile device, old and

new. With these two existing apps already available any solution that I build would have to that major WOW factor, something that

makes it easier. This is when Cognitive Custom Vision Services had my complete and undivided attention. The simple concept of upload,

train and evaluate is brilliant. In addition, through it's active learning images evaluated through your custom vision model become part

of a feedback loop you can use to keep improving your classifier. Hence I decided to pursue that path and came up with my LEGO

Inventory App 2.0

 

This video demonstrates how the 2.0 iteration of the app works

 

Microsoft PowerApps with Cognitive Custom Vision

 

 

From a design and build standpoint the difference between OCR and Custom Vision was the change in the FLOW.

The below screenshot displays how the OCR and Custom Vision action is called and how it's values are us

 

Capture2.PNG

 

Currently Custom Vision is in preview hence there are some limitations but the overall P.O.C. was very successful.

 

URL:

https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/

https://azure.microsoft.com/en-us/services/cognitive-services/custom-vision-service/

About the Author
  • Experienced Consultant with a demonstrated history of working in the information technology and services industry. Skilled in Office 365, Azure, SharePoint Online, PowerShell, Nintex, K2, SharePoint Designer workflow automation, PowerApps, Microsoft Flow, PowerShell, Active Directory, Operating Systems, Networking, and JavaScript. Strong consulting professional with a Bachelor of Engineering (B.E.) focused in Information Technology from Mumbai University.
  • I am a Microsoft Business Applications MVP and a Senior Manager at EY. I am a technology enthusiast and problem solver. I work/speak/blog/Vlog on Microsoft technology, including Office 365, Power Apps, Power Automate, SharePoint, and Teams Etc. I am helping global clients on Power Platform adoption and empowering them with Power Platform possibilities, capabilities, and easiness. I am a leader of the Houston Power Platform User Group and Power Automate community superuser. I love traveling , exploring new places, and meeting people from different cultures.
  • Read more about me and my achievements at: https://ganeshsanapblogs.wordpress.com/about MCT | SharePoint, Microsoft 365 and Power Platform Consultant | Contributor on SharePoint StackExchange, MSFT Techcommunity
  • Encodian Owner / Founder - Ex Microsoft Consulting Services - Architect / Developer - 20 years in SharePoint - PowerPlatform Fan
  • Founder of SKILLFUL SARDINE, a company focused on productivity and the Power Platform. You can find me on LinkedIn: https://linkedin.com/in/manueltgomes and twitter http://twitter.com/manueltgomes. I also write at https://www.manueltgomes.com, so if you want some Power Automate, SharePoint or Power Apps content I'm your guy 🙂
  • I am the Owner/Principal Architect at Don't Pa..Panic Consulting. I've been working in the information technology industry for over 30 years, and have played key roles in several enterprise SharePoint architectural design review, Intranet deployment, application development, and migration projects. I've been a Microsoft Most Valuable Professional (MVP) 15 consecutive years and am also a Microsoft Certified SharePoint Masters (MCSM) since 2013.
  • Big fan of Power Platform technologies and implemented many solutions.
  • Passionate #Programmer #SharePoint #SPFx #M365 #Power Platform| Microsoft MVP | SharePoint StackOverflow, Github, PnP contributor
  • Web site – https://kamdaryash.wordpress.com Youtube channel - https://www.youtube.com/channel/UCM149rFkLNgerSvgDVeYTZQ/