cancel
Showing results for 
Search instead for 
Did you mean: 
Elowy

How to connect Google Dialogflow with Microsoft Flow (without coding)

google-home-speakers.jpgEveryone who has experimented with Google voice dialogs might experience that there is no out of the box way to connect the Google dialogs with automation software. There are a lot of programming opportunities in Dialogflow, but building a automation solution without coding seems to be difficult.

I took the challenge to connect Dialogflow with Microsoft Flow. From there, it is easy to build a solution that interacts with data or other functionalities. For example; make through your Google Home a reservation in a restaurant, get information about the latest support call or send a white paper that is requested in the voice dialog. That could all be possible if the voice dialog could be connected to Microsoft Flow automation software.

In this blog I dive into connecting both worlds. The blog does not explain how to make a Dialogflow dialog, nor how to retrieve (for example) data from Powerapps within Microsoft Flow.

Disclaimer; I’m not a Dialogflow expert, just curious.

Fulfillment webhook in Dialogflow

Dialogflow provides a “fulfillment” functionality that enables the interaction with external systems. That can be performed with the use of code, or by using a default webhook. For our (no coding) solution we will use this default webhook.

Dialogflow uses “Intents” to define the response on certain questions. The logic of this intent -what should be the response to a question-, is defined within that same intent. When Fulfillment is enabled for the intent, the logic will be processed by an external system. Dialogflow sends the request (and variable that are collected during the dialog, called “Entities”) to the external system by using the webhook. The webhook processes the question and variables so it can respond back to Dialogflow.

Only one webhook can be defined in Dialogflow, so requests for all fulfillment enabled webhooks are send to the same interface.

In the following diagram the above explained:

Fulfillment1-1500x681.png

And this is what the Fulfillment request and response look like:

Fulfillment2.png

It is easy to find out what JSON call is (or will be) send to the webhook. In Dialogflow you can try the dialog you created and during the intent where fulfillment will be enabled, select “Diagnostic info” in the bottom right of the screen.

Enable Fulfillment in Dialogflow

Enabling Fulfillment in general can be done using the fulfillment menu entry:

Fulfillment3.png

The URL that should be provided over here is defined by Microsoft Flow, later on in this tutorial.

Per Intent for which fulfillment should be enabled:

Fulfillment4.png

The option “slot filling” is being used when fulfillment is also used for processing the filling of variables in the intent that are not filled instantly. So when you have for example a mandatory variable “Time of reservation” in your Intent, and that variable is not filled directly. The question to the user can be defined by Dialogflow, or by the external system (Enable webhook for slot filling).

Logic in Microsoft Flow

Luckily Microsoft flow has the ability the create an API webhook in Azure that remains online and send all requests to the defined flow. This is done by the trigger “When a HTTP request is received”, in combination with the “Response” action to send back the response.

Because there will be only one flow (and one webhook) for all fulfillment enabled Intents, Microsoft flow has to determine from which Intent the request is coming first. From there it can define a response to the request and send that back.

So, the flow will be like:

Fulfillment5.png

 

Step 1: When a HTTP request is received

To enable the webhook API, this trigger may be used. Upon saving the trigger, a HTTP post URL in Azure is created that is connected to this trigger. This is the URL that has to be copied to the Dialogflow general fulfillment settings.

Fulfillment6.png

The JSON schema in “When  a HTTP request is received” can be generated with a sample payload. This payload can be generated by testing the Dialogflow in Dialogflow and at the end of the fulfillment intent select the button “Diagnostic info” in the bottom right. From there the sample payload can be copied.

But be careful; if you want Microsoft Flow to handle more than one fulfillment action, the JSONs can be different with various variables. In that case you should merge both fulfillment schemas so all variables are included.

In my use case the merged variables in the parameters section of the JSON schema look like:

Fulfillment7.png

Phonenumber in this example is used in another dialog than NumberPersons.

Step 2: Switch based on the Intent

To determine what Intent is requesting the flow, I use the JSON property “displayName”:

Fulfillment8.png

From there is easy to define the specific automation of an Intent in one of the branches of the Switch.

Step 3: Do some magic

Based on the Intent and the needed automation, the magic varies ofcourse. This is an example of my testcase where opening hours of a restaurant are requested from Powerapps (the CDS):

Fulfillment9.png

Step 4: Send a response back to Dialogflow

In the last step Microsoft Flow sends back a response to Dialogflow so the response from the logic can be given back to the Dialogflow user.

Fulfillment10.png

In this example only text is give back (saved in the variable “Response”). There are more responses possible in Dialogflow like images (not in a voice dialog of course) or cards.

 

Thanks for reading! 

Comments
About the Author
  • Experienced Consultant with a demonstrated history of working in the information technology and services industry. Skilled in Office 365, Azure, SharePoint Online, PowerShell, Nintex, K2, SharePoint Designer workflow automation, PowerApps, Microsoft Flow, PowerShell, Active Directory, Operating Systems, Networking, and JavaScript. Strong consulting professional with a Bachelor of Engineering (B.E.) focused in Information Technology from Mumbai University.
  • I am a Microsoft Business Applications MVP and a Senior Manager at EY. I am a technology enthusiast and problem solver. I work/speak/blog/Vlog on Microsoft technology, including Office 365, Power Apps, Power Automate, SharePoint, and Teams Etc. I am helping global clients on Power Platform adoption and empowering them with Power Platform possibilities, capabilities, and easiness. I am a leader of the Houston Power Platform User Group and Power Automate community superuser. I love traveling , exploring new places, and meeting people from different cultures.
  • Read more about me and my achievements at: https://ganeshsanapblogs.wordpress.com/about MCT | SharePoint, Microsoft 365 and Power Platform Consultant | Contributor on SharePoint StackExchange, MSFT Techcommunity
  • Encodian Owner / Founder - Ex Microsoft Consulting Services - Architect / Developer - 20 years in SharePoint - PowerPlatform Fan
  • Founder of SKILLFUL SARDINE, a company focused on productivity and the Power Platform. You can find me on LinkedIn: https://linkedin.com/in/manueltgomes and twitter http://twitter.com/manueltgomes. I also write at https://www.manueltgomes.com, so if you want some Power Automate, SharePoint or Power Apps content I'm your guy 🙂
  • I am the Owner/Principal Architect at Don't Pa..Panic Consulting. I've been working in the information technology industry for over 30 years, and have played key roles in several enterprise SharePoint architectural design review, Intranet deployment, application development, and migration projects. I've been a Microsoft Most Valuable Professional (MVP) 15 consecutive years and am also a Microsoft Certified SharePoint Masters (MCSM) since 2013.
  • Big fan of Power Platform technologies and implemented many solutions.
  • Passionate #Programmer #SharePoint #SPFx #M365 #Power Platform| Microsoft MVP | SharePoint StackOverflow, Github, PnP contributor
  • Web site – https://kamdaryash.wordpress.com Youtube channel - https://www.youtube.com/channel/UCM149rFkLNgerSvgDVeYTZQ/