cancel
Showing results for 
Search instead for 
Did you mean: 

How to connect Google Dialogflow with Microsoft Flow (without coding)

google-home-speakers.jpgEveryone who has experimented with Google voice dialogs might experience that there is no out of the box way to connect the Google dialogs with automation software. There are a lot of programming opportunities in Dialogflow, but building a automation solution without coding seems to be difficult.

I took the challenge to connect Dialogflow with Microsoft Flow. From there, it is easy to build a solution that interacts with data or other functionalities. For example; make through your Google Home a reservation in a restaurant, get information about the latest support call or send a white paper that is requested in the voice dialog. That could all be possible if the voice dialog could be connected to Microsoft Flow automation software.

In this blog I dive into connecting both worlds. The blog does not explain how to make a Dialogflow dialog, nor how to retrieve (for example) data from Powerapps within Microsoft Flow.

Disclaimer; I’m not a Dialogflow expert, just curious.

Fulfillment webhook in Dialogflow

Dialogflow provides a “fulfillment” functionality that enables the interaction with external systems. That can be performed with the use of code, or by using a default webhook. For our (no coding) solution we will use this default webhook.

Dialogflow uses “Intents” to define the response on certain questions. The logic of this intent -what should be the response to a question-, is defined within that same intent. When Fulfillment is enabled for the intent, the logic will be processed by an external system. Dialogflow sends the request (and variable that are collected during the dialog, called “Entities”) to the external system by using the webhook. The webhook processes the question and variables so it can respond back to Dialogflow.

Only one webhook can be defined in Dialogflow, so requests for all fulfillment enabled webhooks are send to the same interface.

In the following diagram the above explained:

Fulfillment1-1500x681.png

And this is what the Fulfillment request and response look like:

Fulfillment2.png

It is easy to find out what JSON call is (or will be) send to the webhook. In Dialogflow you can try the dialog you created and during the intent where fulfillment will be enabled, select “Diagnostic info” in the bottom right of the screen.

Enable Fulfillment in Dialogflow

Enabling Fulfillment in general can be done using the fulfillment menu entry:

Fulfillment3.png

The URL that should be provided over here is defined by Microsoft Flow, later on in this tutorial.

Per Intent for which fulfillment should be enabled:

Fulfillment4.png

The option “slot filling” is being used when fulfillment is also used for processing the filling of variables in the intent that are not filled instantly. So when you have for example a mandatory variable “Time of reservation” in your Intent, and that variable is not filled directly. The question to the user can be defined by Dialogflow, or by the external system (Enable webhook for slot filling).

Logic in Microsoft Flow

Luckily Microsoft flow has the ability the create an API webhook in Azure that remains online and send all requests to the defined flow. This is done by the trigger “When a HTTP request is received”, in combination with the “Response” action to send back the response.

Because there will be only one flow (and one webhook) for all fulfillment enabled Intents, Microsoft flow has to determine from which Intent the request is coming first. From there it can define a response to the request and send that back.

So, the flow will be like:

Fulfillment5.png

 

Step 1: When a HTTP request is received

To enable the webhook API, this trigger may be used. Upon saving the trigger, a HTTP post URL in Azure is created that is connected to this trigger. This is the URL that has to be copied to the Dialogflow general fulfillment settings.

Fulfillment6.png

The JSON schema in “When  a HTTP request is received” can be generated with a sample payload. This payload can be generated by testing the Dialogflow in Dialogflow and at the end of the fulfillment intent select the button “Diagnostic info” in the bottom right. From there the sample payload can be copied.

But be careful; if you want Microsoft Flow to handle more than one fulfillment action, the JSONs can be different with various variables. In that case you should merge both fulfillment schemas so all variables are included.

In my use case the merged variables in the parameters section of the JSON schema look like:

Fulfillment7.png

Phonenumber in this example is used in another dialog than NumberPersons.

Step 2: Switch based on the Intent

To determine what Intent is requesting the flow, I use the JSON property “displayName”:

Fulfillment8.png

From there is easy to define the specific automation of an Intent in one of the branches of the Switch.

Step 3: Do some magic

Based on the Intent and the needed automation, the magic varies ofcourse. This is an example of my testcase where opening hours of a restaurant are requested from Powerapps (the CDS):

Fulfillment9.png

Step 4: Send a response back to Dialogflow

In the last step Microsoft Flow sends back a response to Dialogflow so the response from the logic can be given back to the Dialogflow user.

Fulfillment10.png

In this example only text is give back (saved in the variable “Response”). There are more responses possible in Dialogflow like images (not in a voice dialog of course) or cards.

 

Thanks for reading! 

Comments

Hi @Elowy 

I tried working with your example but I keep getting a response

"Webhook call failed. Error: Failed to parse webhook JSON response: Expect message object but got: null."

Can you please share your response body JSON Schema.  

Hi @SirSajid ,

I left the JSON schema empty. Just added the response in the body text, as described in step 4.

Good luck,

Elowy.

Meet Our Blog Authors
  • Working daily with Microsoft Cloud to deliver the needs of my company, my customers and various Microsoft communities and forums. | Office 365 | Flow | PowerShell | PowerApps | SharePoint |
  • Co-founder of https://plumsail.com, Office 365 and SharePoint expert. Passionate about design and development of easy to use, convenient and flexible products.
  • Microsoft Business Apps MVP. Owner of ThriveFast, an Office 365 consulting company.
  • 7x Microsoft Business Solutions MVP (CRM)
  • I'm keen in MS technologies, SharePoint, Office 365 and development for them
  • Daniel is a Business Productivity Consultant & Microsoft Business Solutions MVP who is very enthusiastic about all things Office 365, Microsoft Flow, PowerApps, Azure & SharePoint (Online). Since the preview, Daniel has been working with Microsoft Flow and later on with Microsoft PowerApps. That led to him being awarded an MVP Award for Business Solutions. He loves to blog, present and evangelize about improving productivity in the modern workspace with these amazing tools!
  • Michelle is an Office 365 solution architect in Twin Cities, MN. She has been delivering business collaboration solutions for years with her focus on SharePoint and Office 365. Michelle is a recent board member of the Minnesota Office 365 User Group and has been a member of the SharePoint community since 2009. She is a frequent speaker at MNSPUG and SharePoint Saturday and co-chaired the Legal SharePoint User Group for 4 years. Her most frequent projects have involved rolling out a large deployment of Office 365, SharePoint Online intranet, build of a "CHAMPS" Office 365 user adoption program and most recently, SharePoint On-Premise to Online Migration. Michelle is very excited about cloud technology as it is shifting her IT Pro focus to collaboration strategy and technical adoption.
  • I'm a Microsoft Office Servers and Services MVP with a special interest in SharePoint, Office 365, Microsoft Flow, Microsoft Teams and PowerApps. I work at Triad Group Plc ( https://triad.co.uk)
  • Passionate #Programmer #SharePoint #SPFx #Office365 #MSFlow | C-sharpCorner MVP | SharePoint StackOverflow, Github, PnP contributor