cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
HenryJammes
Copilot Studio
Copilot Studio

Integrate a PVA chatbot with Azure OpenAI ChatGPT using the Chat Completion API format

OpenAI ChatGPT and GPT-4 models are both optimized for conversational interfaces. This means they accept input formatted as a conversation. The main benefit is retaining the conversation context (i.e. the history of past questions and answers), so that any new follow up question is made in the context of past interactions.

 

Example in ChatGPT, where the question "Who was it before?" only makes sense in the context of the previous questions and answers: 

HenryJammes_1-1683653156588.png

 

To replicate this behavior in Power Virtual Agents, you integrate with the Azure OpenAI Service APIs using a Power Automate cloud flow.

For each of the requests you make to Azure OpenAI, you need to retain the context of previous questions and answers. 

The Chat Completion API is a new dedicated API for interacting with the ChatGPT and GPT-4 models in conversational manner. 

Chat Completion is optimized and will lead to better results than the previous Chat Markup Language (ChatML) format.

 

The format allows you to set the system prompt (with instructions, rules, data, etc.) as well as a format for the following question and answers, between the bot and the user. It's formatted in JSON.

Below is an example of the above example conversation:

 

[
    {
        "role": "system",
        "content": "You are an AI assistant that helps people find information."
    },
    {
        "role": "user",
        "content": "Who was Microsoft CEO in 2020?"
    },
    {
        "role": "assistant",
        "content": "In 2020, the CEO of Microsoft was Satya Nadella. He has been the CEO of Microsoft since 2014."
    },
    {
        "role": "user",
        "content": "Who was it before?"
    },
    {
        "role": "assistant",
        "content": "Before Satya Nadella, Steve Ballmer was the CEO of Microsoft. He served as the CEO from 2000 to 2014, following Bill Gates who co-founded Microsoft and served as CEO for many years."
    }
]

 

You can learn more about Chat Completion API here:

How to work with the ChatGPT and GPT-4 models (preview) - Azure OpenAI Service | Microsoft Learn

 

You cannot keep an infinitely large history of past interactions. Each model has a token limit. 

You can learn more on how to remain the thoken limit here.

 

In the below example, I show how to integrate Power Virtual Agents with Azure OpenAI with Power Automate, and use the Chat Completion API to retain conversation context.

  • In the first post, I show how to do this using the Power Virtual Agents new unified authoring canvas.
  • In the second post, I show how to do this using the classic version of Power Virtual Agents (using Bot Framework Composer).

 

Important note:

 

Prerequisites for the below articles:

55 REPLIES 55

@HenryJammes ,

Henry, I wanted to let you know that I found the error! It seems that there was no problem with the JSON or the flow at all. The only thing I overlooked was including the "=" sign when assigning a value to a property in Bot Framework Composer....

Thank you for your help and for working through this with me. I truly appreciate your support. And, of course, thank you once again for providing such a helpful tutorial!

Sravan_999
Frequent Visitor

Hi @HenryJammes ,
I created the dialogs on BFC for initialize and update dialogs.
but it was showing blank on the flow run -"FullDialog" ,
Please share your suggestions, how to fix this issue

Sravan_999_0-1683798329325.png

Thanks

Not sure what's wrong here @Sravan_999. Easiest way for you to troubleshoot this is to download the solution (at the end of post on PVA 1 + BFC) and compare with how you've done your customizations.

Ohron
Frequent Visitor

I think the problem with the JSON Parse error has to do with the double quotes being returned from the GPT engine within its reply.  When continuing the conversation, the double quotes are submitted back as part of your reply.  The Parser is reading the double quote as part of the JSON structure and not as part of the data being returned.  I have not yet figured out how to escape the " with a backslash.  I think the right method to use is replace([MyJSONData],'"','\"') but have not been able to dial it in just yet.


@Ohron wrote:

I think the problem with the JSON Parse error has to do with the double quotes being returned from the GPT engine within its reply.  When continuing the conversation, the double quotes are submitted back as part of your reply.  The Parser is reading the double quote as part of the JSON structure and not as part of the data being returned.  I have not yet figured out how to escape the " with a backslash.  I think the right method to use is replace([MyJSONData],'"','\"') but have not been able to dial it in just yet.


Nice, thanks for  catching this @Ohron,

I think I was able to work around this.

 

To make sure I don't pass unescaped double quotes to Power Automate, I now wrap my UnrecognitzedTriggerPhrase input with this Power Fx formula in PVA:

 

Substitute(System.Activity.Text,"""","\""")

HenryJammes_1-1683845574202.png

 

I do the same with the GPT output at the end of PVA logic, when I update the FullDialog:

Global.FullDialog 
& "{""role"": ""user"", ""content"": """ & Topic.OriginalUnrecognizedTriggerPhrase & """},"
& "{""role"": ""assistant"", ""content"": """ & Substitute(Topic.GeneratedAnswer,"""","\""") & """},"

HenryJammes_2-1683845642943.png


I can now have conversations full of "

 

HenryJammes_0-1683845977725.png

 

Henry, Thank you for the update.  This fix appears to be working. 

Renad
Helper II
Helper II

Hi there @HenryJammes, is this feasible if only PVA on Microsoft teams is available? (There's no subscription to the web app)

Hi @Renad,


No, as PVA for Teams doesn’t allow Bot Framework Composer extensions (useful here for variable management) nor the use of premium connectors in Power Automate (required for the HTTP connector or for the use of a custom connector). 

This is outlined here:

https://learn.microsoft.com/en-us/power-virtual-agents/requirements-licensing-subscriptions#power-vi...

Hi Henry,Thank you very much for your sharing. I have learned a lot from reading your articles. I would like to know how to manually reset the conversation with the bot or start a new conversation without inheriting the previous history.Because The content included in the answer seems to have caused an error in the 'Fulldialog' process, and my chat with the bot has entered a dead loop,Do you have any suggestions for it?

RayHao_0-1684226133006.png

Hi @RayHao, that's an excellent point.

 

There are indeed many situations where you'd want the conversation to reset:

  • Reaching the token limit of the GPT model (with each question asked, and answer received, the messages array grows in size. The token limit for gpt-35-turbo is 4096 tokens)
  • Asking new questions with no relationship with previous questions/answers, and you don't want ChatGPT to generate an answer based on past interactions.
  • Having chatbot topic logic where it doesn't make sense to retain the Full Dialog history.
  • Etc.

 

The token limit should be something you handle in the Power Automate side, as each response back by GPT provides the total number of tokens you've used so far. In PVA, you should have a mechanism to warn they have reached the limit and need to start over.

 

In PVA, you can reset variables in 3 ways:

  • Have the user trigger the Restart Conversation topic, for example by saying "Start over".

    This topic resets the conversation by clearing variable values and forcing the bot to use the latest published bot content.

  • By directly calling the "Clear all variables" action, in Topic management.
  • In our example, you can "Set a variable" value to Global.FullDialog. If you set it to "", it will act as a new conversation.

 

Helpful resources

Announcements

Super User of the Month | Drew Poggemann

As part of a new monthly feature in the Community, we are excited to share that Drew Poggemann is our featured Super User for the month of February 2024. If you've been in the Community for a while, we're sure Drew's name is familiar to you, as he is one of our most active contributors--he's been a Super User for five consecutive seasons!   Since authoring his first reply 5 years ago to his 514th solution authored, Drew has helped countless Community members with his insights and expertise. In addition to being a Super User, Drew is also a User Group leader and a Microsoft MVP. His contributions to our Super User sessions and to the new SUIT program are always welcome--as well as his sense of humor and fun-loving way of sharing what he knows with others.   When Drew is not solving problems and authoring solutions, he's busy overseeing the Solution Architecture team at HBS, specializing in application architecture and business solution strategy--something he's been doing for over 30 years. We are grateful for Drew and the amazing way he has used his talent and skills to help so many others in the Community. If you are part of the SUIT program, you got to hear some great tips from Drew at the first SUIT session--and we know he still has much more to share!You can find him in the Community and on LinkedIn. Thank you for all you do, Drew!

Announcing Power Apps Copilot Cookbook Gallery

We are excited to share that the all-new Copilot Cookbook Gallery for Power Apps is now available in the Power Apps Community, full of tips and tricks on how to best use Microsoft Copilot as you develop and create in Power Apps. The new Copilot Cookbook is your go-to resource when you need inspiration--or when you're stuck--and aren't sure how to best partner with Copilot while creating apps.   Whether you're looking for the best prompts or just want to know about responsible AI use, visit Copilot Cookbook for regular updates you can rely on--while also serving up some of your greatest tips and tricks for the Community. Our team will be reviewing posts using the new "Copilot Studio" label to ensure we highlight and amplify the most relevant and recent content, so you're assured of high-quality content every time you visit. If you share a post that gets featured in the curated gallery, you'll get a PM in the Community to let you know!The curated gallery is ready for you to experience now, so visit the new Copilot Cookbook for Power Apps today: Copilot Cookbook - Power Platform Community. We can't wait to see what you "cook" up!    

Celebrating a New Season of Super Users with Charles Lamanna, CVP Microsoft Business Applications

February 8 was the kickoff to the 2024 Season One Super User program for Power Platform Communities, and we are thrilled to welcome back so many returning Super Users--as well as so many brand new Super Users who started their journey last fall. Our Community Super Users are the true heroes, answering questions, providing solutions, filtering spam, and so much more. The impact they make on the Communities each day is significant, and we wanted to do something special to welcome them at our first kickoff meeting of the year.   Charles Lamanna, Microsoft CVP of Business Applications, has stressed frequently how valuable our Community is to the growth and potential of Power Platform, and we are honored to share this message from him to our 2024 Season One Super Users--as well as anyone who might be interested in joining this elite group of Community members.     If you want to know more about Super Users, check out these posts for more information today:    Power Apps: What is A Super User? - Power Platform CommunityPower Automate: What is A Super User? - Power Platform Community Copilot Studio: What is A Super User? - Power Platform Community Power Pages: What is A Super User? - Power Platform Community

February 2024 User Group Update: Welcoming New Groups and Highlighting Upcoming Events

It's a new month and a brand-new year, which means another opportunity to celebrate our amazing User Groups!Each month, we highlight the new User Groups that have joined the community. It's been a busy season for new groups, because we are thrilled to welcome 15 New User Groups! Take a look at the list below, shared by the different community categories. If your group is listed here, give this post a kudo so we can celebrate with you!   We love our User Groups and the difference they make in the lives of our Community! Thank you to all the new User Groups, new User Group leaders--we look forward to hearing about your successes and the impact you will leave!   In addition to our monthly User Group spotlight, it's a great time to share some of the latest events happening in our User Group community! Take a look at the list below to find one that fits your schedule and need! There's a great combination of in-person and virtual events to choose from. Also, don't forget to review the many events happening near you or virtually! It's a great time of year to connect and engage with User Groups both locally and online. Please Welcome Our NEW User Groups   Power Platform: Heathcare Power Platform User Group Power Platform Connect Hub Power Platform Usergroup Denmark Mexico Norte- Power Platform User Group Pune Power User Group Sudbury Power Platform User GroupMicrosoft User Group GhanaMPPBLR - Microsoft Power Platform Bengaluru User Group Power Apps:   Myrtle Beach Power Platform User GroupAnanseTechWB PowerApps Copilot Studio: Pathfinders Power Platform Community Dynamics365: Cairo, Egypt MSD 365 Business Central/NAV/F&O User GruopMS Dynamics 365 Business Central LatamCincinnati OH D365 F&O User Group February User Group Events February 2024 Cleveland Power Platform User GroupPortallunsj - Februar 2024Indiana D365/AX February User Group MeetingQ1 2024 KC Power Platform and Dynamics 365 CRM Users Group 

Super Users 2024 Season One is Here!

   We are excited to announce the first season of our 2024 Super Users is here! Our kickoff to the new year welcomes many returning Super Users and several new faces, and it's always exciting to see the impact these incredible individuals will have on the Community in 2024! We are so grateful for the daily difference they make in the Community already and know they will keep staying engaged and excited for all that will happen this year.   How to Spot a Super User in the Community:Have you ever written a post or asked for help in the Community and had it answered by a user with the Super User icon next to their name? It means you have found the actual, real-life superheroes of the Power Platform Community! Super Users are our heroes because of the way they consistently make a difference in the Community. Our amazing Super Users help keep the Community a safe place by flagging spam and letting the Community Managers know about issues. They also make the Community a great place to find answers, because they are often the first to offer solutions and get clarity on questions. Finally, Super Users share valuable insights on ways to keep the Community growing, engaging, and looking ahead!We are honored to reveal the new badges for this season of Super Users! Congratulations to all the new and returning Super Users!     To better answer the question "What is a Super User?" please check out this article: Power Apps: What is A Super User? - Power Platform CommunityPower Automate: What is A Super User? - Power Platform Community Copilot Studio: What is A Super User? - Power Platform Community Power Pages: What is A Super User? - Power Platform Community

Did You Attend the Microsoft Power Platform Conference in 2022 or 2023? Claim Your Badge Today!

If you were one of the thousands of people who joined us at the first #MPPC Microsoft Power Platform Conference in 2022 in Orlando--or attended the second-annual conference in Las Vegas in 2023--we are excited to honor you with a special community badge! Show your support for #MPPC Microsoft Power Platform Conference this year by claiming your badge!           Just follow this link to claim your badge for attending #MPPC in 2022 and/or 2023: MPPCBadgeRequest    Want to earn your badge for 2024? Just keep watching our News & Announcements for the latest updates on #MPPC24.

Top Kudoed Authors
Users online (3,314)