Solved: Retain conversation context when interacting with ...

HenryJammes

I previously shared how to maintain past context when interacting with an OpenAI GPT model, by setting and updating a global variable using Bot Framework Composer, here: Set and update global variables using Bot Framework Composer and how to retain conversation context ....

Today I'd like to show how to do the same, but this time using Azure OpenAI ChatGPT model that just came out.

I find that model much better suited for conversation experiences and automatically formats code samples:

Couple of useful resources on this topic:

HenryJammes

I'm going to use the Fallback topic so that I can use Power Automate to query the Azure OpenAI ChatGPT model (note: you can also make HTTP requests directly in Bot Framework Composer) whenever PVA doesn't find a matching topic for a user utterance.

It's good to understand Chat Markup Language to understand how to scope the conversation and provide instructions to ChatGPT prior to it returning an answer. That way, you can give personality traits to the bot, rules to follow when providing answers, and data or information needed for the model.

Here's an example:

<|im_start|>system
Assistant is an intelligent chatbot designed to help users answer their tax related questions.
<|im_end|>
<|im_start|>user
When do I need to file my taxes by?
<|im_end|>
<|im_start|>assistant
In 2023, you will need to file your taxes by April 18th. The date falls after the usual April 15th deadline because April 15th falls on a Saturday in 2023. For more details, see https://www.irs.gov/filing/individuals/when-to-file
<|im_end|>
<|im_start|>user
How can I check the status of my tax refund?
<|im_end|>
<|im_start|>assistant
You can check the status of your tax refund by visiting https://www.irs.gov/refunds
<|im_end|>

For our example, I'll use a very basic system message that you can customize

<|im_start|>system
Assistant helps Contoso employees with their questions.
<|im_end|>

The idea for our PVA ChatGPT bot is to keep the full conversation when sending new prompts to the ChatGPT API.

To manage a global variable keeping track of the conversation with Azure OpenAI ChatGPT, I'm going to use Bot Framework Composer to do 2 things:

Create an InitializeFullDialog dialog, that will create and set a new variable, FullDialog, in Power Virtual Agents, that will initially contain the system message.
Create a SetFullDialog dialog, that will update the FullDialog variable so that it contains the history of interactions (past questions and answers provided by Azure OpenAI ChatGPT).

I start by opening the Bot Framework Composer

If you need help setting things up, refer to this: Getting started with Bot Framework Composer - Power Virtual Agents | Microsoft Learn

I then add a new dialog to my chatbot

I call it "InitializeFullDialog"

I create one new Output for it, that I call FullDialog, of type string:

In BeginDialog, I add a Manage properties > Set a property node, in order set the FullDialog variable to my system message as well as the necessary chat markup language before I add the user utterence:

Property: dialog.result.FullDialog
Value: ='<|im_start|>system\nAssistant helps Contoso employees with their questions.\n<|im_end|>\n<|im_start|>user\n'

I then add another dialog, SetFullDialog, this time with no outputs.

In the BeginDialog, I add a Manage properties > Set a property node, in order set the Power Virtual Agents FullDialog variable as a concatenation of the previous FullDialog value as well as new values that Power Automate pass me back (I'm showing this later in this article, but they're OK to set up in advance) and some chat markup language to piece it together:

Property: virtualagent.ConversationFullDialog
Value: =concat(virtualagent.FullDialog,virtualagent.OriginalTriggerPhrase,'\n<|im_end|>\n<|im_start|>assistant',virtualagent.ChatGPTResponse,'\n<|im_end|>\n<|im_start|>user\n')

Then I publish my bot in Bot Framework Composer:

As I interact in this example with the Azure OpenAI ChatGPT API, I'm using the Power Virtual Agents Fallback topic. This means that whenever a user asks a question that PVA doesn't know how to answer with an existing topic (i.e. there is no match with the topics trigger phrases), it goes to the Fallback topic, and in the Fallback topic I'm calling the Azure OpenAI ChatGPT API.

If you haven't enabled the Fallback topic in your chatbot, learn how here: Use a system fallback topic - Power Virtual Agents | Microsoft Learn

In the Fallback topic, I start by removing the existing nodes as I'm changing its behavior.

Just after the 'Trigger Phrases' node, I add a "Redirect to another topic" node, and select the InitializeFullDialog dialog.

I click on the variable name, name it FullDialog, and change its usage to "Bot (any topic can access)" and also check "External sources can set values":

Next step is to call the Azure OpenAI ChatGPT API using Power Automate.

I add a new node, select Call an action > Create a flow.

I assume you already have an Azure OpenAI account and an API key, but if you don't, check these:

In your cloud flow, you'll first want to add 2 text inputs, UnrecognizedTriggerPhrase and ConversationFullDialog:

Then, to call the Azure OpenAI GPT API, I use the HTTP action, with this configuration

Method: POST

URI: you can get the URI from the ChatGPT playground in the Azure OpenAI Studio, in view code, in curl

Headers:

api-key: {YourAPIKey}
Content-Type: application/json

Body:

{
"prompt": "{ExpressionBelow}",
"temperature": 0.7,
"top_p": 0.95,
"frequency_penalty": 0,
"presence_penalty": 0,
"max_tokens": 800,
"stop": [
"<|im_end|>"
]
}

For the prompt, this is where I need to pass the user question.

To keep context, the trick is to provide past questions and answers, as well as the new question.

So, I append the FullConversation with the UnrecognizedTriggerPhrase and a bit of chat language markup to let it know I expect a response from the bot.:

concat(
triggerBody()['text_1'],
triggerBody()['text'],
'\n<|im_end|>\n<|im_start|>assistant'
)

Now I need to pass back a few things to PVA.

The ChatGPTResponse is the response from the ChatGPT API.
This is the expression I'm using:
```
body('HTTP:_Azure_OpenAI_ChatGPT')?['choices'][0]?['text']
```
The OriginalTriggerPhrase - that's the individual question that was raised in this flow, that I also need to append to the ConversationFullDialog variable. I use the UnrecognizedTriggerPhrase value in dynamic content.

I give my flow a name, and save it.

Back in PVA, still in my Fallback topic, I select my flow, and can map the UnrecognizedTriggerPhrase and FullDialog to the Power Automate inputs.

I can see that Power Automate has 2 ouputs. I set their Usage to "Bot (any topic can access)":

I now add a new Show a message node, where I select the bot.ChatGPTResponse variable:

The final step is to add a "Redirect to another topic" node, and select the SetFullDialog dialog, that will take care of updating the FullDialog variable with the past questions and answers.

And that's it! 🎉

View solution in original post

MattJimison

This is awesome, @HenryJammes !

I just went through a very similar exercise, but am using Unified Canvas instead, so I thought I'd share the code view of my Fallback topic, since it's a breeze copy/pasting a topic in the Unified Canvas! 😀

A few notes to get it set up:

disable / remove any unnecessary existing topics to avoid triggering them if your intent is to handle all requests via ChatGPT
The Power Automate Flow is the same as the example shown above with Bot Composer except I'm not passing the user's query into the flow and back out since I'm just concatenating with that value in the same topic.
You'll need to get the ID of your flow in your environment and paste it into the part below where I've entered an empty Guid (all 0's) You could also manually recreate this part in the UI if desired
Just open up your fallback topic, go to code view, replace the Flow Id as noted above, select all text, delete, and copy in the below

kind: AdaptiveDialog
beginDialog:
  kind: OnUnknownIntent
  id: main
  actions:
    - kind: SetVariable
      id: setVariable_Kyy1Yh
      variable: Topic.UserQuery
      value: =System.Activity.Text

    - kind: ConditionGroup
      id: conditionGroup_Nuj40I
      conditions:
        - id: conditionItem_iRCr9Y
          condition: =IsBlank(Global.FullConversation)
          actions:
            - kind: SetVariable
              id: 36cfgS
              variable: Global.FullConversation
              value: <|im_start|>system\nI am a virtual assistant that can answer questions\n<|im_end|>\n<|im_start|>user\n

    - kind: InvokeFlowAction
      id: invokeFlowAction_Ho6dKr
      input:
        binding:
          text: =Topic.UserQuery
          text_1: =Global.FullConversation

      output:
        binding:
          response: Topic.Response

      flowId: 00000000-0000-0000-0000-000000000000

    - kind: SendMessage
      id: sendMessage_UgHog9
      message: "{Topic.Response}"

    - kind: SetVariable
      id: setVariable_0LLSEn
      variable: Global.FullConversation
      value: =Concatenate(Global.FullConversation,Topic.UserQuery,"\n<|im_end|>\n<|im_start|>assistant", Topic.Response, "\n<|im_end|>\n<|im_start|>user\n")