Showing results for 
Search instead for 
Did you mean: 
Regular Visitor

Copilot generative response inconsistencies

I've got a copilot using generative AI on an internal SharePoint site. I've configured manual authentication and SSO per the online guides and all seems to be working ok having published the Copilot to Teams - we can ask the copilot questions and it is sometimes answering with content from the SharePoint site.  


What we are finding though is that sometimes the copilot responds with a "I'm sorry, I'm not sure how to help with that. Can you try rephrasing?" when there is definitely an answer on the SharePoint.  Sometimes asking the exact same question a second time will result in the copilot providing the correct answer from the SharePoint but, not always.  Sometimes issuing a "restart" and then asking the exact same question again also seems to result with the correct answer from the SharePoint.


I've enabled Application Insights and it is logging away but, the only thing I notice is that when I ask a question and get a "I'm sorry, I'm not sure how to help with that. Can you try rephrasing?" response, Application Insights seems to just show the copilot using the fallback topic with no real information as to why.  Are there any other logs that I can check which might highlight any reasons for the inconsistencies or is this expected behaviour when using generative AI?

Copilot Studio
Copilot Studio

Hi @MarkB_UK 

By definition there could be a little variance between each response due to the LLM internal mechanism, and sometimes asking the same questions will produce "I don't know" or the actual answer.


Our product group is working on limiting this kind of behavior and they just released a fix this week regarding this issue, could you try again?


Another thing that could help is to lower the moderation level.

New Member

I also faced same issue even today. Even after putting lower the moderation level same issue is persisting. Can you reply with any other alternative solution. I have uploaded data in excel, one of the column has name. I am searching for all the rows which contains a particular keyword. With Same question it says "i don't know", other time it fetches the answer.


The second screenshot is in tabular format, as given in instructions. So, it's correct to get that data. But behavior remains inconsistent. Please help


Hi, @remidyon ,


I'm encountering a similar issue. I have created a model through Azure Open AI Studio, which accesses my SharePoint via an index. When using the model in the playground or the webchat app I created (Contoso), the responses are elaborate and correct.


However, when I add the model to Copilot Studio, the behavior is completely different. The responses seem more limited, and sometimes the information provided is very basic. Instead of generating an answer, as it does in the playground, it often responds with preset phrases from one of the default topics.


Is there any explanation to this?


I would appreciate very much your help on this 🙂

Hi @Freddie_B 

Currently the generative answer node in Copilot Studio will rewrite / summarize / moderate the answers generated by Azure OpenAI (basically the answer is going through a second layer of LLM with a different set of system prompts) - our product group is looking at allowing users to avoid that rewrite process, stay tune for more information on this.


Many thanks @remidyon for you reply and clarification. I'll be waiting for more updates.



Hi @remidyon ,


I'm attempting to send an HTTP request to OpenAI's service. It seems to function correctly overall. However, when I dynamically assign the content of the user field in the JSON payload to a variable where the question is stored, the response indicates that the information isn't found in the retrieved data. Strangely, if I manually input the same question into the content field, I receive the correct response.


Would you know why this is hapenning?




This is the code:








kind: AdaptiveDialog
  kind: OnUnknownIntent
  id: main
  priority: -1
    - kind: SetVariable
      id: setVariable_J8onVi
      variable: Topic.UserQuestion
      value: =System.Activity.Text

    - kind: HttpRequestAction
      id: vdrcdy
      method: Post

        kind: JsonRequestContent
          kind: ValueExpression
              - parameters:
                      - Value: content

                    contentFieldsSeparator: |
                    filepathField: metadata_spo_item_name
                    titleField: Name

                  indexName: sharepoint-index
                  inScope: true
                  queryType: simple
                  roleInformation: You are an AI assistant and you are part of our organization. You need to be polite when you give the answers. The information you will be giving is from an internal SharePoint.
                  semanticConfiguration: default
                  strictness: 3
                  topNDocuments: 5

                type: AzureCognitiveSearch

            deployment: gpt-35-turbo-16k
            max_tokens: 800
              - content: You are an AI assistant and you are part of our organization. You need to be polite when you give the answers. The information you will be giving is from an internal SharePoint.
                role: system

              - content: ${Topic.UserQuestion}
                role: user

            stream: false
            temperature: 0
            top_p: 1

      response: Topic.Response
        kind: Record
              kind: Table
                index: Number
                    kind: Table
                      content: String
                      end_turn: Boolean
                      index: Number
                      role: String

          created: Number
          id: String
          model: String
          object: String
              kind: Record
                completion_tokens: Number
                prompt_tokens: Number
                total_tokens: Number

    - kind: SendActivity
      id: sendActivity_CUSi4i
      activity: "{Last(Last(Topic.Response.choices).messages).content}"

    - kind: EndDialog
      id: a1Vse4





Helpful resources


Check out the Copilot Studio Cookbook today!

We are excited to announce our new Copilot Cookbook Gallery in the Copilot Studio Community. We can't wait for you to share your expertise and your experience!    Join us for an amazing opportunity where you'll be one of the first to contribute to the Copilot Cookbook—your ultimate guide to mastering Microsoft Copilot. Whether you're seeking inspiration or grappling with a challenge while crafting apps, you probably already know that Copilot Cookbook is your reliable assistant, offering a wealth of tips and tricks at your fingertips--and we want you to add your expertise. What can you "cook" up?   Click this link to get started:   Don't miss out on this exclusive opportunity to be one of the first in the Community to share your app creation journey with Copilot. We'll be announcing a Cookbook Challenge very soon and want to make sure you one of the first "cooks" in the kitchen.   😊Don't miss your moment--start submitting in the Copilot Cookbook Gallery today!     Thank you,  Engagement Team

Announcing Power Apps Copilot Cookbook Gallery

We are excited to share that the all-new Copilot Cookbook Gallery for Power Apps is now available in the Power Apps Community, full of tips and tricks on how to best use Microsoft Copilot as you develop and create in Power Apps. The new Copilot Cookbook is your go-to resource when you need inspiration--or when you're stuck--and aren't sure how to best partner with Copilot while creating apps.   Whether you're looking for the best prompts or just want to know about responsible AI use, visit Copilot Cookbook for regular updates you can rely on--while also serving up some of your greatest tips and tricks for the Community. Check Out the new Copilot Cookbook for Power Apps today: Copilot Cookbook - Power Platform Community.  We can't wait to see what you "cook" up!      

Welcome to the Copilot Studio Community!

You are now a part of a vibrant growing group of peers and industry experts who are here to network, share knowledge, and even have a little fun!   If you have not yet registered, please visit here to learn how to register!   Now that you are a member, you can enjoy the following resources:   The Microsoft Copilot Studio Community Forums If you are looking for support with any part of Microsoft Copilot Studio, our forums are the place to go. They are titled "Forums" and there you will find technical professionals with years of experience who are ready and eager to answer your questions. You now have the ability to post, reply and give "kudos" on the Copilot Studio Community forums! Make sure you conduct a quick search before creating a new post because your question may have already been asked and answered!   Microsoft Copilot Studio Ideas Do you have an idea to improve the Microsoft Copilot Studio experience, or a feature request for future product updates? Then the "Copilot Studio Ideas" section is where you can contribute your suggestions and vote for ideas posted by other Community members. We constantly look to the most voted Ideas when planning updates, so your suggestions and votes will always make a difference.   News & Announcements The Copilot Studio Community News & Announcements is your stop to get all the latest news around Community events and announcements. This is where we share with the Community what is going on and how to participate!   Copilot Studio Samples, Learning and Video Galleries We have galleries that can assist you with information on creating a bot in our Webinars and Video Gallery,  and the ability to share the bots you have created in our Bot Sharing Gallery.  Check out the awesome content being shared there today!   Again, we are excited to welcome you to the Microsoft Copilot Studio Community family! Whether you are brand new to the world of bot creation or you are a seasoned Copilot Studio veteran, our goal is to shape the Community to be your ‘go to’ for support, networking, education, inspiration and encouragement as we enjoy this adventure together!   Let us know in the Community Feedback if you have any questions or comments about your Community experience. To learn more about the Community and your account be sure to visit our Community Support Area boards to learn more!   We look forward to seeing you in the Copilot Studio Community!The Copilot Studio Community Team

Tuesday Tip | How to Report Spam in Our Community

It's time for another TUESDAY TIPS, your weekly connection with the most insightful tips and tricks that empower both newcomers and veterans in the Power Platform Community! Every Tuesday, we bring you a curated selection of the finest advice, distilled from the resources and tools in the Community. Whether you’re a seasoned member or just getting started, Tuesday Tips are the perfect compass guiding you across the dynamic landscape of the Power Platform Community.   As our community family expands each week, we revisit our essential tools, tips, and tricks to ensure you’re well-versed in the community’s pulse. Keep an eye on the News & Announcements for your weekly Tuesday Tips—you never know what you may learn!   Today's Tip: How to Report Spam in Our Community We strive to maintain a professional and helpful community, and part of that effort involves keeping our platform free of spam. If you encounter a post that you believe is spam, please follow these steps to report it: Locate the Post: Find the post in question within the community.Kebab Menu: Click on the "Kebab" menu | 3 Dots, on the top right of the post.Report Inappropriate Content: Select "Report Inappropriate Content" from the menu.Submit Report: Fill out any necessary details on the form and submit your report.   Our community team will review the report and take appropriate action to ensure our community remains a valuable resource for everyone.   Thank you for helping us keep the community clean and useful!

The Copilot Studio community is thrilled to share some exciting news!

We are embarking on a journey to enhance your experience by transitioning to a new community platform. Our team has been diligently working to create a fresh community site, leveraging the very Dynamics 365 and Power Platform tools that our community advocates for.  We're proud to announce that the Copilot Studio community will pioneer this migration starting in June 2024. The move will mark the beginning of a new chapter, and we're eager for you to be a part of it. Following our lead, the rest of the Power Platform product sites will join us over the summer.   Stay tuned for more updates as we get closer to the launch. We can't wait to welcome you to our new community space, designed with you in mind, to connect, learn, and grow together.   Here's to new beginnings and endless possibilities!   If you have any questions, observations or concerns throughout this process please go to To stay up to date on the latest details of this migration and other important Community updates subscribe to our News and Announcements forums: Copilot Studio, Power Apps, Power Automate, Power Pages  

Top Solution Authors
Users online (5,623)