-
Notifications
You must be signed in to change notification settings - Fork 145
Description
Hey, team
I am from Azure App Configuration team. My team invest a lot on maintaining our public docs.
We are currently working on an AI evaluation initiative to assess whether AI agents, when equipped with the MS Learn MCP tools, can correctly answer product-specific questions based on our documentation. The results of this evaluation will help us identify gaps and improve our content quality, making our doc more AI-friendly.
Today, when we add new documentation, we submit a pull request to the MicrosoftDocs/azure-docs-pr repository. The content only becomes searchable via the MS Learn MCP doc search tool after the PR is merged and the document is officially published. We would like to validate before merge whether new or updated documentation is sufficiently AI-friendly. However, since preview/staging documents are not discoverable through the doc search tool, we cannot run this evaluation until after release.
We would like to ask whether it is possible to support searching preview or staging documentation (for example, documents in PRs) via the MS Learn MCP doc search tool. We believe native support for preview/staging docs in the doc search tool would significantly improve documentation quality validation.
For our current internal usage, we have a workaround to create a custom MCP server that proxy calls to the MS Learn MCP tools and inject our newly added docs to the search result. While this approach works to some extent, it is complex to maintain and inevitably diverges from the behavior of the official service over time. Do you have any guidance or suggestion about this?
CC @jimmyca15