Skip to content

Commit ca43a50

Browse files
KryptosAIclaude
andauthored
feat: improve discoverability for registries and AI agents (#62)
- Lead README with "first testing tool that is itself an MCP server" - Add npm downloads badge and Smithery badge - Add smithery.yaml for Smithery registry listing - Update server.json to v0.8.2 with agent-optimized description - Rewrite all MCP tool descriptions for agent self-discovery - Add keywords: mcp-server, ai-agent, ai-tools, developer-tools, ci-cd, schema-drift - Update MCP Server Mode section with "when to use" table - Bold "Works as MCP server" in comparison table Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 5a3bb63 commit ca43a50

File tree

5 files changed

+53
-29
lines changed

5 files changed

+53
-29
lines changed

README.md

Lines changed: 23 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -12,13 +12,15 @@
1212

1313
[![CI](https://github.com/KryptosAI/mcp-observatory/actions/workflows/ci.yml/badge.svg)](https://github.com/KryptosAI/mcp-observatory/actions/workflows/ci.yml)
1414
[![npm](https://img.shields.io/npm/v/@kryptosai/mcp-observatory)](https://www.npmjs.com/package/@kryptosai/mcp-observatory)
15+
[![npm downloads](https://img.shields.io/npm/dm/@kryptosai/mcp-observatory)](https://www.npmjs.com/package/@kryptosai/mcp-observatory)
1516
[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](./LICENSE)
1617
[![Node >= 20](https://img.shields.io/badge/node-%3E%3D20-339933)](./package.json)
18+
[![Smithery](https://smithery.ai/badge/@kryptosai/mcp-observatory)](https://smithery.ai/server/@kryptosai/mcp-observatory)
1719
[![mcp-observatory MCP server](https://glama.ai/mcp/servers/KryptosAI/mcp-observatory/badges/score.svg)](https://glama.ai/mcp/servers/KryptosAI/mcp-observatory)
1820

19-
Find problems in your MCP servers before your users do.
21+
**The first testing tool that is itself an MCP server.** Your AI agent can scan, test, record, replay, and verify other MCP servers autonomously — catching regressions, schema drift, and security issues without human intervention.
2022

21-
You update a server, a tool silently breaks, and your agent starts failing. MCP Observatory catches that. It connects to your servers, checks every capability, actually calls tools to make sure they work, and diffs runs to catch what changed.
23+
Use it as a CLI, a CI action, or give it to your agent as an MCP server and let it test your other servers for you.
2224

2325
<p align="center">
2426
<img src="./docs/demo.svg" alt="MCP Observatory scan output" width="820">
@@ -164,19 +166,25 @@ The action runs checks on every PR, comments a markdown report, and blocks merge
164166

165167
## MCP Server Mode
166168

167-
When running as an MCP server (`serve`), your AI agent gets the same capabilities as the CLI:
169+
**No other testing tool is itself an MCP server.** Add Observatory as a server and your AI agent can autonomously test, diagnose, and monitor your other MCP servers.
168170

169-
| Tool | What it does |
170-
|------|-------------|
171-
| `scan` | Discover and check all configured servers |
172-
| `check_server` | Check a specific server by command |
173-
| `record` | Record a server session to a cassette file |
174-
| `replay` | Replay a cassette offline — no live server needed |
175-
| `verify` | Verify a live server still matches a cassette |
176-
| `watch` | Run checks and diff against the previous run |
177-
| `diff_runs` | Compare two saved run artifacts |
178-
| `get_last_run` | Return the most recent run for a target |
179-
| `suggest_servers` | Scan your environment and recommend servers you're missing |
171+
```bash
172+
claude mcp add mcp-observatory -- npx -y @kryptosai/mcp-observatory serve
173+
```
174+
175+
Your agent gets 9 tools:
176+
177+
| Tool | When to use it |
178+
|------|---------------|
179+
| `scan` | Check if all your configured MCP servers are healthy |
180+
| `check_server` | Test a specific server before installing or after updating |
181+
| `record` | Capture a baseline of a working server for future comparison |
182+
| `replay` | Test against a recorded session — no live server needed |
183+
| `verify` | Confirm a server update didn't break anything |
184+
| `watch` | Check a server and see what changed since the last check |
185+
| `diff_runs` | Find regressions between two check results |
186+
| `get_last_run` | Retrieve previous check results for a server |
187+
| `suggest_servers` | Discover MCP servers that match your project stack |
180188

181189
An AI tool that checks other AI tools. It's a tool testing tools that serve tools.*
182190

@@ -259,7 +267,7 @@ npx @kryptosai/mcp-observatory run --target ./target.json
259267
| Benchmarking / latency | — | — | ✅ | — |
260268
| Jest integration | — | — | — | ✅ |
261269
| MCP proxy mode | — | ✅ | — | — |
262-
| Works as MCP server | | — | — | — |
270+
| **Works as MCP server** | **✅** | — | — | — |
263271

264272
Each tool has strengths. Observatory focuses on regression detection and CI-friendly workflows. mcp-recorder is great as a transparent proxy. MCPBench is the go-to for performance benchmarking. mcp-jest is ideal if you're already in a Jest workflow.
265273

package.json

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,11 @@
5151
},
5252
"keywords": [
5353
"mcp",
54+
"mcp-server",
5455
"model-context-protocol",
56+
"ai-agent",
57+
"ai-tools",
58+
"developer-tools",
5559
"cli",
5660
"regression-testing",
5761
"interoperability",
@@ -62,7 +66,9 @@
6266
"vcr",
6367
"mcp-testing",
6468
"security",
65-
"github-action"
69+
"ci-cd",
70+
"github-action",
71+
"schema-drift"
6672
],
6773
"dependencies": {
6874
"@modelcontextprotocol/sdk": "^1.27.1",

server.json

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,17 @@
11
{
22
"$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
33
"name": "io.github.KryptosAI/mcp-observatory",
4-
"description": "Regression testing for MCP servers. Checks capabilities, invokes tools, detects schema drift.",
4+
"description": "The first testing tool that is itself an MCP server. AI agents can scan, test, record, replay, and verify other MCP servers autonomously — catching regressions, schema drift, and security issues without human intervention.",
55
"repository": {
66
"url": "https://github.com/KryptosAI/mcp-observatory",
77
"source": "github"
88
},
9-
"version": "0.3.2",
9+
"version": "0.8.2",
1010
"packages": [
1111
{
1212
"registryType": "npm",
1313
"identifier": "@kryptosai/mcp-observatory",
14-
"version": "0.3.2",
14+
"version": "0.8.2",
1515
"transport": {
1616
"type": "stdio"
1717
},

smithery.yaml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
startCommand:
2+
type: stdio
3+
configSchema:
4+
type: object
5+
properties: {}
6+
commandFunction: |-
7+
(config) => ({
8+
command: "npx",
9+
args: ["-y", "@kryptosai/mcp-observatory", "serve"]
10+
})

src/server.ts

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ export async function startServer(): Promise<void> {
9797

9898
server.tool(
9999
"scan",
100-
"Auto-discover MCP servers from config files and run checks against each one. Returns a summary of tools/prompts/resources status for every discovered server.",
100+
"Use this to check if all your MCP servers are healthy. Auto-discovers servers from Claude config files, connects to each one, and verifies tools/prompts/resources respond correctly. Use with deep=true to also invoke tools and confirm they actually execute. Returns pass/fail status for every server.",
101101
{
102102
config: z.string().optional().describe("Path to a specific MCP config file. If omitted, scans default locations."),
103103
deep: z.boolean().optional().describe("Also invoke safe tools to verify they execute."),
@@ -136,7 +136,7 @@ export async function startServer(): Promise<void> {
136136

137137
server.tool(
138138
"check_server",
139-
"Run checks against a specific MCP server by command. Example: check_server({ command: 'npx -y @modelcontextprotocol/server-everything' })",
139+
"Use this to test a specific MCP server before installing or after updating it. Launches the server by command, checks all capabilities, and saves a run artifact for future comparison. Example: check_server({ command: 'npx -y @modelcontextprotocol/server-everything' }). Use deep=true to invoke tools, security=true to analyze schemas for vulnerabilities.",
140140
{
141141
command: z.string().describe("The command to launch the MCP server (e.g. 'npx -y @modelcontextprotocol/server-everything')."),
142142
args: z.array(z.string()).optional().describe("Additional arguments for the command."),
@@ -174,7 +174,7 @@ export async function startServer(): Promise<void> {
174174

175175
server.tool(
176176
"score_server",
177-
"Score an MCP server's health (0-100) including protocol compliance, schema quality, security, reliability, and performance. Returns grade A-F with detailed breakdown.",
177+
"Use this to get a quick health grade for an MCP server. Runs all checks (capabilities, tool invocation, security) and returns a 0-100 score with A-F grade and detailed breakdown across protocol compliance, schema quality, security, reliability, and performance.",
178178
{
179179
command: z.string().describe("The command to launch the MCP server."),
180180
args: z.array(z.string()).optional().describe("Additional arguments for the command."),
@@ -224,7 +224,7 @@ export async function startServer(): Promise<void> {
224224

225225
server.tool(
226226
"diff_runs",
227-
"Compare two run artifact files and return the diff showing regressions, recoveries, and schema drift.",
227+
"Use this to find what changed between two server checks. Compares two run artifacts and surfaces regressions (things that broke), recoveries (things that got fixed), schema drift (added/removed/changed tool parameters), and gate status changes. Essential after updating a server.",
228228
{
229229
base: z.string().describe("Path to the base run artifact JSON file."),
230230
head: z.string().describe("Path to the head run artifact JSON file."),
@@ -260,7 +260,7 @@ export async function startServer(): Promise<void> {
260260

261261
server.tool(
262262
"get_last_run",
263-
"Return the most recent run artifact for a given target ID. Searches the default runs directory.",
263+
"Use this to retrieve the last check results for a server. Finds the most recent run artifact by target ID so you can review previous results or diff against a new run.",
264264
{
265265
targetId: z.string().describe("The target ID to find the last run for (e.g. server name or command)."),
266266
},
@@ -305,7 +305,7 @@ export async function startServer(): Promise<void> {
305305

306306
server.tool(
307307
"suggest_servers",
308-
"Gather context about the current environment to help recommend MCP servers. Returns currently configured servers, detected languages/frameworks/databases/services, and available servers from the MCP registry.",
308+
"Use this when setting up a project or wondering what MCP servers to add. Scans the working directory for languages, frameworks, databases, and cloud providers, lists currently configured servers, and cross-references the MCP registry to recommend servers you're missing.",
309309
{
310310
cwd: z.string().optional().describe("Working directory to scan for environment signals. Defaults to process.cwd()."),
311311
},
@@ -412,7 +412,7 @@ export async function startServer(): Promise<void> {
412412

413413
server.tool(
414414
"record",
415-
"Record a live MCP server session to a cassette file. The cassette captures all JSON-RPC traffic and can be replayed offline or used to verify future server versions.",
415+
"Use this to capture a baseline of a working MCP server. Records all JSON-RPC traffic to a cassette file that can be replayed offline (no server needed) or used to verify future versions haven't broken anything. Like VCR for MCP.",
416416
{
417417
command: z.string().describe("The command to launch the MCP server."),
418418
args: z.array(z.string()).optional().describe("Additional arguments for the command."),
@@ -457,7 +457,7 @@ export async function startServer(): Promise<void> {
457457

458458
server.tool(
459459
"replay",
460-
"Replay a cassette file offline — no live server needed. Runs all checks against the recorded responses.",
460+
"Use this to test a server without running it. Replays a previously recorded cassette offline and runs all checks against the recorded responses. Useful in CI or when the live server is unavailable.",
461461
{
462462
cassette: z.string().describe("Path to a cassette JSON file."),
463463
},
@@ -510,7 +510,7 @@ export async function startServer(): Promise<void> {
510510

511511
server.tool(
512512
"verify",
513-
"Verify a live server still matches a recorded cassette. Connects to the server, replays the same requests, and compares responses.",
513+
"Use this after updating a server to confirm nothing broke. Connects to the live server, sends the same requests from a recorded cassette, and compares responses. Reports exactly what changed — added tools, removed parameters, different response shapes.",
514514
{
515515
cassette: z.string().describe("Path to a cassette JSON file."),
516516
command: z.string().describe("The command to launch the MCP server."),
@@ -554,7 +554,7 @@ export async function startServer(): Promise<void> {
554554

555555
server.tool(
556556
"watch",
557-
"Run checks against a server repeatedly and report when results change. Returns the initial check and starts monitoring. Note: in MCP server mode this runs a single check and diff against the previous run rather than a persistent loop.",
557+
"Use this to check a server and see what changed since the last check. Runs all checks, saves the result, and diffs against the previous run for the same target. Shows regressions, recoveries, and schema drift in one call.",
558558
{
559559
command: z.string().describe("The command to launch the MCP server."),
560560
args: z.array(z.string()).optional().describe("Additional arguments for the command."),

0 commit comments

Comments
 (0)