[mcp-analysis] MCP Structural Analysis - 2025-12-15

[github-actions[bot]]

This analysis evaluates GitHub MCP tool response sizes and structural usefulness for agentic workflows. Testing 9 representative tools across different toolsets reveals significant variations in efficiency and value for autonomous agents.

Key Findings: Most tools (7 of 9) received excellent usefulness ratings (5/5), demonstrating well-designed APIs. However, list_code_scanning_alerts stands out as the most bloated tool, consuming 9,500 tokens due to embedded educational content. The most efficient tool is get_label at just 35 tokens. Context authentication issues prevent get_me from being useful in workflow environments.

Full Structural Analysis Report

Executive Summary

Metric	Value
Tools Analyzed	9
Toolsets Covered	9
Total Tokens (Today)	18,355
Average Usefulness Rating	4.3/5
Best Rated Tools	get_file_contents, list_issues, list_workflows, list_discussions, get_label, search_repositories (5/5)
Context-Efficient Champions	get_label (35 tokens), list_discussions (120 tokens), list_workflows (180 tokens)
Most Bloated Tool	list_code_scanning_alerts (9,500 tokens)

Usefulness Ratings for Agentic Work

⭐⭐⭐⭐⭐ Excellent Tools (Rating: 5)

Tool	Toolset	Tokens	Assessment
get_file_contents	repos	750	Clean file delivery with SHA. Text returned as resource. Perfect for reading files.
list_issues	issues	850	Rich data with full body, labels, user info. GraphQL pagination. Complete and actionable.
list_workflows	actions	180	Minimal, essential fields only. Excellent efficiency for workflow discovery.
list_discussions	discussions	120	Clean structure, category embedded. Highly efficient for discovery.
get_label	labels	35	Four essential fields. Most efficient tool tested. Perfect for label ops.
search_repositories	search	420	Minimal output default. Essential metadata only. Well-structured.

⭐⭐⭐⭐ Good Tools (Rating: 4)

Tool	Toolset	Tokens	Assessment
list_pull_requests	pull_requests	6,500	Comprehensive but verbose. Includes full repo objects in head/base. High token cost but complete data. Could be more efficient with minimal_output option.

⭐⭐⭐ Adequate Tools (Rating: 3)

Tool	Toolset	Tokens	Assessment
list_code_scanning_alerts	code_security	9,500	Extremely bloated. Full rule documentation, help text, examples in every alert. Useful data buried in educational content. Desperately needs minimal_output option.

⭐ Poor Tools (Rating: 1)

Tool	Toolset	Tokens	Assessment
get_me	context	0	403 error - not accessible by integration in workflow context. Cannot be used.

Schema Analysis

Tool	Schema Type	Depth	Key Fields	Structure Notes
get_file_contents	text_resource	1	content, SHA	Direct text delivery
list_issues	object_with_array	4	issues, pageInfo, number, title, body, labels	GraphQL-style pagination
list_pull_requests	array	5	id, number, state, head, base, assignees	Nested repo objects
list_workflows	object_with_array	2	total_count, workflows, id, name, path	Clean, flat structure
list_code_scanning_alerts	array	4	number, rule, tool, state, location	Bloated with docs
list_discussions	object_with_array	3	discussions, pageInfo, number, title, category	GraphQL pagination
get_label	object	1	color, description, id, name	Simple, flat
search_repositories	object_with_array	3	total_count, items, name, full_name, description	Clean search results
get_me	error	0	-	Auth failure

Response Size Analysis

Average Tokens by Toolset

Toolset	Avg Tokens	Efficiency Rating
code_security	9,500	❌ Poor - Extremely bloated
pull_requests	6,500	⚠️ Fair - Verbose but comprehensive
issues	850	✅ Good - Rich but reasonable
repos	750	✅ Good - Clean delivery
search	420	✅ Excellent - Efficient search
actions	180	✅ Excellent - Minimal data
discussions	120	✅ Excellent - Highly efficient
labels	35	✅ Outstanding - Most efficient
context	0	❌ Broken - Auth failure

Tool-by-Tool Detailed Analysis

🏆 Champion: get_label (35 tokens, 5/5)

Why it's excellent: Four essential fields (color, description, id, name), flat structure, zero bloat. Perfect example of efficient API design.

🥈 Runner-up: list_discussions (120 tokens, 5/5)

Why it's excellent: Clean GraphQL pagination, category info embedded, minimal nesting. Discovery-optimized.

🥉 Third Place: list_workflows (180 tokens, 5/5)

Why it's excellent: Essential workflow metadata only. No unnecessary verbosity. Perfect for agents discovering workflows.

📉 Biggest Offender: list_code_scanning_alerts (9,500 tokens, 3/5)

Why it's problematic:

• Includes full CodeQL rule documentation in every alert
• Help text with multiple code examples embedded
• References, CWE links, and educational content
• Useful data (number, state, location) buried in bloat
• Recommendation: Add minimal_output parameter like search tools

⚠️ Mixed Bag: list_pull_requests (6,500 tokens, 4/5)

Why it's verbose:

• Includes complete repository objects for head and base branches
• Full repo metadata repeated in nested structures
• Comprehensive but token-intensive
• Recommendation: Provide minimal_output option or flatten nested repos

❌ Broken: get_me (0 tokens, 1/5)

Why it fails: 403 error in GitHub Actions workflow context. Tool requires different authentication scope than available to integration.

30-Day Trend Summary

Metric	Value
Total Data Points	138
Date Range	2025-11-26 to 2025-12-15
Analysis Days	20 days
Average Daily Token Usage	~18,000 tokens
Toolsets Tracked	9
Tools Tracked	9-14 unique tools

Trend Observation: Token usage has remained relatively stable over the 30-day window. Most tools maintain consistent usefulness ratings, indicating stable API design. The code_security toolset consistently shows the highest token consumption.

Recommendations

For Agent Developers

High-value, efficient tools (use these first):

• ✅ get_label - Ultra-efficient label operations
• ✅ list_discussions - Efficient discussion discovery
• ✅ list_workflows - Minimal workflow listing
• ✅ search_repositories - Efficient repo search with minimal_output
• ✅ get_file_contents - Clean file reading

High-value but token-intensive (use with pagination):

• ⚠️ list_issues - Rich issue data, use small perPage values
• ⚠️ list_pull_requests - Comprehensive PR data, paginate carefully

Avoid or use sparingly:

• ❌ list_code_scanning_alerts - Extremely bloated, consider alternatives
• ❌ get_me - Broken in workflow context

For MCP Server Maintainers

Priority improvements:

1. Add minimal_output to code_security tools

   • Strip educational content (help, references, examples)
   • Keep only: number, rule.id, rule.name, state, location, message
   • Would reduce tokens by ~90%

2. Add minimal_output to pull_requests tools

   • Flatten nested repository objects
   • Provide option to exclude full repo metadata
   • Would reduce tokens by ~50%

3. Fix context tools in workflow environment

   • Document authentication limitations
   • Provide alternative methods for workflow context

4. Consider response compression

   • Add fields parameter to select specific fields
   • Implement GraphQL-style field selection
   • Allow clients to request only needed data

Context Efficiency Matrix

Best Practices for Agents:

Use Case	Recommended Tool	Token Cost	Efficiency
Get user info	❌ get_me	N/A	Broken
Read file	✅ get_file_contents	750	Excellent
List issues	✅ list_issues (perPage=5)	~850	Good
List PRs	⚠️ list_pull_requests (perPage=1)	6,500	Fair
Find workflows	✅ list_workflows	180	Excellent
Security scan	❌ list_code_scanning_alerts	9,500	Poor
List discussions	✅ list_discussions	120	Excellent
Get label	✅ get_label	35	Outstanding
Search repos	✅ search_repositories	420	Excellent

Visualizations

Response Size by Toolset

Code security tools consume 52x more tokens than label tools on average.

Usefulness Ratings by Toolset

Most toolsets achieve good-to-excellent ratings (4-5/5), with only code_security dropping to adequate (3/5) due to bloat.

Daily Token Usage Trend (30 Days)

Token usage remains stable over time, indicating consistent API behavior and testing methodology.

Token Size vs Usefulness Rating

The scatter plot reveals an efficiency sweet spot: tools with 100-1000 tokens achieve the highest usefulness ratings. Tools beyond 5000 tokens show diminishing returns.

Individual Tool Ratings

Six of nine tools achieve perfect 5/5 ratings, demonstrating generally excellent API design with a few outliers.

Methodology

Tools were tested with minimal parameters (perPage=1 where applicable) to analyze response structure rather than gather extensive data. Token counts estimated at 1 token ≈ 4 characters. Usefulness ratings based on completeness, actionability, clarity, efficiency, and relationship handling. Analysis covers 30-day rolling window with daily trend tracking.

---

Analysis Date: 2025-12-15
Tools Analyzed: 9 across 9 toolsets
Historical Data: 138 measurements over 20 days

AI generated by GitHub MCP Structural Analysis

[github-actions[bot]]

⚓ Avast! This discussion be marked as outdated by GitHub MCP Structural Analysis.
🗺️ A newer treasure map awaits ye at Discussion #6621.
Fair winds, matey! 🏴‍☠️