feat: add validation and prompt generation utilities #1483

neubig · 2025-12-22T16:19:04Z

Summary

This PR adds validation and prompt generation utilities for skills, following the AgentSkills standard.

Depends on PR #1482 (resource directories support)

Changes

1. Enhanced `SkillValidationError`

Now includes a list of specific errors:

from openhands.sdk.context.skills import SkillValidationError

try:
    # ... validation code
except SkillValidationError as e:
    print(f"Validation failed: {e}")
    for error in e.errors:
        print(f"  - {error}")

2. New `validate_skill()` Function

Validates a skill directory according to AgentSkills spec:

from openhands.sdk.context.skills import validate_skill

errors = validate_skill(Path("skills/my-skill"))
if errors:
    print("Validation errors:")
    for error in errors:
        print(f"  - {error}")
else:
    print("Skill is valid!")

Validates:

Directory exists
SKILL.md file present
Skill name format (lowercase alphanumeric with hyphens)
Description length (max 1024 chars)
Frontmatter structure (mcp_tools, triggers, inputs)
.mcp.json validity if present

3. New `to_prompt()` Function

Generates XML prompt block for available skills:

from openhands.sdk.context.skills import Skill, to_prompt

skills = [
    Skill(name="pdf-tools", content="...", description="Extract text from PDFs."),
    Skill(name="code-review", content="...", description="Review code for bugs."),
]

prompt = to_prompt(skills)
print(prompt)

Output:

<available_skills>
  <skill name="pdf-tools">Extract text from PDFs.</skill>
  <skill name="code-review">Review code for bugs.</skill>
</available_skills>

Features:

Uses description field if available
Falls back to first content line (skipping headers)
Escapes XML special characters
Truncates long descriptions to 200 chars

Backward Compatibility

All changes are backward compatible
SkillValidationError still works with single message
New functions are additive

Testing

Added 24 new tests covering:

SkillValidationError with errors list
validate_skill() for various validation scenarios
to_prompt() for XML generation

Related Issues

Closes #1478

Part of #1473 (Support AgentSkills standard)

Diff from Previous PR

View changes from PR #1482

@neubig can click here to continue refining the PR

Agent Server images for this PR

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Architectures	Base Image	Docs / Tags
java	amd64, arm64	`eclipse-temurin:17-jdk`	Link
python	amd64, arm64	`nikolaik/python-nodejs:python3.12-nodejs22`	Link
golang	amd64, arm64	`golang:1.21-bookworm`	Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:4194530-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-4194530-python \
  ghcr.io/openhands/agent-server:4194530-python

All tags pushed for this build

ghcr.io/openhands/agent-server:4194530-golang-amd64
ghcr.io/openhands/agent-server:4194530-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:4194530-golang-arm64
ghcr.io/openhands/agent-server:4194530-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:4194530-java-amd64
ghcr.io/openhands/agent-server:4194530-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:4194530-java-arm64
ghcr.io/openhands/agent-server:4194530-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:4194530-python-amd64
ghcr.io/openhands/agent-server:4194530-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:4194530-python-arm64
ghcr.io/openhands/agent-server:4194530-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:4194530-golang
ghcr.io/openhands/agent-server:4194530-java
ghcr.io/openhands/agent-server:4194530-python

About Multi-Architecture Support

Each variant tag (e.g., 4194530-python) is a multi-arch manifest supporting both amd64 and arm64
Docker automatically pulls the correct architecture for your platform
Individual architecture tags (e.g., 4194530-python-amd64) are also available if needed

Add support for AgentSkills standard fields (https://agentskills.io/specification): - description: Brief description of what the skill does - license: License under which the skill is distributed - compatibility: Environment requirements or compatibility notes - metadata: Arbitrary key-value metadata for extensibility - allowed_tools: List of pre-approved tools for the skill Also adds skills-ref as an optional dependency for future validation and prompt generation utilities. Closes #1474 Co-authored-by: openhands <openhands@all-hands.dev>

The skills-ref library will be added when validation and prompt generation utilities are implemented (issue #1478). Co-authored-by: openhands <openhands@all-hands.dev>

- Add find_skill_md() function to locate SKILL.md files (case-insensitive) - Add validate_skill_name() function for AgentSkills spec validation - Update load_skills_from_dir() to support skill-name/SKILL.md directories - Add directory_name and validate_name parameters to Skill.load() - Export new functions from __init__.py - Add 27 unit tests for new functionality Closes #1475 Co-authored-by: openhands <openhands@all-hands.dev>

- Add find_mcp_config() function to locate .mcp.json files - Add expand_mcp_variables() for variable expansion (, default) - Add load_mcp_config() to parse and validate .mcp.json files - Add mcp_config_path field to Skill model - Update Skill.load() to auto-load .mcp.json from SKILL.md directories - .mcp.json takes precedence over mcp_tools frontmatter - Support variable expansion - Export new functions from __init__.py - Add 19 unit tests for new functionality Closes #1476 Co-authored-by: openhands <openhands@all-hands.dev>

- Add SkillResources model to track resource directories - Add discover_skill_resources() function to scan for resources - Add RESOURCE_DIRECTORIES constant for standard directory names - Add resources field to Skill model - Update Skill.load() to auto-discover resources from SKILL.md directories - Export new classes and functions from __init__.py - Add 21 unit tests for new functionality Closes #1477 Co-authored-by: openhands <openhands@all-hands.dev>

- Enhance SkillValidationError with errors list - Add validate_skill() function for skill directory validation - Add to_prompt() function for XML prompt generation - Export new functions from __init__.py - Add 24 unit tests for new functionality Closes #1478 Co-authored-by: openhands <openhands@all-hands.dev>

github-actions · 2025-12-22T16:21:49Z

Coverage Report •

File	Stmts	Miss	Cover	Missing
openhands-sdk/openhands/sdk/context/skills
exceptions.py	10	5	50%	15–16, 19–21
skill.py	504	409	18%	64, 68–69, 73–74, 78–79, 91–96, 113, 115–119, 121, 137–140, 142–146, 159, 161–163, 165–166, 168–169, 174–175, 177, 189–194, 214–215, 218, 221, 223–225, 228–233, 235, 237–238, 257, 259–265, 267–268, 273–275, 278, 281–284, 286, 303–304, 307–309, 312–315, 318–320, 323–326, 329–330, 333–335, 340–342, 345–347, 350–352, 354–355, 358–363, 365, 387–388, 390–391, 393–394, 396–397, 399–402, 404–408, 413, 522–528, 534–538, 544–551, 579, 581–582, 584–585, 587, 598–599, 601–604, 607, 610–612, 617–622, 624–625, 631–635, 638, 640–641, 643, 665–667, 670–671, 675, 677–680, 683, 685, 711, 714, 722, 727–729, 735, 737–743, 746, 758–759, 772–776, 794, 796–797, 809–810, 815, 822, 832–833, 835, 841–842, 844, 853–856, 858–861, 869–871, 879–881, 896–897, 900, 902–905, 907–909, 914–916, 928–936, 949–958, 979, 983–987, 989, 1012–1013, 1015–1018, 1021–1025, 1028–1029, 1034–1035, 1040–1041, 1045–1046, 1052, 1073–1074, 1076–1079, 1081–1083, 1088–1092, 1094, 1098–1099, 1101, 1104, 1122–1123, 1125–1126, 1129, 1134–1136, 1139, 1141–1143, 1148–1152, 1154, 1159–1160, 1164, 1167, 1181–1183, 1201, 1203–1207, 1214, 1221–1225, 1230–1232, 1234, 1249, 1251, 1253–1257, 1260–1263, 1297, 1299, 1301–1302, 1304–1306, 1309–1312, 1315, 1317, 1320–1322, 1326–1332, 1334–1335, 1337, 1340
TOTAL	14617	6972	52%

Reduce test code by 74% while maintaining essential coverage: - test_agentskills_fields.py: 299 → 73 lines - test_skill_md_convention.py: 405 → 86 lines - test_mcp_json_config.py: 301 → 89 lines - test_resource_directories.py: 300 → 83 lines - test_validation_prompt.py: 332 → 101 lines Total: 1637 → 432 lines Co-authored-by: openhands <openhands@all-hands.dev>

- Merged field validators from main (_parse_allowed_tools, _convert_metadata_values) - Kept new fields (mcp_config_path, resources) from the PR branch - Updated tests to use Pydantic ValidationError for description validation - Removed redundant _parse_agentskills_fields method (functionality moved to validators) Co-authored-by: openhands <openhands@all-hands.dev>

openhands-ai · 2025-12-22T23:07:46Z

Looks like there are a few issues preventing this PR from being merged!

GitHub Actions are failing:
- Agent Server

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #1483 at branch `feat/validation-prompt-utils`

Feel free to include any additional details that might help me get this PR into a better state.

_{^{You can manage your notification settings}}

neubig · 2025-12-31T13:18:36Z

@OpenHands resolve merge conflicts with main

openhands-ai · 2025-12-31T13:18:45Z

I'm on it! neubig can track my progress at all-hands.dev

Merged changes from main branch (fd98c16) which refactored Skill.load() to use helper methods (_load_agentskills_skill, _load_legacy_openhands_skill, _create_skill_from_metadata) and added agent_skills as a third category. Key changes: - Integrated validation and prompt utilities with main's refactored structure - Made find_skill_md and validate_skill_name public functions (exported from __init__.py) - Added mcp_config_path and resources fields to Skill model - Updated _load_agentskills_skill to load MCP config and resources - Updated _create_skill_from_metadata to accept mcp_tools, mcp_config_path, resources - Updated tests to use new API (removed directory_name parameter) Co-authored-by: openhands <openhands@all-hands.dev>

openhands-ai · 2025-12-31T13:27:13Z

Summary: Merge Conflicts Resolved

I successfully resolved the merge conflicts between the feat/validation-prompt-utils branch and origin/main.

Conflicts Resolved

openhands-sdk/openhands/sdk/context/skills/skill.py - Main branch had refactored Skill.load() into helper methods (_load_agentskills_skill, _load_legacy_openhands_skill, _create_skill_from_metadata) and added agent_skills as a third category in load_skills_from_dir().
tests/sdk/context/skill/test_skill_md_convention.py - Updated imports to use public functions.

Key Integration Changes

Made utility functions public: find_skill_md() and validate_skill_name() are now exported from __init__.py (previously private _find_skill_md and _validate_skill_name)
Added new Skill model fields: mcp_config_path and resources for AgentSkills support
Updated _load_agentskills_skill(): Now loads MCP config from .mcp.json and discovers resource directories
Updated _create_skill_from_metadata(): Accepts mcp_tools, mcp_config_path, and resources parameters
Fixed tests: Removed obsolete directory_name parameter from test calls (main branch auto-detects SKILL.md files)

Verification

✅ All 88 tests in tests/sdk/context/skill/ pass
✅ Pre-commit hooks pass (ruff format, ruff lint, pycodestyle, pyright)
✅ Changes pushed to origin/feat/validation-prompt-utils

The PR #1483 is now updated and ready for review.

View full conversation

openhands-agent added 6 commits December 22, 2025 15:55

chore: remove unused agentskills optional dependency

c033966

The skills-ref library will be added when validation and prompt generation utilities are implemented (issue #1478). Co-authored-by: openhands <openhands@all-hands.dev>

openhands-agent added 2 commits December 22, 2025 16:25

Merge branch 'main' into feat/validation-prompt-utils

c633cd7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: add validation and prompt generation utilities #1483

feat: add validation and prompt generation utilities #1483

Uh oh!

neubig commented Dec 22, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Dec 22, 2025 •

edited

Loading

Uh oh!

openhands-ai bot commented Dec 22, 2025

Uh oh!

neubig commented Dec 31, 2025

Uh oh!

openhands-ai bot commented Dec 31, 2025

Uh oh!

openhands-ai bot commented Dec 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

feat: add validation and prompt generation utilities #1483

Are you sure you want to change the base?

feat: add validation and prompt generation utilities #1483

Uh oh!

Conversation

neubig commented Dec 22, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

1. Enhanced SkillValidationError

2. New validate_skill() Function

3. New to_prompt() Function

Backward Compatibility

Testing

Related Issues

Diff from Previous PR

Uh oh!

github-actions bot commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openhands-ai bot commented Dec 22, 2025

Uh oh!

neubig commented Dec 31, 2025

Uh oh!

openhands-ai bot commented Dec 31, 2025

Uh oh!

openhands-ai bot commented Dec 31, 2025

Summary: Merge Conflicts Resolved

Conflicts Resolved

Key Integration Changes

Verification

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

neubig commented Dec 22, 2025 •

edited by github-actions bot

Loading

1. Enhanced `SkillValidationError`

2. New `validate_skill()` Function

3. New `to_prompt()` Function

github-actions bot commented Dec 22, 2025 •

edited

Loading