Skip to content

[Feature] Rigorous copilot testing framework and domain-organized tool hierarchy #4

@pskeshu

Description

@pskeshu

Description

Establish rigorous testing for copilot intelligence and reorganize tools into a proper microscopy domain model.

Current State

  • 40+ tools registered via decorators
  • Basic category organization (ACQUISITION, MOVEMENT, etc.)
  • No systematic testing of copilot behavior

Requirements

1. Tool Domain Model

Reorganize tools into microscopy-specific domains:

tools/
├── sample/           # Sample operations (embryos, wells, dishes)
│   ├── detection.py
│   ├── tracking.py
│   └── state.py
├── acquisition/      # Image acquisition
│   ├── volume.py
│   ├── timelapse.py
│   └── snapshot.py
├── hardware/         # Direct hardware control
│   ├── stage.py
│   ├── optics.py
│   └── illumination.py
├── analysis/         # Image analysis
│   ├── focus.py
│   ├── detection.py
│   └── measurement.py
└── experiment/       # Experiment management
    ├── session.py
    └── workflow.py

2. Copilot Testing Framework

  • Scenario-based tests with expected tool call sequences
  • Mock hardware for offline testing
  • Evaluation metrics: task completion, tool selection accuracy
  • Regression test suite for known behaviors

3. Tool Expansion

  • Gap analysis for missing microscopy operations
  • Compound tools for common workflows
  • Better tool documentation for Claude context

Technical Approach

  • Refactor gently/agent/tools/ directory structure
  • Create tests/copilot/ with scenario tests
  • Use pytest with async support
  • Mock QueueServerClient for offline tests

Key Files

  • gently/agent/tool_registry.py
  • gently/agent/tools/*.py
  • New: tests/copilot/

Metadata

Metadata

Assignees

No one assigned

    Labels

    copilotAI copilot functionalityenhancementNew feature or requestpriority-highCritical for core functionalitytestingTesting and evaluation

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions