Debug mode

[hawkeyexl]

This PRD defines the requirements for implementing a comprehensive debug mode in Doc Detective that enables developers to pause test execution, inspect state, and control test flow for enhanced debugging capabilities. The feature will integrate seamlessly with the existing test execution pipeline while maintaining full backward compatibility.

Background & Problem Statement

Current State

Doc Detective currently executes tests in a linear, automated fashion with limited visibility into intermediate states. When tests fail, developers have minimal insight into what went wrong at each step, making debugging complex test scenarios challenging.

Problem

• Limited visibility: Developers cannot inspect test state between steps
• Difficult debugging: Failed tests provide minimal context for troubleshooting
• No execution control: Cannot pause, retry, or skip problematic steps
• Poor developer experience: Debugging requires modifying test files and re-running entire suites

Target Users

• Technical writers creating and maintaining Doc Detective tests
• QA engineers debugging complex test scenarios
• Developers integrating Doc Detective into CI/CD pipelines
• DevOps engineers troubleshooting documentation automation

Goals & Success Metrics

Primary Goals

1. Enable granular test execution control - Allow pausing at specific steps or running step-by-step
2. Improve debugging experience - Provide clear visibility into test state and execution context
3. Reduce debugging time - Enable retry/skip functionality for failed steps
4. Maintain backward compatibility - Ensure existing tests continue to work unchanged

Success Metrics

• Adoption Rate: 25% of active users enable debug mode within 3 months
• Time to Resolution: 40% reduction in average time to debug failing tests
• User Satisfaction: 90% positive feedback on debug mode usability
• Compatibility: 100% of existing test files continue to function

User Stories & Acceptance Criteria

Epic: Core Debug Mode Implementation

Story 1: Step-by-Step Execution

As a technical writer
I want to execute tests one step at a time
So that I can observe the behavior at each stage

Acceptance Criteria:

• ✅ Config option "debug": "stepThrough" pauses after every step
• ✅ CLI argument --stepThrough enables step-through mode
• ✅ Interactive prompt shows step details and allows continuation
• ✅ User can choose to continue or stop test execution
• ✅ Browser state is preserved between pauses

Story 2: Breakpoint Support

As a QA engineer
I want to set breakpoints on specific test steps
So that I can pause execution at critical points

Acceptance Criteria:

• ✅ Step schema includes "breakpoint": true/false property (default: false)
• ✅ Config option "debug": true enables breakpoint debugging
• ✅ CLI argument --debug enables breakpoint mode
• ✅ Execution pauses only at steps marked with "breakpoint": true
• ✅ Interactive prompt provides step information and control options

Story 3: Error Handling & Recovery

As a developer
I want to recover from failed test steps
So that I can continue testing without restarting the entire suite

Acceptance Criteria:

• ✅ Automatic pause when any step fails during debug mode
• ✅ Interactive prompt offers three options: Retry Step, Skip Step, Stop Test
• ✅ "Retry Step" re-executes the failed step with same parameters
• ✅ "Skip Step" marks step as skipped and continues with next step
• ✅ "Stop Test" terminates test execution gracefully
• ✅ Test context and variables are preserved during recovery actions

Story 4: State Inspection (Nice-to-Have)

As a technical writer
I want to view current test variables and context
So that I can understand the test state at any point

Acceptance Criteria:

• ✅ Debug prompt displays current environment variables
• ✅ Variable values are truncated for readability (max 50 characters)
• ✅ Step details include action type and parameters
• ✅ Clear visual formatting distinguishes different information types

Technical Requirements

Schema Updates

Configuration Schema (config_v3.schema.json)

{
  "properties": {
    "debug": {
      "type": ["boolean", "string"],
      "description": "Debug mode configuration. `true` enables breakpoint debugging, `stepThrough` pauses at every step, `false` disables debugging.",
      "enum": [true, false, "stepThrough"],
      "default": false
    }
  }
}

Step Schema (step_v3.schema.json)

{
  "properties": {
    "breakpoint": {
      "type": "boolean",
      "description": "If `true`, execution will pause at this step when debug mode is enabled.",
      "default": false
    }
  }
}

CLI Integration

New Command Line Arguments

• --debug: Enable breakpoint debugging mode
• --stepThrough: Enable step-through debugging mode
• Arguments override config file settings
• Mutually exclusive (stepThrough takes precedence)

Execution Flow Integration

Debug Manager Component

• Location: Integrate within existing test execution pipeline
• Responsibilities:
  • Determine when to pause execution
  • Display interactive prompts
  • Handle user input and control flow
  • Preserve test context during pauses

Pause Conditions

1. Breakpoints: Step has "breakpoint": true and debug: true
2. Step-through: Every step when debug: "stepThrough"
3. Failures: Any step failure when debug mode is enabled

User Interface Requirements

• Library: Use inquirer for interactive prompts
• Styling: Use chalk for colored console output
• Information Display:
  • Step ID and description
  • Action type and parameters
  • Execution status (breakpoint/failed/step-through)
  • Current environment variables (truncated)
• Action Choices:
  • Normal execution: Continue, Stop Test
  • Failed steps: Continue, Retry Step, Skip Step, Stop Test

Dependencies

New Package Dependencies

{
  "inquirer": "^9.2.0",
  "chalk": "^4.1.2"
}

Non-Functional Requirements

Performance

• Pause Response Time: < 100ms from step completion to debug prompt
• Memory Usage: No significant increase during normal (non-debug) execution
• Timeout Handling: Leverage existing 10-minute timeout window

Compatibility

• Backward Compatibility: 100% compatibility with existing test files
• Schema Versioning: Follow existing v3 schema patterns
• Default Behavior: Debug mode disabled by default

Security

• Variable Display: Truncate potentially sensitive environment variables
• Input Validation: Validate all user inputs through inquirer prompts
• Error Handling: Graceful handling of interrupted debug sessions

Usability

• Clear Visual Hierarchy: Use colors and formatting to distinguish information types
• Intuitive Controls: Self-explanatory prompt options
• Consistent Terminology: Align with existing Doc Detective vocabulary

Implementation Guidelines

Code Organization

src/
├── debug/
│   ├── DebugManager.js       # Core debug functionality
│   ├── prompts.js            # Inquirer prompt definitions
│   └── display.js            # Console output formatting
├── utils.js                  # CLI argument parsing updates
└── schemas/
    ├── config_v3.schema.json # Updated config schema
    └── step_v3.schema.json   # Updated step schema

Error Handling Strategy

• Graceful Degradation: If inquirer fails, continue without debug mode
• User Interruption: Handle Ctrl+C gracefully during debug prompts
• Invalid Input: Re-prompt user for invalid selections
• Timeout Handling: Respect existing test timeout mechanisms

Testing Strategy

• Unit Tests: Debug manager functionality and prompt handling
• Integration Tests: Debug mode with actual test execution
• Regression Tests: Ensure non-debug execution unchanged
• User Acceptance Tests: Manual testing of debug workflows

Documentation Requirements

User Documentation

• Configuration Guide: How to enable debug mode in config files
• CLI Reference: New debug command line arguments
• Debugging Tutorial: Step-by-step guide for using debug features
• Troubleshooting Guide: Common debug mode issues and solutions

Developer Documentation

• Architecture Overview: How debug mode integrates with execution pipeline
• API Reference: Debug manager interface and methods
• Extension Guide: How to add debug support to new actions

Schema Documentation

• Reference Updates: Document new debug and breakpoint properties
• Migration Guide: How to update existing tests for debug support
• Examples: Sample test files demonstrating debug features

Acceptance Criteria

Definition of Done

• ✅ All user stories implemented with acceptance criteria met
• ✅ Schema updates deployed and validated
• ✅ CLI arguments functional and documented
• ✅ Interactive debug prompts working as specified
• ✅ Error recovery flows implemented and tested
• ✅ Backward compatibility verified with existing test suite
• ✅ Documentation updated and published
• ✅ Comprehensive test coverage (>90%) for new functionality

Release Criteria

• ✅ No regressions in existing functionality
• ✅ Performance benchmarks maintained
• ✅ User acceptance testing completed successfully
• ✅ Documentation review and approval completed
• ✅ Security review passed for user input handling

Risks & Mitigation

Technical Risks

Risk	Impact	Probability	Mitigation
Integration complexity with execution pipeline	High	Medium	Thorough architecture review and incremental implementation
Performance impact during normal execution	Medium	Low	Minimal overhead design and performance testing
Browser state management during pauses	Medium	Medium	Leverage existing timeout mechanisms

Product Risks

Risk	Impact	Probability	Mitigation
User adoption lower than expected	Medium	Medium	Comprehensive documentation and tutorials
Feature complexity overwhelming users	High	Low	Progressive disclosure and clear defaults
Compatibility issues with edge cases	High	Low	Extensive regression testing

Future Considerations

Potential Enhancements (Out of Scope)

• Visual Debug Mode: Browser-based debugging interface
• Debug Session Recording: Save and replay debug sessions
• Remote Debugging: Debug tests running on remote systems
• Advanced Variable Editing: Modify variables during debug sessions
• Conditional Breakpoints: Breakpoints based on variable values

Maintenance Considerations

• Schema Evolution: Plan for future debug-related schema additions
• CLI Expansion: Framework for additional debug command line options
• Integration Points: Design for future debugging tool integrations

---

Approval Required From:

• Engineering Lead: Architecture and implementation approach
• Product Owner: Feature scope and user experience
• QA Lead: Testing strategy and acceptance criteria
• Documentation Team: Documentation requirements and standards

Implementation Timeline: 2-3 sprint cycles (4-6 weeks)
Priority: High - Developer experience improvement
Risk Level: Medium - New user-facing functionality with backward compatibility requirements