Mixed “Journey” Scenarios (Agentic Chat + Deterministic REST) with OpenAPI-Aware Validation

[gusfraser]

Context

ReplicantX currently supports two scenario “levels” with YAML-driven execution (Level 1 basic, Level 2 agent). The next step is to support end-to-end user workflows that combine:

• Agentic / non-deterministic conversational steps (Replicant behaves like a human user)
• Deterministic REST steps (Replicant mimics a web/mobile app making API calls)

This capability must be generic: it should work for any product that exposes a chat-like endpoint and/or REST APIs, without requiring the product to add new “artifact” fields to responses.

Instead of asserting on assistant prose, Journey tests should validate:

1. response structure (preferably via OpenAPI schemas and JSONPath checks), and
2. system state via deterministic API calls.

---

Goals

1. Add a new scenario level: level: journey
2. Allow mixing chat and REST steps in a single steps: list.
3. Support OpenAPI ingestion to:
   • resolve endpoints by operationId
   • validate request bodies/params
   • validate response shapes when schemas exist
4. Provide robust assertions based on structured responses and downstream state, not keyword matches.
5. Be CI-friendly: deterministic where possible, bounded where not.

---

Non-goals (initial release)

• Full browser automation (Playwright/Selenium).
• Requiring target services to change response payloads (no “artifacts” requirement).
• Perfect determinism of LLM behaviour.

---

High-level design

A Journey is a step-by-step orchestration with a shared runtime variable context (vars). Steps may include:

• chat/agentic message sending
• HTTP API calls (OpenAPI operationId or method+path)
• polling (only for eventual consistency / async processes)
• extraction of IDs/values from responses
• assertions on responses and system state

Important: HTTP calls already block until they return a response. A poll step is only required when the system’s state changes asynchronously after a successful response (e.g. background jobs, eventual consistency, approvals).

---

YAML: Repo-Conventional Shape

Top-level schema (MVP)

name: "..."
level: journey

base_url: "{{ env.BASE_URL }}"           # existing convention (primary host)

# Optional: allow different base for REST calls; default = base_url host
http_base_url: "{{ env.HTTP_BASE_URL }}"

auth:
  provider: noop|jwt|supabase
  # existing auth config

replicant:
  # reuse existing Replicant config patterns (LLM settings, facts, payload_format, session_mode, etc.)
  # existing documented options should remain valid

openapi:
  source: "https://.../openapi.json"     # or "file:./openapi.json"
  validate_requests: true
  validate_responses: true

vars:
  # optional scenario inputs (can also be extracted during run)

steps:
  - ...

Variable templating

Support {{ env.* }} (already used across ReplicantX) and add {{ vars.* }} for extracted values.

Resolution priority:

1. env.*
2. vars.*
3. run.* (generated values like run.id, run.timestamp)
4. steps.<step_id>.extract.* (optional aliasing to vars)

---

Step Types (MVP)

1) Chat step (chat)

Two modes:

A. Fixed user message

- id: chat_trip_request
  chat:
    user: "Trip to Madrid on Friday returning Monday."
    expect:
      status: 200
      # If chat endpoint is in OpenAPI:
      schema: "openapi:ask:200"
      jsonpath_exists:
        - "$.message"

B. Agentic user message (Replicant generates)

- id: chat_add_hotel
  chat:
    agentic: true
    instruction: "Ask to add a hotel in Madrid near the centre for the same dates."
    expect:
      status: 200
      schema: "openapi:ask:200"
      jsonpath_exists:
        - "$.message"

Chat assertions (preferred)

Replace simplistic expect_contains with structured assertions:

• expect.status (required)
• expect.schema (OpenAPI validation, if available)
• expect.jsonpath_* checks (minimal stable checks)

expect_contains and expect_regex may remain as fallback for plain-text APIs, but should not be the recommended default for Journey tests.

Chat extraction

Allow extracting from:

• response.json (JSONPath)
• response.headers
• response.text (regex fallback)

    extract:
      conversation_id:
        from: response.headers
        key: "x-conversation-id"

---

2) HTTP step (http)

Preferred: OpenAPI operationId. Fallback: method + path.

- id: create_trip
  http:
    operationId: "createTrip"
    json:
      title: "Madrid Trip {{ run.id }}"
    expect:
      status: 201
      schema: "openapi:createTrip:201"
      jsonpath_exists:
        - "$.id"
    extract:
      trip_id:
        from: response.json
        jsonpath: "$.id"

Fallback example:

- id: submit_trip
  http:
    method: POST
    path: "/trips/{{ vars.trip_id }}/submit"
    json: {}
    expect:
      status: 200

HTTP assertions

• status
• schema (OpenAPI response schema, if defined)
• JSONPath checks to assert critical fields / invariants

---

3) Poll step (poll)

Use only when async / eventual consistency exists.

- id: poll_trip_status
  poll:
    every_seconds: 2
    max_attempts: 20
    http:
      operationId: "getTrip"
      path_params:
        trip_id: "{{ vars.trip_id }}"
    until:
      jsonpath_in:
        "$.status": ["APPROVED", "BOOKED"]

---

OpenAPI Integration

Inputs

openapi.source supports:

• URL: https://.../openapi.json
• file: file:./openapi.json

Behaviours

1. Parse OpenAPI and index:

   • operationId → {method, path, request schema, response schemas}

2. If http.operationId is used:

   • resolve method/path automatically

3. Validate requests (optional but recommended):

   • path_params, query, and json against request schema

4. Validate responses (optional but recommended):

   • validate body against the response schema for the returned status code

Schema references in YAML

Use a stable reference string:

• openapi:<operationId>:<status>
  Example:
• openapi:createTrip:201

---

When OpenAPI response schemas are incomplete

Some APIs do not declare response models fully; OpenAPI responses can be {} or missing.

Recommendation (ReplicantX optional feature)

Add an opt-in schema capture mode:

• CLI flag: --capture-schemas

• If OpenAPI response schema is missing/empty:

  • generate a lightweight JSON Schema from sample response
  • cache in .replicantx/schema-cache/<operationId>/<status>.schema.json

• On subsequent runs:

  • optionally validate responses loosely against the cached shape
  • use cached shape to power better JSONPath error messages and IDE hints

This remains generic and requires no changes to the target API.

---

Determinism strategy for mixed-mode journeys

Agentic steps

• Use bounded behaviour:

  • max turns
  • timeouts
  • optional retries (small)

• Assert primarily on shape, not exact text

• Validate outcomes via deterministic REST steps

Deterministic steps

• Prefer OpenAPI operationIds
• Assert on IDs and state transitions via JSONPath
• Use poll only for async completion

---

Reporting & Debugging (Must-have)

Per step, capture:

• step id, type
• request summary (URL/method, redacted headers, payload preview)
• response status, latency, preview
• OpenAPI validation results
• extracted variables (keys by default; values under --debug)

Redaction rules:

• redact headers: Authorization, Cookie, anything matching /token|secret|key/i
• redact JSON fields matching /password|token|secret/i

---

CLI additions (minimal)

• replicantx run <journey.yml> (same entrypoint as other levels)
• replicantx validate <journey.yml> (YAML + template sanity)
• optional: replicantx openapi pull --url ... --out ...
• optional: --capture-schemas flag

---

Example: Generic “Madrid Trip” Journey YAML (Mixed Chat + REST)

NOTE: operationIds are placeholders; replace with your API’s operationIds.

name: "Journey - Madrid trip (chat + REST)"
level: journey

base_url: "https://{{ env.REPLICANTX_TARGET }}"
http_base_url: "https://{{ env.REPLICANTX_TARGET }}"

auth:
  provider: jwt
  token: "{{ env.JWT_TOKEN }}"

openapi:
  source: "https://{{ env.REPLICANTX_TARGET }}/openapi.json"
  validate_requests: true
  validate_responses: true

replicant:
  goal: "Arrange a trip to Madrid and submit it for approval"
  facts:
    traveler_name: "Test User"
  payload_format: openai
  session_mode: auto

vars:
  depart_date: "2026-01-30"
  return_date: "2026-02-02"

steps:
  # 1) Agentic chat: request trip
  - id: chat_trip_request
    chat:
      agentic: true
      instruction: "Ask to travel to Madrid on Friday and return Monday."
      expect:
        status: 200
        schema: "openapi:ask:200"
        jsonpath_exists:
          - "$.message"
      extract:
        conversation_id:
          from: response.headers
          key: "x-conversation-id"

  # 2) REST: create a draft booking (deterministic)
  - id: create_draft_booking
    http:
      operationId: "createDraftBooking"
      json:
        destination: "Madrid"
        depart_date: "{{ vars.depart_date }}"
        return_date: "{{ vars.return_date }}"
        conversation_id: "{{ vars.conversation_id }}"
      expect:
        status: 201
        schema: "openapi:createDraftBooking:201"
        jsonpath_exists:
          - "$.id"
    extract:
      booking_id:
        from: response.json
        jsonpath: "$.id"

  # 3) REST: create trip
  - id: create_trip
    http:
      operationId: "createTrip"
      json:
        title: "Madrid Trip {{ run.id }}"
        start_date: "{{ vars.depart_date }}"
        end_date: "{{ vars.return_date }}"
      expect:
        status: 201
        schema: "openapi:createTrip:201"
        jsonpath_exists:
          - "$.id"
    extract:
      trip_id:
        from: response.json
        jsonpath: "$.id"

  # 4) REST: add booking to trip
  - id: add_booking_to_trip
    http:
      operationId: "addBookingToTrip"
      json:
        trip_id: "{{ vars.trip_id }}"
        booking_id: "{{ vars.booking_id }}"
      expect:
        status: 200
        schema: "openapi:addBookingToTrip:200"

  # 5) Agentic chat: add hotel
  - id: chat_add_hotel
    chat:
      agentic: true
      instruction: "Ask to add a hotel in Madrid near the centre for the same dates."
      expect:
        status: 200
        schema: "openapi:ask:200"
        jsonpath_exists:
          - "$.message"

  # 6) REST: submit trip
  - id: submit_trip
    http:
      operationId: "submitTrip"
      path_params:
        trip_id: "{{ vars.trip_id }}"
      expect:
        status: 200
        schema: "openapi:submitTrip:200"

  # 7) Poll only if approval/booking completion is async
  - id: poll_trip_status
    poll:
      every_seconds: 2
      max_attempts: 20
      http:
        operationId: "getTrip"
        path_params:
          trip_id: "{{ vars.trip_id }}"
      until:
        jsonpath_in:
          "$.status": ["APPROVED", "BOOKED"]

---

Acceptance Criteria

• level: journey scenarios can be executed via existing CLI (replicantx run ...) alongside basic and agent.

• A journey can alternate chat and http steps in one scenario.

• http.operationId resolves via OpenAPI and validates request/response schemas (when present).

• Structured expect block supported for chat and http:

  • status
  • schema (OpenAPI)
  • jsonpath assertions

• extract populates vars.* and templating works across steps.

• poll exists for async state transitions; no “wait” step is required for normal sync HTTP calls.

• Step-level reporting includes OpenAPI validation results, redacted logs, and clear failure localisation.

[gusfraser]

@CodeRabbit Plan

[coderabbitai]

Coding Plan

Approach

Implement a new level: journey scenario type that allows mixing agentic chat steps with deterministic REST API steps in a single YAML-driven workflow. This includes OpenAPI integration for endpoint resolution and schema validation, a shared variable context for cross-step data extraction and templating, and structured assertions based on response status, JSONPath, and OpenAPI schemas.

Observations

ReplicantX is a test scenario framework with two existing levels: basic (deterministic step-by-step API testing) and agent (AI-driven conversational testing). The architecture uses Pydantic models for configuration validation, Typer for CLI, and a factory pattern to select scenario runners based on TestLevel. Key components include BasicScenarioRunner and AgentScenarioRunner in replicantx/scenarios/, authentication providers in replicantx/auth/, HTTP utilities in replicantx/tools/, and Markdown/JSON reporters in replicantx/reporters/. The existing ReplicantAgent in replicant.py provides LLM-based message generation that can be reused for agentic chat steps.

Assumptions

Assumption 1: OpenAPI library selection

Options Considered:

1. openapi-core - integrated OpenAPI validation library with request/response validation built-in
2. pyyaml + jsonschema - parse OpenAPI manually, use jsonschema for validation (lightweight, more control)

Chosen Option: 2

Rationale: Manual parsing with jsonschema provides more control over error messages and validation behavior, aligning with the codebase's lightweight tooling approach.

Assumption 2: JSONPath library selection

Options Considered:

1. jsonpath-ng - mature library supporting full JSONPath specification with $.path syntax
2. jmespath - different query syntax (not $.path), would require documentation changes

Chosen Option: 1

Rationale: jsonpath-ng matches the $.path syntax used throughout the ticket examples and is the standard Python JSONPath implementation.

Assumption 3: Agentic chat integration approach

Options Considered:

1. Create a new lightweight message generator specifically for journey steps
2. Reuse the existing ResponseGenerator from replicant.py for single-turn message generation

Chosen Option: 2

Rationale: Reuse ResponseGenerator for consistency with existing LLM integration patterns. Journey chat steps perform single-turn generation (instruction → message) rather than full conversation loops.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: Data Models & Variable Templating

Establish the foundational data structures for journey configuration and implement the variable context system that enables cross-step data sharing. This phase creates the schema definitions that all subsequent phases depend on.

Task 1: Extend TestLevel Enum and ScenarioConfig

Add the journey level to the existing type system and extend the main configuration model to support journey-specific fields.

• Update replicantx/models.py to add JOURNEY = "journey" to the TestLevel enum
• Add optional fields to ScenarioConfig: http_base_url, openapi, vars (Dict[str, Any] for initial variables)
• Add @model_validator to enforce journey-level requirements: must have steps when level is journey
• Create OpenAPIConfig model with fields: source (str), validate_requests (bool, default True), validate_responses (bool, default True)

Task 2: Create Journey Step Models

Define Pydantic models for the three step types (chat, http, poll) and their shared components (expect, extract).

• Create models in replicantx/models.py for step configuration:
  • ExpectConfig: status (int), schema (Optional[str]), jsonpath_exists (Optional[List[str]]), jsonpath_equals (Optional[Dict[str, Any]]), jsonpath_in (Optional[Dict[str, List[Any]]]), plus legacy fallbacks contains, regex
  • ExtractConfig: from_ (Literal["response.json", "response.headers", "response.text"]), jsonpath (Optional[str]), key (Optional[str]), regex (Optional[str])
  • ChatStepConfig: user (Optional[str]), agentic (bool, default False), instruction (Optional[str]), expect (ExpectConfig), extract (Optional[Dict[str, ExtractConfig]])
  • HttpStepConfig: operationId (Optional[str]), method (Optional[str]), path (Optional[str]), path_params (Optional[Dict]), query (Optional[Dict]), json (Optional[Dict]), headers (Optional[Dict]), expect (ExpectConfig), extract (Optional[Dict[str, ExtractConfig]])
  • PollStepConfig: every_seconds (float), max_attempts (int), http (HttpStepConfig without expect), until (ExpectConfig)
  • JourneyStep: id (str), chat (Optional[ChatStepConfig]), http (Optional[HttpStepConfig]), poll (Optional[PollStepConfig]) with validator ensuring exactly one step type is specified

Task 3: Implement Variable Context and Templating Engine

Create a runtime context manager and extend the existing environment variable substitution to support {{ vars.* }} and {{ run.* }} templating.

• Create replicantx/tools/variable_context.py with VariableContext class:
  • Store env (from os.environ), vars (mutable dict initialized from config), run (dict with id, timestamp)
  • Implement resolve_template(value: str) -> str method that handles {{ env.* }}, {{ vars.* }}, {{ run.* }} with priority order
  • Implement set_var(name: str, value: Any) for extraction results
  • Support recursive resolution in dicts and lists
• Update replicantx/cli.py::substitute_env_vars() to optionally accept a VariableContext for journey scenarios, falling back to current behavior for basic/agent

🤖 Prompt for AI agents

**Goal:** Create the foundational Pydantic models for journey-level
configuration and implement the variable templating system that enables
cross-step data sharing.

**Task 1: Extend TestLevel Enum and ScenarioConfig**
- In `replicantx/models.py`, add `JOURNEY = "journey"` to the `TestLevel` enum
- Add optional fields to `ScenarioConfig`:
  - `http_base_url` (Optional[str])
  - `openapi` (Optional[OpenAPIConfig])
  - `vars` (Optional[Dict[str, Any]]) for initial variables
- Create `OpenAPIConfig` Pydantic model with:
  - `source` (str) - URL or file path
  - `validate_requests` (bool, default=True)
  - `validate_responses` (bool, default=True)
- Add `@model_validator` to `ScenarioConfig` that enforces journey-level
requirements: when `level == TestLevel.JOURNEY`, the `steps` field must be
present

**Task 2: Create Journey Step Models**
- Create the following Pydantic models in `replicantx/models.py`:
- `ExpectConfig`: Fields for `status` (int), `schema` (Optional[str]),
`jsonpath_exists` (Optional[List[str]]), `jsonpath_equals` (Optional[Dict[str,
Any]]), `jsonpath_in` (Optional[Dict[str, List[Any]]]), and legacy fields
`contains`, `regex` (both Optional[str])
- `ExtractConfig`: Fields for `from_` (Literal["response.json",
"response.headers", "response.text"]), `jsonpath` (Optional[str]), `key`
(Optional[str]), `regex` (Optional[str])
- `ChatStepConfig`: Fields for `user` (Optional[str]), `agentic` (bool,
default=False), `instruction` (Optional[str]), `expect` (ExpectConfig),
`extract` (Optional[Dict[str, ExtractConfig]])
- `HttpStepConfig`: Fields for `operationId` (Optional[str]), `method`
(Optional[str]), `path` (Optional[str]), `path_params` (Optional[Dict]), `query`
(Optional[Dict]), `json` (Optional[Dict]), `headers` (Optional[Dict]), `expect`
(ExpectConfig), `extract` (Optional[Dict[str, ExtractConfig]])
- `PollStepConfig`: Fields for `every_seconds` (float), `max_attempts` (int),
`http` (HttpStepConfig without expect field), `until` (ExpectConfig)
- `JourneyStep`: Fields for `id` (str), `chat` (Optional[ChatStepConfig]),
`http` (Optional[HttpStepConfig]), `poll` (Optional[PollStepConfig]). Add a
`@model_validator` that ensures exactly one of chat/http/poll is specified

**Task 3: Implement Variable Context and Templating Engine**
- Create `replicantx/tools/variable_context.py` with `VariableContext` class:
  - Initialize with three namespaces:
    - `env`: populated from `os.environ`
    - `vars`: mutable dict initialized from config, used for cross-step data
    - `run`: dict with `id` and `timestamp` for run metadata
  - Implement `resolve_template(value: str) -> str`:
- Parse and replace `{{ env.* }}`, `{{ vars.* }}`, `{{ run.* }}` template
expressions
    - Follow priority order: vars > run > env
- Handle nested resolution (templates within dict/list structures recursively)
- Implement `set_var(name: str, value: Any)` to add/update variables in the
`vars` namespace
- Update `replicantx/cli.py::substitute_env_vars()` to optionally accept a
`VariableContext` parameter:
  - When provided (journey scenarios), use it for template resolution
- When not provided (basic/agent scenarios), fall back to current environment
variable behavior

Phase 2: OpenAPI Integration

Build the OpenAPI parsing and validation infrastructure that enables operationId resolution and schema-based request/response validation.

Task 1: Create OpenAPI Parser and Index

Implement a tool that loads OpenAPI specifications from URL or file and indexes operations by operationId for efficient lookup.

• Create replicantx/tools/openapi_client.py with OpenAPIClient class:
  • Constructor accepts source string (URL or file:./path)
  • Implement async load() method that fetches/reads and parses OpenAPI JSON/YAML
  • Build internal index: Dict[operationId, OperationInfo] where OperationInfo contains method, path, request_body_schema, response_schemas (by status code), path_parameters, query_parameters
  • Implement get_operation(operationId: str) -> OperationInfo for lookup
  • Implement resolve_path(path_template: str, path_params: Dict) -> str for path parameter substitution
  • Handle both OpenAPI 3.0 and 3.1 formats (JSON Schema differences)

Task 2: Implement Schema Validation

Add JSON Schema validation capabilities for request bodies and response bodies using the indexed OpenAPI schemas.

• Extend OpenAPIClient in replicantx/tools/openapi_client.py:
  • Implement validate_request(operationId: str, body: Dict) -> List[ValidationError] using jsonschema library
  • Implement validate_response(operationId: str, status_code: int, body: Any) -> List[ValidationError]
  • Implement parse_schema_ref(ref: str) -> Optional[Tuple[operationId, status]] for parsing openapi:operationId:status references
  • Return structured validation errors with JSONPath to the failing field
  • Handle missing schemas gracefully (return empty list if schema not defined)

🤖 Prompt for AI agents

**Goal:** Implement OpenAPI specification parsing with operationId indexing and
JSON Schema validation capabilities for requests and responses.

**Task 1: Create OpenAPI Parser and Index**
- Create `replicantx/tools/openapi_client.py` with `OpenAPIClient` class:
  - Constructor accepts `source` (str) - can be a URL or `file:./path` format
  - Implement `async load()` method:
    - Detect if source is URL (http/https) or file path (file: prefix)
    - Fetch OpenAPI spec from URL using HTTP client or read from file system
    - Parse JSON or YAML content (support both formats)
- Build internal index as `Dict[operationId, OperationInfo]` by iterating
through paths and operations
- Handle both OpenAPI 3.0 and 3.1 formats (note JSON Schema differences between
versions)
  - Define `OperationInfo` dataclass/model containing:
    - `method` (str)
    - `path` (str)
    - `request_body_schema` (Optional[Dict])
    - `response_schemas` (Dict[int, Dict]) - keyed by status code
    - `path_parameters` (List)
    - `query_parameters` (List)
- Implement `get_operation(operationId: str) -> OperationInfo` for operationId
lookup, raising error if not found
- Implement `resolve_path(path_template: str, path_params: Dict) -> str` that
substitutes `{param}` placeholders with actual values from path_params dict

**Task 2: Implement Schema Validation**
- Extend `OpenAPIClient` in `replicantx/tools/openapi_client.py`:
- Implement `validate_request(operationId: str, body: Dict) ->
List[ValidationError]`:
    - Lookup operation by operationId
    - Get request body schema
    - Use `jsonschema` library to validate body against schema
- Catch validation exceptions and convert to structured `ValidationError` list
with JSONPath to failing fields
    - Return empty list if no schema is defined (graceful handling)
- Implement `validate_response(operationId: str, status_code: int, body: Any) ->
List[ValidationError]`:
    - Lookup operation by operationId
    - Get response schema for the specific status code
    - Use `jsonschema` library to validate body against schema
    - Convert validation exceptions to structured errors with JSONPath details
    - Return empty list if no schema is defined for that status code
- Implement `parse_schema_ref(ref: str) -> Optional[Tuple[operationId,
status]]`:
    - Parse `openapi:operationId:status` reference format
    - Return tuple of (operationId, status_code) or None if format is invalid

Phase 3: Step Execution Framework

Implement the assertion engine, extraction engine, and individual step executors that process each step type in a journey.

Task 1: Implement Assertion Engine

Create a unified assertion engine that validates responses against the structured expect configuration.

• Create replicantx/scenarios/journey/assertions.py with AssertionEngine class:
  • Constructor accepts optional OpenAPIClient for schema validation
  • Implement validate(expect: ExpectConfig, response: HTTPResponse) -> List[AssertionResult]:
    • Status code check (required)
    • Schema validation via OpenAPIClient if expect.schema specified
    • JSONPath existence checks using jsonpath-ng
    • JSONPath equality checks
    • JSONPath membership checks (value in list)
    • Legacy contains/regex checks as fallback
  • Return List[AssertionResult] with detailed failure messages including actual vs expected values
  • Add new assertion types to AssertionType enum: STATUS_CODE, SCHEMA, JSONPATH_EXISTS, JSONPATH_EQUALS, JSONPATH_IN

Task 2: Implement Extraction Engine

Create an extraction engine that pulls values from responses and populates the variable context.

• Create replicantx/scenarios/journey/extraction.py with ExtractionEngine class:
  • Implement extract(extracts: Dict[str, ExtractConfig], response: HTTPResponse) -> Dict[str, Any]:
    • response.json + jsonpath: Parse response body, apply JSONPath query
    • response.headers + key: Get header value by key
    • response.text + regex: Apply regex with capture group to response text
  • Return dict of extracted key-value pairs
  • Raise descriptive errors when extraction fails (JSONPath not found, header missing, regex no match)

Task 3: Implement HTTP Step Executor

Create the executor for HTTP steps that handles both operationId and method+path modes.

• Create replicantx/scenarios/journey/executors/http_executor.py with HttpStepExecutor class:
  • Constructor accepts HTTPClient, OpenAPIClient (optional), VariableContext, AssertionEngine, ExtractionEngine
  • Implement async execute(step: HttpStepConfig, step_id: str) -> JourneyStepResult:
    • If operationId specified: resolve method/path from OpenAPIClient, substitute path_params
    • If method+path specified: use directly with variable templating
    • Apply variable templating to json body, query params, headers
    • Optionally validate request against OpenAPI schema
    • Execute HTTP request via HTTPClient
    • Run assertions via AssertionEngine
    • Extract values via ExtractionEngine, update VariableContext
    • Return step result with request/response details, assertion results, extracted vars

Task 4: Implement Chat Step Executor

Create the executor for chat steps that handles both fixed and agentic message modes.

• Create replicantx/scenarios/journey/executors/chat_executor.py with ChatStepExecutor class:
  • Constructor accepts HTTPClient, ReplicantConfig, VariableContext, AssertionEngine, ExtractionEngine, OpenAPIClient (optional)
  • Implement async execute(step: ChatStepConfig, step_id: str) -> JourneyStepResult:
    • If agentic=false: use step.user as message (with variable templating)
    • If agentic=true: use existing ResponseGenerator from replicant.py to generate message from step.instruction
    • Format payload using existing payload formatter based on replicant.payload_format
    • Send to chat endpoint (base_url)
    • Run assertions and extraction same as HTTP executor
    • Track conversation history for subsequent agentic steps
  • Reuse existing session management if configured in replicant config

Task 5: Implement Poll Step Executor

Create the executor for poll steps that wraps HTTP execution with retry logic and condition checking.

• Create replicantx/scenarios/journey/executors/poll_executor.py with PollStepExecutor class:
  • Constructor accepts HttpStepExecutor (delegates HTTP execution)
  • Implement async execute(step: PollStepConfig, step_id: str) -> JourneyStepResult:
    • Loop up to max_attempts times with every_seconds delay between attempts
    • Execute inner HTTP step (without its own assertions)
    • Check until conditions using AssertionEngine
    • If conditions met: return success with final response
    • If max attempts exhausted: return failure with last response and attempts summary
    • Track all attempts in result for debugging

🤖 Prompt for AI agents

**Goal:** Build the assertion and extraction engines, then implement executors
for each step type (HTTP, chat, poll) that orchestrate validation and data
extraction.

**Task 1: Implement Assertion Engine**
- Create `replicantx/scenarios/journey/assertions.py` with `AssertionEngine`
class:
  - Constructor accepts optional `OpenAPIClient` for schema-based validation
- Implement `validate(expect: ExpectConfig, response: HTTPResponse) ->
List[AssertionResult]`:
    - Always check status code matches `expect.status` (required assertion)
- If `expect.schema` is specified: parse schema reference and validate response
body via `OpenAPIClient.validate_response()`
- If `expect.jsonpath_exists` is specified: use `jsonpath-ng` to check each path
exists in response JSON
- If `expect.jsonpath_equals` is specified: use `jsonpath-ng` to extract values
and compare for equality
- If `expect.jsonpath_in` is specified: use `jsonpath-ng` to extract values and
check membership in expected list
    - Fall back to legacy `contains`/`regex` checks if specified
- Collect all assertion results with detailed failure messages showing actual vs
expected values
- Update `AssertionType` enum to include: `STATUS_CODE`, `SCHEMA`,
`JSONPATH_EXISTS`, `JSONPATH_EQUALS`, `JSONPATH_IN`
- Return `List[AssertionResult]` with pass/fail status and descriptive messages

**Task 2: Implement Extraction Engine**
- Create `replicantx/scenarios/journey/extraction.py` with `ExtractionEngine`
class:
- Implement `extract(extracts: Dict[str, ExtractConfig], response: HTTPResponse)
-> Dict[str, Any]`:
    - Iterate through each extraction config
- If `from_ == "response.json"` and `jsonpath` is specified: parse response body
as JSON, apply JSONPath query using `jsonpath-ng`, extract matched value
- If `from_ == "response.headers"` and `key` is specified: get header value by
key from response headers
- If `from_ == "response.text"` and `regex` is specified: apply regex pattern
with capture group to response text, extract matched group
- Raise descriptive errors when extraction fails (JSONPath not found, header
missing, regex no match)
    - Return dict mapping variable names to extracted values

**Task 3: Implement HTTP Step Executor**
- Create `replicantx/scenarios/journey/executors/http_executor.py` with
`HttpStepExecutor` class:
- Constructor accepts: `HTTPClient`, `OpenAPIClient` (optional),
`VariableContext`, `AssertionEngine`, `ExtractionEngine`
- Implement `async execute(step: HttpStepConfig, step_id: str) ->
JourneyStepResult`:
    - Determine method and path:
- If `step.operationId` is specified: call `OpenAPIClient.get_operation()` to
resolve method/path, then use `OpenAPIClient.resolve_path()` to substitute path
parameters
      - If `step.method` and `step.path` are specified: use directly
- Apply variable templating via `VariableContext.resolve_template()` to json
body, query params, headers
- If OpenAPI validation is enabled and operationId is available: validate
request body via `OpenAPIClient.validate_request()`
- Execute HTTP request using `HTTPClient` with resolved method, path, and
parameters
    - Run assertions using `AssertionEngine.validate(step.expect, response)`
- If `step.extract` is specified: extract values using
`ExtractionEngine.extract()`, then update `VariableContext` with extracted
values
- Return `JourneyStepResult` containing request summary, response summary,
assertion results, and extracted variable keys

**Task 4: Implement Chat Step Executor**
- Create `replicantx/scenarios/journey/executors/chat_executor.py` with
`ChatStepExecutor` class:
- Constructor accepts: `HTTPClient`, `ReplicantConfig`, `VariableContext`,
`AssertionEngine`, `ExtractionEngine`, `OpenAPIClient` (optional)
- Implement `async execute(step: ChatStepConfig, step_id: str) ->
JourneyStepResult`:
    - Determine user message:
- If `step.agentic == False`: use `step.user` directly, applying variable
templating via `VariableContext`
- If `step.agentic == True`: use existing `ResponseGenerator` from
`replicant.py` to generate message from `step.instruction`
- Format payload using existing payload formatter based on
`replicant.payload_format` from config
    - Send request to chat endpoint (`base_url` from replicant config)
    - Run assertions using `AssertionEngine.validate(step.expect, response)`
- If `step.extract` is specified: extract values using
`ExtractionEngine.extract()`, update `VariableContext`
    - Track conversation history internally for subsequent agentic steps
    - Reuse existing session management if configured in replicant config
- Return `JourneyStepResult` with request/response details, assertion results,
and extracted vars

**Task 5: Implement Poll Step Executor**
- Create `replicantx/scenarios/journey/executors/poll_executor.py` with
`PollStepExecutor` class:
  - Constructor accepts `HttpStepExecutor` for delegating HTTP execution
- Implement `async execute(step: PollStepConfig, step_id: str) ->
JourneyStepResult`:
    - Initialize attempt counter and results list
    - Loop up to `step.max_attempts` times:
- Execute inner HTTP step via `HttpStepExecutor` (disable the inner step's own
expect assertions)
- Check `step.until` conditions using `AssertionEngine.validate(step.until,
response)`
- If all conditions are met: return success with final response and attempt
count
      - If conditions not met: wait `step.every_seconds` before next attempt
- If max attempts exhausted without success: return failure with last response
and summary of all attempts
- Track all attempt details in `JourneyStepResult.poll_attempts` for debugging

Phase 4: Journey Runner & Integration

Create the main journey orchestrator, integrate with CLI, and update reporters to handle journey-specific output.

Task 1: Implement JourneyScenarioRunner

Create the main runner that orchestrates step execution with shared context.

• Create replicantx/scenarios/journey/runner.py with JourneyScenarioRunner class:
  • Inherit patterns from existing BasicScenarioRunner and AgentScenarioRunner
  • Constructor accepts ScenarioConfig, debug, watch, verbose flags
  • Implement async run() -> ScenarioReport:
    • Initialize auth provider using existing factory pattern
    • Create HTTPClient(s): one for base_url, optionally another for http_base_url
    • Initialize VariableContext with config vars and run metadata
    • Load OpenAPI spec if configured via OpenAPIClient
    • Initialize executors: ChatStepExecutor, HttpStepExecutor, PollStepExecutor
    • Execute steps sequentially, passing shared context
    • Stop on first failure (fail-fast) or collect all results based on config
    • Generate ScenarioReport with all step results
• Export runner from replicantx/scenarios/__init__.py

Task 2: Create Journey Step Result Model

Extend the result models to capture journey-specific execution details.

• Add to replicantx/models.py:
  • JourneyStepResult extending StepResult with: step_id, step_type (chat/http/poll), request_summary (method, url, redacted headers), response_summary (status, latency, body preview), openapi_validation_results, extracted_vars (keys only by default), poll_attempts (for poll steps)
  • Update ScenarioReport to handle List[JourneyStepResult] alongside existing List[StepResult]
• Implement redaction utility in replicantx/tools/redaction.py:
  • Redact headers matching Authorization, Cookie, /token|secret|key/i
  • Redact JSON fields matching /password|token|secret/i

Task 3: Integrate with CLI

Wire the journey level into the existing CLI dispatch logic.

• Update replicantx/cli.py:
  • Add journey case to runner selection in run_scenarios_sequential() and _execute_scenario():

    elif config.level == TestLevel.JOURNEY:
        runner = JourneyScenarioRunner(config, debug=debug, watch=watch, verbose=verbose)
    

  • Ensure validate command handles journey YAML validation (Pydantic does this automatically)
  • No new CLI flags needed for MVP (existing flags apply)

Task 4: Update Reporters for Journey Steps

Extend both Markdown and JSON reporters to properly format journey step results.

• Update replicantx/reporters/markdown.py:
  • Add journey step formatting showing: step id, type, request summary (method/URL), response status/latency
  • Include OpenAPI validation results (pass/fail with details)
  • Show extracted variable keys (values only in debug mode)
  • Format poll step attempts as sub-steps
  • Use existing emoji conventions (✅/❌)
• Update replicantx/reporters/json.py:
  • Add serialization for JourneyStepResult fields
  • Include full OpenAPI validation details in structured format
  • Include extracted vars (keys always, values in debug mode)

🤖 Prompt for AI agents

**Goal:** Create the main journey orchestration runner, integrate it with the
CLI, define journey-specific result models with redaction, and update reporters
to format journey step output.

**Task 1: Implement JourneyScenarioRunner**
- Create `replicantx/scenarios/journey/runner.py` with `JourneyScenarioRunner`
class:
- Follow patterns from existing `BasicScenarioRunner` and `AgentScenarioRunner`
- Constructor accepts: `ScenarioConfig`, `debug` (bool), `watch` (bool),
`verbose` (bool)
  - Implement `async run() -> ScenarioReport`:
- Initialize auth provider using existing factory pattern from
`replicantx/auth/`
    - Create `HTTPClient` for `base_url` (chat endpoint)
- If `http_base_url` is specified in config, create a second `HTTPClient` for
REST API calls
- Initialize `VariableContext` with `config.vars` (initial variables) and run
metadata (id, timestamp)
- If `config.openapi` is specified: create `OpenAPIClient`, call `await load()`
to parse and index spec
    - Initialize engines and executors:
      - Create `AssertionEngine` (with optional OpenAPIClient)
      - Create `ExtractionEngine`
- Create `HttpStepExecutor`, `ChatStepExecutor`, `PollStepExecutor` with shared
context
    - Iterate through `config.steps` sequentially:
      - Determine step type (chat/http/poll) and call appropriate executor
      - Pass shared `VariableContext` to each executor
      - Collect `JourneyStepResult` from each execution
- Implement fail-fast behavior: stop on first failure or continue based on
config
    - Generate `ScenarioReport` containing all step results
- Export `JourneyScenarioRunner` from `replicantx/scenarios/__init__.py`

**Task 2: Create Journey Step Result Model**
- Add to `replicantx/models.py`:
  - Create `JourneyStepResult` class extending `StepResult`:
    - `step_id` (str)
    - `step_type` (Literal["chat", "http", "poll"])
    - `request_summary` (dict) with method, url, redacted headers
    - `response_summary` (dict) with status, latency, body preview
    - `openapi_validation_results` (Optional[List]) for validation errors
    - `extracted_vars` (dict) - keys only by default, values in debug mode
    - `poll_attempts` (Optional[int]) for poll step retry count
- Update `ScenarioReport` to handle `List[JourneyStepResult]` alongside existing
`List[StepResult]`
- Create `replicantx/tools/redaction.py` with redaction utilities:
- Implement header redaction: redact headers matching `Authorization`, `Cookie`,
or regex pattern `/token|secret|key/i`
- Implement JSON field redaction: redact fields matching regex pattern
`/password|token|secret/i`
  - Return redacted copies of data structures

**Task 3: Integrate with CLI**
- Update `replicantx/cli.py`:
- Locate runner selection logic in `run_scenarios_sequential()` and
`_execute_scenario()` functions
  - Add journey case:
    ```python
    elif config.level == TestLevel.JOURNEY:
runner = JourneyScenarioRunner(config, debug=debug, watch=watch,
verbose=verbose)
    ```
- Verify `validate` command handles journey YAML validation (Pydantic model
validation is automatic)
- No new CLI flags needed - existing flags (debug, watch, verbose) apply to
journey level

**Task 4: Update Reporters for Journey Steps**
- Update `replicantx/reporters/markdown.py`:
  - Add journey step formatting logic:
    - Display step id, step type (chat/http/poll)
    - Show request summary: HTTP method and URL
    - Show response summary: status code and latency
- Include OpenAPI validation results with pass/fail status and detailed error
messages
- Display extracted variable keys (show values only when debug mode is enabled)
    - Format poll steps with sub-items showing individual attempt details
    - Use existing emoji conventions (✅ for pass, ❌ for fail)
- Update `replicantx/reporters/json.py`:
  - Add serialization logic for `JourneyStepResult` fields
- Include full OpenAPI validation details in structured format (validation
errors with JSONPath details)
  - Include extracted vars: always show keys, show values only in debug mode

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

**Goal:** Create the foundational Pydantic models for journey-level
configuration and implement the variable templating system that enables
cross-step data sharing.

**Task 1: Extend TestLevel Enum and ScenarioConfig**
- In `replicantx/models.py`, add `JOURNEY = "journey"` to the `TestLevel` enum
- Add optional fields to `ScenarioConfig`:
  - `http_base_url` (Optional[str])
  - `openapi` (Optional[OpenAPIConfig])
  - `vars` (Optional[Dict[str, Any]]) for initial variables
- Create `OpenAPIConfig` Pydantic model with:
  - `source` (str) - URL or file path
  - `validate_requests` (bool, default=True)
  - `validate_responses` (bool, default=True)
- Add `@model_validator` to `ScenarioConfig` that enforces journey-level
requirements: when `level == TestLevel.JOURNEY`, the `steps` field must be
present

**Task 2: Create Journey Step Models**
- Create the following Pydantic models in `replicantx/models.py`:
- `ExpectConfig`: Fields for `status` (int), `schema` (Optional[str]),
`jsonpath_exists` (Optional[List[str]]), `jsonpath_equals` (Optional[Dict[str,
Any]]), `jsonpath_in` (Optional[Dict[str, List[Any]]]), and legacy fields
`contains`, `regex` (both Optional[str])
- `ExtractConfig`: Fields for `from_` (Literal["response.json",
"response.headers", "response.text"]), `jsonpath` (Optional[str]), `key`
(Optional[str]), `regex` (Optional[str])
- `ChatStepConfig`: Fields for `user` (Optional[str]), `agentic` (bool,
default=False), `instruction` (Optional[str]), `expect` (ExpectConfig),
`extract` (Optional[Dict[str, ExtractConfig]])
- `HttpStepConfig`: Fields for `operationId` (Optional[str]), `method`
(Optional[str]), `path` (Optional[str]), `path_params` (Optional[Dict]), `query`
(Optional[Dict]), `json` (Optional[Dict]), `headers` (Optional[Dict]), `expect`
(ExpectConfig), `extract` (Optional[Dict[str, ExtractConfig]])
- `PollStepConfig`: Fields for `every_seconds` (float), `max_attempts` (int),
`http` (HttpStepConfig without expect field), `until` (ExpectConfig)
- `JourneyStep`: Fields for `id` (str), `chat` (Optional[ChatStepConfig]),
`http` (Optional[HttpStepConfig]), `poll` (Optional[PollStepConfig]). Add a
`@model_validator` that ensures exactly one of chat/http/poll is specified

**Task 3: Implement Variable Context and Templating Engine**
- Create `replicantx/tools/variable_context.py` with `VariableContext` class:
  - Initialize with three namespaces:
    - `env`: populated from `os.environ`
    - `vars`: mutable dict initialized from config, used for cross-step data
    - `run`: dict with `id` and `timestamp` for run metadata
  - Implement `resolve_template(value: str) -> str`:
- Parse and replace `{{ env.* }}`, `{{ vars.* }}`, `{{ run.* }}` template
expressions
    - Follow priority order: vars > run > env
- Handle nested resolution (templates within dict/list structures recursively)
- Implement `set_var(name: str, value: Any)` to add/update variables in the
`vars` namespace
- Update `replicantx/cli.py::substitute_env_vars()` to optionally accept a
`VariableContext` parameter:
  - When provided (journey scenarios), use it for template resolution
- When not provided (basic/agent scenarios), fall back to current environment
variable behavior
===============================================================================

Task: 2

**Goal:** Implement OpenAPI specification parsing with operationId indexing and
JSON Schema validation capabilities for requests and responses.

**Task 1: Create OpenAPI Parser and Index**
- Create `replicantx/tools/openapi_client.py` with `OpenAPIClient` class:
  - Constructor accepts `source` (str) - can be a URL or `file:./path` format
  - Implement `async load()` method:
    - Detect if source is URL (http/https) or file path (file: prefix)
    - Fetch OpenAPI spec from URL using HTTP client or read from file system
    - Parse JSON or YAML content (support both formats)
- Build internal index as `Dict[operationId, OperationInfo]` by iterating
through paths and operations
- Handle both OpenAPI 3.0 and 3.1 formats (note JSON Schema differences between
versions)
  - Define `OperationInfo` dataclass/model containing:
    - `method` (str)
    - `path` (str)
    - `request_body_schema` (Optional[Dict])
    - `response_schemas` (Dict[int, Dict]) - keyed by status code
    - `path_parameters` (List)
    - `query_parameters` (List)
- Implement `get_operation(operationId: str) -> OperationInfo` for operationId
lookup, raising error if not found
- Implement `resolve_path(path_template: str, path_params: Dict) -> str` that
substitutes `{param}` placeholders with actual values from path_params dict

**Task 2: Implement Schema Validation**
- Extend `OpenAPIClient` in `replicantx/tools/openapi_client.py`:
- Implement `validate_request(operationId: str, body: Dict) ->
List[ValidationError]`:
    - Lookup operation by operationId
    - Get request body schema
    - Use `jsonschema` library to validate body against schema
- Catch validation exceptions and convert to structured `ValidationError` list
with JSONPath to failing fields
    - Return empty list if no schema is defined (graceful handling)
- Implement `validate_response(operationId: str, status_code: int, body: Any) ->
List[ValidationError]`:
    - Lookup operation by operationId
    - Get response schema for the specific status code
    - Use `jsonschema` library to validate body against schema
    - Convert validation exceptions to structured errors with JSONPath details
    - Return empty list if no schema is defined for that status code
- Implement `parse_schema_ref(ref: str) -> Optional[Tuple[operationId,
status]]`:
    - Parse `openapi:operationId:status` reference format
    - Return tuple of (operationId, status_code) or None if format is invalid
===============================================================================

Task: 3

**Goal:** Build the assertion and extraction engines, then implement executors
for each step type (HTTP, chat, poll) that orchestrate validation and data
extraction.

**Task 1: Implement Assertion Engine**
- Create `replicantx/scenarios/journey/assertions.py` with `AssertionEngine`
class:
  - Constructor accepts optional `OpenAPIClient` for schema-based validation
- Implement `validate(expect: ExpectConfig, response: HTTPResponse) ->
List[AssertionResult]`:
    - Always check status code matches `expect.status` (required assertion)
- If `expect.schema` is specified: parse schema reference and validate response
body via `OpenAPIClient.validate_response()`
- If `expect.jsonpath_exists` is specified: use `jsonpath-ng` to check each path
exists in response JSON
- If `expect.jsonpath_equals` is specified: use `jsonpath-ng` to extract values
and compare for equality
- If `expect.jsonpath_in` is specified: use `jsonpath-ng` to extract values and
check membership in expected list
    - Fall back to legacy `contains`/`regex` checks if specified
- Collect all assertion results with detailed failure messages showing actual vs
expected values
- Update `AssertionType` enum to include: `STATUS_CODE`, `SCHEMA`,
`JSONPATH_EXISTS`, `JSONPATH_EQUALS`, `JSONPATH_IN`
- Return `List[AssertionResult]` with pass/fail status and descriptive messages

**Task 2: Implement Extraction Engine**
- Create `replicantx/scenarios/journey/extraction.py` with `ExtractionEngine`
class:
- Implement `extract(extracts: Dict[str, ExtractConfig], response: HTTPResponse)
-> Dict[str, Any]`:
    - Iterate through each extraction config
- If `from_ == "response.json"` and `jsonpath` is specified: parse response body
as JSON, apply JSONPath query using `jsonpath-ng`, extract matched value
- If `from_ == "response.headers"` and `key` is specified: get header value by
key from response headers
- If `from_ == "response.text"` and `regex` is specified: apply regex pattern
with capture group to response text, extract matched group
- Raise descriptive errors when extraction fails (JSONPath not found, header
missing, regex no match)
    - Return dict mapping variable names to extracted values

**Task 3: Implement HTTP Step Executor**
- Create `replicantx/scenarios/journey/executors/http_executor.py` with
`HttpStepExecutor` class:
- Constructor accepts: `HTTPClient`, `OpenAPIClient` (optional),
`VariableContext`, `AssertionEngine`, `ExtractionEngine`
- Implement `async execute(step: HttpStepConfig, step_id: str) ->
JourneyStepResult`:
    - Determine method and path:
- If `step.operationId` is specified: call `OpenAPIClient.get_operation()` to
resolve method/path, then use `OpenAPIClient.resolve_path()` to substitute path
parameters
      - If `step.method` and `step.path` are specified: use directly
- Apply variable templating via `VariableContext.resolve_template()` to json
body, query params, headers
- If OpenAPI validation is enabled and operationId is available: validate
request body via `OpenAPIClient.validate_request()`
- Execute HTTP request using `HTTPClient` with resolved method, path, and
parameters
    - Run assertions using `AssertionEngine.validate(step.expect, response)`
- If `step.extract` is specified: extract values using
`ExtractionEngine.extract()`, then update `VariableContext` with extracted
values
- Return `JourneyStepResult` containing request summary, response summary,
assertion results, and extracted variable keys

**Task 4: Implement Chat Step Executor**
- Create `replicantx/scenarios/journey/executors/chat_executor.py` with
`ChatStepExecutor` class:
- Constructor accepts: `HTTPClient`, `ReplicantConfig`, `VariableContext`,
`AssertionEngine`, `ExtractionEngine`, `OpenAPIClient` (optional)
- Implement `async execute(step: ChatStepConfig, step_id: str) ->
JourneyStepResult`:
    - Determine user message:
- If `step.agentic == False`: use `step.user` directly, applying variable
templating via `VariableContext`
- If `step.agentic == True`: use existing `ResponseGenerator` from
`replicant.py` to generate message from `step.instruction`
- Format payload using existing payload formatter based on
`replicant.payload_format` from config
    - Send request to chat endpoint (`base_url` from replicant config)
    - Run assertions using `AssertionEngine.validate(step.expect, response)`
- If `step.extract` is specified: extract values using
`ExtractionEngine.extract()`, update `VariableContext`
    - Track conversation history internally for subsequent agentic steps
    - Reuse existing session management if configured in replicant config
- Return `JourneyStepResult` with request/response details, assertion results,
and extracted vars

**Task 5: Implement Poll Step Executor**
- Create `replicantx/scenarios/journey/executors/poll_executor.py` with
`PollStepExecutor` class:
  - Constructor accepts `HttpStepExecutor` for delegating HTTP execution
- Implement `async execute(step: PollStepConfig, step_id: str) ->
JourneyStepResult`:
    - Initialize attempt counter and results list
    - Loop up to `step.max_attempts` times:
- Execute inner HTTP step via `HttpStepExecutor` (disable the inner step's own
expect assertions)
- Check `step.until` conditions using `AssertionEngine.validate(step.until,
response)`
- If all conditions are met: return success with final response and attempt
count
      - If conditions not met: wait `step.every_seconds` before next attempt
- If max attempts exhausted without success: return failure with last response
and summary of all attempts
- Track all attempt details in `JourneyStepResult.poll_attempts` for debugging
===============================================================================

Task: 4

**Goal:** Create the main journey orchestration runner, integrate it with the
CLI, define journey-specific result models with redaction, and update reporters
to format journey step output.

**Task 1: Implement JourneyScenarioRunner**
- Create `replicantx/scenarios/journey/runner.py` with `JourneyScenarioRunner`
class:
- Follow patterns from existing `BasicScenarioRunner` and `AgentScenarioRunner`
- Constructor accepts: `ScenarioConfig`, `debug` (bool), `watch` (bool),
`verbose` (bool)
  - Implement `async run() -> ScenarioReport`:
- Initialize auth provider using existing factory pattern from
`replicantx/auth/`
    - Create `HTTPClient` for `base_url` (chat endpoint)
- If `http_base_url` is specified in config, create a second `HTTPClient` for
REST API calls
- Initialize `VariableContext` with `config.vars` (initial variables) and run
metadata (id, timestamp)
- If `config.openapi` is specified: create `OpenAPIClient`, call `await load()`
to parse and index spec
    - Initialize engines and executors:
      - Create `AssertionEngine` (with optional OpenAPIClient)
      - Create `ExtractionEngine`
- Create `HttpStepExecutor`, `ChatStepExecutor`, `PollStepExecutor` with shared
context
    - Iterate through `config.steps` sequentially:
      - Determine step type (chat/http/poll) and call appropriate executor
      - Pass shared `VariableContext` to each executor
      - Collect `JourneyStepResult` from each execution
- Implement fail-fast behavior: stop on first failure or continue based on
config
    - Generate `ScenarioReport` containing all step results
- Export `JourneyScenarioRunner` from `replicantx/scenarios/__init__.py`

**Task 2: Create Journey Step Result Model**
- Add to `replicantx/models.py`:
  - Create `JourneyStepResult` class extending `StepResult`:
    - `step_id` (str)
    - `step_type` (Literal["chat", "http", "poll"])
    - `request_summary` (dict) with method, url, redacted headers
    - `response_summary` (dict) with status, latency, body preview
    - `openapi_validation_results` (Optional[List]) for validation errors
    - `extracted_vars` (dict) - keys only by default, values in debug mode
    - `poll_attempts` (Optional[int]) for poll step retry count
- Update `ScenarioReport` to handle `List[JourneyStepResult]` alongside existing
`List[StepResult]`
- Create `replicantx/tools/redaction.py` with redaction utilities:
- Implement header redaction: redact headers matching `Authorization`, `Cookie`,
or regex pattern `/token|secret|key/i`
- Implement JSON field redaction: redact fields matching regex pattern
`/password|token|secret/i`
  - Return redacted copies of data structures

**Task 3: Integrate with CLI**
- Update `replicantx/cli.py`:
- Locate runner selection logic in `run_scenarios_sequential()` and
`_execute_scenario()` functions
  - Add journey case:
    ```python
    elif config.level == TestLevel.JOURNEY:
runner = JourneyScenarioRunner(config, debug=debug, watch=watch,
verbose=verbose)
    ```
- Verify `validate` command handles journey YAML validation (Pydantic model
validation is automatic)
- No new CLI flags needed - existing flags (debug, watch, verbose) apply to
journey level

**Task 4: Update Reporters for Journey Steps**
- Update `replicantx/reporters/markdown.py`:
  - Add journey step formatting logic:
    - Display step id, step type (chat/http/poll)
    - Show request summary: HTTP method and URL
    - Show response summary: status code and latency
- Include OpenAPI validation results with pass/fail status and detailed error
messages
- Display extracted variable keys (show values only when debug mode is enabled)
    - Format poll steps with sub-items showing individual attempt details
    - Use existing emoji conventions (✅ for pass, ❌ for fail)
- Update `replicantx/reporters/json.py`:
  - Add serialization logic for `JourneyStepResult` fields
- Include full OpenAPI validation details in structured format (validation
errors with JSONPath details)
  - Include extracted vars: always show keys, show values only in debug mode

💡 Iterate on the plan with: @coderabbitai <feedback>

Example Feedback
- `@coderabbitai` You can skip phase 3. Add a simple unit test case for phase 2.
- `@coderabbitai` For assumption 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!