Skip to content

fix(e2e): improve test reliability and fix broken assertions#398

Merged
FL4TLiN3 merged 5 commits intomainfrom
fix/e2e-test-reliability
Jan 9, 2026
Merged

fix(e2e): improve test reliability and fix broken assertions#398
FL4TLiN3 merged 5 commits intomainfrom
fix/e2e-test-reliability

Conversation

@FL4TLiN3
Copy link
Contributor

@FL4TLiN3 FL4TLiN3 commented Jan 9, 2026

Summary

  • Update streaming event names to match state-machine-redesign changes (startStreamingReasoning, completeStreamingReasoning)
  • Fix lazy-init.toml to use local e2e-mcp-server path instead of private npm package
  • Add --run-id option to runtime CLI (was missing)
  • Refactor PDF/image tests to use flow-based assertions instead of flaky LLM keyword matching
  • Add --config to publish.test.ts unpublish commands
  • Add infrastructure failure detection helper for Docker tests to handle MCP connection issues gracefully
  • Support additionalVolumes in Docker runtime for local package mounting

Test plan

  • Run full E2E suite: 142 passed, 2 skipped
  • Verify streaming tests pass with new event names
  • Verify lazy-init tests work with local server path
  • Verify PDF/image tests pass reliably without keyword flakiness
  • Verify Docker tests handle infrastructure failures gracefully

🤖 Generated with Claude Code

- Update streaming event names (startStreamingReasoning, completeStreamingReasoning)
- Fix lazy-init.toml to use local e2e-mcp-server path instead of npx
- Add --run-id option to runtime CLI
- Refactor PDF/image tests to use flow-based assertions instead of flaky keyword matching
- Add --config to publish.test.ts unpublish commands
- Add infrastructure failure detection helper for Docker tests
- Support additionalVolumes in Docker runtime for local package mounting

All 142 E2E tests now pass reliably.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@codecov
Copy link

codecov bot commented Jan 9, 2026

Codecov Report

❌ Patch coverage is 46.15385% with 7 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
packages/runtimes/docker/src/compose-generator.ts 45.45% 4 Missing and 2 partials ⚠️
packages/core/src/schemas/run-command.ts 0.00% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

FL4TLiN3 and others added 4 commits January 9, 2026 09:36
- Add runId?: string to RunParamsInput for CLI --run-id option
- Add changeset for E2E test reliability fixes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@FL4TLiN3 FL4TLiN3 merged commit 0515dd9 into main Jan 9, 2026
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant