Skip to content

feat: add OMOS PR1 scoring v2 and resolver foundation#4

Open
Daltonganger wants to merge 1 commit intofeat/dynamic-provider-fallback-pr3from
feat/omos-pr1-scoring-resolver
Open

feat: add OMOS PR1 scoring v2 and resolver foundation#4
Daltonganger wants to merge 1 commit intofeat/dynamic-provider-fallback-pr3from
feat/omos-pr1-scoring-resolver

Conversation

@Daltonganger
Copy link
Copy Markdown
Owner

Summary

  • add modular Scoring Engine V2 foundation under src/cli/scoring-v2 (features, weights, engine, explainable score breakdown)
  • add deterministic precedence resolver with provenance metadata and integrate it into dynamic plan compilation
  • add scoringEngineVersion flag support (v1 | v2-shadow | v2) with shadow plumbing that compares V1/V2 while still applying V1 in shadow mode
  • extend dynamic plan metadata with scoring/provenance outputs and keep existing runtime wiring backward compatible
  • add regression tests for scoring determinism, precedence resolution, shadow behavior, and config schema loading

Notes

  • this is a stacked PR targeting feat/dynamic-provider-fallback-pr3
  • manual menu and /omos UX are intentionally out of scope for this PR

Validation

  • bun test src/cli/scoring-v2/engine.test.ts src/cli/precedence-resolver.test.ts src/cli/dynamic-model-selection.test.ts src/cli/providers.test.ts src/config/loader.test.ts
  • bun run check:ci (passes with existing Biome schema-version info message)

Introduce modular V2 scoring components, shadow-mode plumbing, and deterministic precedence resolution with provenance while keeping V1 as the applied default. Adds schema flag support and regression tests for determinism and fallback compilation.
@kilo-code-bot
Copy link
Copy Markdown

kilo-code-bot bot commented Feb 7, 2026

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (9 files)
  • src/cli/config-manager.ts
  • src/cli/dynamic-model-selection.ts
  • src/cli/dynamic-model-selection.test.ts
  • src/cli/precedence-resolver.ts
  • src/cli/precedence-resolver.test.ts
  • src/cli/scoring-v2/engine.ts
  • src/cli/scoring-v2/features.ts
  • src/cli/scoring-v2/weights.ts
  • src/cli/scoring-v2/types.ts
  • src/cli/types.ts
  • src/config/loader.test.ts
  • src/config/schema.ts

Summary

This PR introduces a new scoring engine (v2) with shadow mode support for A/B testing model selection algorithms. The implementation is clean and well-tested:

  • Precedence Resolver: Clean layer-based resolution with proper fallback handling
  • Scoring V2 Engine: Feature-weighted scoring with agent-specific weight overrides
  • Shadow Mode: Allows comparing v1 vs v2 results without affecting production behavior
  • Type Safety: All new types are properly defined and exported
  • Test Coverage: Good test coverage for new functionality

The code follows project conventions (Biome formatting, TypeScript strict mode, proper exports).

@golammowela251wq
Copy link
Copy Markdown

@copilot review this with a free model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants