Skip to content

Conversation

@marc-romu
Copy link
Member

Description

Implements the unified model management & selection architecture (formerly Suggestion 2) with a full refactor of model registration, capability storage, and selection flow.
Key changes:

  • Centralized model management in ModelManager:
    • ModelManager.SelectBestModel(...) is the single authority for selection.
    • Capability data stored via AIModelCapabilityRegistry as storage-only.
    • Exact name and alias matching only in the registry (no wildcard resolution).
  • Provider-scoped selection:
    • New IAIProvider.SelectModel(requiredCapability, requestedModel) encapsulates selection.
    • AIProvider.SelectModel(...) delegates to ModelManager.SelectBestModel(...) while honoring provider defaults/settings.
    • AIRequestBase.GetModelToUse() now calls provider.SelectModel(...).
  • Simplified selection policy (capability-first):
    1. User-specified model (pass-through if unknown; use if known and capable)
    2. Provider-preferred default (settings), if capable
    3. Default for the requested capability
    4. Best of the remaining capable models (verified, rank, non-deprecated, name)
    • Note: removed the separate “default-compatible” tier to reduce complexity.
  • Documentation aligned with the new architecture:
    • docs/Providers/ModelSelection.md — updated selection policy.
    • docs/Architecture.md — clarified registry is exact/alias only and storage-only.
    • docs/Providers/AIModelCapabilities.md — best practices updated (no wildcards; use Aliases).
    • docs/Providers/index.md, docs/Providers/AIProvider*.md — guidance for provider-scoped selection.
    • docs/Suggestions.md — Suggestion 2 removed (fully implemented).
  • Tests and fixes:
    • ModelManagerTests and AdvancedConfigTests updated.
    • Fix: missing Image2Image capability.
    • Fix: incorrect fallback to default model when component defines a specific model.
      Touched areas (non-exhaustive):
  • src/SmartHopper.Infrastructure/AIModels/ModelManager.cs
  • src/SmartHopper.Infrastructure/AIProviders/AIProvider.cs
  • src/SmartHopper.Infrastructure/AICall/AIRequestBase.cs
  • src/SmartHopper.Infrastructure/AIModels/{AIModelCapabilities.cs, AIModelCapabilityRegistry.cs, IAIProviderModels.cs}
  • Providers: OpenAIProviderModels.cs, MistralAIProviderModels.cs, DeepSeekProviderModels.cs
  • Components and tests updated accordingly

Breaking Changes

  • Provider model retrieval must be centralized:
    • Removed legacy per-provider model retrieval/selection paths. Providers must expose models via RetrieveModels() and rely on ModelManager for selection.
    • No wildcard entries in registry; use concrete model names and Aliases for alternative names.
  • External API remains the same for end users/components; changes primarily affect provider implementations.

Testing Done

  • Updated unit tests:
    • src/SmartHopper.Infrastructure.Tests/ModelManagerTests.cs
    • src/SmartHopper.Infrastructure.Tests/AdvancedConfigTests.cs
  • Manual verification:
    • Selection respects user-specified models, provider defaults, capability validation, and tie-breakers (Verified, Rank, Deprecated, Name).
    • Components using provider “Default” and explicit models behave as expected.

Checklist

  • Focused on a single, scoped refactor (model manager and selection)
  • CHANGELOG.md updated under [Unreleased]
  • PR title follows Conventional Commits
  • Docs updated under docs/ to reflect the new behavior

Motivation

Completes the unified model management & selection design by:

  • Making ModelManager the single source of truth for capabilities and selection
  • Delegating selection to providers for proper scoping and settings awareness
  • Simplifying policy (removed default-compatible tier; no wildcards) for predictability and maintainability

Copilot AI review requested due to automatic review settings August 22, 2025 17:24
@github-actions
Copy link
Contributor

🏷️ This PR has been automatically assigned to milestone 0.6.0-alpha based on the version in Solution.props.

@github-actions github-actions bot added this to the 0.6.0-alpha milestone Aug 22, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements a comprehensive refactor of model management and selection across SmartHopper, centralizing authority in ModelManager while introducing provider-scoped selection. The changes establish a unified architecture where ModelManager serves as the single source of truth for model capabilities and selection logic, while providers expose models through a standardized RetrieveModels() interface and handle scoped selection via SelectModel().

Key changes:

  • Centralized model management with ModelManager.SelectBestModel() as the single authority for selection decisions
  • Provider-scoped selection through new IAIProvider.SelectModel() method that encapsulates model resolution while honoring provider settings
  • Simplified selection policy using capability-first ordering with concrete models only (no wildcard resolution)

Reviewed Changes

Copilot reviewed 38 out of 39 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/SmartHopper.Providers.OpenAI/OpenAIProviderModels.cs Replaced API-based model retrieval with hardcoded model definitions using new RetrieveModels() interface
src/SmartHopper.Providers.MistralAI/MistralAIProviderModels.cs Converted from dynamic API calls to static model metadata using RetrieveModels() pattern
src/SmartHopper.Providers.DeepSeek/DeepSeekProviderModels.cs Simplified to return concrete model capabilities via RetrieveModels() method
src/SmartHopper.Infrastructure/AIProviders/IAIProvider.cs Added SelectModel() method for provider-scoped model selection
src/SmartHopper.Infrastructure/AIProviders/AIProviderModels.cs Streamlined to single RetrieveModels() method, removing legacy retrieval patterns
src/SmartHopper.Infrastructure/AIProviders/AIProvider.cs Implemented SelectModel() delegation to ModelManager and updated initialization flow
src/SmartHopper.Infrastructure/AIModels/ModelManager.cs Added centralized SelectBestModel() logic and SetDefault() capability management
src/SmartHopper.Infrastructure/AIModels/IAIProviderModels.cs Simplified interface to focus on RetrieveModels() for metadata retrieval
src/SmartHopper.Infrastructure/AIModels/AIModelCapabilityRegistry.cs Removed wildcard matching, enforced exact name/alias matching with thread-safety
src/SmartHopper.Infrastructure/AIModels/AIModelCapabilities.cs Extended with verification, ranking, streaming, and alias metadata fields
src/SmartHopper.Infrastructure/AIModels/AICapability.cs Added Image2Image capability and cleaned up imports
src/SmartHopper.Infrastructure/AICall/AIRequestBase.cs Refactored GetModelToUse() to delegate selection to provider.SelectModel()

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@marc-romu marc-romu merged commit e1a1b4a into dev Aug 23, 2025
10 checks passed
@marc-romu marc-romu deleted the refactor/0.6.0-model-manager branch August 23, 2025 13:59
@marc-romu marc-romu modified the milestones: 0.6.0-alpha, 1.0.0-alpha Oct 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants