🚨 CRITICAL: Models without cost configuration provide free service

[nickna]

Problem

When a model has no associated cost configuration, the system returns 0 cost and continues processing - effectively providing free service. This occurs for:

• New models not yet configured
• Misconfigured model mappings
• Models with inactive cost mappings

Evidence

• CostCalculationService.cs:87-90: Returns 0 when no model cost found
• UsageTrackingMiddleware.cs:216-220: Skips billing when cost is 0
• ModelCostService.cs:67-69: Caches null cost result for 1 hour

Impact

• Severity: CRITICAL
• Revenue Impact: 100% loss for unconfigured models
• Current State: Any model without explicit cost config runs free

Solution Requirements

Per your decision: "Effectively a model mapping is disabled until it has a cost record, even if the cost is free."

1. Enforce Cost Configuration:

   • Models MUST have an active cost configuration
   • Even free models need explicit $0 cost config
   • Reject requests for unconfigured models

2. Implementation Points:

   • Check cost availability BEFORE processing request
   • Return 503 Service Unavailable for unconfigured models
   • Clear error message indicating model configuration issue

Implementation Details

1. Update Conduit.cs or request pipeline:

public async Task<ChatCompletionResponse> CreateChatCompletionAsync(
    ChatCompletionRequest request, ...)
{
    // Validate model has cost configuration
    var modelCost = await _modelCostService.GetCostForModelAsync(request.Model);
    if (modelCost == null)
    {
        throw new ServiceUnavailableException(
            $"Model '{request.Model}' is not available. Cost configuration pending.");
    }
    
    // Continue with normal processing...
}

2. Update ModelProviderMappingService:

public async Task<bool> IsModelAvailableAsync(string modelAlias)
{
    var mapping = await GetByModelNameAsync(modelAlias);
    if (mapping == null) return false;
    
    // Check if model has active cost configuration
    var hasCost = await _modelCostRepository.HasActiveCostAsync(mapping.Id);
    return hasCost;
}

3. Add health check:

• Monitor models without cost configuration
• Alert operations team when new models lack costs

4. Migration for existing data:

• Audit all current model mappings
• Ensure all active models have cost records
• Add $0 cost records where intentionally free

Testing

• Unit tests for cost enforcement
• Integration tests with missing costs
• Verify proper error messages
• Test free models with $0 config work correctly

Breaking Change Notice

⚠️ This is a breaking change that will reject previously working requests for unconfigured models. Requires:

1. Audit of all existing models
2. Add cost configs before deployment
3. Clear communication to operations team

Priority

P0 - CRITICAL: Direct revenue impact, implement immediately

Related Issues

• Depends on model cost configuration UI/API
• Part of billing guarantee system

[nickna]

📋 Technical Investigation Report

I've conducted a comprehensive codebase investigation to verify the claims in this issue and identify all locations that would need changes if this work proceeds.

✅ Issue Verification

The core vulnerability is confirmed, though some line numbers cited are inaccurate:

Issue Claim	Actual Location	Behavior
CostCalculationService.cs:87-90	Lines 97-105	Returns 0m when modelCost == null ✅
UsageTrackingMiddleware.cs:216-220	Lines 293-305	Only updates spend when cost > 0 ✅
ModelCostService.cs:67-69	Lines 91-103	Caches results (15min memory, 1hr distributed) ✅

🔴 Files Requiring Changes

1. New Authorization Attribute (Create)

Following the pattern of RequireBalanceAttribute.cs:

• Services/ConduitLLM.Gateway/Authorization/RequireCostConfigurationAttribute.cs (NEW)

2. Gateway Controllers (Add attribute)

5 controllers handle billable operations and need cost validation:

• Services/ConduitLLM.Gateway/Controllers/ChatController.cs
• Services/ConduitLLM.Gateway/Controllers/EmbeddingsController.cs
• Services/ConduitLLM.Gateway/Controllers/ImagesController.cs (+ Sync/Async partials)
• Services/ConduitLLM.Gateway/Controllers/VideosController.cs
• Services/ConduitLLM.Gateway/Controllers/FunctionsController.cs

3. Configuration/Settings (Add new setting)

To make blocking configurable:

• Shared/ConduitLLM.Configuration/Interfaces/IGlobalSettingsCacheService.cs - Add GetRequireCostConfigurationAsync() method
• Shared/ConduitLLM.Configuration/Services/GlobalSettingsCacheService.cs - Implement method
• Database seed script for default value

4. Cost Calculation Service (Modify behavior)

Option A: Throw exception instead of returning 0

• Shared/ConduitLLM.Core/Services/CostCalculationService.cs (lines 97-105)

Option B: Add separate validation method

• Shared/ConduitLLM.Core/Interfaces/ICostCalculationService.cs - Add ValidateCostConfigurationExistsAsync()
• Shared/ConduitLLM.Core/Services/CostCalculationService.cs - Implement method

5. Model Cost Service (Optional: Don't cache null)

• Shared/ConduitLLM.Configuration/Services/ModelCostService.cs (line 95) - Consider not caching null results

6. Prometheus Metrics (Already exists, enhance)

• Services/ConduitLLM.Gateway/Middleware/UsageMetrics.cs - ZeroCostEvents counter already exists
• Consider adding alert rules in monitoring configuration

🧪 Test Files Requiring Updates

16 CostCalculation test files:

• Tests/ConduitLLM.Tests/Core/Services/CostCalculationService*.cs (16 files)

7 UsageTracking test files:

• Tests/ConduitLLM.Tests/Gateway/Middleware/UsageTrackingMiddleware*.cs (7 files)

New test files needed:

• Tests/ConduitLLM.Tests/Gateway/Authorization/RequireCostConfigurationAttributeTests.cs (NEW)

📝 Implementation Recommendations

1. Make it configurable: Add RequireCostConfiguration GlobalSetting (default: false for backwards compatibility)

2. Phased rollout:

   • Phase 1: Add observability (alerts on ZeroCostEvents metric)
   • Phase 2: Add soft-fail mode (log ERROR but allow request)
   • Phase 3: Enable blocking (return error response)

3. Status code choice: Consider 400 Bad Request or 422 Unprocessable Entity instead of 503. The issue is configuration, not service availability.

4. Error response format:

{
  "error": {
    "code": "model_cost_not_configured",
    "message": "The model 'gpt-4' does not have cost configuration. Contact your administrator.",
    "model": "gpt-4"
  }
}

⚠️ Breaking Change Considerations

• Existing deployments may have models running without cost configs (intentionally or not)
• Some free-tier/internal models may not have explicit $0 records
• Pre-deployment audit required: Query for all ModelProviderMapping records without corresponding ModelCost entries

📊 Estimated Scope

Category	Count
Controllers to modify	5
Core services to modify	2-3
Configuration files	3
New files to create	2-3
Test files to update	~23
New test files	1

---

Investigation performed using Claude Code