Skip to content

Add integration tests for end-to-end usage tracking #675

@nickna

Description

@nickna

Description

Add comprehensive integration tests to verify the complete usage tracking flow from API request through to cost calculation and spend updates.

Background

The new usage tracking middleware (UsageTrackingMiddleware) has been implemented to track actual token usage from provider responses, replacing the crude character÷4 estimation. While unit tests exist, we need integration tests to ensure the entire flow works correctly.

Acceptance Criteria

  • Create integration tests that verify the complete flow from HTTP request to spend update
  • Test OpenAI response format parsing and tracking
  • Test Anthropic response format parsing (including cached tokens)
  • Test image generation usage tracking
  • Test streaming responses with usage data
  • Test batch spend service integration
  • Test fallback to direct database updates when batch service is unhealthy
  • Verify Prometheus metrics are correctly recorded

Technical Details

  • Tests should use WebApplicationFactory for realistic HTTP pipeline testing
  • Mock external dependencies (providers, database) but keep middleware pipeline intact
  • Verify both successful and error scenarios
  • Test files: SimpleUsageTrackingTest.cs and UsageTrackingMiddlewareTests.cs exist with basic coverage

Priority

Medium - Important for ensuring reliability but core functionality is already unit tested

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions