-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Description
Description
Add comprehensive integration tests to verify the complete usage tracking flow from API request through to cost calculation and spend updates.
Background
The new usage tracking middleware (UsageTrackingMiddleware) has been implemented to track actual token usage from provider responses, replacing the crude character÷4 estimation. While unit tests exist, we need integration tests to ensure the entire flow works correctly.
Acceptance Criteria
- Create integration tests that verify the complete flow from HTTP request to spend update
- Test OpenAI response format parsing and tracking
- Test Anthropic response format parsing (including cached tokens)
- Test image generation usage tracking
- Test streaming responses with usage data
- Test batch spend service integration
- Test fallback to direct database updates when batch service is unhealthy
- Verify Prometheus metrics are correctly recorded
Technical Details
- Tests should use
WebApplicationFactoryfor realistic HTTP pipeline testing - Mock external dependencies (providers, database) but keep middleware pipeline intact
- Verify both successful and error scenarios
- Test files:
SimpleUsageTrackingTest.csandUsageTrackingMiddlewareTests.csexist with basic coverage
Priority
Medium - Important for ensuring reliability but core functionality is already unit tested