Skip to content

[token] Reduce DB reads on tracked validations#268

Merged
mfittko merged 4 commits intomainfrom
perf/reduce-db-reads-tracked-tokens
Jan 6, 2026
Merged

[token] Reduce DB reads on tracked validations#268
mfittko merged 4 commits intomainfrom
perf/reduce-db-reads-tracked-tokens

Conversation

@mfittko
Copy link
Contributor

@mfittko mfittko commented Jan 6, 2026

Summary

Reduce unnecessary DB reads on the uncached request path by ensuring token caching applies to tracked validations, while keeping quota enforcement correct.

Changes

  • Populate token cache after successful ValidateTokenWithTracking() to avoid repeated token DB lookups.
  • For cached limited tokens, perform usage increment (quota enforcement) without an extra DB read.
  • Tighten IncrementTokenUsage DB update gating for is_active and expires_at, and return semantic errors.
  • Add/adjust tests for limited-token caching behavior and inactive/expired increment behavior.

Testing

  • make test
  • make lint

Notes

This targets proxy-side latency under concurrency for uncached calls; limited tokens still require a per-request DB write to enforce max_requests.

- Populate token cache after ValidateTokenWithTracking
- For cached limited tokens: increment usage without extra DB read
- Tighten DB IncrementTokenUsage gating (inactive/expired/quota) and return semantic errors

Testing: make test, make lint
Copilot AI review requested due to automatic review settings January 6, 2026 20:53
@mfittko mfittko self-assigned this Jan 6, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes token validation performance by extending cache usage to tracked validations (previously only used for non-tracking validation). The key insight is that for limited tokens, the cached metadata (active/expiry/project) can be reused while still enforcing max_requests through a synchronous DB write that avoids an extra DB read.

  • Caches tokens after successful tracked validation to avoid redundant DB reads on subsequent requests
  • For cached limited tokens, uses cached metadata but still calls IncrementTokenUsage for quota enforcement
  • Tightens DB-level enforcement by adding is_active and expires_at checks to the UPDATE's WHERE clause, returning semantic errors when these constraints aren't met

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
internal/token/cache.go Implements limited-token caching by calling IncrementTokenUsage directly on cache hits; replaces IsValid() with new isCacheableTokenValid() helper that excludes quota checks; populates cache after successful tracking
internal/database/token.go Adds is_active and expires_at guards to the UPDATE WHERE clause in IncrementTokenUsage; expands error-path SELECT to return ErrTokenInactive or ErrTokenExpired when appropriate
internal/token/cache_test.go Adds countingStore mock and test verifying that limited tokens hit the cache on second call with only one increment and no extra DB read
internal/token/token_integration_test.go Updates MockStore.IncrementTokenUsage to match new DB behavior (check active/expired/quota before incrementing)
internal/database/token_test.go Adds test cases verifying IncrementTokenUsage returns ErrTokenInactive and ErrTokenExpired when appropriate

- Use SQL boolean literal TRUE for is_active check (works across sqlite/mysql/postgres)
- Tighten cache test assertion per review
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

- Invalidate cached limited-token entries when IncrementTokenUsage returns rate-limit/inactive/expired
- Treat any non-NULL expires_at as authoritative (remove IsZero guard)

Testing: go test ./internal/token ./internal/database
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@mfittko mfittko merged commit dcb6e33 into main Jan 6, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants