Skip to content

feat(routing): add HTTP policy provider with revision-aware caching#846

Open
JasonOA888 wants to merge 1 commit intokatanemo:mainfrom
JasonOA888:feat/issue-812-http-policy-provider
Open

feat(routing): add HTTP policy provider with revision-aware caching#846
JasonOA888 wants to merge 1 commit intokatanemo:mainfrom
JasonOA888:feat/issue-812-http-policy-provider

Conversation

@JasonOA888
Copy link
Copy Markdown

Implements #812 - Support loading routing policy from external HTTP endpoint

Summary

  • Add PolicyProvider module with HTTP endpoint fetching and caching
  • Support policy_id + revision for revision-aware cache invalidation
  • Add extract_routing_metadata() to extract policy_id/revision from requests
  • Resolution order: inline policy > external provider > default
  • Configurable TTL and custom headers for policy endpoint

Configuration

routing:
  policy_provider:
    url: "https://my-service.internal/v1/routing-policy"
    headers:
      Authorization: "Bearer $POLICY_API_KEY"
    ttl_seconds: 300

Request Format

{
  "messages": [...],
  "policy_id": "customer-abc-123",
  "revision": 42
}

Caching Behavior

  • Cache key: policy_id with stored revision
  • If cached revision >= requested revision, use cache
  • Cache entries also expire via TTL as safety net

Flow

  1. Request comes with policy_id and revision
  2. Check local cache for policy_id
  3. If cached and revision matches, use cached policy
  4. Otherwise, fetch from HTTP endpoint
  5. Validate response (policy_id match, schema_version supported)
  6. Cache result and pass to router

Fixes #812

Implements katanemo#812 - Support loading routing policy from external HTTP endpoint

Changes:
- Add PolicyProvider module with HTTP endpoint fetching and caching
- Support policy_id + revision for revision-aware cache invalidation
- Add extract_routing_metadata() to extract policy_id/revision from requests
- Resolution order: inline policy > external provider > default
- Configurable TTL and custom headers for policy endpoint

The policy provider fetches routing preferences from an external HTTP
endpoint when policy_id is provided in the request. Policies are cached
by policy_id with revision comparison for efficient updates.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support loading routing policy from external HTTP endpoint

2 participants