fix(python): add org_id to prompt cache key for cross-org isolation #1268
Conversation
The prompt cache was using a key format of `{project_name}:{slug}:{version}`
which did not include the organization ID. This caused a cross-org cache
collision bug where two organizations with the same project name and prompt
slug could retrieve each other's cached prompts.
This was reported by HubSpot (Pylon case #10589) where a UI-based Python
scorer returned a "legal-related prompt" instead of their SDC coverage
prompt because the disk cache returned a prompt from a different org.
Changes:
- Add `org_id` parameter to `_create_cache_key()` function
- Update cache key format to `{org_id}:{prefix}:{slug}:{version}` when
org_id is available
- Add `org_id` parameter to `PromptCache.get()` and `PromptCache.set()`
- Pass `_state.org_id` from `load_prompt()` to cache operations
- Add tests for cross-org cache isolation
The fix is backward compatible - existing cached prompts without org_id
in the key will simply miss and trigger a fresh API fetch.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Hey Olmo, this came up because a customer realized the wrong prompt was getting sent to OpenAI. I am attempting to fix the issue with this. I also have a repro script if that's helpful. |
|
|
||
|
|
||
| def _create_cache_key( | ||
| org_id: str | None, |
There was a problem hiding this comment.
if org_id is required to prevent cross org issues, it should be required (e.g. not none)
| """Creates a unique cache key from project identifier, slug and version, or from ID.""" | ||
| """Creates a unique cache key from org ID, project identifier, slug and version, or from prompt ID. | ||
|
|
||
| The org_id is included to ensure cache isolation between organizations. Without it, |
There was a problem hiding this comment.
i think the org_id can always be in the cache key
| try: | ||
| if id: | ||
| return _state._prompt_cache.get(id=id) | ||
| return _state._prompt_cache.get(id=id, org_id=_state.org_id) |
There was a problem hiding this comment.
you can revert this line since id will always win maybe even combine the next statement for the same reason into one
| _state._prompt_cache.set( | ||
| prompt, | ||
| id=id, | ||
| org_id=_state.org_id, |
There was a problem hiding this comment.
same here combine with the next statement
| raise ValueError("Slug must be provided when not using ID") | ||
|
|
||
| # Include org_id in cache key if available to ensure cross-org isolation | ||
| if org_id: |
There was a problem hiding this comment.
looks like login is called L1696 before updating/setting the prompt. you should be able to assume org_id exists .. perhaps raise if not set
4 participants
org_idto cache key format for proper organization isolationThe prompt cache was using a key format of
{project_name}:{slug}:{version}which did not include the organization ID. This caused a cross-org cache collision bug where two organizations with the same project name and prompt slug could retrieve each other's cached prompts.Test plan
test_handle_different_orgs_with_same_project_and_slug- verifies prompts from different orgs with same project/slug are isolatedtest_org_id_isolation_with_disk_cache- verifies isolation works after memory eviction (via disk cache)