Extract shared OnePasswordClient into fides.common.onepassword#7698
Extract shared OnePasswordClient into fides.common.onepassword#7698
Conversation
Move the duplicated 1Password SDK integration from test helpers into a shared, instance-based client class. Each consumer (SaaS test secrets, seed profile resolution) can instantiate its own client with different tokens and vault IDs. The existing test helper module-level API is preserved as thin wrappers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
|
/code-review |
There was a problem hiding this comment.
The extraction is clean and the refactored test helper is a nice reduction in duplication. A few things to address before merging:
Critical
_get_clientand_ensure_mappingsboth have a check-then-act race condition that will cause duplicate authentication/listing calls when two coroutines call them concurrently. SinceOnePasswordClientis now infides.commonand targeted at production use (fidesplus seed profiles), this should be protected withasyncio.Lockbefore it lands.
Suggestions
- The sync wrappers (
get_secrets_sync,get_item_notes_json_sync) create a fresh event loop on every call but reuseself._client(the SDKClientobject) across those loops. If the SDK holds loop-bound connections internally, subsequent calls will fail. Consider usingasyncio.run()and resettingself._clientbetween calls, or verifying the SDK handles this gracefully. - Duplicate vault item titles are silently overwritten — a warning log would make this easier to diagnose.
Nice to have
- Unit tests for
OnePasswordClient(mock the SDKClient). - A comment on the dual-condition lookup in
get_item_notes(purpose == "NOTES"vsid == "notesPlain").
| async def _get_client(self) -> Client: | ||
| """Lazily authenticate and return the 1Password SDK client.""" | ||
| if self._client is None: | ||
| self._client = await Client.authenticate( | ||
| auth=self._token, | ||
| integration_name=self._integration_name, | ||
| integration_version=self._integration_version, | ||
| ) | ||
| return self._client |
There was a problem hiding this comment.
Critical: Race condition under concurrent async usage
Both _get_client and _ensure_mappings use a check-then-act pattern without any lock:
if self._client is None:
self._client = await Client.authenticate(...) # yields to event loop hereIf two coroutines call _get_client() concurrently, both can pass the is None check before either sets self._client. The same applies to _ensure_mappings — both coroutines could pass if self._mappings_initialized: return and then both call client.items.list(...).
Fix with an asyncio.Lock:
def __init__(self, ...):
...
self._client_lock = asyncio.Lock()
self._mappings_lock = asyncio.Lock()
async def _get_client(self) -> Client:
if self._client is None:
async with self._client_lock:
if self._client is None: # double-checked locking
self._client = await Client.authenticate(...)
return self._clientThe same pattern applies to _ensure_mappings.
|
|
||
| def get_secrets_sync(self, title: str) -> Dict[str, str]: | ||
| """Synchronous wrapper around :meth:`get_secrets`.""" | ||
| loop = asyncio.new_event_loop() | ||
| try: | ||
| return loop.run_until_complete(self.get_secrets(title)) | ||
| finally: |
There was a problem hiding this comment.
Suggestion: Reusing SDK client across event loops
Each call to get_secrets_sync creates a fresh asyncio event loop, but self._client (the underlying 1Password SDK Client) is cached after the first call. On subsequent calls the cached client — authenticated in a now-closed loop — is reused in a new loop.
Whether this works depends on the SDK's internals. If it uses aiohttp or a similar library that stores loop-specific connections, subsequent calls will fail with RuntimeError: Event loop is closed or similar.
Two options:
- Don't cache
self._clientacross sync wrapper calls (drop the lazy cache, authenticate per call). Expensive but safe. - Use
asyncio.run()instead ofnew_event_loop(), and clearself._clientinfinallyso eachasyncio.run()call starts fresh.
asyncio.run() is also the modern, recommended replacement for the manual new_event_loop()/run_until_complete()/close() pattern (available since Python 3.7).
| for item in items: | ||
| logger.debug(f"1PW: indexed item '{item.title}'") | ||
| self._title_to_id[item.title] = { | ||
| "item_id": item.id, | ||
| "category": item.category, | ||
| } |
There was a problem hiding this comment.
Suggestion: Duplicate vault item titles silently drop one
If the vault contains two items with the same title, the second silently overwrites the first in _title_to_id. Since get_item later uses this mapping, one item becomes permanently inaccessible with no indication why.
Consider adding a warning:
if item.title in self._title_to_id:
logger.warning(f"1PW: duplicate item title '{item.title}' in vault {self._vault_id}; overwriting")
self._title_to_id[item.title] = {"item_id": item.id, "category": item.category}| if not item: | ||
| return None | ||
|
|
||
| for field in item.get("fields", []): |
There was a problem hiding this comment.
Nice to have: Dual-condition lookup deserves a comment
The or field.get("id") == "notesPlain" fallback is non-obvious. A brief comment explaining why both checks exist would help future readers:
# Check "purpose" first (canonical); fall back to the well-known field ID
# used by 1Password for plain-text notes items.
if field.get("purpose") == "NOTES" or field.get("id") == "notesPlain":| """ | ||
| Instance-based 1Password client for retrieving secrets and items. | ||
|
|
||
| Provides lazy client initialization and vault item lookup by title. | ||
| Each instance is parameterized by its own service account token and vault ID, | ||
| so multiple consumers (e.g. SaaS test secrets, seed profile resolution) | ||
| can coexist with different credentials. | ||
| """ | ||
|
|
||
| import asyncio | ||
| import json | ||
| from typing import Any, Dict, List, Optional | ||
|
|
||
| from loguru import logger | ||
| from onepassword.client import Client # type: ignore[import-untyped] | ||
|
|
||
|
|
There was a problem hiding this comment.
Nice to have: No unit tests for the new class
OnePasswordClient is a new production module in fides.common but there are no corresponding tests. The 1Password SDK Client is straightforward to mock — a few tests covering get_item (found / not found), get_secrets (filters empty fields), get_item_notes (purpose vs. id fallback), and get_item_notes_json (valid JSON / invalid JSON) would give good coverage without needing a real vault.
Description Of Changes
Extract the duplicated 1Password SDK integration from test helpers into a shared, instance-based
OnePasswordClientclass infides.common.onepassword. This enables multiple consumers (SaaS test secrets, seed profile resolution in fidesplus) to each instantiate their own client with different service account tokens and vault IDs.The existing test helper module-level API (
get_secrets,get_item_by_title,list_available_items) is preserved as thin wrappers so no callers need to change.Code Changes
src/fides/common/onepassword/__init__.py- New package, exportsOnePasswordClientsrc/fides/common/onepassword/client.py- New instance-based client with lazy init,get_item(),get_secrets(),get_item_notes(),get_item_notes_json(),list_items(), plus sync wrapperstests/ops/test_helpers/onepassword_client.py- Refactored to thin wrappers aroundOnePasswordClientSteps to Confirm
from fides.common.onepassword import OnePasswordClientimports cleanlyonepassword_client.get_secrets()still work (the module-level API is preserved)OnePasswordClientcan be instantiated with different tokens/vault IDs for independent consumersPre-Merge Checklist
CHANGELOG.mdupdated🤖 Generated with Claude Code