-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Description
🚀 Describe the new functionality needed
The record-integration-tests.yml workflow currently references secrets.GOOGLE_APPLICATION_CREDENTIALS and secrets.VERTEX_AI_PROJECT / secrets.VERTEX_AI_LOCATION for the vertexai provider, but these secrets are not configured in the repository. As a result, the Vertex AI recording job silently runs without valid credentials and fails.
The fix is to adopt GCP Workload Identity Federation (OIDC-based, keyless authentication) instead of static service account credentials.
What needs to change:
-
GCP side: Set up a GCP project with a Workload Identity Pool and Provider that trusts the
llamastack/llama-stackGitHub repo's OIDC tokens. -
GitHub repo side: Add two repository secrets:
GCP_WORKLOAD_IDENTITY_PROVIDER— the full provider resource name (projects/<id>/locations/global/workloadIdentityPools/<pool>/providers/<provider>)VERTEX_AI_PROJECT— the GCP project ID (already referenced but not set)
-
Workflow changes in
record-integration-tests.yml:- Add
id-token: writepermission (required for OIDC token exchange) - Add a
google-github-actions/authstep (pinned SHA) before the test run step for thevertexaiprovider:- name: Authenticate to Google Cloud (Vertex AI) if: matrix.provider.setup == 'vertexai' uses: google-github-actions/auth@7c6bc770dae815cd3e89ee6cdf493a5fab2cc093 # v3 with: project_id: ${{ secrets.VERTEX_AI_PROJECT }} workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
- Remove the
GOOGLE_APPLICATION_CREDENTIALSsecret reference (the auth action setsGOOGLE_APPLICATION_CREDENTIALSautomatically via a generated credentials file) - Set
VERTEX_AI_LOCATIONto a hardcoded value (e.g.,global) rather than a secret, since it's not sensitive
- Add
-
Security: Fork PRs and Dependabot PRs should skip the Vertex AI auth step (OIDC tokens are not available).
💡 Why is this needed? What if we don't build it?
Without this, the vertexai provider in the recording matrix is effectively dead — it appears in the workflow but can never authenticate. This means:
- Vertex AI integration test recordings cannot be auto-generated or updated via CI
- Contributors must record Vertex AI tests manually with their own credentials
- The workflow gives a false sense of coverage by listing
vertexaias a provider
Workload Identity Federation is the recommended approach for GitHub Actions ↔ GCP auth (no long-lived keys to rotate, no secret file management).
Other thoughts
- The
vertexaiprovider is already gated behindworkflow_dispatch(not auto-triggered on PRs), so theid-token: writepermission only applies to manual runs, minimizing security surface. - The existing security model (fork PR blocking, read-only
pull_requesttrigger) is preserved. VERTEX_AI_LOCATIONshould beglobalfor Gemini models (regional endpoints don't support them).