Add --force-refresh support for Databricks CLI token fetching#1377
Add --force-refresh support for Databricks CLI token fetching#1377mihaimitrea-db wants to merge 1 commit intomainfrom
Conversation
1d03f6d to
9e82d18
Compare
Range-diff: main (1d03f6d -> 9e82d18)
Reproduce locally: |
9e82d18 to
3195f5d
Compare
Range-diff: main (9e82d18 -> 3195f5d)
Reproduce locally: |
3195f5d to
4115f37
Compare
Range-diff: main (3195f5d -> 4115f37)
Reproduce locally: |
4115f37 to
469cb44
Compare
Range-diff: main (4115f37 -> 469cb44)
Reproduce locally: |
469cb44 to
32c2cbd
Compare
Range-diff: main (469cb44 -> 32c2cbd)
Reproduce locally: |
32c2cbd to
194256c
Compare
Range-diff: main (32c2cbd -> 194256c)
Reproduce locally: |
When the SDK's cached CLI token is stale, try `databricks auth token --force-refresh` to get a freshly minted token from the IdP. If the installed CLI is too old to recognise the flag, fall back to regular `auth token` and remember the capability for future refreshes. Centralise unknown-flag detection in CliTokenSource._exec_cli_command() via UnsupportedCliFlagError so the same classifier is reused by both the legacy --profile fallback and the new --force-refresh downgrade path in DatabricksCliTokenSource. See: databricks/cli#4767
194256c to
cd6c876
Compare
Range-diff: main (194256c -> cd6c876)
Reproduce locally: |
|
If integration tests don't run automatically, an authorized user can run them manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
🥞 Stacked PR
Use this link to review incremental changes.
Summary
Pass
--force-refreshto the Databricks CLIauth tokencommand so the SDK always receives a fresh token instead of a potentially stale one from the CLI's internal cache.See: databricks/cli#4767
Why
The SDK manages its own token caching via
Refreshable. When the SDK decides it needs a new token and shells out todatabricks auth token, the CLI may return a cached token that is about to expire (or has already expired from the SDK's perspective). This creates unnecessary refresh failures and retry loops.The CLI recently added a
--force-refreshflag (databricks/cli#4767) that bypasses its internal cache. By using this flag, the SDK is guaranteed a freshly minted token every time it asks for one, eliminating the stale-token problem.What changed
Interface changes
None.
CliTokenSourceandDatabricksCliTokenSourceare not part of the public API surface.Behavioral changes
--force-refreshwhen invokingdatabricks auth tokenwith a--profileargument. If the CLI is too old to support this flag, the SDK falls back to the plain--profilecommand (and then to--hostif--profileis also unsupported).--force-refreshis only paired with--profile, never with--host. When--hostis the only selector (no profile configured), no--force-refreshis attempted."Databricks CLI does not support --force-refresh. Please upgrade your CLI to the latest version."--profile→--hostfallback warning is unchanged.Internal changes
DatabricksCliTokenSourcenow holds a_force_cmdfield (--profile+--force-refresh). This isNonewhen no profile is configured.DatabricksCliTokenSource.refresh()tries_force_cmdfirst. On"unknown flag: --force-refresh"or"unknown flag: --profile", it logs a warning and delegates tosuper().refresh(), which preserves the existingcmd→fallback_cmdchain. When_force_cmdisNone, it delegates directly tosuper().refresh().CliTokenSourceis unchanged. The existingcmd/fallback_cmd/refresh()logic is preserved as-is.AzureCliTokenSourcedoes not use_force_cmd.How is this tested?
Unit tests in
tests/test_credentials_provider.py:test_force_refresh_tried_first_with_profile—_force_cmdsucceeds, no further commands are tried.test_host_only_skips_force_refresh— when only host is configured,--force-refreshis not used.test_force_refresh_fallback_when_unsupported—_force_cmdfails with"unknown flag: --force-refresh", verifies fallback to plain--profilecommand.test_profile_fallback_when_unsupported—_force_cmdfails with"unknown flag: --profile"(very old CLI), verifies fallback cascades through--profileto--host.test_two_step_downgrade_both_flags_unsupported— both--force-refreshand--profilefail, verifies the full chain:_force_cmd→cmd→fallback_cmd.test_real_auth_error_does_not_trigger_fallback— non-flag error from_force_cmdis raised immediately without fallback.