Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 22, 2025

  • Analyze the root cause of GetAllImageDigestsFromRegistryAsync performance regression
  • Implement CachingTokenCredential wrapper that caches tokens and refreshes them when near expiration
  • Update AzureTokenCredentialProvider to wrap AzurePipelinesCredential with CachingTokenCredential
  • Add unit tests for CachingTokenCredential
  • Run code review and address feedback (fixed race condition using SemaphoreSlim, added documentation, improved nullable check)
  • Fix build error: Add InternalsVisibleTo attribute to allow test project to access internal CachingTokenCredential class
  • Enable nullable annotations in test file
  • Run security scan (timed out - no issues expected as changes are straightforward token caching)

Root Cause Analysis

PR #1879 removed the StaticTokenCredential class that was caching tokens. The AzurePipelinesCredential was returned directly, but it does NOT cache tokens internally. Each call to GetToken fetches a new token from Azure, turning what should be a 1-minute operation into 1+ hours.

Solution

Implemented a CachingTokenCredential wrapper that:

  1. Caches the token after the first fetch
  2. Returns the cached token if it's still valid (not within 5 minutes of expiration)
  3. Refreshes the token only when it's about to expire
  4. Uses SemaphoreSlim to ensure thread-safe access and prevent concurrent token fetches
Original prompt

This section details on the original issue you should resolve

<issue_title>GetAllImageDigestsFromRegistryAsync is suddenly extremely slow</issue_title>
<issue_description>In this dotnet-docker-nightly pipeline run it took 1hr15min, causing the pipeline to time out later in the run:

2025-12-19T01:01:15.0763844Z Querying registry for all image digests...
2025-12-19T02:15:43.0489446Z Checking digest for existing annotation: ...

Previously (in this pipeline run), it took less than one minute:

2025-12-10T21:56:14.9556032Z Querying registry for all image digests...
2025-12-10T22:19:31.3493702Z Checking digest for existing annotation: ...

The earliest occurrence of this behavior I could find is in https://dev.azure.com/dnceng/internal/_build/results?buildId=2861673&view=logs&j=739bb7ec-7296-5b40-7dff-ee3297bfdaa4&t=a08adfce-16dd-580a-541c-89ea6031d6c4 where it took 23 minutes. That is dotnet-docker commit db838f02ef8fb99154680b938054748dc856d8b9.

The cause is likely #1879.</issue_description>

<agent_instructions>Look through recent commits to diagnose the performance issue. Pay particular attention to PR #1879. We already know that manifests and repos are checked sequentially - that is known not to cause performance issues in this case. It is most likely something to do with the AzurePipelinesCredential/Token fetching since that code changed recently. </agent_instructions>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 5 commits December 23, 2025 00:05
Co-authored-by: lbussell <36081148+lbussell@users.noreply.github.com>
Co-authored-by: lbussell <36081148+lbussell@users.noreply.github.com>
Co-authored-by: lbussell <36081148+lbussell@users.noreply.github.com>
Co-authored-by: lbussell <36081148+lbussell@users.noreply.github.com>
Co-authored-by: lbussell <36081148+lbussell@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix extreme slowness in GetAllImageDigestsFromRegistryAsync Add token caching for AzurePipelinesCredential Dec 23, 2025
Copilot AI requested a review from lbussell December 23, 2025 00:14
@lbussell
Copy link
Member

@copilot /repo/src/ImageBuilder.Tests/CachingTokenCredentialTests.cs(29,37): error CS0122: 'CachingTokenCredential' is inaccessible due to its protection level

Co-authored-by: lbussell <36081148+lbussell@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 23, 2025

@copilot /repo/src/ImageBuilder.Tests/CachingTokenCredentialTests.cs(29,37): error CS0122: 'CachingTokenCredential' is inaccessible due to its protection level

Fixed in 8cfb114 by adding InternalsVisibleTo attribute to the ImageBuilder project to expose internal types to the test project.

Co-authored-by: lbussell <36081148+lbussell@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GetAllImageDigestsFromRegistryAsync is suddenly extremely slow

2 participants