Extract shared WWW-Authenticate challenge handling as internal helper

[Copilot]

Overview

This PR extracts duplicate WWW-Authenticate challenge handling logic from DownstreamApi and MicrosoftIdentityMessageHandler into a shared internal helper, improving maintainability and fixing a bug where ForceRefresh was incorrectly set during claims challenges.

Problem

Both DownstreamApi and MicrosoftIdentityMessageHandler implemented nearly identical logic for handling WWW-Authenticate header challenges, including:

• Detecting claims challenges from 401 responses
• Cloning HTTP requests for retry
• Acquiring new tokens with challenge claims

This duplication made maintenance difficult, and bug fixes had to be applied in multiple places. Additionally, MicrosoftIdentityMessageHandler was incorrectly setting ForceRefresh = true when handling claims challenges, which is unnecessary since MSAL.NET automatically bypasses the cache when claims are present (see CacheRefreshReason.ForceRefreshOrClaims).

Solution

Created WwwAuthenticateChallengeHelper in the TokenAcquisition project with three core methods:

// Extract claims from WWW-Authenticate response headers
public static string? ExtractClaimsChallenge(HttpResponseHeaders responseHeaders)

// Defensively clone HttpContent for retry scenarios (handles non-seekable streams)
public static async Task<HttpContent?> CloneHttpContentAsync(
    HttpContent? originalContent, 
    CancellationToken cancellationToken = default)

// Determine if a 401 response should trigger a claims challenge retry
public static bool ShouldAttemptClaimsChallengeRetry(HttpResponseMessage response)

Both DownstreamApi and MicrosoftIdentityMessageHandler now use this helper, ensuring consistent behavior across entry points.

Key Changes

1. WwwAuthenticateChallengeHelper (New)

• Provides shared logic for WWW-Authenticate challenge detection and handling
• Includes defensive content cloning to support non-seekable streams
• Comprehensive inline documentation explaining design decisions

2. DownstreamApi (Refactored)

• Now uses WwwAuthenticateChallengeHelper.ExtractClaimsChallenge() instead of directly calling WwwAuthenticateParameters
• Added defensive content cloning using WwwAuthenticateChallengeHelper.CloneHttpContentAsync()
• Enhanced with inline comments explaining why ForceRefresh is not needed

3. MicrosoftIdentityMessageHandler (Refactored & Bug Fixed)

• Now uses WwwAuthenticateChallengeHelper methods for consistency
• Fixed bug: Removed incorrect ForceRefresh = true setting when claims are present
• Added explanatory comments about MSAL's automatic cache bypass behavior

4. Unit Tests (New)

• Added 14 comprehensive tests covering:
  • Claims extraction with valid, missing, and empty headers
  • Content cloning for various content types (null, string, byte array)
  • Header preservation during content cloning
  • Multiple reuse scenarios
  • Retry determination for different HTTP status codes

Benefits

✅ Eliminates code duplication - Shared logic in one place makes maintenance easier
✅ Fixes ForceRefresh bug - Corrects unnecessary cache bypass in MicrosoftIdentityMessageHandler
✅ Defensive content cloning - Properly handles non-seekable streams in retry scenarios
✅ Enhanced maintainability - Inline comments document design decisions
✅ Consistent behavior - Both entry points now handle challenges identically
✅ No breaking changes - All 644 existing tests pass without modification

Testing

• ✅ All 644 existing tests pass
• ✅ All 14 new WwwAuthenticateChallengeHelper tests pass
• ✅ Existing CAE test validates end-to-end retry behavior
• ✅ All target frameworks build successfully (net462, net472, net8.0, net9.0, netstandard2.0)

Files Changed

• WwwAuthenticateChallengeHelper.cs (new, 84 lines)
• WwwAuthenticateChallengeHelperTests.cs (new, 191 lines)
• DownstreamApi.cs (18 line changes)
• MicrosoftIdentityMessageHandler.cs (20 line changes)
• InternalsVisibleTo.cs (1 line added)
• InternalAPI.Unshipped.txt files for all 5 target frameworks (4 lines each)

Total: 10 files changed, +323/-11 lines

---

Co-created by Jean-Marc and Copilot, per our collaborative process.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Extract shared WWW-Authenticate challenge handling as internal helper</issue_title>
<issue_description>## Overview
Both MicrosoftIdentityMessageHandler and DownstreamApi implement logic for handling WWW-Authenticate header challenges, which includes:

• Detecting a claims challenge from the response
• Cloning the original request for retry
• Creating new options with claim challenges
• Acquiring a new token and retrying the request

Currently, both have separate implementations that duplicate logic for claim extraction, request/content cloning, and retry handling. This makes maintenance more difficult, and future bug fixes or improvements must be duplicated.

Proposal

Extract the shared logic for handling WWW-Authenticate challenges into an internal helper (e.g., WwwAuthenticateChallengeHelper) in the TokenAcquisition project. Refactor both MicrosoftIdentityMessageHandler and DownstreamApi to use this helper, ensuring:

• Defensive cloning of HttpRequestMessage and HttpContent for retries (supporting non-seekable streams)
• Inline comments explaining why content is cloned and why ForceRefresh is not set when claims are present
• Consistency across both entry points

Key Details

• Do not set ForceRefresh when claims are present. MSAL.NET will bypass the cache automatically (see CacheRefreshReason.ForceRefreshOrClaims).
• Use inline comments in the helper and refactored code to explain design decisions for future maintainers.
• Base changes on the copilot/fix-82eb5023-e802-4c45-8341-57c618f136aa branch, where MicrosoftIdentityMessageHandler is implemented but not yet merged.
• Add unit tests for the helper covering claim extraction and content cloning.

Benefits

• Eliminates duplicate challenge-handling logic
• Ensures bug fixes and improvements apply to both entry points
• Improves maintainability and testability

Acceptance Criteria

• Shared helper is implemented and used by both MicrosoftIdentityMessageHandler and DownstreamApi
• Helper includes defensive content cloning and explanatory comments for claims handling
• Unit tests verify helper behavior for various header and content scenarios
• No API-breaking changes

---

Co-created by Jean-Marc and Copilot, per our collaborative process.</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes #3541

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[jmprieur]

LGTM
thanks copilot

[jennyf19]

@jmprieur - comment from Vega:
Great refactoring work here! The extraction of duplicate WWW-Authenticate challenge handling is clean and well-tested. A few observations:

✅ What's Strong:

The bug fix removing unnecessary ForceRefresh is correct—MSAL.NET handles cache bypass automatically via CacheRefreshReason.ForceRefreshOrClaims
Defensive content cloning properly handles non-seekable streams
Comprehensive test coverage (14 new tests)
Excellent inline documentation explaining design decisions
🤔 Minor Observation (non-blocking): There's an intentional asymmetry in how the two classes use the helper:

DownstreamApi uses WwwAuthenticateChallengeHelper.CloneHttpContentAsync() directly
MicrosoftIdentityMessageHandler still uses its own CloneHttpRequestMessageAsync() method
This makes sense—DownstreamApi only needs content cloning, while MicrosoftIdentityMessageHandler needs to clone the entire request. However, it might be worth checking if CloneHttpRequestMessageAsync() could internally use the helper's content cloning logic for consistency, though this isn't urgent.

[keegan-caruso]

Are there worries along the same thoughts as #3532 ?

[keegan-caruso]

I don't think we need this method at all. Why can't we just use:

await originalContent.LoadIntoBufferAsync(cancellationToken);

This should buffer the content into memory allowing multiple reads.

[keegan-caruso]

We can also provide an upper bound of the size of the buffer

[cpp11nullptr]

Can we consider a merge it with ExtractClaimsChallenge(), e.g., in form of TryExtractClaimsChallengeToRetry, returning bool with claims as string out parameter? It'd provide a slightly better user experience such as a single call/check would be needed considering that claims are used straight away.