[Issue] context canceled for azd up (but azd deploy works)

[giuliohome]

Output from azd version
azd version 1.22.5

Describe the bug
azd up fails with 'context canceled' on second deployment step

To Reproduce

I'm working with https://github.com/Azure-Samples/azure-ai-travel-agents

giulio@myhome:~/csharp/azd-multiagent$azd up
...
...
...
ERROR: error executing step command 'deploy api-llamaindex-ts': failed to authenticate: server response error:
 Get "https://login.microsoftonline.com/common/discovery/instance?api-version=1.1&authorization_endpoint=https%3A%2F%2Flogin.microsoftonline.com%2Forganizations%2Foauth2%2Fv2.0%2Fauthorize": context canceled
giulio@myhome:~/csharp/azd-multiagent$

Expected behavior

It should not fail indeed azd deploy does work fine!

Environment
Information on your environment:
* ssh session
* fedora 42
* bash

Additional context

I've already tried clearing auth cache,

rm -rf ~/.azd/auth
rm -rf ~/.azure/accessTokens.json ~/.azure/azureProfile.json

azd auth logout
azd auth login --use-device-code

disabling IPv6, and curl to the login endpoint works

My finding that azd deploy works correctly (deploying all services in one go), while azd up fails during the deployment phase, narrows down the problem significantly: the issue is specifically within the azd up command's orchestration, particularly in the handoff or state management between its provision and subsequent deploy stages. It's not a general azd deploy issue, nor a single-service deployment issue, nor a networking issue for individual calls.

[giuliohome]

Hi @vhvb1989. I've seen your reply to a similar issue:

"This strongly suggests a transient CLI stall, not a permanent configuration problem."

However, in my case this is consistently reproducible - the issue pattern repeats every single time I attempt this operation. I can provide additional diagnostics if that would help narrow down the root cause.

Hi @diberry. The issue persists for me as well - curious if you're seeing the same consistent reproduction?

[giuliohome]

I believe I've identified the root cause by examining the workflow runner code.

In cli/azd/pkg/workflow/runner.go, the Run method creates a child context for each workflow step:

for _, step := range workflow.Steps {
    // Create a child context for this step to enable automatic handler cleanup
    stepCtx, cancel := context.WithCancel(ctx)
    
    if len(step.AzdCommand.Args) > 0 {
        r.azdRunner.SetArgs(step.AzdCommand.Args)
    }
    
    err := r.azdRunner.ExecuteContext(stepCtx)
    cancel()  // Cancel immediately after step execution
    
    if err != nil {
        return fmt.Errorf("error executing step command '%s': %w", ...)
    }
}

The problem: When cancel() is called after each step completes, it cancels the stepCtx context. However, there may be async operations (like credential refresh, token acquisition, or Azure SDK calls) that were initiated during the step but haven't completed yet.

During azd up:

1. Provision step completes → cancel() called
2. First deploy (api) completes → cancel() called
3. Second deploy (app) starts → tries to authenticate using credentials/tokens that were obtained in previous steps, but those background operations reference the now-canceled contexts

This explains why:

• ✅ azd deploy works standalone (fresh context, no premature cancellation)
• ❌ azd up fails on the 2nd+ deploy (context from credential acquisition was canceled)
• ❌ The error is "context canceled" during authentication

Suggested fix: The cancel() should be deferred or the context lifecycle needs to respect async operations:

stepCtx, cancel := context.WithCancel(ctx)
defer cancel()  // Let it live until the loop iteration ends naturally

Or better yet, let the parent context manage the lifecycle since these are sequential blocking operations anyway.

This change was introduced 2 months ago in commit #5960 "Adds automatic handler removal on context cleanup" - which likely inadvertently introduced this regression.

---

Analysis provided by Claude (Anthropic)

[vhvb1989]

That's a really awesome analysis @giuliohome . Thank you.

Adding @wbreza for review

[giuliohome]

@vhvb1989 Confirmed! I've tested the fix locally and can verify it resolves the issue completely.

Changes made:
In cli/azd/pkg/workflow/runner.go, line ~44:

stepCtx, cancel := context.WithCancel(ctx)
defer cancel()  // Changed from immediate cancel() after ExecuteContext

After this one-line change, azd up now completes successfully through all deployment steps without the "context canceled" error.

Run results:

• ✅ azd up - All steps complete (provision → deploy api → deploy app)
• ✅ No authentication errors
• ✅ No context cancellation issues

Happy to provide more details or test additional scenarios if needed.

The repositories involved with code diff

giulio@myhome:~/csharp/azd-multiagent$ git remote -v
fork	https://github.com/giuliohome-org/azure-ai-travel-agents.git (fetch)
fork	https://github.com/giuliohome-org/azure-ai-travel-agents.git (push)
giulio@myhome:~/csharp/azd-multiagent$ cd ~/golang/azure-dev/cli/azd
giulio@myhome:~/golang/azure-dev/cli/azd$ git remote -v
origin	https://github.com/Azure/azure-dev.git (fetch)
origin	https://github.com/Azure/azure-dev.git (push)
giulio@myhome:~/golang/azure-dev/cli/azd$ git diff
diff --git a/cli/azd/pkg/workflow/runner.go b/cli/azd/pkg/workflow/runner.go
index 51a9fc1f..36f40d91 100644
--- a/cli/azd/pkg/workflow/runner.go
+++ b/cli/azd/pkg/workflow/runner.go
@@ -46,7 +46,8 @@ func (r *Runner) Run(ctx context.Context, workflow *Workflow) error {

                // Cancel the step context to trigger automatic cleanup of any handlers
                // registered during this step execution
-               cancel()
+               // fix https://github.com/Azure/azure-dev/issues/6431#issuecomment-3699427805
+               defer cancel()

                if err != nil {
                        return fmt.Errorf("error executing step command '%s': %w", strings.Join(step.AzdCommand.Args, " "), err)
giulio@myhome:~/golang/azure-dev/cli/azd$

My shell output log

giulio@myhome:~/csharp/azd-multiagent$ ~/golang/azure-dev/cli/azd/azd up
Note: Running custom 'up' workflow from azure.yaml

Provisioning Azure resources (azd provision)
Provisioning Azure resources can take some time.

Subscription: Sottoscrizione di Azure 1 (9634d46e-508a-4bf9-809a-64146f590e34)
Location: Sweden Central

  You can view detailed progress in the Azure Portal:
  https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2F9634d46e-508a-4bf9-809a-64146f590e34%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Fgiulio-azd2-multiagent-1767124631

  (✓) Done: Resource group: rg-giulio-azd2-multiagent (2.792s)
  (✓) Done: Log Analytics workspace: log-ca5x2xgysr7hq (25.385s)
  (✓) Done: Container Registry: crca5x2xgysr7hq (24.003s)
  (✓) Done: Application Insights: appi-ca5x2xgysr7hq (3.154s)
  (✓) Done: Foundry: cog-ca5x2xgysr7hq (23.323s)
  (✓) Done: Azure AI Services Model Deployment: cog-ca5x2xgysr7hq/gpt-5-mini (1.547s)
  (✓) Done: Portal dashboard: dash-ca5x2xgysr7hq (2.259s)
  (✓) Done: Container Apps Environment: cae-ca5x2xgysr7hq (54.917s)
  (✓) Done: Container App: mcp-destination-recommendation (19.707s)
  (✓) Done: Container App: api-llamaindex-ts (19.307s)
  (✓) Done: Container App: mcp-customer-query (20.505s)
  (✓) Done: Container App: api-langchain-js (18.316s)
  (✓) Done: Container App: api-maf-python (18.213s)
  (✓) Done: Container App: mcp-echo-ping (20.004s)
  (✓) Done: Container App: ui-angular (18.825s)
  (✓) Done: Container App: mcp-itinerary-planning (35.216s)

Deploying services (azd deploy)

  (✓) Done: Deploying service api-langchain-js
  - Endpoint: https://api-langchain-js.icyplant-1949a013.swedencentral.azurecontainerapps.io/

  (-) Skipped: Deploying service mcp-customer-query
  (-) Skipped: Deploying service mcp-destination-recommendation
  (-) Skipped: Deploying service mcp-echo-ping
  (-) Skipped: Deploying service mcp-itinerary-planning
  (-) Skipped: Deploying service ui-angular

Deploying services (azd deploy)

  (-) Skipped: Deploying service api-langchain-js
  (-) Skipped: Deploying service mcp-customer-query
  (-) Skipped: Deploying service mcp-destination-recommendation
  (-) Skipped: Deploying service mcp-echo-ping
  (-) Skipped: Deploying service mcp-itinerary-planning
  (✓) Done: Deploying service ui-angular
  - Endpoint: https://ui-angular.icyplant-1949a013.swedencentral.azurecontainerapps.io/


Deploying services (azd deploy)

  (-) Skipped: Deploying service api-langchain-js
  (✓) Done: Deploying service mcp-customer-query
  - Endpoint: https://mcp-customer-query.internal.icyplant-1949a013.swedencentral.azurecontainerapps.io/

  (-) Skipped: Deploying service mcp-destination-recommendation
  (-) Skipped: Deploying service mcp-echo-ping
  (-) Skipped: Deploying service mcp-itinerary-planning
  (-) Skipped: Deploying service ui-angular

Deploying services (azd deploy)

  (-) Skipped: Deploying service api-langchain-js
  (-) Skipped: Deploying service mcp-customer-query
  (✓) Done: Deploying service mcp-destination-recommendation
  - Endpoint: https://mcp-destination-recommendation.internal.icyplant-1949a013.swedencentral.azurecontainerapps.io/

  (-) Skipped: Deploying service mcp-echo-ping
  (-) Skipped: Deploying service mcp-itinerary-planning
  (-) Skipped: Deploying service ui-angular

Deploying services (azd deploy)

  (-) Skipped: Deploying service api-langchain-js
  (-) Skipped: Deploying service mcp-customer-query
  (-) Skipped: Deploying service mcp-destination-recommendation
  (-) Skipped: Deploying service mcp-echo-ping
  (✓) Done: Deploying service mcp-itinerary-planning
  - Endpoint: https://mcp-itinerary-planning.internal.icyplant-1949a013.swedencentral.azurecontainerapps.io/

  (-) Skipped: Deploying service ui-angular

Deploying services (azd deploy)

  (-) Skipped: Deploying service api-langchain-js
  (-) Skipped: Deploying service mcp-customer-query
  (-) Skipped: Deploying service mcp-destination-recommendation
  (✓) Done: Deploying service mcp-echo-ping
  - Endpoint: https://mcp-echo-ping.internal.icyplant-1949a013.swedencentral.azurecontainerapps.io/

  (-) Skipped: Deploying service mcp-itinerary-planning
  (-) Skipped: Deploying service ui-angular

SUCCESS: Your up workflow to provision and deploy to Azure completed in 36 minutes 26 seconds.

[giuliohome]

P.S. Correction: In my local test, I placed the defer cancel() after ExecuteContext(), which still fixed the immediate issue but would fail to cleanup on panic. The correct placement is immediately after context creation:

stepCtx, cancel := context.WithCancel(ctx)
defer cancel()  // Must be here, not after ExecuteContext

if len(step.AzdCommand.Args) > 0 {
    r.azdRunner.SetArgs(step.AzdCommand.Args)
}

err := r.azdRunner.ExecuteContext(stepCtx)

This ensures cleanup even if ExecuteContext panics.

[vhvb1989]

Thanks @giuliohome .

Looking at context.WithCancel(parent context.Context), there is this note in the docs:

Canceling this context releases resources associated with it, so code should call cancel as soon as the operations running in this [Context] complete.

I think that explains why the call to cancel() was there. But the problem seems to be invoking the function inside the loop.
By adding the defer cancel(), we are holding the call to release resources to the end of Run(...) .

What I don't understand is why are we using WithCancel(...) to create the child context in the first place.
Looking at the root context in main.go, it uses the background context, which might have been a wrong decision for us, as it won't stop/cancel.
But what if instead of creating a new cancellable context for each step, we would just use the main parent context?
Hopefully @wbreza can help here.

[giuliohome]

Minor note about the root context

Looking at the root context in main.go, it uses the background context, which might have been a wrong decision for us, as it won't stop/cancel.

@vhvb1989 I think there might be a misunderstanding about context.Background().

context.Background() doesn't mean "never stops" - it just means it's the root context with no parent and no built-in timeout. When azd exits (after the workflow completes or fails), everything stops anyway since the process terminates.
Therefore yes, as you wrote, the fix options are:

1. Use parent context directly (removes the per-step context entirely)
2. Defer child context cancellation (keeps per-step contexts but delays cleanup)

Quick broader architectural questions:

I noticed that azd deploy (without service name) and azd pipeline work fine because they don't use the workflow runner - they deploy services directly in their own loops.

Meanwhile, azd up with custom workflows in azure.yaml goes through pkg/workflow/runner.go where this bug exists.

Questions:

1. Is the workflow runner path (pkg/workflow/runner.go) intended to eventually replace the direct deployment path?
2. Or are they meant to coexist for different use cases?
3. Should we consider making azd up use the same direct deployment path when there's no custom workflow defined?

Understanding the architectural intent would help determine the best fix:

• If workflows are the future → fix the workflow runner properly
• If direct deployment is preferred → maybe azd up shouldn't use workflows by default
• If both should coexist → we need to ensure they share the same context lifecycle patterns

This context cancellation bug only exists in the workflow path, which suggests the two code paths have different (incompatible?) assumptions about context lifecycle.

Thanks for any clarification!

[wbreza]

I will review this issue.

There really shouldn't be any async work happening within the stepCtx after the runner has completed executing the step. We cannot wait/defer to cleanup till the end of the workflow because there are per workflow steps cleanup steps that need to be performed.

I will report back once I know more.