File upload failed even after retry

[jeffschwMSFT]

Build

https://dev.azure.com/dnceng/internal/_build/results?buildId=2365835

Error:

Fail to upload '/mnt/vss/_work/1/a/IntermediateArtifacts/MonoRuntimePacks/Shipping/Microsoft.NETCore.App.Ref.9.0.0-preview.2.24080.4.symbols.nupkg' due to 'Bad Request'.
Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException: Bad Request
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.FileContainer.Client.FileContainerHttpClient.UploadFileAsync(Int64 containerId, String itemPath, Stream fileStream, Byte[] contentId, Int64 fileLength, Boolean isGzipped, Guid scopeIdentifier, CancellationToken cancellationToken, Int32 chunkSize, Int32 chunkRetryTimes, Boolean uploadFirstChunk, Object userState)
   at Microsoft.VisualStudio.Services.FileContainer.Client.FileContainerHttpClient.UploadFileAsync(Int64 containerId, String itemPath, Stream fileStream, Guid scopeIdentifier, CancellationToken cancellationToken, Int32 chunkSize, Boolean uploadFirstChunk, Object userState, Boolean compressStream)
   at Microsoft.VisualStudio.Services.Agent.Worker.Build.FileContainerServer.UploadAsync(IAsyncCommandContext context, Int32 uploaderId, CancellationToken token) in /home/vsts/work/1/s/src/Agent.Worker/Build/FileContainerServer.cs:line 250
Detail upload trace for file that fail to upload: IntermediateArtifacts/MonoRuntimePacks/Shipping/Microsoft.NETCore.App.Ref.9.0.0-preview.2.24080.4.symbols.nupkg

Chunk '2' attempt '1' of file 'IntermediateArtifacts/MonoRuntimePacks/Shipping/Microsoft.NETCore.App.Ref.9.0.0-preview.2.24080.4.symbols.nupkg' fail to send request to server. Error: Microsoft.VisualStudio.Services.Common.VssServiceException: TF203009: The size of the file that you uploaded does not match the size that the client computer provided.
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.FileContainer.Client.FileContainerHttpClient.UploadFileAsync(Int64 containerId, String itemPath, Stream fileStream, Byte[] contentId, Int64 fileLength, Boolean isGzipped, Guid scopeIdentifier, CancellationToken cancellationToken, Int32 chunkSize, Int32 chunkRetryTimes, Boolean uploadFirstChunk, Object userState).

##[error]File upload failed even after retry.

Build leg reported

No response

Pull Request

No response

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "##[error]File upload failed even after retry",
    "BuildRetry": false,
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Report

Build	Definition	Step Name	Console log	Pull Request
1273572	dotnet/maui	publish artifacts	Log
1273563	dotnet/maui	publish artifacts	Log	dotnet/maui#33786
1273512	dotnet/maui	publish artifacts	Log	dotnet/maui#33774
1271298	dotnet/maui	publish artifacts	Log
1270990	dotnet/maui	publish artifacts	Log
1270314	dotnet/arcade	Unix Build / Publish	Log	dotnet/arcade#16482
1269495	dotnet/maui	publish artifacts	Log
2888946	dotnet-maui	publish artifacts	Log
1269322	dotnet/maui	publish artifacts	Log	dotnet/maui#33552
1269126	dotnet/maui	publish artifacts	Log	dotnet/maui#33721
1265196	dotnet/maui	publish artifacts	Log
2886211	dotnet-runtime	🔒 Publish intermediate artifacts	Log
1264453	dotnet/maui	publish artifacts	Log
1262921	dotnet/maui	publish artifacts	Log	dotnet/maui#33552
1263011	dotnet/maui	publish artifacts	Log	dotnet/maui#33632
1259366	dotnet/maui	publish artifacts	Log
1255920	dotnet/maui	publish artifacts	Log	dotnet/maui#33563
1253566	dotnet/xharness	Build and run tests	Log
2879329	dotnet-runtime	🔒 Publish intermediate artifacts	Log
2878454	dotnet-runtime	🔒 Publish intermediate artifacts	Log
2878455	dotnet-runtime	🔒 Publish intermediate artifacts	Log
1251708	dotnet/maui	publish artifacts	Log	dotnet/maui#33363
1251323	dotnet/xharness	Build and run tests	Log	dotnet/xharness#1533
1250661	dotnet/maui	publish artifacts	Log	dotnet/maui#33459
1249844	dotnet/maui	publish artifacts	Log	dotnet/maui#33160
1250005	dotnet/maui	publish artifacts	Log
1249917	dotnet/maui	publish artifacts	Log
1249861	dotnet/maui	publish artifacts	Log	dotnet/maui#33441
1249944	dotnet/xharness	Build and run tests	Log
1248106	dotnet/maui	publish artifacts	Log
1248060	dotnet/maui	publish artifacts	Log
1248036	dotnet/maui	publish artifacts	Log
1247511	dotnet/maui	publish artifacts	Log
1246000	dotnet/maui	publish artifacts	Log	dotnet/maui#33075
1246024	dotnet/arcade	Unix Build / Publish	Log
2873216	dotnet-runtime	🔒 Publish intermediate artifacts	Log

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
3	11	36

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng/internal/_build/results?buildId=2365835
Error message validated: [##[error]File upload failed even after retry]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 2/7/2024 2:03:27 AM UTC

[lewing]

This is often an from multiple lanes attempting to upload the same artifact

[markwilkie]

@lewing is right - there's usually not a whole lot dnceng can do to help.

[missymessa]

@jeffschwMSFT @lewing is this still an issue?

[lewing]

@jeffschwMSFT @lewing is this still an issue?

It still happens but it is typically a (mostly harmless) repository issue. Perhaps this issue should be updated to explain the cause and suggest fixes. It is clearly a recurring problem.