upgrade aws-sdk in @uppy/companion #6180
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
this fixes #6167
AI summary :
The Bug: S3 Multipart Upload Fails with InvalidPart
What happened
When uploading large files (200MB+) from Google Drive → Companion → S3 using multipart upload, the upload would reach ~96-100% and then fail with S3's
InvalidParterror onCompleteMultipartUpload.Root cause
A version mismatch between two AWS SDK packages caused by yarn dedupe.
Commit
3c3034b40("Dedupe dependencies #6085") ranyarn dedupe. Companion had@aws-sdk/client-s3and@aws-sdk/lib-storageboth at"^3.338.0", which previously both resolved to3.600.0. But the@uppy/transloaditpackage elsewhere in the monorepo had"@aws-sdk/client-s3": "^3.891.0". Dedupe merged the resolutions —client-s3jumped to3.896.0whilelib-storagestayed at3.600.0.Why the mismatch matters
AWS SDK
client-s3@3.896.0introduced a breaking behavioral change in its checksum middleware:• Old (3.600.0): Default checksum = MD5, no auto-injection.
UploadPartrequests sent no checksum header.• New (3.896.0): Default checksum = CRC32,
requestChecksumCalculationdefaults to"WHEN_SUPPORTED". The middleware auto-injectsx-amz-checksum-crc32on everyUploadPartrequest.The old
lib-storage@3.600.0didn't know about this new behavior. It calledCreateMultipartUploadwithout settingChecksumAlgorithm. Thenclient-s3@3.896.0added CRC32 checksums to eachUploadPart. S3 saw parts with checksums that weren't declared at creation time → rejected withInvalidPartatCompleteMultipartUpload.How the fixes work
Option A —
requestChecksumCalculation: 'WHEN_REQUIRED'(workaround):Tells the new
client-s3to only add checksums when S3 requires them (which it doesn't forUploadPart). This restores the old behavior — no checksum headers, no mismatch.Option B — Upgrade all AWS SDK packages to
^3.986.0(proper fix):The new
lib-storage@3.986.0is aware of the CRC32 checksum behavior. When it detectsrequestChecksumCalculation = "WHEN_SUPPORTED", it explicitly setsChecksumAlgorithm: "CRC32"on theCreateMultipartUploadcall. Now S3 expects CRC32 checksums on the parts, the parts have CRC32 checksums → everything is consistent → upload succeeds.TL;DR
yarn dedupeupgradedclient-s3but notlib-storage. The newclient-s3started adding CRC32 checksums to parts, but the oldlib-storagedidn't declare CRC32 at multipart creation time. S3 rejected the mismatch. Fix: upgrade both packages together so they agree on checksum behavior.