Restore #6284: AWS S3 rewrite by qxprakash · Pull Request #6345 · transloadit/uppy

qxprakash · 2026-06-17T08:58:20Z

restore AWS S3 rewrite #6284 to a6eddb1
sign all the commits which were authored by me and were unsigned

- base branch cleanup and ran `yarn && yarn dedupe` - client copied and tests wired up

- added `getCredentials` callback in `S3Config` - simplified signer test util ( `sigv4-signer.js` ) and moved to `src/signer.ts` - all tests run in a new non root minio user ( required for sts ) , which gets created before the tests start see `tests/s3-client/setup.js` - tests added for sts , using `@aws-sdk/client-sts` for getting creds. - credential caching inspired from the current implementation in `@aws-s3` plugin , see `#getTemporarySecurityCredentials` inside `@uppy/aws-s3/src/index.ts` --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…er.ts

#### This PR replaces signed headers with presigned urls , for signing. - I implemented this along with the uploader rewrite , but raised PR for this it separately to reduce diffs. - `signer.ts` now returns signed url , which is directly used by `_presignedRequest` method to make calls to s3 - tested it with uploader rewrite working fine. --------- Co-authored-by: Murderlon <merlijn@soverin.net>

Working : - Simple uploads (putObject) - Multipart uploads - Pause / Resume / Abort , Resume works using listParts as before - Progress tracking per chunk ( `#onProgress` ) - this is temporary , until we implement xhr based progress tracking as - before. **added new examples to examples/aws-nodejs to test the rewrite , old examples will eventually be removed** Not Yet Implemented - Rate limiting – Currently uploads are sequential - Retries - Remote file uploads – Files from Google Drive, Dropbox, etc. XHR progress - full wiring with uppy state ( I saw few state bugs while testing it manually ) --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Murderlon <merlijn@soverin.net>

Cherry-picked as the previous branch got messy ( from last PR ) - XHR-based uploads using `@uppy/utils/fetcher` - Silent offline detection (without treating it as an error) with automatic resume ( inspired from old plugin code ) - Retry with backoff using fetcher defaults: - Max retries: 3 - Delays: 300ms → 600ms → 1200ms - Retries on 5xx and 429; skips other 4xx errors - S3 Error Mappings

shouldUseMultipart: true was not working due to signature mismatch AI summary : This pull request improves support for specifying the `contentType` during S3 uploads, ensuring that the correct content type is set when generating presigned URLs for both single and multipart uploads. It also updates the client-side and server-side code to handle this new parameter. **Backend and API enhancements:** - Updated the `/s3/presign` endpoint in `index.js` to accept an optional `contentType` parameter and to include it in the S3 `PutObjectCommand` and `CreateMultipartUploadCommand` requests, defaulting to `'application/octet-stream'` if not provided. [[1]](diffhunk://#diff-0c4471088c62917198e1438777b4c7e78fd4d87d450d2a7c78b86b7c647ac97bL360-R360) [[2]](diffhunk://#diff-0c4471088c62917198e1438777b4c7e78fd4d87d450d2a7c78b86b7c647ac97bR384-R391) **Client-side and types improvements:** - Added `contentType` to the `presignableRequest` type to ensure type safety and clarity when passing this parameter. - Modified the S3 client (`S3.ts`) to include `contentType` when making presign requests. - Updated the example HTML (`rewrite-test.html`) to send `contentType` in presign requests and enabled multipart uploads for testing. [[1]](diffhunk://#diff-ca9bc28a71d2d8ac4e0aa6844698f2244594a12e695fc3e153db36674ccb2b52L103-R103) [[2]](diffhunk://#diff-ca9bc28a71d2d8ac4e0aa6844698f2244594a12e695fc3e153db36674ccb2b52R114)

- queue support added for file level concurrency.

- removed old examples , entirely remove withCustomEndpoints.html as every s3 operations are now handled by s3mini so no need of it. - add new examples and update README.md

right now whenever you upload any file through multipart upload you'll find empty body returned in `upload-success` event cause : response from completeMultipartRequest returned in CamelCase which was not parsed correctly ```xml <?xml version="1.0" encoding="UTF-8"?> <CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Location>https://testbucketnewfix.s3.eu-north-1.amazonaws.com/13febe01-8d5b-4ed3-bda1-9620fc37c694-VSCode-darwin-universal.dmg</Location><Bucket>testbucketnewfix</Bucket><Key>13febe01-8d5b-4ed3-bda1-9620fc37c694-VSCode-darwin-universal.dmg</Key><ETag>"be4129821e06a893b453f1373995202b-49"</ETag><ChecksumCRC64NVME>Wjr7mcJOfU0=</ChecksumCRC64NVME><ChecksumType>FULL_OBJECT</ChecksumType></CompleteMultipartUploadResult> ``` --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

- add support for remote uploads - add support for restore using golden-retriever - update examples - #6181 still persists --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Mikael Finstad <finstaden@gmail.com>

- update outdated examples based on latest api changes

changeset-bot · 2026-06-17T08:58:35Z

⚠️ No Changeset found

Latest commit: 260719f

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 49398593bd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot

Pull request overview

This PR restores and modernizes the @uppy/aws-s3 implementation (per the referenced AWS S3 rewrite), introducing a new browser-friendly S3 client/signing flow, Golden Retriever resume support for multipart uploads, and expanded test coverage (including MinIO-based integration tests).

Changes:

Replaces the legacy multipart implementation with a new S3Uploader + s3-client/* stack (client-side signing via STS credentials, server-side signing via signRequest, or Companion-backed signing).
Adds MinIO-backed integration tests (including Playwright browser tests) and CI wiring for VITE_MINIO_CONFIG.
Updates examples and dev dashboards to use the new option naming (companionEndpoint) and new signing flows.

Reviewed changes

Copilot reviewed 43 out of 44 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
turbo.json	Passes `VITE_MINIO_CONFIG` through Turbo test tasks to support MinIO tests.
private/dev/Dashboard.js	Updates AwsS3 dev usage to `companionEndpoint`.
packages/@uppy/utils/src/fetcher.ts	Adjusts `body` defaulting behavior in XHR fetcher.
packages/@uppy/core/src/Uppy.ts	Fixes comment typos and adds `s3Multipart` placeholder on file state.
packages/@uppy/aws-s3/vitest.config.ts	Introduces multi-project Vitest config (jsdom + Playwright browser) and global setup for MinIO.
packages/@uppy/aws-s3/tests/test-utils/browser-crypto.ts	Adds WebCrypto-based `randomBytes` helper for browser contexts.
packages/@uppy/aws-s3/tests/s3-client/setup.ts	Adds MinIO Docker compose global setup/teardown for tests.
packages/@uppy/aws-s3/tests/s3-client/minio.test.ts	Adds MinIO integration test suite for `S3mini` + STS signing.
packages/@uppy/aws-s3/tests/s3-client/docker.ts	Adds Docker helpers for starting/stopping/exec’ing in compose services.
packages/@uppy/aws-s3/tests/s3-client/config.ts	Reads `VITE_MINIO_CONFIG` and provides test credentials/config.
packages/@uppy/aws-s3/tests/s3-client/compose.minio.yaml	Adds MinIO docker-compose definition used by tests.
packages/@uppy/aws-s3/tests/index.test.ts	Adds extensive unit tests for new AwsS3 behavior and Golden Retriever resume state.
packages/@uppy/aws-s3/src/utils.ts	Removes `AwsBody` type from utils (moved to main entry).
packages/@uppy/aws-s3/src/S3Uploader.ts	New uploader handling simple + multipart uploads and resume state persistence (`s3Multipart`).
packages/@uppy/aws-s3/src/s3-client/utils.ts	Adds XML parsing, SigV4 helpers, URL sanitizers, and custom error types.
packages/@uppy/aws-s3/src/s3-client/types.ts	Adds types for presignable requests and credential/signing APIs.
packages/@uppy/aws-s3/src/s3-client/signer.ts	Adds SigV4 presigner (with backward-compatible alias).
packages/@uppy/aws-s3/src/s3-client/S3mini.ts	Adds browser-capable S3 client supporting `signRequest` or `getCredentials`.
packages/@uppy/aws-s3/src/s3-client/S3Client.ts	Adds shared XHR + retry/offline handling base class.
packages/@uppy/aws-s3/src/s3-client/index.ts	Exports `S3mini` and related types.
packages/@uppy/aws-s3/src/s3-client/consts.ts	Adds shared constants and error messages for S3 client/signing.
packages/@uppy/aws-s3/src/s3-client/CompanionS3.ts	Adds Companion-backed S3 client implementation.
packages/@uppy/aws-s3/src/MultipartUploader.ts	Deletes legacy multipart uploader implementation.
packages/@uppy/aws-s3/src/index.ts	Rewrites AwsS3 plugin API/implementation to use new uploader + signing modes.
packages/@uppy/aws-s3/src/index.test.ts	Deletes legacy AwsS3Multipart tests (replaced by `tests/index.test.ts`).
packages/@uppy/aws-s3/src/HTTPCommunicationQueue.ts	Deletes legacy HTTP communication queue.
packages/@uppy/aws-s3/src/createSignedURL.ts	Deletes legacy presigning implementation.
packages/@uppy/aws-s3/src/createSignedURL.test.ts	Deletes legacy presigning tests.
packages/@uppy/aws-s3/package.json	Adds `@aws-sdk/client-sts` dev dependency.
examples/companion-digitalocean-spaces/README.md	Updates docs links from `aws-s3-multipart` to `aws-s3`.
examples/companion-digitalocean-spaces/main.js	Updates example option to `companionEndpoint`.
examples/aws-php/s3-sign.php	Updates PHP signer example to new `signRequest` shape and improves CORS handling.
examples/aws-php/main.js	Switches example from `getUploadParameters` to `signRequest`.
examples/aws-nodejs/routes/sts.js	Adds STS credentials endpoint for client-side signing example.
examples/aws-nodejs/routes/presign.js	Adds server-side presign endpoint for `signRequest` example.
examples/aws-nodejs/README.md	Updates documentation to describe both signing modes.
examples/aws-nodejs/public/withCustomEndpoints.html	Removes the older “custom endpoints” example page.
examples/aws-nodejs/public/index.html	Updates browser example UI + wiring for both signing modes + GoldenRetriever.
examples/aws-nodejs/index.js	Refactors Node example server to use the new `routes/*` and injects bucket/region into HTML.
examples/aws-companion/main.js	Updates example option to `companionEndpoint`.
.github/workflows/ci.yml	Adds `VITE_MINIO_CONFIG` to CI env.
.github/workflows/bundlers.yml	Adds debug step to ensure packaged `uppy` doesn’t ship `workspace:*` deps.
.github/CONTRIBUTING.md	Documents how to run MinIO-backed tests locally.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

…store/pr-6284

Copilot

Pull request overview

Copilot reviewed 42 out of 43 changed files in this pull request and generated 6 comments.

socket-security · 2026-06-19T20:09:26Z

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	npm/@aws-sdk/client-sts@3.1071.0

View full report

…dback

qxprakash · 2026-06-19T20:33:42Z

@mifi most of the major comments have been addressed , for resolving merge conflicts in yarn.lock I checked out the yarn.lock of the main branch ( since it had the latest upgraded deps ) and then ran yarn install in the s3-rewrite branch to update the lockfile.

qxprakash · 2026-06-19T20:34:36Z

can you check why CI is not running ?

mifi · 2026-06-22T10:46:47Z

@copilot why is CI not running?

Copilot · 2026-06-22T10:52:32Z

@copilot why is CI not running?

Looking at the GitHub Actions history for this repo, no workflows have run at all since June 4, 2026 — this affects every branch and PR across the repository, not just this one. The CI, Bundlers, and Lockfile checks workflows all stopped triggering on that date.

This is a repository/organization-level issue, not caused by any code change in this PR. To fix it, please check:

GitHub Actions settings – Settings → Actions → General – verify "Allow all actions and reusable workflows" is selected and that Actions are not disabled.
Organization-level Actions settings – the organization may have paused/disabled Actions.
Billing / minutes quota – if the organization's paid Actions minutes were exhausted, workflows stop triggering until the quota resets or is topped up.

Once the account-level or org-level Actions issue is resolved, this PR's CI should trigger automatically on the next push/sync.

mifi · 2026-06-22T11:41:57Z

@copilot github actions was disabled repo-wise. how can i re-trigger the actions for PRs like this one now that i've enabled it?

mifi

I think I already reviewed most of this code in pretty good detail before, so I assume the code hasn't been changed after that

workspace:* entries rather than just logging it

…of upload

…partUpload request was resolved

mifi · 2026-07-16T13:11:46Z

@copilot can you make a PR that merges into this PR's branch that does the following:

Rewrite S3Client.ts methods putObject, createMultipartUpload, uploadPart, listParts, completeMultipartUpload, abortMultipartUpload, deleteObject to use a single argument (object parameters) instead of positional arguments. also include signal (abortcontroller)) in all the above methods. yarn check, yarn typecheck has to be successful when you're done.

and make all methods have single arg params

qxprakash · 2026-07-18T07:04:35Z

@copilot can you make a PR that merges into this PR's branch that does the following:

Rewrite S3Client.ts methods putObject, createMultipartUpload, uploadPart, listParts, completeMultipartUpload, abortMultipartUpload, deleteObject to use a single argument (object parameters) instead of positional arguments. also include signal (abortcontroller)) in all the above methods. yarn check, yarn typecheck has to be successful when you're done.

@mifi I think copilot is testing your patience with AI 😂

mifi · 2026-07-18T09:49:29Z

now it's testing your patience too

qxprakash and others added 14 commits June 17, 2026 13:54

add s3mini client to @uppy/aws-s3 plugin (#6103)

6b9521f

- base branch cleanup and ran `yarn && yarn dedupe` - client copied and tests wired up

give credit

db9506a

remove HTTPCommunication queue and clean index.ts and MultipartUplaod…

e7f62dd

…er.ts

remove tests

a3ca750

@uppy/aws-s3 add queue (#6171)

779c19c

- queue support added for file level concurrency.

update aws-nodejs examples (#6187)

50de6d1

- removed old examples , entirely remove withCustomEndpoints.html as every s3 operations are now handled by s3mini so no need of it. - add new examples and update README.md

@uppy/aws-s3: update examples (#6311)

4939859

- update outdated examples based on latest api changes

remove .claude/worktrees gitlink

1c0c402

chatgpt-codex-connector Bot reviewed Jun 17, 2026

View reviewed changes

Comment thread packages/@uppy/aws-s3/src/index.ts Outdated

Comment thread packages/@uppy/aws-s3/src/s3-client/S3mini.ts

Comment thread packages/@uppy/aws-s3/src/S3Uploader.ts Outdated

qxprakash requested a review from Copilot June 17, 2026 09:12

Copilot started reviewing on behalf of qxprakash June 17, 2026 09:12 View session

qxprakash commented Jun 17, 2026

View reviewed changes

Comment thread .github/workflows/bundlers.yml Outdated

Copilot AI reviewed Jun 17, 2026

View reviewed changes

qxprakash and others added 3 commits June 20, 2026 00:49

exclude minio.test.ts from jsdom test

113a5c5

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

revert body = null in fetcher

f88e037

Merge branch 'restore/pr-6284' of github.com:transloadit/uppy into re…

1b36024

…store/pr-6284

qxprakash requested a review from Copilot June 19, 2026 19:56

Copilot started reviewing on behalf of qxprakash June 19, 2026 19:56 View session

Copilot AI reviewed Jun 19, 2026

View reviewed changes

Merge remote-tracking branch 'origin/main' into restore/pr-6284

5585a79

update comments , update type casing and update examples based on fee…

9f88450

…dback

fix lint based on new biome rules after merge

1d41bcf

qxprakash requested a review from mifi June 19, 2026 20:35

Copilot started work on behalf of mifi June 22, 2026 10:47 View session

Copilot finished work on behalf of mifi June 22, 2026 10:52

mifi reviewed Jun 23, 2026

View reviewed changes

qxprakash added 12 commits July 1, 2026 13:21

Merge branch 'main' into restore/pr-6284

ad8295c

make "Debug packaged uppy dependencies" fail when package.json has

1a255a8

workspace:* entries rather than just logging it

remove utils.ts and remove unused consts from s3-client/consts.ts

9b070d4

apply feedback

9b24d14

update uppy/aws-s3 with vitest v4

842b19e

apply review feedback : add uuid-file-name in presigned and sts mode …

8b6becc

…of upload

apply review feedback : abort upload if it has been after createMulti…

9e24d64

…partUpload request was resolved

Merge branch 'main' into restore/pr-6284

227c86a

Merge branch 'main' into restore/pr-6284

17da535

Merge "main" into "restore/pr-6284 and resolve conflicts

e51c1ad

update imports after merge remove stale files

0c7f02a

lint fix

de7d9c2

mifi and others added 3 commits July 16, 2026 21:03

put signal into missing methods

260716b

and make all methods have single arg params

simplify and fix raxce condtitios

260716d

remove the test to abort upload createMultipartUpload in the edgecase

f28d496

fix and simplify

260719f

Uh oh!

Conversation

qxprakash commented Jun 17, 2026

Uh oh!

changeset-bot Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

socket-security Bot commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

qxprakash commented Jun 19, 2026

Uh oh!

qxprakash commented Jun 19, 2026

Uh oh!

mifi commented Jun 22, 2026

Uh oh!

Copilot AI commented Jun 22, 2026

Uh oh!

mifi commented Jun 22, 2026

Uh oh!

mifi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mifi commented Jul 16, 2026

Uh oh!

qxprakash commented Jul 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mifi commented Jul 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

changeset-bot Bot commented Jun 17, 2026 •

edited

Loading

socket-security Bot commented Jun 19, 2026 •

edited

Loading

qxprakash commented Jul 18, 2026 •

edited

Loading