feat(nextcloud): trigger endpoint for bulk migrations#4728
Open
feat(nextcloud): trigger endpoint for bulk migrations#4728
Conversation
Twake users migrating from Nextcloud need a way to import their entire file tree into their Cozy. Konnectors and stack workers are bounded by job timeouts, so the heavy lifting has to live outside the stack. This endpoint is the user-facing entry point: it validates the credentials synchronously, persists the account, creates a tracking document, and hands the work off to a separate migration service via RabbitMQ. The service drives the existing /remote/nextcloud/:account/* routes and updates the tracking document, which the Settings UI watches over realtime.
641380f to
e01ad4c
Compare
Managed Nextcloud providers (e.g. thegood.cloud) strip the optional user_status app, so the probe returned 404 on otherwise-valid instances. Combined with the coarse "non-200 = invalid auth" classifier, every migration request against these hosts surfaced as a misleading 401 "credentials invalid". Probe /ocs/v2.php/cloud/user (OCS Core, cannot be disabled) and narrow auth-failure classification to 401/403 only.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
POST /remote/nextcloud/migrationto start a Nextcloud-to-Cozy bulk migration from the Settings UI.The endpoint probes the supplied credentials against the Nextcloud OCS
cloud/userendpoint, upserts anio.cozy.accountsdocument with the resolved WebDAV user ID, creates anio.cozy.nextcloud.migrationstracking document inpendingstate, and publishes anextcloud.migration.requestedcommand to the newmigrationRabbitMQ exchange.A separate migration service consumes the command and drives the existing
/remote/nextcloud/:account/*routes to perform the transfer, writing progress back to the tracking document.The Settings UI monitors the document in real time to display progress. Only one migration can be in flight per instance; failed migrations do not block new attempts.
API
POST /remote/nextcloud/migration(requiresPOST io.cozy.nextcloud.migrations){ "nextcloud_url": "https://nextcloud.example.com", "nextcloud_login": "alice", "nextcloud_app_password": "xxxxx-xxxxx-xxxxx-xxxxx-xxxxx", "source_path": "/" }source_pathis optional and defaults to/. The password should be a Nextcloud app password.201 Created
{ "data": { "id": "d4e5f6a7b8c94d0ea1b2c3d4e5f6a7b8", "type": "io.cozy.nextcloud.migrations", "attributes": { "status": "pending", "target_dir": "/Nextcloud", "progress": { "files_imported": 0, "files_total": 0, "bytes_imported": 0, "bytes_total": 0 }, "errors": [], "skipped": [], "started_at": null, "finished_at": null } } }Error responses
pendingorrunningmigration already exists for this instancefailedbefore returningRabbitMQ contract
Exchange
migration, routing keynextcloud.migration.requested. Payload (the consumer is responsible for declaring its queue and binding):{ "migrationId": "d4e5f6a7b8c94d0ea1b2c3d4e5f6a7b8", "workplaceFqdn": "alice.cozy.example.com", "accountId": "a1b2c3d4e5f6", "sourcePath": "/", "timestamp": 1712563200 }Credentials are never in the payload; they live in the
io.cozy.accountsdocument referenced byaccountId.MessageIDis set to the migration ID for cross-system tracing.Credentials probe
The OCS probe itself is not new. It already existed as a lazy fallback behind
(nc *NextCloud).fetchUserID(), called fromfillUserIDonly when an existing Nextcloud account document was missing its cachedwebdav_user_id. In practice that path was almost never exercised, so a latent bug sat there undetected: the probe targeted/ocs/v2.php/apps/user_status/api/v1/user_status, and any non-200 response (including a 404 from a managed Nextcloud host that strips the optionaluser_statusapp) was turned intowebdav.ErrInvalidAuth.What this PR adds is the synchronous, user-facing path: the migration trigger endpoint calls
FetchUserIDWithCredentialson the caller-supplied credentials before persisting anything, so the probe is now on the critical path of every trigger. That exposure turned the latent classification bug into a user-visible "credentials are invalid" 401 for anyone pointing the feature at a managed Nextcloud likethegood.cloud.The follow-up commit on this branch fixes it by switching the probe to OCS Core (
/ocs/v2.php/cloud/user, which cannot be disabled by an admin) and narrowing the auth-failure classification to 401/403 only, so other non-2xx statuses bubble up with their real cause instead of being silently relabelled. Both code paths (the old lazy fallback and the new synchronous call) benefit from the fix.Validated end-to-end against a real Nextcloud instance: the probe, the encrypted-at-rest account persistence, the tracking document, and the RabbitMQ publish with no credentials in the payload all behave as documented.