[wip] rekey all cosmos data from .id=UUID to .id=resourceID #3651
Conversation
deads2k
force-pushed
the
cs-76-standardize-metadata
branch
from
4f8d363 to
56657dd
Compare
|
/retest |
deads2k
force-pushed
the
cs-76-standardize-metadata
branch
4 times, most recently
from
c04ac35 to
2ac4245
Compare
|
/retest |
This change performs a data migration on read to switch from reading based on a cosmos search of fields matching a pattern to direct item reads based on key. On a failure to lookup by cosmosID and success reading based on existing search, we create a new item and delete the original. Since old frontends use the cosmosUID as fully opaque, the new ID doesn't cause problems for old frontends even on rollback. When the frontend starts, we find all subscriptions, then search for all resources in each subscription using the untyped client listRecursive, then read each item to trigger the migration.
deads2k
force-pushed
the
cs-76-standardize-metadata
branch
from
2ac4245 to
80634a3
Compare
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: deads2k The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@deads2k: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
This change performs a data migration on read to switch from reading based on a cosmos search of fields matching a pattern to direct item reads based on key. On a failure to lookup by cosmosID and success reading based on existing search, we create a new item and delete the original.
Since old frontends use the cosmosUID as fully opaque, the new ID doesn't cause problems for old frontends even on rollback.
When the frontend starts, we find all subscriptions, then search for all resources in each subscription using the untyped client listRecursive, then read each item to trigger the migration.
If the create succeeds and the delete fails, then resulting GETs will produce an ambiguous result error (a pre-existing handled error, so old frontends are ok) and we'll have to resolve the conflict manually.
Still needs more test coverage for types, but since we merged #3662 we can see this works properly for clusters and operations. Need to add coverage (in a separate PR first to prove it works) for nodepools and externalauths.