Conversation
…-status index: this is supposed to be handled explicitly.
…ry log when refreshing the migration status at the end.
asnare
added
enhancement
asnare
had a problem deploying
to
account-admin
GitHub Actions
Failure
|
✅ 56/56 passed, 6 flaky, 4 skipped, 4h11m42s total Flaky tests:
Running from acceptance #7841 |
nfx
requested changes
Nov 11, 2024
It should return all the tables as a snapshot.
asnare
had a problem deploying
to
account-admin
GitHub Actions
Failure
…ng the migration index is available during encoding.
asnare
had a problem deploying
to
account-admin
GitHub Actions
Failure
asnare
had a problem deploying
to
account-admin
GitHub Actions
Failure
asnare
had a problem deploying
to
account-admin
GitHub Actions
Failure
asnare
had a problem deploying
to
account-admin
GitHub Actions
Failure
asnare
had a problem deploying
to
account-admin
GitHub Actions
Failure
asnare
had a problem deploying
to
account-admin
GitHub Actions
Failure
asnare
had a problem deploying
to
account-admin
GitHub Actions
Failure
asnare
had a problem deploying
to
account-admin
GitHub Actions
Failure
JCZuurmond
approved these changes
Dec 12, 2024
JCZuurmond
reviewed
Dec 12, 2024
| @job_task(job_cluster="user_isolation") | ||
| def verify_progress_tracking_prerequisites(self, ctx: RuntimeContext) -> None: | ||
| """Verify the prerequisites for running this job on the table migration cluster are fulfilled.""" | ||
| ctx.verify_progress_tracking.verify(timeout=dt.timedelta(hours=1)) |
Contributor
There was a problem hiding this comment.
This forces the UCX catalog to be created before table migration, while it was not a pre-requiste before
1 task
asnare
had a problem deploying
to
account-admin
GitHub Actions
Failure
JCZuurmond
had a problem deploying
to
account-admin
GitHub Actions
Failure
1 task
github-merge-queue bot
pushed a commit
that referenced
this pull request
Dec 13, 2024
## Changes Exclude TACL migration in table migration integration tests because these were not asserted, and to speed up the tests and reduce flakiness ### Linked issues Attempt to reduce flakiness blocking CI in #3239 Similar to #3437 in the sense that both PR scope integration tests to a smaller set of resources ### Tests - [x] modified integration tests
Contributor
|
@asnare : When you are back, rebase with main and check if the CI still fails, the linked PR make the tests more robust |
Merged
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
The table-migration workflows already contained tasks at the end that log information about tables that still need to be migrated. The primary purpose of this PR is to update these workflows so they also capture updated progress information into the history log.
Other changes include:
Updating the (singleton) encoder for table-history so that initialisation doesn't trigger an implicit refresh of theMoved to Refactor refreshing of migration-status information for tables, eliminate another redundant refresh. #3270.TableMigrationStatusdata. Instead this is controlled at the workflow level, as intended.Linked issues
Conflicts with #3200 (will need rebasing).(Resolved.)Functionality
updated documentation
modified existing workflows:
migrate-tablesmigrate-external-hiveserde-tables-in-place-experimentalmigrate-external-tables-ctasscan-tables-in-mounts-experimentalmigrate-tables-in-mounts-experimentalTests