Add unified cake_migrations table support with BC autodetect by dereuromark · Pull Request #965 · cakephp/migrations

dereuromark · 2025-11-22T23:04:39Z

Summary

This PR introduces a consolidated migration tracking approach for v5.0, addressing #822.

Note: this is just one possible approach that combines BC with a clear forwards-approach.
@markstory I am still curious about your approach and where it could be a better fit.

New Features

cake_migrations table: Single table with plugin column to track all migrations (app + plugins)
UnifiedMigrationsTableStorage: New storage class for unified table operations
migrations upgrade command: Migrates data from legacy phinxlog tables to the new unified table

Backward Compatibility (Zero Breaking Changes)

Autodetect mode (default): If any phinxlog or *_phinxlog table exists, legacy mode is used automatically
Fresh installations: Automatically use the new cake_migrations table
No action required on upgrade - existing apps continue working with their phinxlog tables

Configuration

// config/app.php
'Migrations' => [
    'legacyTables' => null,  // (default) Autodetect - BC for existing, new table for fresh
    // 'legacyTables' => false, // Force unified cake_migrations table
    // 'legacyTables' => true,  // Force legacy phinxlog tables
]

Upgrade Workflow (for existing apps wanting to migrate)

Upgrade to v5.0 (existing apps continue working with phinxlog - no changes needed)
When ready, run bin/cake migrations upgrade --dry-run to preview
Run bin/cake migrations upgrade to migrate data
Set 'Migrations' => ['legacyTables' => false] in config
Application now uses unified cake_migrations table

New Table Schema

CREATE TABLE cake_migrations (
    id INT AUTO_INCREMENT PRIMARY KEY,
    version BIGINT NOT NULL,
    migration_name VARCHAR(100) NULL,
    plugin VARCHAR(100) NULL,        -- NULL for app, 'PluginName' for plugins
    start_time TIMESTAMP NULL,
    end_time TIMESTAMP NULL,
    breakpoint TINYINT(1) NOT NULL DEFAULT 0,
    UNIQUE KEY version_plugin_unique (version, plugin)
);

Test Plan

TODO

Add documentation for the upgrade process

This change introduces a consolidated migration tracking approach for v5.0: **New Features:** - `cake_migrations` table: Single table with `plugin` column to track all migrations - `UnifiedMigrationsTableStorage`: New storage class for unified table operations - `migrations upgrade` command: Migrates data from legacy phinxlog tables **Backward Compatibility:** - Autodetect mode (default): If any `phinxlog` or `*_phinxlog` table exists, legacy mode is used automatically - no breaking changes on upgrade - Fresh installations automatically use the new `cake_migrations` table **Configuration:** - `Migrations.legacyTables = null` (default): Autodetect - `Migrations.legacyTables = false`: Force unified table - `Migrations.legacyTables = true`: Force legacy phinxlog tables **Upgrade workflow:** 1. Upgrade to v5.0 (existing apps continue working with phinxlog) 2. Run `bin/cake migrations upgrade` to migrate data 3. Set `Migrations.legacyTables = false` in config 4. Application now uses unified `cake_migrations` table Refs #822 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

src/Db/Adapter/AbstractAdapter.php

markstory · 2025-11-23T04:20:59Z

src/Db/Adapter/AbstractAdapter.php

+     */
+    protected function isUsingUnifiedTable(): bool
+    {
+        $config = Configure::read('Migrations.legacyTables');


This should be added to the documentation and the upgrade guide, as we'll need a guide so folks know how to upgrade and rollback between schema if necessary.

src/Db/Adapter/UnifiedMigrationsTableStorage.php

src/Util/UtilTrait.php

tests/bootstrap.php

src/Command/MigrationsUpgradeCommand.php

- Simplify NULL handling using 'plugin IS' => $this->plugin pattern - Remove cache complexity from UtilTrait - Replace empty() with explicit === [] check - Simplify upgradeTable() to no-op for new table format - Use in_array for phinxlog detection (simpler than loop) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Uses the new hasTable() method from CakePHP 5.3 as suggested in the PR feedback. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

- Add LEGACY_TABLES CI matrix option to test both modes - Use schemaDialect()->hasTable() for phinxlog detection - Add documentation for unified table upgrade process - Add basic test for UnifiedMigrationsTableStorage - Update bootstrap to read LEGACY_TABLES from environment 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

When `Migrations.legacyTables` is set to `false`, the upgrade command is not needed since the user has already opted into the unified table. Only show it when in autodetect mode (null) or legacy mode (true). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

dereuromark · 2025-11-23T23:32:45Z

I tested it on dereuromark/cakephp-sandbox#99
and it seems to work already really well.
The config flag also makes sure it is clean afterwards (no legacy stuff remaining).

A rerun with

bin/cake migrations upgrade --drop-tables

also cleans out the old empty ones.

Added LEGACY_TABLES=false matrix option for MySQL PHP 8.3

The unified table feature works, but tests have extensive hardcoded phinxlog assumptions (99 failures). Removing CI matrix for now. Partial fixes to BakeMigrationDiffCommandTest to show the pattern: - Add UtilTrait to get correct schema table name - Update table cleanup to include both table types - Update queries to use dynamic table name TODO: Follow-up PR needed to update all tests for both table modes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

- Add helper methods to TestCase for migration table operations: - getMigrationsTableName() - gets correct table name based on mode - clearMigrationRecords() - clears records with plugin filtering - getMigrationRecordCount() - counts records with plugin filtering - insertMigrationRecord() - inserts with correct structure - isUsingUnifiedTable() - checks current mode - Update command tests to use new helper methods: - MigrateCommandTest - RollbackCommandTest - StatusCommandTest - MarkMigratedTest - DumpCommandTest - BakeMigrationDiffCommandTest - Add cleanup for both phinxlog and cake_migrations tables - Re-add LEGACY_TABLES=false CI matrix option 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Check Migrations.legacyTables config to use the correct table name. When set to false, use the new 'cake_migrations' table instead of 'phinxlog'. See: cakephp/migrations#965 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

- Update Util::tableName() to return cake_migrations when unified table mode is enabled - Update Manager::cleanupMissingMigrations() to use correct table name and filter by plugin - Skip cake_migrations table in bake snapshot/diff commands (like phinxlog tables) - Update TableFinder to skip cake_migrations table - Update Migrator to not drop cake_migrations during table cleanup - Update tests to use helper methods for migration record operations - Add mode-aware assertions in adapter tests - Skip some Migrator tests in unified mode that test legacy-specific behavior 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

markstory

I got part way through.

src/Command/UpgradeCommand.php

src/Command/MigrationsUpgradeCommand.php

src/Command/UpgradeCommand.php

docs/en/upgrading-to-builtin-backend.rst

Co-authored-by: Mark Story <mark@mark-story.com>

I thought the conditional logic in Manager, and unwrapping decorators pointed to a missing abstraction method. Since we're in a major anyways, I could extend the interface and have better layering as well.

Table creation we broken for everything except mysql. Now it isn't.

Tests fail without this directory being around.

markstory · 2025-12-07T04:08:05Z

@dereuromark Do you know what is up with the tests? They don't fail for me locally.

dereuromark · 2025-12-07T04:17:09Z

This one lock file is super annoying to me too
The directory MigrationsDiffDecimalChange exists but only has a .gitkeep file. The test copies the initial migration file from the comparisons folder to this directory, but then the cleanup in setUp() or tearDown() is deleting these files.

I will take another quick look

The checkSync() method compares the last migration file version against the last migrated version. After the test deletes the migration record from the phinxlog table, the migration file still exists, causing checkSync() to fail because it sees an unmigrated file. The fix deletes the migration file after running migrate and before baking the diff, so checkSync() passes correctly.

dereuromark · 2025-12-07T07:49:41Z

claude fixed it:

Summary

Root Cause: The runDiffBakingTest() method was failing because the checkSync() method in BakeMigrationDiffCommand checks if the last migration file version matches the last migrated version. The test was:

Copying the diff migration file (e.g., 20160415220805_TheDiffDefaultMysql.php)
Running migrate (which applies all migrations including the copied one)
Deleting the migration record for version 20160415220805 from the database
Trying to bake a new diff

After step 3, the migration FILE still existed with version 20160415220805, but the last migrated version was now 20160414193900 (the previous migration). The checkSync() method saw this discrepancy and failed with "Your migrations history is not in sync with your migrations files."

Fix: Added unlink($destination) after deleting the migration record from the database. This removes the migration file so that checkSync() sees the last file as version 20160414193900, which matches the last migrated version.

markstory · 2025-12-07T20:52:31Z

Thanks for figuring out what was going on there. I wasn't able to reproduce locally.

After step 3, the migration FILE still existed with version 20160415220805, but the last migrated version was now 20160414193900 (the previous migration).

I don't see the 20160414193900 migration entry locally. This might be why I was getting different behavior.

markstory reviewed Nov 23, 2025

View reviewed changes

ADmad reviewed Nov 23, 2025

View reviewed changes

src/Command/MigrationsUpgradeCommand.php Outdated Show resolved Hide resolved

dereuromark and others added 6 commits November 23, 2025 11:06

Use hasTable() for phinxlog detection

79ec7d4

Uses the new hasTable() method from CakePHP 5.3 as suggested in the PR feedback. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Remove development notes

a034f31

Merge branch '5.x' into 5.x-unified-migrations-table

d756e1f

dereuromark and others added 4 commits November 24, 2025 00:41

Cleanup.

c0ca79a

Remove undefined resetOutput() call from StatusCommandTest

42f498f

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

dereuromark mentioned this pull request Nov 24, 2025

Support unified cake_migrations table dereuromark/cakephp-test-helper#30

Closed

2 tasks

markstory reviewed Nov 25, 2025

View reviewed changes

dereuromark added this to the 5.x (CakePHP 5.x, builtin-backend) milestone Nov 25, 2025

dereuromark added the enhancement label Nov 25, 2025

dereuromark and others added 10 commits November 29, 2025 18:57

Update src/Command/MigrationsUpgradeCommand.php

7b9be94

Co-authored-by: Mark Story <mark@mark-story.com>

Update src/Command/MigrationsUpgradeCommand.php

ab0af12

Co-authored-by: Mark Story <mark@mark-story.com>

Update docs/en/upgrading-to-builtin-backend.rst

b1b795c

Co-authored-by: Mark Story <mark@mark-story.com>

Update docs/en/upgrading-to-builtin-backend.rst

dc48661

Co-authored-by: Mark Story <mark@mark-story.com>

Update docs/en/upgrading-to-builtin-backend.rst

6fc996b

Co-authored-by: Mark Story <mark@mark-story.com>

Update src/Command/MigrationsUpgradeCommand.php

c1fc1b9

Co-authored-by: Mark Story <mark@mark-story.com>

Update src/Db/Adapter/AbstractAdapter.php

7643c3b

Co-authored-by: Mark Story <mark@mark-story.com>

Feedback and fixes

62738aa

I thought the conditional logic in Manager, and unwrapping decorators pointed to a missing abstraction method. Since we're in a major anyways, I could extend the interface and have better layering as well.

Merge branch '5.x' into 5.x-unified-migrations-table

0c3406c

Add more tests.

62ca3fe

markstory added 7 commits December 5, 2025 00:27

Move files around and add tests

f53fcdd

Table creation we broken for everything except mysql. Now it isn't.

Add loadbearing gitkeep file.

567aaa5

Tests fail without this directory being around.

See if this test is causing CI to fail and emit warnings.

d38c20e

Narrow type with assert()

6e09f62

Unskip test, it was the problem after all.

2f27c91

Fix quotes

6b4ffb7

Add another directory.

8fe27e2

dereuromark marked this pull request as ready for review December 7, 2025 07:47

dereuromark added 2 commits December 7, 2025 08:50

Remove file.

7e6b59c

Auto create folder. ensures a more stable setup.

96883d1

markstory merged commit 79f7554 into 5.x Dec 7, 2025
14 checks passed

markstory deleted the 5.x-unified-migrations-table branch December 7, 2025 20:58

Conversation

dereuromark commented Nov 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

New Features

Backward Compatibility (Zero Breaking Changes)

Configuration

Upgrade Workflow (for existing apps wanting to migrate)

New Table Schema

Test Plan

TODO

Uh oh!

Uh oh!

Uh oh!

markstory Nov 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dereuromark commented Nov 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

markstory left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

markstory commented Dec 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dereuromark commented Dec 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dereuromark commented Dec 7, 2025

Uh oh!

markstory commented Dec 7, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dereuromark commented Nov 22, 2025 •

edited

Loading

dereuromark commented Nov 23, 2025 •

edited

Loading

markstory commented Dec 7, 2025 •

edited

Loading

dereuromark commented Dec 7, 2025 •

edited

Loading