Skip to content

feat: Round-trip parity testing framework (DB → YAML → DB) #39

@Lazialize

Description

@Lazialize

Problem

When a schema is exported from an existing DB via strata export and then used as input for strata generate, diffs are detected even though no actual changes exist. This is caused by information loss during export (type conversions, missing constraints, etc.), and as shown by multiple existing bugs (#23-#28), there is no systematic testing infrastructure to catch these issues.

Proposed Solution

Build a test framework that automatically verifies DB → YAML → DB round-trip fidelity to guarantee export accuracy.

Test Strategy

  1. Round-trip test: Schema YAML → DB apply → export → diff detection → verify zero diff
  2. Reverse round-trip test: Existing DB → export → generate from that YAML → verify zero diff
  3. Cross-dialect test: Apply the same schema to PostgreSQL/MySQL/SQLite and verify export fidelity for each

Test Case Matrix

Test Target PostgreSQL MySQL SQLite
Basic types (INTEGER, VARCHAR, TEXT, etc.)
DECIMAL (precision, scale)
BOOLEAN defaults (true/false)
ENUM types -
DialectSpecific types
AUTO_INCREMENT / SERIAL
CHECK constraints
FOREIGN KEY (ON DELETE)
UNIQUE constraints
Indexes (normal, unique)
Default values (string, number, expression)
NULL / NOT NULL
Views

Implementation Plan

  1. Test helper (src/cli/tests/helpers/roundtrip.rs — new)

    /// Apply schema YAML to DB, export, and verify zero diff
    pub async fn assert_roundtrip_parity(
        schema_yaml: &str,
        dialect: Dialect,
    ) -> Result<()> {
        // 1. Parse schema YAML
        // 2. Generate migration SQL via generate
        // 3. Apply to DB via apply
        // 4. Export to YAML via export
        // 5. Compare original schema with exported schema
        // 6. Assert zero diff
    }
  2. Test files (src/cli/tests/)

    • e2e_roundtrip_postgres.rs — PostgreSQL round-trip tests
    • e2e_roundtrip_mysql.rs — MySQL round-trip tests
    • e2e_roundtrip_sqlite.rs — SQLite round-trip tests
  3. Test schema templates (src/cli/tests/fixtures/roundtrip/)

    • YAML schema files for each test case
    • Dialect-specific expected value files
  4. CI integration: Use testcontainers to automatically run PostgreSQL/MySQL round-trip tests in CI

Existing Bug Regression Coverage

This framework will automate regression tests for the following existing bugs:

Files Affected

  • src/cli/tests/helpers/roundtrip.rs (new)
  • src/cli/tests/e2e_roundtrip_*.rs (new)
  • src/cli/tests/fixtures/roundtrip/ (new)

Alternatives Considered

  • Snapshot tests: Save export result snapshots and compare. High maintenance cost when schemas change
  • SQL-level comparison: Compare generated SQL directly. Dialect SQL syntax differences are too large; YAML-level comparison is more appropriate

Additional Context

  • Corresponds to "Round-trip parity tests (DB → YAML → DB)" in ROADMAP.md
  • Can leverage the existing testcontainers infrastructure (testcontainers 0.26, testcontainers-modules 0.14)
  • Build on existing E2E test patterns (e2e_*.rs)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions