Conversation
* Feat/automate unified doc creation (#3) * refactor: Move all files located in the root to a dedicated folder * chore: Setup development tools and packages * refactor/fix: Remove redundant metadata and fix reference from .json documents - Removed `$schema` and `$id` from nested .json documents - Fixed the reference inside the oneOf block for PSQL v15.0 in the file `specs.json` to point to the correct `spec.json` * feat: implement database schema specification generator with conditional resolution Implement a comprehensive database schema specification generator that processes JSON Schema files with conditional oneOf logic for different database engines and versions. ### Core Features **Schema Resolution System:** - Add JSONRefResolver for handling $ref references with circular detection - Implement ConditionalMerger for oneOf conditional logic resolution - Support both if/then and direct property constraint formats - Add VariantExtractor for database variant discovery from oneOf blocks **Validation Framework:** - Add SchemaValidator with JSON Schema Draft 7 compliance checking - Implement custom validation rules for project-specific requirements - Include validation for required fields and schema structure **Logging Infrastructure:** - Add centralized logging configuration with JSON-based setup - Support both console and file handlers with detailed formatting - Include queue-based logging for performance **CLI Generator:** - Implement SchemaGenerator for end-to-end schema processing - Support multiple database variants (PostgreSQL, MySQL) - Generate resolved schemas without oneOf conditionals * feat: Add generation of all available files in .json format to `output/vmap.json` - Introduced new functionality to generate a .json file that maps all available engines and their corresponndiing versions to an url. The map is saved in the output folder in a singleton file. - Introduced a new configuration module to manage application settings. - Added support for loading configuration from a .env file. - Updated the schema generator and output manager to utilize the new config. - Created example .env file and updated README with configuration instructions. - Implemented tests for configuration handling and output manager functionality. * feat: Add CI workflow and setup actions for Python and uv * Feature/versioned releases (#6) * fix: enforce required environment variables in configuration & pass them into generated schemas dynamically * fix: reorder top-level keys in generated schemas to prioritize $id and $schema fields * fix: moved description field from snapshot to schema definition * fix: enable overriding of environment variables when loading .env file * feat: add Dockerfile, .dockerignore for schema generation and workflow to create images * feat: added workflow to create production images & cleaned workflow to create staging images * refactor(config): Remove unecessary python-dotenv dependency - Removed python-dotenv dependency and lines utilizing it. The pydantic_settings managed the environment and should fail on missing dependencies. * refactor(Dockerfile): Pin UV version and simplify building stage - Refactiored the dockerfile to have a pinned version of UV for consistent builds. - Simplified dockerfile by utilizing UV's prebuilt image * refactor(config): Remove leftover comments --------- Co-authored-by: Matheus-OAMK <t2fema00@students.oamk.fi> * feat: consolidate CI/CD workflows by creating service version images and removing staging image workflow (#9) * feat: consolidate CI/CD workflows by creating service version images and removing staging image workflow * feat: remove staging branch from CI workflow triggers --------- Co-authored-by: Matheus Oliveira <t2fema00@students.oamk.fi>
RepoRover
added
enhancement
There was a problem hiding this comment.
Pull request overview
This PR introduces a significant architectural refactoring that replaces the conditional schema merging approach with a registry-based system. Instead of using a single root schema with oneOf conditionals that get resolved at generation time, the system now directly loads engine/version-specific spec files from a registry. This simplifies the generation logic by eliminating the ConditionalMerger component entirely.
Key changes:
- Registry-based discovery using
_registry_.jsoninstead of rootspecs.jsonwith conditional merging - Direct loading of engine/version-specific specs from
schemas/engines/{engine}/{version}/spec.json - New output structure including schema map (
smap.json), manifest schema, and config schemas with injected$idfields
Reviewed changes
Copilot reviewed 36 out of 36 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_resolver.py | Added type ignore comment for method assignment |
| tests/test_output_manager.py | Renamed version_map to engine_map, vmap to smap, updated test expectations |
| tests/test_integration_production.py | Updated paths to include 'v' prefix, added project schema validations, simplified assertions |
| tests/test_conditional_merger*.py | Deleted - ConditionalMerger no longer exists |
| tests/conftest.py | Restructured to match new architecture with registry and project schemas |
| docs/specs.json | Deleted - replaced by registry-based approach |
| docs/schemas/registry.json | New engine/version registry file |
| docs/schemas/project/* | New project-level schemas (manifest, config/base, config/engines) |
| docs/schemas/engines/postgresql/v15.0/spec.json | Updated to be self-contained with required fields |
| docs/schemas/engines/postgresql/v15.0/components/*.json | Changed number to integer, added primary_key constraint type, updated foreign key structure |
| docs/schemas/base/*.json | Deleted - replaced by project schemas and engine specs |
| docs/examples/*.json | Deleted - replaced by YAML examples |
| docs/examples/* | New YAML example files demonstrating schema format |
| docs/README.md | Updated to document new architecture and schema map |
| docs/ARCHITECTURE.md | New comprehensive architecture documentation |
| database_schema_spec/resolution/variant_extractor.py | Updated to read from registry file instead of database schema file |
| database_schema_spec/resolution/resolver.py | Removed ConditionalMerger logic and oneOf resolution |
| database_schema_spec/resolution/conditional_merger.py | Deleted - no longer needed |
| database_schema_spec/resolution/init.py | Removed ConditionalMerger from exports |
| database_schema_spec/io/output_manager.py | Added docs_dir, write_project_schema, write_schema_map methods; renamed version_map to engine_map |
| database_schema_spec/core/config.py | Updated file naming configuration for new structure |
| database_schema_spec/cli/generator.py | Refactored to generate project schemas and schema map; simplified variant generation |
| README.md | Updated to document new output structure and schema map |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request refactors the schema generation workflow and output structure to support a more modular, engine/version-aware approach. The changes remove conditional merging logic, introduce a registry-based schema discovery, and add new output files including a schema map and project-level schemas. The code now generates fully resolved engine/version specs and organizes outputs for better discoverability and extensibility.
Major changes to schema generation and output:
docs/schemas/with a_registry_.jsonfile for engine/version registration, replacing the previous single root schema. Output files include a schema map (smap.json), manifest, config schemas, and fully resolved engine/version specs. [1] [2]ConditionalMerger; variant resolution is handled directly by loading engine/version-specific spec files. [1] [2] [3]Improvements to output management and file naming:
OutputManagernow receives both output and docs directories, writes project schemas with injected$idfields, and generates a comprehensive schema map instead of a version map. Reserved directory names are excluded from engine discovery. [1] [2] [3]General codebase simplification: