Skip to content

init: schema generation logic & CI automation#10

Merged
RepoRover merged 3 commits intomainfrom
dev
Sep 2, 2025
Merged

init: schema generation logic & CI automation#10
RepoRover merged 3 commits intomainfrom
dev

Conversation

@RepoRover
Copy link
Contributor

This pull request introduces a comprehensive setup for a Python-based database schema specification generator, focusing on project structure, environment configuration, automation, and developer experience. The most important changes are grouped below:

Project Initialization and Structure

  • Added a new README.md with clear instructions, project overview, development workflow, environment variable documentation, and output expectations.
  • Introduced __init__.py files and package-level docstrings to support test discovery and clarify module exports, including the main SchemaGenerator class. [1] [2] [3]

Core Functionality

  • Implemented the main orchestration class SchemaGenerator in database_schema_spec/cli/generator.py, which manages variant extraction, schema resolution, conditional logic, validation, and output generation.

Environment and Dependency Management

  • Added .env.example for environment variable configuration, notably the required BASE_URL.
  • Specified Python version in .python-version and created a robust .dockerignore for cleaner Docker builds. [1] [2]
  • Provided a multi-stage Dockerfile for efficient schema generation and output-only final images.

Automation and Quality Assurance

  • Established CI workflows for dependency installation, linting, formatting, type checking, and testing using GitHub Actions (ci.yaml and create-service-version-image.yaml). [1] [2]
  • Added a pre-commit configuration for automated code quality checks (linting, formatting, import sorting, type checking) via uv, ruff, and pyright.

References:
[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]

Matheus-OAMK and others added 3 commits August 18, 2025 10:16
* refactor: Move all files located in the root to a dedicated  folder

* chore: Setup development tools and packages

* refactor/fix: Remove redundant metadata and fix reference from .json documents

- Removed `$schema` and `$id` from nested .json documents
- Fixed the reference inside the oneOf block for PSQL v15.0 in the file `specs.json` to point to the correct `spec.json`

* feat: implement database schema specification generator with conditional resolution

Implement a comprehensive database schema specification generator that processes JSON Schema files with conditional oneOf logic for different database engines and versions.

### Core Features

**Schema Resolution System:**
- Add JSONRefResolver for handling $ref references with circular detection
- Implement ConditionalMerger for oneOf conditional logic resolution
- Support both if/then and direct property constraint formats
- Add VariantExtractor for database variant discovery from oneOf blocks

**Validation Framework:**
- Add SchemaValidator with JSON Schema Draft 7 compliance checking
- Implement custom validation rules for project-specific requirements
- Include validation for required fields and schema structure

**Logging Infrastructure:**
- Add centralized logging configuration with JSON-based setup
- Support both console and file handlers with detailed formatting
- Include queue-based logging for performance

**CLI Generator:**
- Implement SchemaGenerator for end-to-end schema processing
- Support multiple database variants (PostgreSQL, MySQL)
- Generate resolved schemas without oneOf conditionals

* feat: Add generation of all available files in .json format to `output/vmap.json`

- Introduced new functionality to generate a .json file that maps all available engines and their corresponndiing versions to an url. The map is saved in the output folder in a singleton file.
- Introduced a new configuration module to manage application settings.
- Added support for loading configuration from a .env file.
- Updated the schema generator and output manager to utilize the new config.
- Created example .env file and updated README with configuration instructions.
- Implemented tests for configuration handling and output manager functionality.

* feat: Add CI workflow and setup actions for Python and uv
* fix: enforce required environment variables in configuration & pass them into generated schemas dynamically

* fix: reorder top-level keys in generated schemas to prioritize $id and $schema fields

* fix: moved description field from snapshot to schema definition

* fix: enable overriding of environment variables when loading .env file

* feat: add Dockerfile, .dockerignore for schema generation and workflow to create images

* feat: added workflow to create production images & cleaned workflow to create staging images

* refactor(config): Remove unecessary python-dotenv dependency

- Removed python-dotenv dependency and lines utilizing it. The pydantic_settings managed the environment and should fail on missing dependencies.

* refactor(Dockerfile): Pin UV version and simplify building stage

- Refactiored the dockerfile to have a pinned version of UV for consistent builds.
- Simplified dockerfile by utilizing UV's prebuilt image

* refactor(config): Remove leftover comments

---------

Co-authored-by: Matheus-OAMK <t2fema00@students.oamk.fi>
…and removing staging image workflow (#9)

* feat: consolidate CI/CD workflows by creating service version images and removing staging image workflow

* feat: remove staging branch from CI workflow triggers
@RepoRover RepoRover merged commit 9bd1124 into main Sep 2, 2025
5 checks passed
RepoRover added a commit that referenced this pull request Dec 10, 2025
* Feat/automate unified doc creation (#3)

* refactor: Move all files located in the root to a dedicated  folder

* chore: Setup development tools and packages

* refactor/fix: Remove redundant metadata and fix reference from .json documents

- Removed `$schema` and `$id` from nested .json documents
- Fixed the reference inside the oneOf block for PSQL v15.0 in the file `specs.json` to point to the correct `spec.json`

* feat: implement database schema specification generator with conditional resolution

Implement a comprehensive database schema specification generator that processes JSON Schema files with conditional oneOf logic for different database engines and versions.

### Core Features

**Schema Resolution System:**
- Add JSONRefResolver for handling $ref references with circular detection
- Implement ConditionalMerger for oneOf conditional logic resolution
- Support both if/then and direct property constraint formats
- Add VariantExtractor for database variant discovery from oneOf blocks

**Validation Framework:**
- Add SchemaValidator with JSON Schema Draft 7 compliance checking
- Implement custom validation rules for project-specific requirements
- Include validation for required fields and schema structure

**Logging Infrastructure:**
- Add centralized logging configuration with JSON-based setup
- Support both console and file handlers with detailed formatting
- Include queue-based logging for performance

**CLI Generator:**
- Implement SchemaGenerator for end-to-end schema processing
- Support multiple database variants (PostgreSQL, MySQL)
- Generate resolved schemas without oneOf conditionals

* feat: Add generation of all available files in .json format to `output/vmap.json`

- Introduced new functionality to generate a .json file that maps all available engines and their corresponndiing versions to an url. The map is saved in the output folder in a singleton file.
- Introduced a new configuration module to manage application settings.
- Added support for loading configuration from a .env file.
- Updated the schema generator and output manager to utilize the new config.
- Created example .env file and updated README with configuration instructions.
- Implemented tests for configuration handling and output manager functionality.

* feat: Add CI workflow and setup actions for Python and uv

* Feature/versioned releases (#6)

* fix: enforce required environment variables in configuration & pass them into generated schemas dynamically

* fix: reorder top-level keys in generated schemas to prioritize $id and $schema fields

* fix: moved description field from snapshot to schema definition

* fix: enable overriding of environment variables when loading .env file

* feat: add Dockerfile, .dockerignore for schema generation and workflow to create images

* feat: added workflow to create production images & cleaned workflow to create staging images

* refactor(config): Remove unecessary python-dotenv dependency

- Removed python-dotenv dependency and lines utilizing it. The pydantic_settings managed the environment and should fail on missing dependencies.

* refactor(Dockerfile): Pin UV version and simplify building stage

- Refactiored the dockerfile to have a pinned version of UV for consistent builds.
- Simplified dockerfile by utilizing UV's prebuilt image

* refactor(config): Remove leftover comments

---------

Co-authored-by: Matheus-OAMK <t2fema00@students.oamk.fi>

* feat: consolidate CI/CD workflows by creating service version images and removing staging image workflow (#9)

* feat: consolidate CI/CD workflows by creating service version images and removing staging image workflow

* feat: remove staging branch from CI workflow triggers

---------

Co-authored-by: Matheus Oliveira <t2fema00@students.oamk.fi>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants