Skip to content

Draft of new multi dataframe synthesis#415

Draft
qubixes wants to merge 1 commit intodevelopfrom
multiframe
Draft

Draft of new multi dataframe synthesis#415
qubixes wants to merge 1 commit intodevelopfrom
multiframe

Conversation

@qubixes
Copy link
Member

@qubixes qubixes commented Feb 3, 2026

Adds primary and foreign key relationships. See #413

Could be separated out into its own package, though the current implementation (without documentation) is about 100 lines of code, so that might be a bit too light for a new package. It also adds a new demo multi dataset which can be used to explain how it works. A new notebook is also there to show the new feature off.

There are several things to be worked out still. A non-exhaustive list:

  • GMF file / serialization. I'm still not 100% convinced that multiple files is the solution. An SQLite database is also a single file, so...
  • The interface of the classes could be improved for sure. It works well enough as it is, but it lacks a lot of elegance.
  • Efficiency might be a bottleneck. Probably not a big deal, but could become an issue in some circumstances. The current approach is to synthesize the metaframes seperately and connect them later. Some columns are synthesized multiple times this way.
  • No documentation
  • No tests
  • Ruff compliance

Adds primary and foreign key relationships.
@qubixes qubixes requested a review from vankesteren February 3, 2026 20:46
@vankesteren
Copy link
Member

first impression: excellent!! Will do more thorough testing and review later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants