Skip to content

feat: Introduce Multi-file Project Configurations#5

Closed
wsidl wants to merge 29 commits intorelease/0.2.0from
feat/multi-file
Closed

feat: Introduce Multi-file Project Configurations#5
wsidl wants to merge 29 commits intorelease/0.2.0from
feat/multi-file

Conversation

@wsidl
Copy link
Copy Markdown
Owner

@wsidl wsidl commented Sep 16, 2024

Summary

The project file will be required to support the generation of a common input sources but requiring different parameters to express alternate and/or conflicting patterns. This feature will introduce a !include tag that would allow including the content of a nearby file.

Current Behaviour

Requires re-implementing a project file with the data sources and making minor modifications. For example, project_1.yaml could have two objects with 2 fields each, where obj2.field2 is a status of obj1.field2:

obj1:
  type: object
  properties:
    field1:
      type: string
    field2:
      type: integer
obj2:
  type: object
  properties:
    field1:
      type: static
      value: "sensor"
    field2:
      type: string
      expression: "['below, 'above'][(obj1().field2 > 0) | int]"

Creating a another project with the same objects, output types, and configurations but with a slight modification (like the following means copying the entire file.

...
obj2:
    ...
    field2:
      type: string
      expression: "['even', 'odd'][obj2().field2 % 2 | int]"

The problem this creates is a maintenance issue if the structure of either object changes, it will need to be changed for all files.

Expected Behaviour

Instead of maintaining all of these individual files, it would be possible to create a single set of object definitions with known parameters and include those into the project file with overridden modifications.

Using the above example, each object can be their own file:

obj1.yaml

type: object
properties:
  field1:
    type: string
  field2:
    type: integer

obj2.yaml

type: object
properties:
  field1:
    type: static
    value: sensor
  field2:
    type: string

Then each project file can include it into their project with modifications.

project1.yaml

obj1: << !include obj1.yaml
obj2: << !include obj2.yaml
  field2:
    expression: "['below, 'above'][(obj1().field2 > 0) | int]"

`project2.yaml

obj1: << !include obj1.yaml
obj2: << !include obj2.yaml
  field2:
    expression: "['even', 'odd'][obj2().field2 % 2 | int]"

so if modifications occur to obj1 that could potentially impact downstream logic, it would be modified in one place with only individual modifications where overrides take place.

Additional Notes

Since YAML also supports multi-document files, the !include tag can also include an index of the document within the file to reference.

@wsidl wsidl self-assigned this Sep 16, 2024
@wsidl wsidl linked an issue Sep 16, 2024 that may be closed by this pull request
@wsidl wsidl changed the base branch from main to release/0.2.0 February 13, 2025 21:35
@wsidl wsidl marked this pull request as draft February 13, 2025 21:57
wsidl and others added 3 commits March 2, 2025 15:37
* fix: added a `force` parameter on the render method to allow re-rendering a value during an iteration.

* feat: `generate` method accepts keyword arguments.

Supports complex objects that should pass context of the parent object to children. Supports a List Generator to pass an index to children generators.
add test workflow for Github Actions
@wsidl wsidl removed this from the Release 0.2.0 - Complex Outputs milestone Mar 5, 2025
wsidl and others added 5 commits March 5, 2025 22:30
* feat: introduce hidden flags on properties
* feat: add XML formatter
# New Features

* XML Formatter
* Hidden Property Flag

# Fixes

* Github reads README from .github dir
* add unit tests for expression datetime filter handling formats
* remove series comparator as more work is necessary to handle multiple values over a series, especially when retrieving the child properties of an object
* add docs on introduced filters and update `iter` filter to support more generic functions
* fix: clear root document in the XML formatter when multiple events are provided
wsidl and others added 5 commits March 9, 2025 22:03
Also introduced custom YAML Tags to flag known document types
# Conflicts:
#	pyproject.toml
#	syntrend/config/model.py
#	syntrend/generators/__init__.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant