Skip to content

Support complex pipelines with grouped inputs (nested imageList / multi-branch composition) #3

@nathabee

Description

@nathabee

Issue: Support complex pipelines with grouped inputs (nested imageList / multi-branch composition)

Problem

Current multi-input support treats all selected images as a single flat imageList.

Example:

  • User loads 6 images

  • Internally represented as:

    imageList = [img1, img2, img3, img4, img5, img6]
    

There is no way to define logical grouping inside that list.

However, real workflows require grouping and staged composition.


Example Use Case

User loads 6 images.

They want:

  • Group1 = [img1, img2]
  • Group2 = [img3, img4]
  • Group3 = [img5, img6]

Now instead of a single vertical pipeline:

imageList → pipeline

They want hierarchical composition:

pipeline3(
  pipeline2(
    pipeline1(Group1),
    Group2
  ),
  Group3
)

Concrete example:

  1. pipeline1: iconize → stack vertically

    • Applies to Group1 (2 images)
    • Result: 1 combined image
  2. pipeline2: stack-on-raw

    • Combine result of pipeline1 with Group2
  3. pipeline3: final composition with Group3

This is not possible with the current flat imageList model.


Root Limitation

The current artifact model supports:

  • image
  • imageList
  • pdf

But not:

  • imageList of imageList
  • grouping metadata
  • branching pipelines
  • nested evaluation

Pipelines are strictly linear.


Required Capability

We need support for grouped inputs and eventually multi-branch pipelines.


Phase 1 — Grouped imageList (minimal viable)

Allow:

type ImageGroup = ImageArtifact[]
type ImageGroupList = ImageGroup[]

Instead of:

imageList: ImageArtifact[]

UI must allow:

  • Select multiple images
  • Group them manually
  • Reorder groups
  • Reorder images inside group

Pipeline start type becomes:

imageGroupList

Operators can then define:

  • imageGroup → image
  • imageGroupList → image
  • imageGroupList → imageGroupList
  • etc.

Phase 2 — Nested pipeline execution

Allow:

  • Pipeline applied to each group independently
  • Composition operators combining results
  • Potentially DAG-style pipelines instead of strict linear list

This may require:

  • Tree-based pipeline model

  • Or explicit “map” operator:

    map(imageGroup, subPipeline)
    

Architectural Questions

  • Do we support nested artifact types?
  • Do we introduce a generic CollectionArtifact<T>?
  • Do we keep linear pipelines but add higher-order operators (map/reduce)?
  • Do we move toward DAG instead of list-of-ops?

Acceptance Criteria

  • User can load 6 images
  • Create 3 groups of 2
  • Apply pipeline to each group independently
  • Combine group results sequentially
  • Maintain type safety in typing system
  • No regression in single-image workflows

Why This Matters

Without grouping:

  • Multi-image support remains shallow
  • Complex compositions require manual pre-processing
  • BeeMage remains linear-only

With grouping:

  • We move toward real compositional pipeline logic
  • Enables batch processing
  • Enables map/reduce-like operators
  • Unlocks true multi-branch architecture

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions