Adding Bundle as another Distribution kind

[DamianReeves]

Idea: Adding Bundle to Distribution

Morphir contains a datastructure known as Distribution which is used to model the definitions and dependencies of a Package.

For reference, the current Elm definition of Distribution looks like the following:

{-| Type that represents a package distribution. Currently the only distribution type we provide is a `Library`.
-}
type Distribution
    = Library PackageName (Dict PackageName (Package.Specification ())) (Package.Definition () (Type ()))

This shows that Distribution currently only has a single case/variant of Library.

The library datastructure allows you to specify the following:

• The package name the library describes
• The dependencies of the library - this is represented by a dictionary of package specifications keyed by the name of the package of the dependency
• The actual definition of the package - this is modeled using a Package.Definition object which captures the definitions of all the Modules in the package.

While this is great for describing a single library, in an ideal world we would have a complimentary Distribution format which describes all the modules used across multiple packages, or put differently, we need to be able to describe not just the specification of dependencies, but the actual full definition of the package dependencies.

What we really want is a way to bundle all the modules/packages together in one distribution.

Suggested Idea

Add an additional case of Distribution known as Bundle that captures all package definitions in the bundle.

This would look something like the following:

type Distribution
    = Library PackageName (Dict PackageName (Package.Specification ())) (Package.Definition () (Type ()))
    | Bundle Dict PackageName (Package.Definition () (Type ()))

Why Add This?

This additional representation, allows us to create the equivalent of a "fatjar" or Javascript bundle (like those produced by webpack or Parcel) which contains all the dependencies of a library in a single "packaged" IR file.

Potential Tooling Support

The Bundle distribution option will allow us to support combining Morphir libraries and standalone IR files using a bundle CLI command.

Imagine a case where we have 2 Morphir IR files:

my-repo
├── App
│   └── morphir-ir.json
└── My.Company.Commons
│   └── morphir-ir.json
└── Shared
    └── morphir-ir.json    

Given the existence of theBundle variant of Distribution we can implement a command such as the one specified below, which would allow you to bundle together two or more Distributions into a single Distribution.

For example:

~/code/my-repo:$ morphir bundle -o dist/morphir-ir.json App/morphir-ir.json My.Company.Commons/morphir-ir.json Shared/morphir-ir.json

Open Question

Would it make more sense to make Bundle modeled more like a dictionary of dependencies and definition (essentially a dictionary of the library contents), like below (or some variation of this)?

type Distribution
    = Library PackageName (Dict PackageName (Package.Specification ())) (Package.Definition () (Type ()))
    | Bundle Dict PackageName { dependencies: (Dict PackageName (Package.Specification ())), definition: (Package.Definition () (Type ()))}

[AttilaMihaly]

Thanks for the great proposal! We just had a similar discussion with @stephengoldbaum where we concluded that we should add a new Distribution type. While the details of what we were thinking of adding are a bit different the fundamental goal is the same: to have a distribution that contains all the information that allows you to run some business functionality.

This means that we need a variant of Distribution that contains the definitions not just the specification. The existing Library variant is catered more towards reusable functionality which is why they provide more flexibility on the dependencies. Maybe we need to do something similar to what Elm does with the split between packages and applications. A package is a reusable set of tools that other packages can depend on while an application is always at the bottom of the dependency graph.

So maybe we need something like this (simplified for readability):

type Distribution
    = Library PackageName (Dict PackageName Package.Specification) Package.Definition
    | Application PackageName (Dict PackageName Package.Definition) Package.Definition

So basically the same structure but dependencies would be definitions instead. We could also aggressively tree-shake the dependencies like Elm does so that it only includes the types/values that the application actually uses (transitively).

When we build such an Application or Bundle though we will need to resolve version conflicts or decide to use a format that allows multiple versions of the same dependency.

Here's a somewhat related discussion for more context: #160

[DamianReeves]

I would also suggest that we allow the inclusion of the Morphir.SDK in the Application bundle,.

[AttilaMihaly]

What do you mean by including? The SDK is full of native functions, what would we do with those?

[DamianReeves]

I'm going to move my thoughts around this and overall thoughts on Application to a new Proposal. Please see #218 (Note I am still drafting the full proposal)