Re-architecture to better fit the modular nature of the project

[kurktchiev]

The project's current state allows for a great showcase of the general idea behind The BACK Stack. However, it poses challenges regarding the lifecycle management of components and the modularity of said components.

To alleviate some of these challenges, I am proposing a re-architecture that will ultimately enhance the original Hub/Spoke idea and allow us to better:

• manage individual hub components
• lifecycle management of those components
• control installation/or not of components
• include new packages as needed
• break out the core code from example implementations

This new design makes a few assumptions:

1. Hub clusters are a single tenant that houses the BACK stack HUB components, nothing else
2. We are basing the install/upgrade/uninstall process through porter
3. Each component is installed through helm
4. Break out the cluster/app claims to their own repository

It also introduces a new concept to the Hub/Spoke, called a Bead. Beads are 3rd party packages that allow an end user to extend the platform to their liking.

Below is the diagram describing the breakouts and decisions around the re-architecture:

Also, the following diagram is my current sketch of how I envision Beads working:

cc @JimBugwadia @patelrit @morey-tech for comments

[morey-tech]

Love the new design! Something to consider:

We are basing the install/upgrade/uninstall process through porter

It's common for Argo CD to manage itself after the initial bootstrapping, and it could also manage the other components of the hub. I wonder if Porter should be solely responsible for the install/uninstall but not the upgrade and ongoing operation of the hub components.

This would reduce the complexity we need in Porter. It could simply create the Hub cluster and bootstrap it with Argo CD. Then, uninstallation is simply deleting the hub cluster. In the long term, we could integrate Porter with Terraform (OpenTofu) to create the hub cluster on cloud providers and clean them up after uninstalling them.

Upgrading would be managed through GitOps, with changes to the manifests in the hub cluster repo. Argo CD can do the deployment. If there's any special logic we need to implement for those upgrades, it should be done using resource hooks in the Applications.

This answers how Beads are managed; Argo CD would handle them just like the other hub components.

[kurktchiev]

I am on board with using Argo as a the source of truth for things, take a look over at https://github.com/back-stack/bead-gke I am working through building out what adding GKE support can look like. I think overall things can be moved into Argo except the backstage stuff which is wonky and I am not sure what the best way to handle the wiring into that is honestly. It requires either manual interventions of a configmap potentially or whole new images for backstage, neither option is super appealing.

[morey-tech]

except the backstage stuff which is wonky and I am not sure what the best way to handle the wiring into that is honestly. It requires either manual interventions of a configmap potentially or whole new images for backstage, neither option is super appealing.

1. What is the official recommendation from the Backstage project for people adding in their customizations?
2. Maybe we should treat Backstage as a bead (e.g., if a user wants to customize it, they have to bring their own Backstage).

[kurktchiev]

There is no official way to specifically enhance it once the image has been built outside of rebuilding. There are ways to extend it to a point (the gke example leans onto that by using a catalog that can be imported to add in functionality), but if there is need for say new plugins to be added etc, it requires a new image altogether.

I am not against backstage being a Bead tbh, it is a nice interface into the system for those that want it but I think the heart of things should be git+argo/porter

[JimBugwadia]

Have we looked at Backstage templates (blog, docs) for extensions?

I feel we need to define a higher level abstraction to add a Service to a BACK Stack deployment, which allows declarative configuration of:

1. Backstage Template
2. ArgoCD Application
3. Crossplane Composite
4. Kyverno Policy Set

Perhaps the minimal requirement is the ArgoCD Application and the rest are optional?

I am still trying to understand what exactly a Bead is....did you have something similar to the above in mind?

This can be formalized as a new CRD.