What if plugins could load other plugins?

[Ramblurr]

Currently all plugins must be defined upfront, as can be seen in the apply-plugins function:

system/src/donut/system/plugin.cljc

Lines 43 to 46 in 04290dc

	(defn apply-plugins
	"update system map according to plugin instructions"
	[{:keys [:donut.system/plugins] :as system}]
	(reduce apply-plugin system plugins))

But what if a plugin could load other plugins?

My use case for this is a "main" plugin that loads most of the core components in the system, but also depends on some plugins itself.

;; in my library
{::dsp/name ::my-main-plugin
 ::dsp/doc
 "Attaches my core components to the system."

 ::dsp/system-defaults
 {::ds/defs {:env   (config/read-env :profile profile)
             :my-ns {:server       server/HTTPServerComponent
                     :ring-handler handler/RingHandlerComponent
                     ;; ..            and more
                     }}
  
  ::ds/plugins [pop/options-validation-plugin] ;; <---- I have a custom options validation plugin that I would like to load from my main plugin.
}}

This allows a user of my library the ability to use it minimally, while still maintaining control of the primary donut system map themselves. Later on they could add their own components

(def system {::ds/defs    {}
             ::ds/plugins [(my-ns/main-plugin opts)]}) 

Plugin Application Semantics

Supporting this opens up a can of worms..

• Depth-First vs. Breadth-First: Should we fully apply a plugin and all its dependencies before moving to the next plugin, or should we apply all plugins at the current level before diving deeper?
• Plugin Identity: How do we determine if a plugin has already been applied? By value or by ::dsp/name ? Does it matter?
• Plugin Ordering: Should plugins be applied in a specific order, or is the order arbitrary? How do we handle dependencies between plugins
• Plugin Removal: What happens if a plugin removes other plugins that haven't been applied yet? Should they be skipped?
• Termination Guarantee: How do we ensure that the plugin application process terminates if plugins keep adding more plugins? Maybe there should be a default depth (like reitit does for its compilation depth)

Here's one possibility, that applies plugins one at a time

• Process one plugin at a time
• After each plugin application, check for new plugins
• Track processed plugins to avoid reapplication
• Continue until all plugins (including newly added ones) have been processed
• Use value equality to determine plugin identity

Is it necessary?

After writing this up and thinking about the implications of this, I wonder if it is worth it. Decisions made regarding the application semantics wouldn't fit everyone's needs all the time.

Maybe my solution should be that my main plugin manually applies its own plugins to itself via :donut.system.plugin/system-update.

Anyways, wanted to submit this as food for thought.