Driver layers

[prdoyle]

Bosk drivers form stackable layers, but those layers are not a first-class object in the system: the layers are implicitly defined by factory lambdas that call other factory lambdas. Once the driver stack is constructed there's no way to identify or reference the drivers it contains. This means, for example, there's no generalized way that the library could describe what the layers are for troubleshooting.

For this reason, and more reasons I'll describe below, I'd like to change the current DriverFactory concept and make a proper DriverLayer abstraction.

Thought experiment: OTel context propagation

Suppose we want OpenTelemetry context to be propagated through bosk drivers. This is trivial for most drivers, because they call the downstream driver synchronously on the same thread, and so thread-local OTel context naturally works; however, for drivers that call asynchronously (even one as simple as BufferingDriver), we'd need to stash the context information in the BoskDiagnosticContext, which is propagated by all drivers¹, and then retrieve it again when the request arrives downstream.

This would apply not just to BufferingDriver, but even more importantly, replicating drivers like MongoDriver and SqlDriver. In general, we'd need to wrap this stash/retrieve logic around any driver that is not known to call the downstream driver synchronously on the same thread.

Here is the thought experiment: imagine we want a bosk-opentelemetry library that, by merely adding it as a dependency of your project, you automatically get context propagation. How would this work? We could use Java's Service Provider Interface (SPI) to discover this library and activate it, but what would it do? It must somehow inspect all the drivers in the stack and wrap them in stash/retrieve logic to propagate the context.

Our current DriverFactory system offers no way to make this possible, even in principle. There's no way any kind of Bosk plugin could be given a chance to inspect and modify the driver stack because the driver stack is not a first-class object; it's defined implicitly by lambdas.

Footnotes

1. As verified by DriverConformanceTest. ↩

[prdoyle]

Our current DriverFactory system offers no way to make this possible, even in principle. There's no way any kind of Bosk plugin could be given a chance to inspect and modify the driver stack because the driver stack is not a first-class object; it's defined implicitly by lambdas.

I suppose I should mention that there is one possibility: an SPI-loaded component could wrap the entire driver stack with a stash/retrieve, which would give context propagation from the submitter of the change to the local driver, and therefore to the hooks.

However, this means the intermediate driver layers have no OTel context, so if they write to a database, that could be missed depending on the driver stack. For example, imagine a BufferingDriver writing to a MongoDriver. The former would lose the OTel thread context, and so when the latter does its database write, that would not be captured in the same trace.

It's also not entirely clear whether we even want hooks to be considered part of the same trace, given that they are inherently asynchronous.