Project Proposal: system packages

[mmanciop]

New project proposal to provide a product-like, idiomatic experience to provide a seamless experience of monitoring applications running on (virtual) hosts through a combination of the OpenTelemetry Injector injecting SDKs and autoinstrumentations, OpenTelemetry eBPF Instrumentation (OBI), and the OpenTelemetry Collector.

[dyladan]

How will it be chosen which autoinstrumentation packages to use? Is there a goal to make the experience cohesive across languages in terms of what is instrumented and how instrumentations are configured, or will that be dictated by the current offerings of each target language?

[mmanciop]

Which processes to instrument and which not will mostly depend on:

(1) which autoinstrumentation packages are installed
(2) allow/deny lists in Injector and OBI

Also, I think we could additionally have:
(3) depending on other configurations, opt-ins / opt-outs via process environment

[mmanciop]

@dyladan Dynatrace has significant experience with the OneAgent in opt-in/opt-out mechanisms for host-wide monitoring. Is there any advice or experience you folks can share?

[jack-berg]

This project will serve a function that doesn't yet exist in OpenTelemetry, acting as a point where we take the various components in the ecosystem and stitch them together into an opinionated set of defaults. There are going to be questions about what components are included, what the threshold for inclusion is in terms of stability, what the default configuration is, how breaking changes of dependent components are managed, and more. Its going to function as the project that takes a discreet set of tools and turns them into cohesive product.

I think for this to be successful, this group is going to need representation and have communication from the various SIGs whose projects it stitches together, including: the language SIGs with auto instrumentation solutions that will be used, the operator SIG which serves a similar function in k8s, the collector SIG which will no doubt be a part of this, the eBPF projects that will likely end up part of this (profiling, OBI), and maybe more.

That's not to say that people from all these groups are necessarily going to do the work, but they need to be in the loop, and offer guidance / feedback to the SIG and bring guidance / feedback back to their respective SIG.

Can we advertise this project to all the SIGs we think will be involved and request that each volunteers one or more maintainers to work in a liaison capacity?

[mmanciop]

I also think we could make the support in the system packages (and OpenTelemetry Injector) a criterion in the Status and releases compatibility matrix for language SIGs, as it would signal to both Language SIGs and end users that OpenTelemetry considers automatic instrumentation as a first-class requirement. (Of course, assuming there is consensus around this last statement.)

[tedsuo]

I agree that updating the status page to include Linux Package Management as a column and a section would be a good way to raise awareness. But we should also reach out to Java, .NET, Python, Ruby, and Go to confirm a liaison who will review and provide feedback.

[svrnm]

are there other CNCF or LF projects we are close to that might be able to help us here?

[mmanciop]

None that I know of

[svrnm]

@pavolloffay @frzifus @brunobat tagging you as you're with RedHat, any chance you or someone you know would be able to assist here?

[svrnm]

@open-telemetry/ebpf-profiler-maintainers please take a look!