Enable container + Wasm workloads running within the same pod

[devigned]

Currently, runwasi only knows how to run Wasm workloads. K8s users often want to run sidecars for service meshes and other traditional container injections within the same pod.

It would empower a more idiomatic K8s experience if runwasi was able to run both Wasm workloads and traditional container workloads within the same pod.

[0xE282B0]

After merging #78 it is possible to register multiple executors for the wasmedge shim. That way it is possible to have wasmedge containers and plain old linux containers on the same pod.
However, it has some drawbacks:

• it's currently only possible for one shim
• it splits the decision of which handler to use between CRI and OCI runtime
• it is harder to debug when an image is not starting

I'm already using this approach with crun, which decides based on a pod annotation if a Wasm runtime should be used or even autodetect. Another approach would be that the CRI selects the handler based on the image annotation according to Wasm Image Spec

What do you think, should we solve this on shim/oci-runtime level and make it available for all shims or push it to the CRI?

[cpuguy83]

@0xE282B0 I think we can look for an ELF header to determine what executor to use.

[0xE282B0]

@cpuguy83 or for the magic number of a bytecode file.

[ipuustin]

IMHO explicit runtime choice sounds better than implicit, because otherwise someone could craft an image to use a potentially less-secure executor?

In a sense it might be a good thing to allow different shims to run within the single pod, because then the ability to have plain old containers in the same pod would not be specific to certain runwasi shims, and also running containers could be bit of a scope creep for runwasi. It seems that the reason for RuntimeClass to be pod-specific (instead of container-specific) is the sandbox resource sharing (https://kubernetes.io/blog/2018/10/10/kubernetes-v1.12-introducing-runtimeclass/#runtimeclass-in-kubernetes-1-12).

If we could somehow pin down the sharing semantics and declare that (for example) runc and runwasi are compatible regarding network and other resources, we might be able to do a meta-runtimeclass such as "wasm-sandbox". The handler could be a small proxy shim for selecting which "real runtime" to use and then execing that binary? I'm not sure but it could be that RuntimeClass KEP means something like this when they speak of "conformant" RuntimeClasses (https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/585-runtime-class#risks-and-mitigations).

[Mossaka]

I have started working on this task. The first step that I took is to use youki's DefaultExecutor to implement runwasi's Instance trait, hoping that it could run any Linux containers fine.

After this step, I will write a detector logic in runwasi that tries to figure out the unpacked image is a wasm image or a non-wasm image, and then invoke the corresponding youki executors resp.

[Mossaka]

My experiment of runwasi + youki for Linux containers is living in this repo https://github.com/Mossaka/runwasi-youki

It is largely the same codebase as the wasmedge shim @utam0k contributed. I swapped the WasmedgeExecutor to DefaultExecutor.

[0xE282B0]

@Mossaka you can actually register both Executors and use the can_handle function to run your detector logic. That was at least working good for me. 😊

[Mossaka]

Ah interesting.. Thanks for sharing! @0xE282B0

[Mossaka]

The above mentioned youki + runwasi experiment is working now! I can run a Linux container as a runwasi instance. Next, I will be working on #110 and then integrate youki's DefaultExecutor to both wasmtime and wasmedge shim so that wasm and linux container can run side-by-side.

[Mossaka]

#156 is merged. Closing this one.