[Sandbox] SpinKube

[endocrimes]

Application contact emails

danielle@fermyon.com, c.voigt@reply.de, fcastelli@suse.com, jiazho@microsoft.com

Project Summary

SpinKube is an open source platform for efficiently running (containerless) Spin-based WebAssembly (Wasm) applications on Kubernetes.

Project Description

SpinKube is a Kubernetes-based platform for running serverless Wasm without a container. It achieves this by combining a few technologies: A containerd-shim-spin that can directly run Spin applications from OCI Artifacts without a Container, A runtime-class-manager (formerly KWasm) operator that manages the lifecycle of WebAssembly runtimes on Kubernetes Hosts, and the Spin Operator which manages the configuration and lifecycle of Spin applications.

This combination provides excellent developer and operator experience when running Wasm on Kubernetes.

SpinKube provides many benefits over container-based serverless compute:

• Change management - with container based deployments, patching system dependencies (such as OpenSSL) needs to happen in every image independently, which can become difficult to manage over time. With SpinKube many of these dependencies are node-local, and the runtime-class-manager provides a kubernetes-native api for patching those on running systems, without application downtime.
• Rightsizing workloads - managing access to shared resources like CPU and memory correctly is difficult, which often leads to over-provisioning resources (over introducing downtime) resulting in low utilization of host resources. With SpinKube, applications consume near-zero resources at idle, and per-request resource isolation reduces the need for broad-strokes resource limits, allowing you to reduce overprovisioning of resources.
• Size - containers are often bigger than needed, and often include extraneous system dependencies by default which are not required to run the application. Spin applications contain only the compiled application and its assets - and does not need per-architecture artifacts, this results in much smaller distribution units and can drastically improve scaling performance.

Org repo URL (provide if all repos under the org are in scope of the application)

https://github.com/spinkube

Project repo URL in scope of application

N/A

Additional repos in scope of the application

No response

Website URL

https://spinkube.dev

Roadmap

Roadmap (view)

Roadmap context

The SpinKube Project aims to integrate WebAssembly applications seamlessly into the Kubernetes and the broader CNCF Landscape. To achieve those goals there are up-and-downstream enhancements to make across Observability, Kubernetes Node management, and scaling systems. Although not yet fully realized in the GitHub Project, the roadmap includes bringing automatic OpenTelemetry instrumentation to Wasm applications and having runtime-class-manager handle the lifecycle of installing and upgrading complex runtimes on Kubernetes nodes.

We believe in the future of WebAssembly as the next wave of efficient Serverless compute.

Contributing Guide

https://www.spinkube.dev/docs/contrib/

Code of Conduct (CoC)

https://github.com/spinkube/governance/blob/main/CODE_OF_CONDUCT.md

Adopters

No response

Contributing or Sponsoring Org

Fermyon Technologies, Microsoft, Liquid Reply, SUSE

Maintainers file

https://github.com/spinkube/governance/blob/main/MAINTAINERS.md

IP Policy

• If the project is accepted, I agree the project will follow the CNCF IP Policy

Trademark and accounts

• If the project is accepted, I agree to donate all project trademarks and accounts to the CNCF

Why CNCF?

The CNCF hosts a vibrant and diverse community of developers and organizations. The CNCF is able to provide a shared governance model and neutral home for the project, allowing for collaboration among various vendors and end-users, fostering the creation of a solution that delivers collective benefits to all stakeholders within the ecosystem.

Benefit to the Landscape

To expand the range of serverless platform options in the CNCF ecosystem and bring the benefits of Wasm to the Kubernetes community. Many end-users are looking for options that allow them to give their developers a serverless solution that is cost-efficient while offering excellent performance. SpinKube offers a seamless way to do this by leveraging the power of Wasm and deeply integrating it into various layers of Kubernetes for integration with the whole ecosystem.

Cloud Native 'Fit'

Landscape: Serverless

TAGs: TAG Runtime + TAG App Delivery

TAG Runtime is a natural fit for much of SpinKube, especially as the home of WG-WASM. We already integrate deeply with ContainerD through runwasi, and aim to work alongside other common runtimes too. These discussions present an opportunity to enhance the Kubernetes ecosystem, especially for lightweight environments with WebAssembly runtimes.

TAG App Delivery is also relevant to SpinKube - Our primary distribution mechanism for Wasm artifacts are OCI Artifacts, which ties into the work of wg-artifacts.

Cloud Native 'Integration'

SpinKube unites Wasm applications and Kubernetes in a way that is natural to developers who use Kubernetes today - by running applications as deployments and creating regular Kubernetes Service’s. This means you can benefit from your existing Service Meshes, O11y Pipeline’s, CI+CD, and scaling solutions while also benefiting from the portability and efficiency of WebAssembly.

SpinKube is also built on the shoulders of giants - including OCI Artifacts, runwasi in the containerd project, Kubebuilder, and OpenTelemetry.

Cloud Native Overlap

The WasmEdge, wasmCloud, knative, and krustlet projects in the CNCF ecosystem have some overlapping objectives with SpinKube. The SpinKube Project aims to integrate WebAssembly applications seamlessly into the Kubernetes and the broader CNCF Landscape.

The CNCF WasmEdge, SpinKube, and CNCF wasmCloud each use a different Wasm runtime, wasmEdge, Spin, and wasmCloud, respectively. While each of them have a differing developer experience for building applications, they each have some level of support for running their applications on Kubernetes.

According to the wasmCloud documentation, "wasmCloud is compatible with, but not dependent on, Kubernetes". With wasmCloud, Pods in a cluster are the wasmCloud "hosts" and contain N Wasm applications.
On the other hand, SpinKube was designed and built for running WebAssembly on Kubernetes - and there is a 1:1 relationship between Pods and Wasm applications. This is due to SpinKube leveraging the containerd/runwasi project to natively execute Wasm applications. Since all Wasm apps are represented as Pods, you can easily use existing tooling such as Cilium, Istio, HPA, KEDA, and Dapr with SpinKube.

WasmEdge is a runtime for embedded functions, microservices, udf, smart contracts, and IoT devices. The WasmEdge Runtime is excellent for packaging inside your software, but is more akin to WasmTime (the runtime used by Spin) than it is to Spin itself. Spin is hyper-focused on event driven serverless applications (that respond to events from HTTP requests/Queue messages/...), and SpinKube relies on that developer experience and programming model as we continue to integrate into the Cloud Native Ecosystem.
Both projects leverage runwasi to run on Kubernetes, and as such SpinKube’s runtime-class-manager not only simplifies configuring the Spin shim but can also manage the WasmEdge shim if that runtime better suits your needs, or you would like to use both.

SpinKube fits in a similar domain as serverless projects in the Cloud Native ecosystem. Like Knative, SpinKube is targeted at engineers who build serverless platforms and want a Kubernetes native experience. The defining distinction is that SpinKube applications are Wasm applications, giving them a greater level of isolation (as every request is fully isolated), fast start-up times, and simplified management of system dependencies.

Krustlet is another project in the CNCF space that runs Wasm on Kubernetes; however, it is no longer maintained - the team that developed it did go on to develop runwasi however.

Similar projects

Knative, wasmCloud, wasmEdge, krustlet

Landscape

Not yet

Business Product or Service to Project separation

N/A.

Project presentations

SpinKube Tutorial - https://www.youtube.com/watch?v=TTXAUSicj04

Project champions

No response

Additional information

No response

[devigned]

I am a big +1 to this proposal. However, I believe it is significant that the namesake project at the core of SpinKube, Spin, is itself not a CNCF project. My main concern regarding this is that if there was a change of licensing for Spin, then the SpinKube collection of projects could be at risk.

[boomskats]

This proposal is really interesting (the demos at kubecon were great), but I do find some of the statements above confusing and potentially misleading.

This opening statement describing the cloud native overlap:

The CNCF WasmEdge, SpinKube, and WasmCloud each use a different Wasm runtime, wasmEdge, Spin, and wasmCloud, respectively.

explicitly clashes with this statement in the paragraph immediately following it:

WasmEdge [...] is more akin to WasmTime (the runtime used by Spin) than it is to Spin itself.

Is Spin itself a runtime that SpinKube depends on (as suggested by the first statement), or does it use Wasmtime (as suggested by the second)?

As far as I'm aware neither Spin or wasmCloud are runtimes. According to Spin's own readme, it is a framework which uses Wasmtime, and according to this blog post, wasmCloud also uses Wasmtime (it does not use 'wasmCloud'). If both wasmCloud and Spin use Wasmtime for their runtimes, what is the purpose of that opening statement?

I'm also not sure why this text refers to WasmEdge as 'CNCF WasmEdge' and wasmCloud as simply wasmCloud, given that they are both CNCF sandbox projects and I've generally seen wasmCloud referred to CNCF wasmCloud elsewhere.

Lastly, Taylor Thomas, one of the original developers of Krustlet that the above text refers to, recently wrote a blog post on this subject which I feel is a relevant read.

[endocrimes]

I'm also not sure why this text refers to WasmEdge as 'CNCF WasmEdge' and wasmCloud as simply wasmCloud, given that they are both CNCF sandbox projects and I've generally seen wasmCloud referred to CNCF wasmCloud elsewhere.

@boomskats that is entirely unintentional - in the place we do that I'll update it for wasmCloud too

Is Spin itself a runtime that SpinKube depends on (as suggested by the first statement), or does it use Wasmtime (as suggested by the second)?

WRT "runtime" - I'm going to try and do some wordsmithing to find a better word, but originally they (Spin and wasmCloud) were referred to as "runtimes" because from an end-user perspective they somewhat are in this context. I'm on holiday until Tuesday so that'll take a bit longer to change.

[kapilt]

It seems a little odd to have an operator and accompanying in a foundation for a base project not in a foundation which is also reliant on a venture backed startup. Ie. It’s unclear why spin is not part of this. Given the zeitgeist and heightened sensitivity to companies changing project licenses to non oss, it seems like a non starter unless the project is also in a foundation. else cncf as a community runs the risk of being foundation laundered on accessories for a core that could end up in the same place, re non oss.

Re other foundation homes for spin, Byte code alliance projects are typically lower level runtime/spec oriented versus orchestration here. Splitting coherency for contributor organizations across foundations doesn’t seem like a good rationale either. Spin is devex and orchestration, and cncf already holds numerous projects to the goals and targeted towards wasm.

It’s not clear what benefit it has to users or community on the split but the risks to the same seem high unless spin is part of the same proposal to move to a foundation.

[endocrimes]

@devigned

I believe it is significant that the namesake project at the core of SpinKube, Spin, is itself not a CNCF project. My main concern regarding this is that if there was a change of licensing for Spin, then the SpinKube collection of projects could be at risk.

I understand that concern and it's one we're currently taking very seriously (sorry for the delay, I was on vacation post KubeCon and wanted to have some internal discussions before posting a response).
We're just as committed to keeping Spin open as we have been from the beginning with the Open Source Promise. We're currently working on defining how best to ensure that Spin will remain open for everyone.

[Mossaka]

We're just as committed to keeping Spin open as we have been from the beginning with the Open Source Promise.

I praise the Fermyon team for pledging to keep Spin open sourced in a good faith commitment and the fact that the team has a track record of building open source tools and platforms. However, we must consider the possibility of an acquisition scenario that a new ownership may change the open-source license to the closed-source model. This scenario isn't unheard of. I am interested in hearing considerations on how to ensure that Spin will remain open in enforceable frameworks.

[angellk]

TAG App Delivery is also relevant to SpinKube - Our primary distribution mechanism for Wasm artifacts are OCI Artifacts, which ties into the work of wg-artifacts.

@endocrimes TAG App Delivery would love to understand more how SpinKube is using OCI Artifacts. Could you please present to and engage with the Artifacts WG?

CNCF slack: #wg-artifacts
Meeting Info/Notes

[devigned]

@angellk I believe @jsturtevant and @thomastaylor312 will be leading a discussion about Wasm OCI Artifacts in the TAG-Runtime Wasm-WG in the upcoming meeting on Tuesday April 16th at 5:00 pm CET/ 8:00 am PST meeting notes / working group details.

For a preview, I believe this doc, WASI OCI Design, tells the gist of the story.

We pursued this in containerd/runwasi#147 so that we could fulfill the promise of Wasm workloads being architecture and OS agnostic in Kubernetes. Previous to this work, one would need to build an OCI image for a given platform / arch (linux/arm, windows/amd64, ...) etc. To target multiple, you'd have to bake multi arch images. That is not Wasm. We believed that K8s Wasm workloads should be free to run on any node in your cluster with a single artifact.

IMHO, it's one of the coolest features of the containerd Wasm shims!

[angellk]

Thanks @devigned - please reach out to the TAG App Delivery Artifacts Working Group to make sure they are invited / involved then since they are specifically called out in the application.

[endocrimes]

For those that are interested outside of the context of this issue, we now have an upstream tracking issue in Spin for governance definition and are making good progress in figuring out the path forwards. Thanks for your patience 💖 : spinframework/spin#2449

[jsturtevant]

Thanks @devigned - please reach out to the TAG App Delivery Artifacts Working Group to make sure they are invited / involved then since they are specifically called out in the application.

As a follow up, @devigned did an overview of the work at the last meeting on Tuesday April 12th https://docs.google.com/document/d/1E7iKPOuyA1jxPe8vDG8aPd8jtnCEbpDpCifXDvDCnA0/edit#heading=h.uylsvgzqikm