CoCo Serverless

This is project is an extention of the coco-serverless repository. The original repository has the goal to deploy Knative on CoCo and run some baseline benchmarks. This project extends its functionality, introducing our custom CoCo implementation with an imporved image pulling mechanism (CoCo-Hybrid). This repository hosts benchmarks for CoCo-Hybrid, providing a means to compare with previously established baselines.

Our CoCo-hybrid mode makes ajustments so several of the CoCo components. The adjusted compontnets are found in the following branches of our forked repositories:

• Nydus-snapshotter
• kata-containers
• guest-components
• nydus

All instructions in this repository assume that you have checked-out the source code, and have activated the python virtual environment:

source ./bin/workon.sh

# List available tasks
inv -l

Pre-Requisites

You will need CoCo's fork of containerd built and running. To this extent you may run:

inv containerd.build
inv containerd.install

You also need all the kubernetes-related tooling: kubectl, kubeadm, and kubelet:

inv k8s.install [--clean]

You may also want to install k9s, a kubernetes monitoring tool:

inv k9s.install-k9s

Quick Start

Deploy a (single-node) kubernetes cluster using kubeadm:

inv kubeadm.create

Second, install both the operator and the CC runtime from the upstream tag. We currently pin to version v0.7.0 (see the COCO_RELEASE_VERSION variable).

inv operator.install
inv operator.install-cc-runtime

Third, update the initrd file to include our patched kata-agent:

inv kata.replace-agent

if it is the first time, you will have to manually build the agent following these instructions.

Then, you are ready to run one of the supported apps:

• Hello World! (Py) - simple HTTP server running in Python to test CoCo and Kata.
• Hello World! (Knative) - same app as before, but invoked over Knatvie.
• Hello Attested World! (Knative + Attestation) - same setting as the Knative hello world, but with varying levels of attestation configured.

If your app uses Knative, you will have to install it first:

inv knative.install

Setting Up CoCo-Hybrid

In order to enable the CoCo-Hybrid mode, the following configuration steps need to be taken:

Our customised nydus-snapshotter binary, linux Kernel and VM initrd image. These can be installed using the following command:

inv hybrid.install-cc-hybrid-deps

The kata configs can then be adjusted to point to the nre kernel and initrd using the following command

inv hybrid.update-configs

Additionally, in ordert to configure the snapshotter to be in our hybrid mode the following commands can be run:

inv nydus-snapshotter.populate_host_sharing_config
inv nydus-snapshotter.toggle_mode --hybrid

All instructions in this repository assume that you have checked-out the source code, and have activated the python virtual environment:

source ./bin/workon.sh

# List available tasks
inv -l

Pre-Requisites

You will need CoCo's fork of containerd built and running. To this extent you may run:

inv containerd.build
inv containerd.install

You also need all the kubernetes-related tooling: kubectl, kubeadm, and kubelet:

inv k8s.install [--clean]

You may also want to install k9s, a kubernetes monitoring tool:

inv k9s.install

Quick Start

Deploy a (single-node) kubernetes cluster using kubeadm:

inv kubeadm.create

Second, install both the operator and the CC runtime from the upstream tag. We currently pin to version v0.7.0 (see the COCO_RELEASE_VERSION variable).

inv operator.install
inv operator.install-cc-runtime

Third, update the initrd file to include our patched kata-agent:

inv kata.replace-agent

if it is the first time, you will have to manually build the agent following these instructions.

Then, you are ready to run one of the supported apps:

• Hello World! (Py) - simple HTTP server running in Python to test CoCo and Kata.
• Hello World! (Knative) - same app as before, but invoked over Knatvie.
• Hello Attested World! (Knative + Attestation) - same setting as the Knative hello world, but with varying levels of attestation configured.

If your app uses Knative, you will have to install it first:

inv knative.install

Evaluation

The goal of the project is to measure the performance of Knative with CoCo, and compare it to other isolation mechanisms using standarised benchmarks. To This extent, we provide a thorough evaluation in the evaluation directory.

Uninstall

In order to uninstall components for debugging purposes, you may un-install the CoCo runtime, and then the operator as follows:

inv operator.uninstall-cc-runtime
inv operator.uninstall

Lastly, you can completely remove the k8s cluster by running:

inv kubeadm.destroy

Further Reading

For further documentation, you may want to check these other documents:

• Attestation - attestation particularities of CoCo and SEV(-ES).
• Guest Components - patch image-rs or other guest components.
• K8s - documentation about configuring a single-node Kubernetes cluster.
• Kata - instructions to build our custom Kata fork and initrd images.
• Key Broker Service - docs on using and patching the KBS.
• Knative - documentation about Knative, our serverless runtime of choice.
• Local Registry - configuring a local registry to store OCI images.
• OVMF - notes on building OVMF and CoCo's OVMF boot process.
• SEV - speicifc documentation to get the project working with AMD SEV machines.
• Troubleshooting - tips to debug when things go sideways.