kata containers & gVistor 安装备忘

## Runtime

### Docker

```
kubelet -> dockerd -> containerd -> containerd-shim -> runC容器
```

### Isolated container
```
kubelet -> (CRI)containerd
                          \-> containerd-shim -> runC 容器
                          \-> containerd-shim-kata-v2 -> runV 安全沙箱容器
                          \-> containerd-shim-runsc-v1 -> runsc 安全沙箱容器
```

## kata containers
### Requirements

- Kubernetes, Kubelet, `kubeadm`
- containerd with `cri` plug-in
- Kata Containers

### Configure Kata

#### Package Installation

```bash
# Step 1: 将 docs 转成 script
source /etc/os-release
curl -fsSL -O https://raw.githubusercontent.com/kata-containers/documentation/master/install/${ID}-installation-guide.md
bash -c "$(curl -fsSL https://raw.githubusercontent.com/kata-containers/tests/master/.ci/kata-doc-to-script.sh) ${ID}-installation-guide.md ${ID}-install.sh"

# Step 2: 执行安装脚本
source /etc/os-release
bash "./${ID}-install.sh"

# Step 3: 验证
command -v kata-runtime
kata-runtime --version
```

### Configure containerd

#### 配置 containerd

```bash
mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml
```
#### 修改缺省配置
```toml
# /etc/containerd/config.toml
# 在缺省配置中做以下修改
[plugins]
  [plugins.cri]
    sandbox_image = "registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1"
    systemd_cgroup = true
    [plugins.cri.registry]
      [plugins.cri.registry.mirrors]
        [plugins.cri.registry.mirrors."docker.io"]
          endpoint = ["https://registry-1.docker.io"]
        [plugins.cri.registry.mirrors."k8s.gcr.io"]
          endpoint = ["https://registry.cn-hangzhou.aliyuncs.com/google_containers"]
[plugins.cri.containerd.runtimes.kata]
 runtime_type = "io.containerd.kata.v2"
```

#### 重启 containerd

```bash
# 重启
service containerd restart
# 查看状态
service containerd status
```

### Configure crictl

```bash
cat <<EOF | tee /etc/crictl.yaml
runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
timeout: 10
debug: false
EOF
```

### Configure Kubelet to use containerd

```bash
mkdir -p /etc/systemd/system/kubelet.service.d/
 
cat << EOF | tee  /etc/systemd/system/kubelet.service.d/0-containerd.conf
[Service]                                                 
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock"
EOF
```

```bash
# Inform systemd about the new configuration
systemctl daemon-reload
systemctl restart kubelet
```

### Usage

#### Install the Runtime Class for gVisor

```bash
# Step 1: Install a RuntimeClass
cat <<EOF | kubectl create -f -
apiVersion: node.k8s.io/v1beta1
kind: RuntimeClass
metadata:
  name: kata
handler: kata
EOF
```

#### Create a Pod with the kata Runtime Class

```bash
# Step 2: Create a pod
# 通过runtimeClassName: kata 指定使用安全沙箱容器运行时。

cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: nginx-untrusted-kata
spec:
  runtimeClassName: kata
  containers:
  - name: nginx-untrusted-kata
    image: docker-reg.basebit.me:5000/base/nginx:1.15.2
EOF
```

### Validate

```bash
[root@host-d10-005 containerd]# crictl exec 37034de5f6577 dmesg |grep Kata
[    1.017406] systemd[1]: Started Kata Containers Agent.
[    1.017505] systemd[1]: Reached target Kata Containers Agent Target.
```

## gVistor

### Configure gVisor

#### Install runsc

```bash
#!/usr/bin/env bash
(
  set -e
  URL=https://storage.googleapis.com/gvisor/releases/release/latest
  wget ${URL}/runsc
  wget ${URL}/runsc.sha512
  sha512sum -c runsc.sha512
  rm -f runsc.sha512
  mv runsc /usr/local/bin
  chown root:root /usr/local/bin/runsc
  chmod 0755 /usr/local/bin/runsc
)
```

#### Install containerd-shim-runsc-v1

通过 https://github.com/google/gvisor-containerd-shim/releases 下载

```bash
#!/usr/bin/env bash
(
  set -e
  URL=https://github.com/google/gvisor-containerd-shim/releases/download/v0.0.4/containerd-shim-runsc-v1.linux-amd64
  wget -O  containerd-shim-runsc-v1 ${URL}
  chmod +x containerd-shim-runsc-v1
  mv containerd-shim-runsc-v1 /usr/local/bin/containerd-shim-runsc-v1
)
```
### Configure containerd

#### 配置 containerd

```bash
mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml
```
#### 修改缺省配置
```toml
# /etc/containerd/config.toml
# 在缺省配置中做以下修改
[plugins]
  [plugins.cri]
    sandbox_image = "registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1"
    systemd_cgroup = true
    [plugins.cri.registry]
      [plugins.cri.registry.mirrors]
        [plugins.cri.registry.mirrors."docker.io"]
          endpoint = ["https://registry-1.docker.io"]
        [plugins.cri.registry.mirrors."k8s.gcr.io"]
          endpoint = ["https://registry.cn-hangzhou.aliyuncs.com/google_containers"]
[plugins.cri.containerd.runtimes.runc]
 runtime_type = "io.containerd.runc.v1"
[plugins.cri.containerd.runtimes.gvisor]
 runtime_type = "io.containerd.runsc.v1"
 runtime_engine = "/usr/local/bin/runsc"
 runtime_root = "/run/containerd/runsc"
```

#### 重启 containerd

```bash
# 重启
service containerd restart
# 查看状态
service containerd status
```

### Configure crictl

```bash
cat <<EOF | tee /etc/crictl.yaml
runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
timeout: 10
debug: false
EOF
```
验证 crictl 是否加载 containerd 最新信息

```bash
# 验证 crictl 加载的 containerd 版本
[root@vm-05-187 ~]# crictl version|grep Runtime
RuntimeName:  containerd
RuntimeVersion:  1.2.13
RuntimeApiVersion:  v1alpha2

# 验证 crictl 加载 containerd 的具体配置
crictl info
```
### Configure kubelet

```bash
cat <<EOF | tee /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS=--cgroup-driver=systemd --runtime-cgroups=/system.slice/containerd.service --container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock
EOF
```

### Usage

#### Install the Runtime Class for gVisor

```bash
# Step 1: Install a RuntimeClass
cat <<EOF | kubectl create -f -
apiVersion: node.k8s.io/v1beta1
kind: RuntimeClass
metadata:
  name: gvisor
handler: gvisor
EOF
```

#### Create a Pod with the gVisor Runtime Class

```bash
# Step 2: Create a pod
# 通过runtimeClassName: gvisor 指定使用安全沙箱容器运行时。

cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: busybox-gvisor
  labels:
    app: busybox-gvisor
spec:
  runtimeClassName: gvisor
  containers:
  - name: busybox-gvisor
    image: registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2
    command:
    - tail
    - -f
    - /dev/null
    resources:
      limits:
        cpu: 1000m
        memory: 512Mi
      requests:
        cpu: 1000m
        memory: 512Mi
EOF
```

#### run the following command to check its value for RuntimeClass:

```bash
kubectl get pods pod-name -o jsonpath='{.spec.runtimeClassName}'
kubectl get pods -o jsonpath=$'{range .items[*]}{.metadata.name}: {.spec.runtimeClassName}\n{end}'
```

### Validate 

```bash
# 验证
crictl pods
crictl inspectp a2fd99d195151

[root@vm-05-187 pods]# crictl ps |grep dataset-untrusted-gvisor
5f0db044cec38       25b5e9d69bb96       25 minutes ago      Running             dataset-untrusted-gvisor    0                   a2fd99d195151

[root@vm-05-187 pods]# crictl exec 5f0db044cec38 dmesg
[    0.000000] Starting gVisor...
[    0.591383] Consulting tar man page...
[    1.040814] Segmenting fault lines...
[    1.269043] Gathering forks...
[    1.323039] Granting licence to kill(2)...
[    1.779303] Constructing home...
[    1.818690] Mounting deweydecimalfs...
[    2.079180] Checking naughty and nice process list...
[    2.480878] Letting the watchdogs out...
[    2.677193] Creating bureaucratic processes...
[    3.031363] Generating random numbers by fair dice roll...
[    3.372694] Ready!
```

kata containers & gVistor 安装备忘 #55

Description

Runtime

Docker

Isolated container

kata containers

Requirements

Configure Kata

Package Installation

Configure containerd

配置 containerd

修改缺省配置

重启 containerd

Configure crictl

Configure Kubelet to use containerd

Usage

Install the Runtime Class for gVisor

Create a Pod with the kata Runtime Class

Validate

gVistor

Configure gVisor

Install runsc

Install containerd-shim-runsc-v1

Configure containerd

配置 containerd

修改缺省配置

重启 containerd

Configure crictl

Configure kubelet

Usage

Install the Runtime Class for gVisor

Create a Pod with the gVisor Runtime Class

run the following command to check its value for RuntimeClass:

Validate

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions