deprecation of devicemapper: need path forward for block devices + Kata

[egernst]

As highlighted in the docker CE release notes, devicemapper is being deprecated with 18.09.

block based graph drivers provide a much more performant solution for Kata Containers, so we will want to make sure we have a driver plugin available for Kata Containers that leverages block devices.

Following relevant PRs for the deprecation:
docker/cli#1455
docker/cli#1424

Discussion can be seen on the topic here: docker/for-linux#452

[egernst]

@caoruidong @gnawux @sameo @jon @WeiZhang555

[jodh-intel]

Note that Docker 18.09 [1] also breaks Kata, so it is inadvisable for users to upgrade at this time. See #926.

---

[1] - Strictly the version of containerd it depends on (v1.2.0).

[WeiZhang555]

I don't have a good suggestion in addition to maintain one devicemapper plugin in Kata community, though that sounds like another burden. Damn it....

[gnawux]

I have been looking at this issue for months actually. Here I conclude some of the situations.

With containerd + shimv2(#572), the snapshot plug-in will pass a Mount interface to the shim, which has the stuff to be mount and the options, which means we could plug the raw device to the VM without mount it if we know it is a block device. As we know the CRI-O folks are working on shimv2 equivalent, then we could think this way works for both of containerd and CRI-O.

Then the issue turns to be "is there a proper block based snapshotter (or say, graph driver) for the future?" Here I have some candidates:

• Maintain a devicemapping based snapshot driver. This should work, however, the dm thin-provisioning pool itself is not designed for containers, and the performance/maintenance complexity of the current docker dm driver is not very good for production usage.
• Add a new local block based driver: rawblock (add new StorageDriver: rawblock hyperhq/hyperd#522) on filesystem might be a good candidate. Here is some comparison:

  rawblock driver vs vfs driver:
    hypervisor handles block better than dir(9p, sharefs).
    rawblock driver supports copy-on-write (on btrfs and recent XFS)
  
  rawblock driver vs devicemapper:
    loopback-based devicemapper is too ugly.
  

• Add a network based solution, such as ceph rbd, which supports copy-on-write snapshot and could be insert to qemu (compiled with librbd) directly without mount it on the host through krbd.

I'd like to learn comments from you guys.

[WeiZhang555]

@gnawux Every option indicates we need to maintain block based graph driver in Kata community

And

rawblock driver vs devicemapper:
  loopback-based devicemapper is too ugly.

It's not fair, direct-lvm based devicemapper is the recommended solution instead of loopback based. Though it's true that devicemapper has a poor quality when used in docker.

[gnawux]

@WeiZhang555

Every option indicates we need to maintain block based graph driver in Kata community

Unfortunately yes, I think we have to maintain a storage driver in case no others provide one for kata.

It's not fair, direct-lvm based devicemapper is the recommended solution instead of loopback based. > Though it's true that devicemapper has a poor quality when used in docker.

Agree it is not fair. I may be biased, I don't trust the dm driver in the production environment personally.

[linxiulei]

I have wrote a scratch feature implementing devicemapper within containerd codes (https://github.com/linxiulei/containerd/tree/devicemapper). it worked but needs more works to perfect it.