no devicemapper support in 18.09.x on ubuntu and debian

[akosyrev]

• This is a bug report
• [x ] I searched existing issues before opening this one

I found that last nightly build from this repo https://download.docker.com/linux/ubuntu/dists/xenial/pool/nightly/amd64/ fails with error, when i'm trying to run it with devicemapper:

{
    "storage-driver": "devicemapper"
}

Error message:

failed to start daemon: error initializing graphdriver: driver not supported```

I recompiled it from sources and error is still here. iI built from commit 7718f802c486607f080ab8158953dc668e3f09fb

Also I noticed that bightly build from 2018-09-26 is less than 2018-09-20: 17M vs 24M 

[akosyrev]

https://github.com/docker/docker-ce-packaging/pull/223/files#diff-6670e1ec6985ef6cc31ddd74f1e52e70R43

[domste]

Have the same problem. Is there a reason why devicemapper is excluded?

[Halo9Pan]

docker/cli#1424
"Deprecate", but it was excluded...

[teadur]

Same here devicemapper is excluded on debian stretch starting from 18.09

[cpuguy83]

ping @andrewhsu

[thaJeztah]

This is being tracked internally; opened an issue for the packaging team.

[nemesis545]

Same here devicemapper is excluded on debian buster starting from Docker version 18.09.0, build 4d60db4

[nemesis545]

i got this:
Unable to locate plugin: devicemapper, retrying in 1s
dockerd -s devicemapper --experimental -D

DEBU[2018-11-13T10:38:00.517663555-05:00] Listener created for HTTP on unix (/var/run/docker.sock) 
WARN[2018-11-13T10:38:00.521002154-05:00] failed to rename /var/lib/docker/tmp for background deletion: rename /var/lib/docker/tmp /var/lib/docker/tmp-old: file exists. Deleting synchronously 
DEBU[2018-11-13T10:38:00.521222922-05:00] Golang's threads limit set to 114570         
INFO[2018-11-13T10:38:00.521525999-05:00] parsed scheme: "unix"                         module=grpc
INFO[2018-11-13T10:38:00.521542834-05:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2018-11-13T10:38:00.521602212-05:00] parsed scheme: "unix"                         module=grpc
INFO[2018-11-13T10:38:00.521616361-05:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2018-11-13T10:38:00.521683461-05:00] ccResolverWrapper: sending new addresses to cc: [{unix:///run/containerd/containerd.sock 0  <nil>}]  module=grpc
DEBU[2018-11-13T10:38:00.521716188-05:00] Using default logging driver json-file       
DEBU[2018-11-13T10:38:00.521746247-05:00] [graphdriver] trying provided driver: devicemapper 
INFO[2018-11-13T10:38:00.521806617-05:00] ccResolverWrapper: sending new addresses to cc: [{unix:///run/containerd/containerd.sock 0  <nil>}]  module=grpc
INFO[2018-11-13T10:38:00.521846492-05:00] ClientConn switching balancer to "pick_first"  module=grpc
INFO[2018-11-13T10:38:00.521727080-05:00] ClientConn switching balancer to "pick_first"  module=grpc
WARN[2018-11-13T10:38:00.521924066-05:00] Unable to locate plugin: devicemapper, retrying in 1s 
INFO[2018-11-13T10:38:00.521923492-05:00] pickfirstBalancer: HandleSubConnStateChange: 0xc42019a160, CONNECTING  module=grpc
INFO[2018-11-13T10:38:00.521959850-05:00] pickfirstBalancer: HandleSubConnStateChange: 0xc42016c9e0, CONNECTING  module=grpc
INFO[2018-11-13T10:38:00.522111703-05:00] pickfirstBalancer: HandleSubConnStateChange: 0xc42016c9e0, READY  module=grpc
INFO[2018-11-13T10:38:00.522124829-05:00] pickfirstBalancer: HandleSubConnStateChange: 0xc42019a160, READY  module=grpc
DEBU[2018-11-13T10:38:00.522207455-05:00] processing event stream                       module=libcontainerd namespace=plugins.moby
WARN[2018-11-13T10:38:01.522307695-05:00] Unable to locate plugin: devicemapper, retrying in 2s 
WARN[2018-11-13T10:38:03.522773842-05:00] Unable to locate plugin: devicemapper, retrying in 4s 
^CINFO[2018-11-13T10:38:04.863216849-05:00] Processing signal 'interrupt'                
WARN[2018-11-13T10:38:07.523305063-05:00] Unable to locate plugin: devicemapper, retrying in 8s 
^CINFO[2018-11-13T10:38:08.095129525-05:00] Processing signal 'interrupt'                
^CINFO[2018-11-13T10:38:08.303084600-05:00] Processing signal 'interrupt'                
^CINFO[2018-11-13T10:38:08.558976666-05:00] Processing signal 'interrupt'                
INFO[2018-11-13T10:38:08.559126689-05:00] Forcing docker daemon shutdown without cleanup; 3 interrupts received ```

[sboeuf]

Hi @thaJeztah, what's the status on this issue? Is this resolved now?

[amshinde]

@thaJeztah We rely on the block storage provided by devicemapper storage driver, so would really like to see this resolved.

[sboeuf]

Also, are you planning to support any alternative to devicemapper? Because block device support provides better performances in our case.

[thaJeztah]

I'm not sure what the current status is; I know this was something that was being looked into, but no details yet if they found a solution.

Also, are you planning to support any alternative to devicemapper?

The current roadmap is to complete the migration to using containerd as a runtime, and for image management (see moby/moby#38043). Currently, the functionality of containerd and dockerd overlaps (as in; they both are capable of image-management, and both implement storage drivers). The goal is to hand over the overlapping parts to containerd. Currently containerd has support for overlay, zfs, btrfs, and aufs as storage drivers ("snapshotters") but additional storage drivers could be added upstream in the containerd project. t.b.h., I'm not sure if devicemapper would find its way in there. Maintenance of devicemapper was largely driven by Red Hat (as it was the only option for their kernels), but now that overlay has become mainstream, focus has shifted in that direction.

Because block device support provides better performances in our case.

Do you have more information about your use-case? Is this for writes or reads ? (for writes, the general recommendation is to use a volume, to skip the CoW performance penalty)