Add ulimits support to docker service and docker stack deploy (carry 2660)

[thaJeztah]

carries #2660
closes #2660

- What I did

1. Bump docker/docker to my own fork to have API changes regarding ulimits support on service endpoints ;
2. Add --ulimit to docker service create ;
3. Add --ulimit-add and --ulimit-rm options to docker service update ;
4. Add Ulimits to docker service inspect --pretty ;
5. Support ulimits in docker-compose files, to make them work with docker stack deploy ;

This is related to moby/moby#40639.

- How I did it

- How to verify it

Given the following docker-compose.yaml:

version: "3"

services:
  test:
    image: debian
    command: /bin/bash -c "ulimit -a && sleep 15"
    ulimits:
      nofile: 100
    deploy:
      mode: replicated

docker service create

$ docker service create --name t2 --ulimit=nofile=100 debian /bin/bash -c "ulimit -a && sleep 30"
$ docker service logs -f t2
...
t2.1.dkm9duj5i4rq@aker    | open files                      (-n) 100
t2.1.dkm9duj5i4rq@aker    | real-time priority              (-r) 0
...

docker service update

$ docker service update --ulimit-rm nofile --ulimit-add rtprio=1 t2
$ docker service logs -f t2
...
t2.1.dkm9duj5i4rq@aker    | open files                      (-n) 1048576
t2.1.dkm9duj5i4rq@aker    | real-time priority              (-r) 1
...

docker service inspect

$ docker service update --ulimit-add nofile=100:200 t2
$ docker service inspect --pretty t2
...
Ulimits:
 nofile: 100:200
 nproc: -1:-1
...

$ docker service inspect t2
...
        "Ulimits": [
            {
                "Name": "nproc",
                "Hard": -1,
                "Soft": -1
            },
            {
                "Name": "nofile",
                "Hard": 200,
                "Soft": 100
            }
        ]
...

docker stack deploy

$ cat docker-compose.yaml
version: "3"

services:
  test:
    image: debian
    command: /bin/bash -c "ulimit -a && sleep 15"
    ulimits:
      nofile: 100
    deploy:
      mode: replicated

$ docker stack deploy --compose-file docker-compose.yaml test
$ docker service logs -f test_test | grep "open files"
test_test.1.jezlw4wt08rz@aker    | open files                      (-n) 100
$ docker inspect $(docker ps --filter=name=test_test -q)
...
        "Ulimits": [
            {
                "Name": "nofile",
                "Hard": 100,
                "Soft": 100
            }
        ],
...

- Description for the changelog

Add ulimits support to docker service create|update|inspect and docker stack deploy

- A picture of a cute animal (not mandatory but encouraged)

[thaJeztah]

Need to update the zsh script as well, but might do so in a follow-up

[thaJeztah]

Because one uses a regular "list" and the other uses a ulimit option, the output for these flags is inconsistent (ulimit vs list); probably something we should fix somehow; not sure what's best

[codecov-commenter]

Codecov Report

Merging #2712 into master will increase coverage by 0.02%.
The diff coverage is 66.10%.

@@            Coverage Diff             @@
##           master    #2712      +/-   ##
==========================================
+ Coverage   58.54%   58.56%   +0.02%     
==========================================
  Files         296      296              
  Lines       21286    21345      +59     
==========================================
+ Hits        12462    12501      +39     
- Misses       7915     7931      +16     
- Partials      909      913       +4     

[cpuguy83]

LGTM

[albers]

Bash completion LGM.
I'll create a follow-up PR that adds completion for the limits.

[vindhya2g]

Hi team,
docker service update throws error saying --ulimit-add is not a valid option. I do not see ulimits when I run inspect as well. My docker version is Docker version 19.03.9, build 9d988398e7

[cpuguy83]

@vindhya2g You need to upgrade docker to 20.10.

[vindhya2g]

Thank you