Unable to use Docker when DNSMasq is running

[UrGuardian4ngel]

Bug Reports

• dlite version in use (run dlite --version):

dlite version 2.0.0-beta9

• expected behavior:

Be able to pull docker images.

• actual behavior:

Error response from daemon: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 192.168.64.1:53: read udp 192.168.64.38:50471->192.168.64.1:53: read: connection refused

• steps to reproduce

While dnsmasq is active, run dlite start.
When you execute e.g. docker pull nginx, you will get the error message above.

---

I know this issue came up briefly in #135, but that thread was closed...
Is there a known solution for this, so that I can use dnsmasq together with dlite?

Because, dlite seems to work a lot faster (and easier) than most other solutions I already tried on macOS... ;-)

Things I already tried

When I stop dnsmasq and restart dlite, the problem goes away.
Docker starts pulling in the images as it should. Great!

And it seems like it keeps working when I restart dnsmasq, after I started dlite?

Already tried updating my dnsmasq.conf for

• binding to ip 127.0.0.1
• restricting it to en0 (Wi-Fi) and en1 (ethernet)
• excluding en5 (an interface which only seems to exists when dlite is running)
• ...

[nlf]

since this has come up a few times, i'm wondering if this is a pattern that's common..

how do you have dnsmasq installed? how is it configured? how do you run it?

if there's some common pattern maybe i can somehow inject configuration to work around it, binding dlite's dns resolver to some alternative port and have dnsmasq forward requests for the tld to it.

[UrGuardian4ngel]

I'm on macOS 10.11.5.

I installed dnsmasq using Homebrew. I have it configured for multiple purposes:

1. route all development under a specific TLD to localhost
2. nuke a lot of ad, spam & tracking websites (just pointing them to localhost)
3. work around some nasty ip resolving issues when switching between vpn / non-vpn at work

for 1., it's something like this:

address=/dev/127.0.0.1

# Multiple clauses here, because we have a very specific url schema for
# dev / staging / production environments, per project, per client
address=/dev.client-1.my-company.be/127.0.0.1
address=/dev.client-2.my-company.be/127.0.0.1
# ...

For 2., I'm just importing multiple host files from an external hosted repository (addn-hosts=/path/to/nuke/etc/hosts-files).

For the rest, I think my config files are pretty much forward. But, if you are interested, I could post an extract of my configuration? If that could be helpful.

[nlf]

do you keep your config file at the standard /usr/local/etc/dnsmasq.conf and use brew services to run the daemon?

i'm wondering if rather than creating the /etc/resolver/docker entry, appending server=/docker/127.0.0.1#1053 to the dnsmasq config and running (or instructing the user to run) sudo brew services restart dnsmasq would do the trick

[nlf]

you might try those steps, actually..

sudo rm /etc/resolver/docker
echo "server=/docker/127.0.0.1#1053" >> /usr/local/etc/dnsmasq.conf
sudo brew services restart dnsmasq

see if everything works after that

note: i'm assuming you kept the default vm hostname of local.docker

[UrGuardian4ngel]

do you keep your config file at the standard /usr/local/etc/dnsmasq.conf and use brew services to run the daemon?

Yes and no.

The configuration file is in ~/.dotfiles/dnsmasq/dnsmasq.conf, but I have a symlink coming from /usr/local/etc/dnsmasq.conf to that file. I'm using sudo brew services start dnsmasq (because port 53 is privileged).

And in my dnsmasq.conf, I have the following lines to change the 'user' to me (instead of letting it run as root):

user=micjacob
group=wheel

[UrGuardian4ngel]

And, apparently that server clause does not do the trick (for me) :(

Silly me! I still had my interface restrictions / exclusions in place.

Now it seems to boot and work correctly! :)

[nlf]

darn. so my assumption here is that dnsmasq isn't listening on the bridge interface that the vm is trying to use, can you configure it to listen on 192.168.64.1 in addition to 127.0.0.1 and try again?

[UrGuardian4ngel]

Yeah, you're right ;)

My bind-interfaces test from before was still in place. So yeah, dnsmasq would indeed refuse requests coming from other interfaces. (en5 or bridge100, I'm guessing?)

[nlf]

should be bridge100 i believe

[UrGuardian4ngel]

Thanks for your quick response time & solve!
I see it's not only a superb project you have running here ;-)

---

Just chiming back in.

As you mentioned above, last night, I added server=/docker/127.0.0.1#1053 to my /usr/local/etc/dnsmasq.conf and restarted the dnsmasq service.

sudo rm /etc/resolver/docker
echo "server=/docker/127.0.0.1#1053" >> /usr/local/etc/dnsmasq.conf
sudo brew services restart dnsmasq

That did the trick of combining my dnsmasq installation + running docker & dlite in harmony. Great!

However, this morning I did a reboot of my computer. And I fell back into the exact same issue...
That is, until I just did a simple sudo brew services restart dnsmasq. Tadaam, it works again.

In the meantime, I think this issue can be closed? Maybe add your 'workaround' somewhere in the README.md. If other people had similar issues when using dnsmasq, it might help them, too?

---

Some additional debugging info

I don't really know why it started working after dnsmasq restart?
Because, in both cases (initial auto-start on system boot + my manually triggered reboot), using nameserver 127.0.0.1#1053 for domain docker appeared in the logs (see below) – even twice?
Even if the dlite dns server was not (yet) operational at that time, it should become available when it launches, no?

Might be a caching issue from my side (cache-size=1000)? Might be some issue with some kind of service priorities / launch order? I don't know at yet, at the moment.
But when I find it, I will report back here.

# extract from /usr/local/var/log/dnsmasq.log
using nameserver 127.0.0.1#1053 for domain docker
read /etc/hosts - 12 addresses
read /var/hosts/data/mvps.org/hosts - 13237 addresses
read /var/hosts/data/yoyo.org/hosts - 2366 addresses
using nameserver 127.0.0.1#1053 for domain docker
using nameserver 208.67.222.222#53
using nameserver 208.67.220.220#53