Port forwarding doesn't work for non-127.0.0.1 localhost

[matthew-nichols]

Description

nerdctl run --rm -p 127.0.0.2:8080:80 nginx does not work as expected:

$ curl 127.0.0.2:8080
curl: (56) Recv failure: Connection reset by peer

nerdctl run --rm -p 127.0.0.1:8080:80 nginx does work, and outputs the appropriate nginx welcome page from curl 127.0.0.1:8080

Can confirm (on another Linux box) that the same functionality (listen on 127.0.0.2:8080) works in rootless Podman.

Note: the output for nerdctl port in general, seems reversed to what I would expect but contains the right address:

$ nerdctl port nginx
80/tcp -> 127.0.0.2:8080
$ nerdctl port nginx
80/tcp -> 127.0.0.1:8080

ss also confirms something is listening on the right address:

$ sudo ss -tupln | grep 8080
tcp   LISTEN 0      4096            127.0.0.2:8080       0.0.0.0:*    users:(("rootlesskit",pid=1820,fd=30))

Steps to reproduce the issue

1.Install and setup rootless nerdctl
2.Run nerdctl run --rm -p 127.0.0.2:8080:80 nginx
3.Attempt curl 127.0.0.2:8080
4.Get reset by peer

Describe the results you received and expected

Received: reset by peer
Expected: to be connected to the service in the container

What version of nerdctl are you using?

nerdctl version 2.0.0-rc.2, rootless, Linux (Pop!_OS 22.04 LTS), install from tar

Are you using a variant of nerdctl? (e.g., Rancher Desktop)

None

Host information

$ nerdctl info
Client:
 Namespace:	default
 Debug Mode:	false

Server:
 Server Version: v2.0.0-rc.4
 Storage Driver: overlayfs
 Logging Driver: json-file
  Cgroup Driver:  : systemd
  Cgroup Version: : 2
 Plugins:
  Log:     fluentd journald json-file syslog
  Storage: native overlayfs stargz
 Security Options:
  apparmor
  seccomp
   Profile:	builtin
  cgroupns
  rootless
 Kernel Version:   6.9.3-76060903-generic
 Operating System: Pop!_OS 22.04 LTS
 OSType:           linux
 Architecture:     x86_64
 CPUs:             16
 Total Memory:     30.56GiB
 Name:             hostname
 ID:               2eb24a77-0449-485d-b84a-e18aca278b1e

WARNING: AppArmor profile "nerdctl-default" is not loaded.
         Use 'sudo nerdctl apparmor load' if you prefer to use AppArmor with rootless mode.
         This warning is negligible if you do not intend to use AppArmor.
WARNING: No cpu cfs period support
WARNING: No cpu cfs quota support
WARNING: No cpu shares support
WARNING: No cpuset support
WARNING: IPv4 forwarding is disabled
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

[apostasie]

@AkihiroSuda tag expert?

From a glance, I am not sure this can be solved inside nerdctl.

[AkihiroSuda]

This is probably a bug of https://github.com/rootless-containers/rootlesskit.

Workarounds:

1. Set Environment="CONTAINERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=slirp4netns" https://rootlesscontaine.rs/getting-started/containerd/#changing-the-port-forwarder
2. Or, use rootful mode (sudo nerdctl)

[fahedouch]

This is probably a bug of https://github.com/rootless-containers/rootlesskit.

Workarounds:

1. Set Environment="CONTAINERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=slirp4netns" https://rootlesscontaine.rs/getting-started/containerd/#changing-the-port-forwarder
2. Or, use rootful mode (sudo nerdctl)

it is not bug. The default ROOTLESSKIT_PORT_DRIVER is builtin which limits network functionality by not propagating source IP addresses. Can we consider slirp4netns as default driver which provide more realistic network behavior ?

[haytok]

Hi, @AkihiroSuda @fahedouch

In the current implementation, when --port-driver is specified as buildin, Rootlesskit Parent NetNS accept() external connections for the address or 127.0.0.1 and port number specified by -p.

On the other hand, Rootlesskit Child NetNS will send the request to 127.0.0.1 regardless of the ip by accept() in Parent NetNS.

• https://github.com/rootless-containers/rootlesskit/blob/master/pkg/port/builtin/child/child.go#L123C1-L158C3

However, that is the expected behavior as described in the document below.

• https://github.com/rootless-containers/rootlesskit/blob/master/docs/port.md

builtin 30.0 Gbps Always 127.0.0.1
The builtin driver is fast, but be aware that the source IP is not propagated and always set to 127.0.0.1.

Question

I think propagation of the ip that is accept() in Parent NetNS to Child NetNS will solve this problem, so I'll create a PR in rootlesskit for propagation of parent ip to improve the behavior of this issue. What do you think?

[AkihiroSuda]

Thanks, does your PR pass the nerdctl CI?
LGTM then.
(The proper LGTM will be after reviewing the actual code)

[fahedouch]

Thanks for your efforts but I'm not sure how you plan to forward traffic from a child network namespace to the host network while preserving the source IP address when using the builtin driver with detached network namespace