moby SWARM Mode - Docker binds ingress ports always on 0.0.0.0 - Go
Description
We have multiple hosts using swarm mode.
Some of this hosts have two or more external IP addresses. Docker swarm services with published ports always bind to 0.0.0.0:
Container not started in swarm mode, are binding to the correct IP. Services in Swarm are always binding published ports to every IP known by the Host (eno1,eno2 and lo). We cannot use the second IP with a port already taken by docker swarm.
Steps to reproduce the issue:
1. Create a Docker Service in Swarm with published Ports
2. netstat -utelpna | grep
Describe the results you received:
Started dockerd with --ip argument:
root 4869 1.1 10.0 2749972 102920 ? Ssl 12:36 0:12 /usr/bin/dockerd -H fd:// --ip=139.59.215.247 --swarm-default-advertise-addr=139.59.215.247
Swarm services bind published ports to every IP known by the system (0.0.0.0).
Describe the results you expected:
Expected swarm services to bind published ports only on IP 139.59.215.247.
Additional information you deem important (e.g. issue happens only occasionally):
Output of docker version:
Client:
Version: 17.09.0-ce
API version: 1.32
Go version: go1.8.3
Git commit: afdb6d4
Built: Tue Sep 26 22:42:09 2017
OS/Arch: linux/amd64
Server:
Version: 17.09.0-ce
API version: 1.32 (minimum version 1.12)
Go version: go1.8.3
Git commit: afdb6d4
Built: Tue Sep 26 22:40:48 2017
OS/Arch: linux/amd64
Experimental: false
Output of docker info:
Containers: 29
Running: 4
Paused: 0
Stopped: 25
Images: 65
Server Version: 17.09.0-ce
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 554
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: active
NodeID: lbvavrurdt9edmfb44eshjbqk
Is Manager: true
ClusterID: 3uthkj04fn1rv2htzljm2imo9
Managers: 1
Nodes: 2
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 3
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 0
Autolock Managers: false
Root Rotation In Progress: false
Node Address: 139.59.215.247
Manager Addresses:
139.59.215.247:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 06b9cb35161009dcb7123345749fef02f7cea8e0
runc version: 3f2f8b84a77f73d38244dd690525642a72156c64
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.0-4-amd64
Operating System: Debian GNU/Linux 9 (stretch)
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 996.6MiB
Name: dev1
ID: CDVW:ZY3R:JMYQ:AW3L:CC6K:F7RL:JRZX:GPXQ:FOD6:4VTI:7TZV:7JDC
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No swap limit support
6 Answer:
@fcrisciani Thanks for the answer. My use case is that my servers are part of two networks. An internal private network which can not be accessed from the internet (in which all servers are in and I via VPN) and an external one which can be accessed via the internet. I wanted to have a very simple but still secure way of accessing docker swarm services in the private network only (without having to deal with the complicated IP table stuff).
I have the same problem.
my solution was to run in host mode.
To be sure it would bind in the address I want in each docker node.
I have to remove docker_gwbridge from each node before join swarm and do something like
docker network create -o "com.docker.network.bridge.hostbindingipv4"="192.168.1.151" docker_gwbridge
192.168.1.151 is the ip I want to expose in this node
On compose I was using
ports:
- target: 80
published: 80
protocol: tcp
mode: host
It would be GREAT have something like that for ingress.
And it sounds really simple to implement.
Hope we have it.
I am a little bit confused now. Checked now a lot of tickets but could not find what I was looking for. So hope somebody here can help me as it seems very closely related. What I want is that all services of a swarm node do not listen to all IP addresses and to a specific one instead. So my server has the IP 10.0.0.1 and 192.0.0.1 I want that it only responds to 10.0.0.1 . That would be for all services running on that node. I can understand that it is complicated on a per service level but for the whole node that should be not that hard. Can that be done or is that also not possible?
@janober on recent versions of docker you can specify a --data-path-addr when initialising the swarm (docker swarm init), or joining the swarm (docker swarm join). That option allows you to specify either an IP-address, or interface to use for networking data (separate from the "control" networking, that's used by managers to control the swarm)
It was implemented in this pull request: https://github.com/moby/moby/pull/32717
Note that this option can only be set on init or join, and cannot be modified afterwards
@thaJeztah thanks a lot for the fast reply. That sounds like exactly what I am looking for. Sadly does it not work for me. I am running Docker version 17.09.0-ce, build afdb6d4. So that option is already there.
However when I start swarm like this:
docker swarm init --advertise-addr=x.x.x.x--data-path-addr=x.x.x.x
and then a container, for example ngnix like that:
docker service create \
--name nginx \
--publish 80:80 \
nginx
I can not just access the web page via http://x.x.x.x I can still access it via the other IP address http://y.y.y.y
Do I understand or do something wrong?
Maybe also my later question is not very clear. I am also not talking about the "--listen-addr" because also that one does not change the "listening" I am talking about. I want to change the IP the docker services respond to.
Read next
- Sorting by date not working - ElasticPress
- qBittorrent Program No Longer Working - Cplusplus
- framework APP_KEY not cached unless config:cache is run twice PHP
- Direction for finding zookeeper issues - Go clickhouse-operator
- App crash with error: "Bad JNI version returned from JNI_OnLoad" when building .apk - Dart flutter-ffmpeg
- darktable minor usability: position of "module order" tickbox in selective copy-paste dialog C
- gpdb Explain analyze coredump while direct dispatch is enabled C
- [feature request] new prompt line for "post-command output" / "error indicator" - Shell powerlevel10k