0

I'm using Docker to manage LXC containers. At the moment Docker configures the underlying LXC container with host-only routing and then uses iptables/DNAT to punch holes into the container, making it reachable via the hosts IP and a "random" port.

This isn't going to work for us for a number of reasons. We'd like each container to be reachable via its own (routable) IP. So I've been playing with using IP tables for this. It works fine for me at home (running on Ubuntu Raring) but not at work (running on Ubuntu Precise). I'm guessing there's some default kernel configuration that's changed that I'm not aware of.

Here's what I'm doing:

1) Setting up a new macvlan virtual interface so I can use DHCP to get an IP:

ip link add container1 link eth0 type macvlan
dhclient container1
ip link set container1 up

At this point, I have a new interface with its own IP address and default gateway, subnet mask, etc. matching the underlying physical interface (eth0).

2) Configure IP tables to DNAT incoming traffic from the new interface's IP. Let's say the new interface has IP 10.10.10.88 and the Docker-controlled LXC container has IP 172.17.0.20.

iptables -t nat -N CONTAINER1
iptables -t nat -A PREROUTING -p all -d 10.10.10.88/32 -j CONTAINER1
iptables -t nat -A CONTAINER1 -p tcp -m tcp --dport 80 -j DNAT --to-destination 172.17.0.20:8080

(Here, I'm also NAT'ing the port from 80 to 8080).

As mentioned, this works fine at home. I can now reach this container from the host as 172.17.0.20:8080 and from another machine on my LAN as 10.10.10.88:80. (Interestingly, I can't reach it on 10.10.10.88:80 from within the host, but I guess I need another rule in, maybe, the OUTPUT chain to catch that.) But it doesn't work at work.

At work, when I try and reach this container using its routable IP (10.10.10.88 in this example), I can see the packets arriving and matching the iptables rules up to the DNAT rule. I can observe this using both the packet counters printed by iptables -L, and also by using the TRACE target on the raw table and watching the kernel log. But after matching the DNAT rule, the packets just vanish. On my home machine (which works), after the DNAT, I can see the packets passing through the INPUT chain in the filter table with the new (NAT'ed) destination IP. On my work machine, this step doesn't happen.

Both machines have IP forwarding turned on.

2
  • I'm confused, are you saying #1 that you cannot reach the container running at home from work? Or are you saying #2 that you have two separate Docker instances + their containers; one at work, one at home; configured otherwise identically apart from the different Ubuntu versions you are using? Oct 30, 2013 at 20:31
  • #2 - except that they're the same version of Linux!
    – dty
    Oct 31, 2013 at 13:28

0

You must log in to answer this question.

Browse other questions tagged .