0

I have identical OSes (Ubuntu 16.04) running on 2 hosts, with identical OS updates.

I also have identical LxD service installed (via sudo apt-get install lxd) and configured (via sudo lxd init) on both hosts.

Now, for both space-saving, portability/work-continuity reasons (because these 2 hosts are in different physical locations, behind their own firewalls that I don't control), I thought, why not install all my LxD images and containers on an external drive that I could plug in one host or the other, interchangeably.

For this, I installed my desired images and set up the desired containers on Host A, and copied the /var/lib/lxd, /var/log/lxd, and /var/cache/lxd folders (and their contents) to the external drive.

When I plugged the external drive into Host B, I could actually see my image- and (stopped) container-lists from Host A just fine. I could even launch new containers on Host B off of original, Host A images.

But when on Host B I tried to start a container, foo, originally started and stopped on Host A, I got the following error:

$ lxc start foo
error: Error calling 'lxd forkstart foo /var/lib/lxd/containers /var/log/lxd/foo/lxc.conf': err='exit status 1'
  lxc 20161217084040.956 ERROR lxc_start - start.c:lxc_spawn:1162 - failed to set up id mapping
  lxc 20161217084041.013 ERROR lxc_start - start.c:__lxc_start:1354 - failed to spawn 'foo'
  lxc 20161217084041.564 ERROR lxc_conf - conf.c:run_buffer:347 - Script exited with status 1
  lxc 20161217084041.564 ERROR lxc_start - start.c:lxc_fini:555 - failed to run post-stop hooks for container 'foo'.
  lxc 20161217084041.566 ERROR lxc_conf - conf.c:userns_exec_1:4334 - Error setting up child mappings
  lxc 20161217084041.566 ERROR lxc_cgfsng - cgroups/cgfsng.c:recursive_destroy:983 - Error destroying /sys/fs/cgroup/systemd//lxc/foo
  lxc 20161217084041.568 ERROR lxc_conf - conf.c:userns_exec_1:4334 - Error setting up child mappings
  lxc 20161217084041.568 ERROR lxc_cgfsng - cgroups/cgfsng.c:recursive_destroy:983 - Error destroying /sys/fs/cgroup/pids//lxc/foo
  lxc 20161217084041.569 ERROR lxc_conf - conf.c:userns_exec_1:4334 - Error setting up child mappings
  lxc 20161217084041.569 ERROR lxc_cgfsng - cgroups/cgfsng.c:recursive_destroy:983 - Error destroying /sys/fs/cgroup/net_cls//lxc/foo
  lxc 20161217084041.571 ERROR lxc_conf - conf.c:userns_exec_1:4334 - Error setting up child mappings
  lxc 20161217084041.571 ERROR lxc_cgfsng - cgroups/cgfsng.c:recursive_destroy:983 - Error destroying /sys/fs/cgroup/hugetlb//lxc/foo
  lxc 20161217084041.573 ERROR lxc_conf - conf.c:userns_exec_1:4334 - Error setting up child mappings
  lxc 20161217084041.573 ERROR lxc_cgfsng - cgroups/cgfsng.c:recursive_destroy:983 - Error destroying /sys/fs/cgroup/cpu//lxc/foo
  lxc 20161217084041.575 ERROR lxc_conf - conf.c:userns_exec_1:4334 - Error setting up child mappings
  lxc 20161217084041.575 ERROR lxc_cgfsng - cgroups/cgfsng.c:recursive_destroy:983 - Error destroying /sys/fs/cgroup/freezer//lxc/foo
  lxc 20161217084041.576 ERROR lxc_conf - conf.c:userns_exec_1:4334 - Error setting up child mappings
  lxc 20161217084041.576 ERROR lxc_cgfsng - cgroups/cgfsng.c:recursive_destroy:983 - Error destroying /sys/fs/cgroup/cpuset//lxc/foo
  lxc 20161217084041.578 ERROR lxc_conf - conf.c:userns_exec_1:4334 - Error setting up child mappings
  lxc 20161217084041.578 ERROR lxc_cgfsng - cgroups/cgfsng.c:recursive_destroy:983 - Error destroying /sys/fs/cgroup/blkio//lxc/foo
  lxc 20161217084041.580 ERROR lxc_conf - conf.c:userns_exec_1:4334 - Error setting up child mappings
  lxc 20161217084041.580 ERROR lxc_cgfsng - cgroups/cgfsng.c:recursive_destroy:983 - Error destroying /sys/fs/cgroup/devices//lxc/foo
  lxc 20161217084041.581 ERROR lxc_conf - conf.c:userns_exec_1:4334 - Error setting up child mappings
  lxc 20161217084041.581 ERROR lxc_cgfsng - cgroups/cgfsng.c:recursive_destroy:983 - Error destroying /sys/fs/cgroup/memory//lxc/foo
  lxc 20161217084041.583 ERROR lxc_conf - conf.c:userns_exec_1:4334 - Error setting up child mappings
  lxc 20161217084041.583 ERROR lxc_cgfsng - cgroups/cgfsng.c:recursive_destroy:983 - Error destroying /sys/fs/cgroup/perf_event//lxc/foo

Try `lxc info --show-log foo` for more info

Questions:

Is some hardware-specific information of the host the container is originally launched on encoded in the on-disk serialized copy of the container (such as RAM, MAC address) making it impossible to start the container on another host with a slightly different hardware spec?

What else can I do for my use-case of portable containers via an external hard drive?

1

The numbers for various login names in /etc/subuid and /etc/subgid on my 2 hosts were not matching, hence the runtime failure on starting the container on the host it was not originally created (or, launched) on.

Once I made these files on the 2 hosts identical, I was able to start the containers just fine.

Note to LxD developers: A higher-level, more user-friendly error message from LxD, with hints to possible causes, would be a welcome addition to this GREAT piece of software!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.