I moved my server from a hacky Debian install to a hopefully maintain- and scalable NixOS installation. However I have most of my services in LXC containers to create at least some seperation and smart network management.
You might ask why I don’t use Docker, here’s why: Docker is a rather nice solution but they accelerated way too fast. Too many people jumped on the band wagon (Microsoft, lol) and the flawed architecture (RUN apt-get install) became the new standard on how to deploy machines to production. Without the huge funding from the dark side of ycombinator, Docker would never have been as “successful” as it is today. It would have been an alternative to LXC with less than 1000 forks on GitHub and something far better could have emerged more quickly. If something groundbreaking surfaces now, the whole world needs to uninstall Docker and convert all machines to the new solution. Software should convince with features not with marketing. </rant>
NixOS seems like a good idea to overcome most of the issues I have with LXC, Docker and Debian. This article will tell you how to start your LXC containers on NixOS.
While the lxc package is already present, containers do not run out of the box since a few folders are missing and need to be created by hand. This article will soon be out of date once I overcome my everlasting procastination phase and I will be able to create a PR that reflects everything I write in this article. Wait? Why am I writing this article again instead of creating the PR right away…?
Since I also use
veth adapters, I will show you some parts
configuration.nix that are responsible for creating the
bridge, iptables entries which looks something like this.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
You will need to create a few directories after running
Once this is done you can migrate or create your containers.
If you have legacy containers from e.g. a debian installation you need
to alter your LXC
config a bit.
Most of these LXC containers include a config like
which cannot be resolved since it lives somewhere in the nix-store.
So you should create a symlink to the file in the nix-store or copy
the file to
Then you want to create a file that you can name something like
/var/lib/lxc/nixos_flavour.custom.conf. In this case
In your container config you can simply use this line to
flavor.common.conf indirectly while adding the
AppArmor option that is currently necessary when using rather
old LXC containers. So for the example above you want
to add this line:
This might not be the ideal solution but currently works for me. In the end you want to put every option into one single config which will not change when the hash of lxc changes in the nix-store.
Now to systemd, our beloved overlord. Starting with version 230, it behaves like a good init daemon should behave:
It kills all processes created within a user session that are not being
killed by the user when the session is destroyed.
So if you start your containers using
lxc-start, systemd will
kill them once you log off or disconnect from your server. meh.
(this also breaks tmux, screen, most vnc servers)
While I see the point of this, I only need the LXC containers
for some weeks until every service is converted to a
expression - so writing a systemd unit for it is not really
Therefore I put this into my
1 2 3