Setting up a systemd-nspawn Fedora container
The systemd-nspawn container runtime is part of the systemd system software. It is exceptionally lightweight, flexible, and integrates deeply with systemd for seamless service management. This article covers how to set up a Fedora container.
|
Status: Awaiting final review. For now, just take it all with a grain of salt. |
The only prerequisite is a fully installed basic Fedora Server system.
1. Setting up the nspawn container infrastructure
-
Create a container storage area
The systemd-nspawn tools like machinectl look for system-wide usable containers in
/var/lib/machines. This directory is also created during the installation of the program if it does not exist.Following the Fedora server storage scheme, create an appropriate storage where to store all of the system-wide nspawn containers. Depending on your choice of storage concept, either create a ressource at /var/lib/machines or let the installation program create the default subdirectory.
If you created this directory yourself, make sure that the correct SELinux labels are in place, that ownership is set to root, and that accessibility is restricted to root only.
# restorecon -vFr /var/lib/machines # chown root:root /var/lib/machines # chmod 700 /var/lib/machines -
Install the software
# dnf install systemd-container # machinectl No machines. -
Add the host configuration directory for nspawn
# mkdir /etc/systemd/nspawn
2. Create an nspawn system container
In this context, the term ‘system container’ specifically refers to a container that has a complete init system and, in particular, its own network connection. It therefore operates in a similar way to a VM, albeit without its own kernel (lightweight VM), and is otherwise largely independent of the host machine.
2.1 Setting up a system containers infrastructure
-
Creating a container subdirectory
The systemd-nspawn tooling expects a subdirectory, using its name as both the container name and the default hostname.
Create a subdirectory according your storage concept.
-
Create a minimal container directory tree
The creation of a container filesystem or the provision of a corresponding image is treated as "out of scope" by systemd-nspawn. There are a number of alternative options. By far the easiest and most efficient way is simply to use the distribution specific bootstrap tool, DNF in case of fedora, in the container’s directory. This is the recommended procedure.
# dnf install dbus dhcp-client dnf fedora-release glibc glibc-langpack-en iputils \ less ncurses passwd systemd systemd-networkd systemd-resolved util-linux \ vim-default-editor --releasever=44 --setopt=install_weak_deps=False \ --best --use-host-config --installroot=/var/lib/machines/{CTNAME}Fedora 44 installs 174 packages (270M) and allocates 527M in the file system.
Optionally, you may add your local language package
# dnf --releasever=44 --best --setopt=install_weak_deps=False --use-host-config --installroot=/var/lib/machines/{CTNAME} install glibc-langpack-{YOUR_LOCALE}The message:
install-info: File or directory not found for /dev/nullappears several times. The cause is that the/dev/file system is not yet initialized at this point. You may savely ignore the message.
2.2 System container configuration and commissioning
-
Setting the password for root
This requires temporarily setting SELinux to permissive, otherwise passwd will not make any changes.
# setenforce 0 # systemd-nspawn -D /var/lib/machines/{ctname} passwd # setenforce 1In the same way you can create any other account. If you want to disallow root create a administrative account instead.
# setenforce 0 # systemd-nspawn -D /var/lib/machines/{ctname} adduser -g wheel <USERNAME> # setenforce 1 -
Configuring host to provision network interfaces
If you need a connection to an internal, protected network is needed (e.g. provided by libvirt) replace the host’s interface by a virtual bridge if not already done, and assign it’s interface name to the container ressources in the host:
# vim /etc/systemd/nspawn/{ctname}.nspawn [Network] Bridge=vbr6s0If a connection to an external public network is required as well, two corresponding interfaces must be provided. For simplicity’s sake, use a Mac VLAN on the host interface instead of a virtual bridge. (replace the host interface names accordingly).
# vim /etc/systemd/nspawn/{ctname}.nspawn (insert) [Network] MACVLAN=enp4s0 Bridge=vbr6s0If the container is not intended for use with an internal interface, leave it off.
-
Optional: Configure the connection to the internal network within the container
# vim /var/lib/machines/{ctname}/etc/systemd/network/20-host0.network (insert) # {ctname}.localnet # internal network interface via bridge # static configuration, no dhcp defined [Match] Name=host0* [Network] DHCP=no Address=10.10.10.yy/24 #Gateway=10.10.10.10 LinkLocalAddressing=no IPv6AcceptRA=noIf the internal network is also to be used for external access via NAT, the gateway entry must be commented in. Otherwise do not!
-
Optional: Configure a connection to the public network via Mac VLAN
In this case, the gateway entry must be commented out in the configuration of the internal network, as mentioned in item 3.
# vim /var/lib/machinec/{ctname}/etc/systemd/network/10-mv.network (insert) # {ctname}.example.org # public interface via mac-vlan # static configuration, no dhcp available [Match] Name=mv-enp* [Link] ARP=True [Network] DHCP=no # IPv4 static configuration, no DHCP configured! Address=www.xxx.yyy.zz/rr Gateway=www.xxx.yyy.vv # without Destination specification # treated as default! #Destination= # IPv6 static configuration Address=2001:638:708:f010::zzz/64 IPv6AcceptRA=True Gateway=2001:638:708:f010::1 # in case of issues possible workaround # cf https://github.com/systemd/systemd/issues/1850 #GatewayOnlink=yes [IPv6AcceptRA] UseOnLinkPrefix=False UseAutonomousPrefix=FalseDon’t forget to adjust interface names and IP addresses accordingly!
-
Boot the container and log in
Check if container boots without error messages
# systemd-nspawn -D /var/lib/machines/{ctname} -b OK Spawning container {ctname} on /var/l…01. OK … {ctname} login: -
Checking the status of systemd-networkd
If inactive, activate and start the service.
# systemctl status systemd-networkd … # systemctl enable systemd-networkd # systemctl start systemd-networkd # systemctl status systemd-networkd -
Check if all network interfaces are available
# ip a -
Check for correct routing
# ip route show -
Configure default DNS search path
Specify a search domain to appended to a unary hostname without domain part, usually the internal network domain name, e.g. example.lan. Adjust the config file according to the pattern below:
# vim /etc/systemd/resolved.conf [Resolve] ... #dns.quad9.net #DNS= #FallbackDNS= #Domains= Domains=example.lan #DNSSEC=no ... -
Check if name resolution is configured correctly
# ls -al /etc/resolv.conf lrwxrwxrwx. 1 root root 39 29. Dez 12:15 /etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.confIf the file is missing or is a text file, correct it.
# cd /etc # rm -f resolv.conf # ln -s ../run/systemd/resolve/stub-resolv.conf resolv.conf # ls -al /etc/resolv.conf # cdEnsure that systemd-resolved service is enabled.
# systemctl status systemd-resolvedActivate the service if necessary.
# systemctl enable systemd-resolved -
Set the intended hostname
# hostnamectl # hostnamectl set-hostname <FQDN> -
Terminate the container
# <CTRL>+]]] Container <CTNAME> terminated by signal KILL.
3. Create an nspawn application container
In this context, the term ‘application container’ specifically refers to the way in which the container shares the host’s network interfaces and possibly other resources, and may have either a reduced or a complete init system. It thus behaves like a standard application in a sandbox on the host.
3.1 Setting up an application containers infrastructure
-
Creating a container subdirectory
The systemd-nspawn tooling expects a subdirectory, using its name as both the container name and the default hostname.
Create a subdirectory according your storage concept.
-
Create a minimal container directory tree
The creation of a container filesystem or the provision of a corresponding image is treated as "out of scope" by systemd-nspawn. There are a number of alternative options. By far the easiest and most efficient way is simply to use the distribution specific bootstrap tool, DNF in case of fedora, in the container’s directory. This is the recommended procedure.
# dnf install dbus dhcp-client dnf fedora-release glibc glibc-langpack-en iputils \ less ncurses passwd util-linux \ vim-default-editor --releasever=44 --setopt=install_weak_deps=False \ --best --use-host-config --installroot=/var/lib/machines/{CTNAME}Fedora 44 installs 17x packages (2x0M) and allocates 5xxM in the file system.
Optionally, you may add your local language package
# dnf --releasever=44 --best --setopt=install_weak_deps=False --use-host-config --installroot=/var/lib/machines/{CTNAME} install glibc-langpack-{YOUR_LOCALE}The message:
install-info: File or directory not found for /dev/nullappears several times. The cause is that the/dev/file system is not yet initialized at this point. You may savely ignore the message.
3.2 Application container configuration and commissioning
-
Setting the password for root
This requires temporarily setting SELinux to permissive, otherwise passwd will not make any changes.
# setenforce 0 # systemd-nspawn -D /var/lib/machines/{ctname} passwd # setenforce 1 -
Adjust container Host’s runtime
Specifying private user configuration and shared network access.
# vim /etc/systemd/nspawn/{ctname}.nspawn (insert) [Exec] PrivateUsers=false [Network] Private=off VirtualEthernet=false -
Boot the container and log in
Check if container boots without error messages
# systemd-nspawn -b -D /var/lib/machines/{ctname} OK Spawning container {ctname} on /var/l…01. OK … {ctname} login: -
Check if all network interfaces are available
# ip aYou should see the same interfaces and IP addresses as on the host system.
-
Set the intended hostname
# hostnamectl # hostnamectl set-hostname <FQDN> -
Optional: modify the Terminalprompt
Include the name of the hostd to get ab indicator for the container
[…]# vim /etc/profile.d/custom.sh #Adjust shell prompt to include container name [ "$PS1" = "\\s-\\v\\\$ " ] && PS1="[\u@\h-mail \W]\\$ " -
Check name resolution
# ping fedoraproject.org # ping {MY_HOST_NAME} -
Check file /etc/resolv.conf
# vim /etc/resolv.conf nameserver 127.0.0.53 options edns0 trust-ad search <YOUR_DOMAIN> -
Terminate the container
# <CTRL>+]]] Container <CTNAME> terminated by signal KILL.
4. Starting a container as a system service for productive operation
-
Booting the container using systemctl
In this step, a separate UID/GID range is automatically created for the container.
# systemctl enable systemd-nspawn@{ctname} # systemctl start systemd-nspawn@{ctname} # systemctl status systemd-nspawn@{ctname} -
Log in to the container
# machinectl login {ctname}Alternatively
# machinectl shell {ctname}The latter provides you with a root shell.
-
Completing and finalizing the container configuration
Within the container, perform other designated software installations and customizations.
-
Logging off from the container
After finishing all further work inside the container press <ctrl>]]] ( Mac: <ctrl><alt>666) to exit the container.
-
Optionally: Autostart of the container at hosts boot up
The well-known 'systemctl enable …' doesn’t work here due to coordination issues of NetworkManager and systemd and network interception of libvirt. There are 2 ways to autostart a container:
-
(Re-)Activate rc.local
-
Create a rc.local file
# vim /etc/rc.d/rc.local #!/usr/bin/bash systemctl start systemd-nspawn@<CONTAINER01> systemctl start systemd-nspawn@<CONTAINER01> systemctl start systemd-nspawn@<CONTAINERnn> # chmod ug+x /etc/rc.d/rc.local -
Create systemd Services for rc.local
# mkdir /etc/systemd/system/rc-local.service.d/ # vim /etc/systemd/system/rc-local.service.d/network.conf # /etc/systemd/system/rc-local.service.d/network.conf [Unit] Wants=network-online.target After=network-online.target -
Activate systemd rc-local service
# /usr/lib/systemd/system-generators/systemd-rc-local-generator /run/systemd/generatorAfter every change to the rc.local file, you must run systemd-rc-local-generator again (specifying the full path and the argument!).
-
-
Using a systemd timer
-
Create a bash script to start each container
# vim /var/lib/machines/autostart.sh #!/usr/bin/bash systemctl start systemd-nspawn@<CONTAINER01> systemctl start systemd-nspawn@<CONTAINER01> systemctl start systemd-nspawn@<CONTAINERnn> # chmod ug+x /var/lib/machines/autostart.sh -
Create a systemd service to execute the script
# vim /etc/systemd/system/autostart-nspawn.service [Unit] Description=Autostart nspawn containers at boot After=network.target [Service] Type=oneshot User=root ExecStart/var/lib/machines/autostart.sh -
Create a systemd timer
# vim /etc/systemd/system/autostart-nspawn.timer [Unit] Description=Timer for autostart nspawn containers at boot [timer] # Starts 5 minutes after booten OnBootSec=5min Unit=autostart-nspawn.service [Install] WantedBy=timer.target -
Activate the timer
# systemctl daemon-reload # systemctl enable --now autostart-nspawn.timer
-
-
5. Troubleshooting
5.1 SELinux
On first boot after installing systemd-container, a SELinux bug currently (Fedora 34/35) blocks execution. The solution is to fix the SELinux label(s).
-
Select the SELinux tab in Cockpit, preferably before booting the container for the first time.
-
There, the AVCs are listed and solutions are offered, such as:
type=AVC msg=audit(1602592088.91:50075): avc: denied { search } for pid=35673 comm="systemd-machine" name="48865" dev="proc" ino=1070782 scontext=system_u:system_r:systemd_machined_t:s0 tcontext=system_u:system_r:unconfined_service_t:s0 tclass=dir permissive=0The proposed solution is roughly as follows:
# ausearch -c 'systemd-machine' --raw | audit2allow -M my-systemdmachine # semodule -i my-systemdmachine.pp -
The operation must be repeated until no SELinux error is reported and the container starts as a service.
Alternatively, the SELinux CLI tool can be used, which also suggests these solutions.
5.2 Error message dev-hugepages
You will find message such as
dev-hugepages.mount: Mount process exited, code=exited, statupdateus=32/n/a
dev-hugepages.mount: Failed with result 'exit-code'.
[FAILED] Failed to mount Huge Pages File System.
See 'systemctl status dev-hugepages.mount' for details.
DFN installs this by default, but it is not applicable inside a container. It is a general kernel configuration that cannot be changed by a container (at least as long as it is not configurable within namespaces).
The messages can be safely ignored.
Want to help? Learn how to contribute to Fedora Docs ›