Adding Virtualization Support
|
Status of this document: Updated to f45, not fully approved yet. |
Qemu-kvm in combination with Libvirt management toolkit is the standard virtualization methodology in Fedora. This optionally includes a local virtual network that you may use for protected communication between the virtual guest systems and between the guests and the host. Its default configuration enables access to the public network via NAT.
Preparation
Hardware requirements
QEMU / KVM require hardware virtualization support. The first thing to do is to make sure that it is available.
# grep -E --color 'vmx|svm' /proc/cpuinfo
The command will return one line per cpu core if virtualization is enabled. If not, you should first check in the BIOS whether virtualization is disabled.
Storage set up
Libvirt stores its data including the image files of the virtual hard disk(s) for the guest systems in /var/lib/libvirt. If you adhere to the default partitioning layout, the libvirt application data is stored in its own logical volume that you have to create in advance. You need to specify the size of the storage area, a unique name, and the accommodating VG (fedora in case of default partitioning). In the new logical volume, create an xfs file system and mount it at /var/lib/libvirt.
Cockpit
The easiest way is to use Cockpit. Start your favorite browser and navigate to your server, named example.com here.
https://example.com:9090
If no valid public certificate has been installed yet, a browser warning will appear and you will need to accept an exception for the self-signed certificate. Log in using your unprivileged administrative user account. Acquire administrative access if this has not already been done.
In the left navigation column, select 'Storage'. A list of storage devices will then be displayed in the centre content area.
To create a logical volume, click on the three dots to the right of the 'Volume group' line (which is 'fedora' or 'fedora-HOSTNAME' by default). Then select 'Create new logical volume'.
In the form that opens, enter the name of the new logical volume in the top field. In this example, enter 'libvirt'. Leave the 'Usage' field as 'Block device for file systems' and adjust the size at the bottom to 500 GiB, for example. Then create the LV.
A new line appears in the 'Logical volumes' list with the LV name. In this example, it is 'libvirt'. Select the three-dot button again and then 'Format'. In the form that opens, enter the name of the new file system (e.g. 'libvirt' in this case) and the mount point (e.g. /var/lib/libvirt). Leave the other fields at their default values. Select 'Format' and Cockpit will handle everything else.
Once complete, the file system becomes immediately available and is permanently configured accordingly.
Command line
Some administrators may prefer the command line for easy scripting. Create a Logical Volume of appropriate size, 50 GiB in this example. It is helpful to create the logical volume in accordance with its purpose, in this case for libvirt. Install either in the system Volume Group (named fedora by default) or in the user data VG if created during installation. Adjust size and VG name as required.
# lvcreate -L 50G -n libvirt fedora
# mkfs.xfs /dev/fedora/libvirt
# mkdir -p /var/lib/libvirt
# echo "UUID=$(blkid -s UUID -o value /dev/mapper/fedora-libvirt) /var/lib/libvirt xfs defaults 0 0" >> /etc/fstab
# systemctl daemon-reload
# mount -a
Installing libvirt virtualization software
Installing the software is quite simple.
# dnf install qemu-kvm-core libvirt virt-install cockpit-machines guestfs-tools
Be sure to install guestfs-tools, not libguestfs-tools (unless you need additional windows guest related software). The package guestfs-tools provides a basic set of various useful tools to maintain virtual disks. Additional packages provide support for specific use cases, e.g. various file systems or forensic support. Use dnf search guestfs to get a list of available packages.
Do not install the group @virtualization onto a Fedora Server. It includes various graphical programs and libraries that are not usable on headless servers.
Next check the SELinux labels
# ls -alZ /var/lib/libvirt
Usually, installation sets the SELinux labels properly. Otherwise, set them manually.
# restorecon -R -vF /var/lib/libvirt
If everything is correct, proceed with the next step.
Activating virtualization
The libvirt Fedora installation procedure provides systemd startup scripts that take care of enabling and starting the various unix sockets and services as needed. This includes support for qemu, xen and lxc. Configuration of vbox is disabled by default. The drivers determine during startup whether the required prerequisites are met and abort otherwise. The default services, qemu and lxc in case of Fedora Server, are started at boot time. If not used for about one minute they are deactivated, but will restart on demand via socket activation as soon as a virtual machine is started (either by command line or Cockpit service). There is no need for administrator intervention at all.
-
Activate (start) the required libvirt modular drivers
Just after installation start libvirt manually to be able to perform the remaining configuration steps without reboot.
# for drv in qemu interface network nodedev nwfilter secret storage ; \ do systemctl start virt${drv}d{,-ro,-admin}.socket ; done -
Check successful start via a status query
# for drv in qemu interface network nodedev nwfilter secret storage ; \ do systemctl status virt${drv}d{,-ro,-admin}.socket ; done ● virtqemud.socket - libvirt QEMU daemon socket Loaded: loaded (/usr/lib/systemd/system/virtqemud.socket; enabled; preset: enabled) Active: active (listening) since Wed 2024-04-... Triggers: ● virtqemud.service Listen: /run/libvirt/virtqemud-sock (Stream) CGroup: /system.slice/virtqemud.socket Apr 10 13:51:33 example.com systemd[1]: Listening on virtqemud.socket - libvirt QEMU daemon socket. ● virtqemud-ro.socket - libvirt QEMU daemon read-only socket Loaded: loaded (/usr/lib/systemd/system/virtqemud-ro.socket; enabled; preset: enabled) Active: active (listening) since Wed 2024-04-... Triggers: ● virtqemud.service Listen: /run/libvirt/virtqemud-sock-ro (Stream) CGroup: /system.slice/virtqemud-ro.socket Apr 10 13:51:33 example.com systemd[1]: Listening on virtqemud-ro.socket - libvirt QEMU daemon read-only socket. ● virtqemud-admin.socket - libvirt QEMU daemon admin socket Loaded: loaded (/usr/lib/systemd/system/virtqemud-admin.socket; enabled; preset: enabled) Active: active (listening) since Wed ... Triggers: ● virtqemud.service Listen: /run/libvirt/virtqemud-admin-sock (Stream) lines 1-23 ... ...
You can use the virtualisation functionality immediately, even if you might not realise it at the moment. As we cannot currently start any libvirt services, the service daemon processes terminate within the first one to two minutes. However, they remain dormant and ready to go via the socket service.
Configuring network services
By default, libvirt includes a configuration for an internal network to enable communication between VMs, as well as between the host and the VMs. It also provides VMs with NAT access to the public network.
Installation and configuration are straightforward. Unfortunately, libvirt bypasses Network Manager and systemd. This disrupts the capabilities of network services and, in particular, systemd-resolved’s split DNS capability. It also adds some quirky firewall rules that significantly hinder port forwarding to VMs in particular.
Alternatively, you can configure the same functionality using NetworkManager, systemd and DNSMasq. Although this is a bit more involved, these tools work together perfectly and result in fewer problems.
The no-internal-libvirt-network alternative
In some usage scenarious you will not need or even not want an internal network for communication between virtual machines. So you have to deactivate the libvirt internal network service.
-
Deactivate libvirt network daemon
Stop libvirt’s virtnetworkd daemon, disable autostart at boot and mask it so it cannot be activated accidentally.
# systemctl stop virtnetworkd{,-ro,-admin}.socket # systemctl stop virtnetworkd.service # systemctl disable virtnetworkd{,-ro,-admin}.socket # systemctl disable virtnetworkd.service -
When libvirt was already running, check and delete any leftovers
Check for the internal bridge interface
# ip a ... <n>: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc htb state DOWN group .... link/ether 52:54:00:91:80:46 brd ff:ff:ff:ff:ff:ff inet 192.168.122.1/24 brd 192.168.122.255 .... valid_lft forever preferred_lft foreverIf the a virbr0 interface exists as above, delete it.
# nmcli con delete virbr0 # nmcli dev delete virbr0 # ip aStop the dnsmasq process started by libvirt
# ps -ef | grep dnsmasq dnsmasq {proc1} 1 . . . /usr/bin/dnsmasq . . . root {proc2} 6787 . . . /usr/bin/dnsmasq root 8968 8060 . . . grep --color=auto dnsmasq # kill -9 {proc2} # kill -9 {proc1} # ps -ef | grep dnsmasq root 8968 8060 . . . grep --color=auto dnsmasq -
Optionally, you may mask the internal network driver so it cannot be activated accidentally
# systemctl mask virtnetworkd{,-ro,-admin}.socket Created symlink '/etc/systemd/system/virtnetworkd.socket' → '/dev/null'. Created symlink '/etc/systemd/system/virtnetworkd-ro.socket' → '/dev/null'. Created symlink '/etc/systemd/system/virtnetworkd-admin.socket' → '/dev/null'. # systemctl mask virtnetworkd.service Created symlink '/etc/systemd/system/virtnetworkd.service' → '/dev/null'. -
Reboot
# rebootCheck that libvirt internal network is not active.
# ip a # ps -ef | grep dnsmasq
Configuration of the internal-libvirt-network alternative
As mentioned above libvirt service processes are shutdown after about 1 minute of inactivity. This includes the default internal virtual network. The internal interface (virbr0) is neither visible nor active. Nevertheless, we can configure it.
-
Check /etc/resolv.conf
[…]# ls -al /etc/resolv.confIt has to be a link and not a file. Fix it if requirred.
[…]# cd /etc […]# rm -f resolv.con […]# systemctl restart systemd-resolved […]# ln -s ../run/systemd/resolve/stub-resolv.conf resolv.conf -
Adjust libvirt internal network configuration
The default configuration of the internal network (virbr0) activates just a DHCP Server. If the virtual machines should also be able to communicate with each other and the host, then adding a DNS server is at least very advantageous.
The first step is to choose a domain name. A top-level ".local" is explicitly not recommended, nor taking one of the official top-level names. But for example, you can take the official domain name and replace the last, top-level part with 'lan' or 'internal' or
localnet. An official domain example.com would translate to an internal domain example.lan. We use that one throughout this tutorial. The host gets the internal name host.example.lan.Use the libvirt tool to adjust the default network. Replace names and placeholders as required. Delete the line with "forward mode = 'nat'" if you do not want to allow access to the public network via the virtual network.
# virsh net-edit default <network> <name>default</name> <uuid>aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee</uuid> <bridge name='virbr0' stp='on' delay='0'/> <mac address='52:54:00:xx:yy:zz'/> <forward mode='nat'/> <mtu size='8000'/> <domain name='example.lan' localOnly='yes'/> <dns forwardPlainNames='no'> <forwarder domain='example.lan' /> <host ip='192.168.122.1'> <hostname>host</hostname> <hostname>host.example.lan</hostname> </host> </dns> <ip address='192.168.122.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.122.2' end='192.168.122.254'/> </dhcp> </ip> </network> -
Activate the modified configuration
# virsh net-destroy default # virsh net-start default -
Check if the DNS resolution works
# nslookup host 192.168.122.1 # dig @192.168.122.1 host.example.com -
Activate a permanent network service
The libvirt services don’t start at boot time, but on demand at first access via socket activation. Therefore, you won’t get an interface virbr0 until some libvirt service requests it. That’s sometimes inconvenient, e.g. if you use that interface for non-libvirt services, too (e.g. lxd or nspawn container). You may prefer to enable the virt-network service anyway. Otherwise you may skip this step.
[…]# systemctl enable virtnetworkd.service --nowAn interface virbr0 as well as a dnsmasq server is now available, regardless of when and if a libvirt service is started.
Finishing Cockpit-machines configuration
Open your browser and connect to the Cockpit instance of your host server. Consult the Postinstallation Customizations guide to learn about the possible connection paths. Log in as root or with your administrative account. In the overview (start) page select Virtual Machines in the left navigation column.
If there is no entry Virtual Machines in the navigation column, the cockpit-machines module was left off in the installation step above. Select Applications further down and then Machines for installation.
|
Currently, it sometimes takes a long time to display additional installation options. You may install the module with |
Storage pools
When first used, the list of virtual machines displayed in the center of the page is empty, of course. A box left above that list displays 0 Storage pools. Libvirt uses pools to determine the location of typical files. The installer has already created the directories. Only the pools need to be defined here.
Typically you use one Pool for installation media, stored at /var/lib/libvirt/boot. "Installation media" would be a suitable descriptive pool name. Select 0 Storage pools in the box and then Create storage pool. A new form opens.
If you are logged in as an administrative user (even if having used sudo su - ), you are asked to select a connection type, "system" or "session". This selection is presented in various configuration forms, so we explain here. Use "system" for production deployments, the common case. Select "session" in the special case of testing, development, and experimentation. The "session" option does not support any custom or advanced networking, but works pretty much everwhere (including containers) and without any privileges. The libvirt project provides additional information for developers. If you are logged in directly as root, this line doesn’t show up. Instead, everything is treated as system; i.e. production deployment (never do development or experimentation as root).
Next enter "Installation media" as the name, "Filesystem directory" as the type, and /var/lib/libvirt/boot as the target path.
In most cases, the (virtual) hard disk used for a virtual machine is a disk image file stored in /var/lib/libvirt/images. Define another pool named "Disk images" accordingly.
Activate both pools in the drop down menu of each pool.
You may create additional pools as needed, e.g. disk images via iSCSI in a SAN or as a logical volume in a volume group (LVM) on the host’s local disk. The latter offers better performance in theory, but the practical gain is usually rather small, if any. We will not go into further detail here for the time being.
Networks
A box on the right above the virtual machines list shows 1 Network and lists the networks managed by libvirt. By default, it contains the internal network default with the interface virbr0. The list does not contain the external interface. It is managed by the server. Nevertheless, it is available for virtual machines.
Completed
Virtualization is now ready to use on the server and you can start setting up guest VMs.
This guide just describes the Fedora specific way to use libvirt. For further information please use the libvirt project documentation.
Want to help? Learn how to contribute to Fedora Docs ›