Creating a virtual machine using a generic disk image – the example of virt-builder

Peter Boy Verze F35-F36 Last review: 2022-05-10

Many people and professional journals describe virt-builder as a way to "quickly build virtual machine images". This is simply wrong. Instead, the program picks up an already existing machine image, referred to as a "template", and customizes it according to the parameters that the system administrator has specified. The virt-builder program does literally build nothing at all. The differentiation may be subtle. But those phrases distract from the fact that the entire process depends entirely on a large binary blob. And for a system built and trusted on free software, the question of how that blob is created deserves attention.

How it works

The creation of a virtual machine image involves two steps.

The System Administrator invokes virt-builder and specifies the desired virtual machine properties via parameters. This includes in particular a user account, if necessary a password for root, hostname, and if desired additional programs to be installed, network configuration and other options. The program usually invoked in a way to save the created image right away in the machine images pool /var/lib/libvirt/images.

Unless explicitly configured otherwise, virt-builder downloads a (binary) disk image built by the fsguest-tools project and infuses modification into the image according to the specified parameters. This binary is created along whatever rules the fsguest-tools project or one of its developers deems useful. The project also provides tools enabling a system administrator to build a local repository, with custom-created images. This is not the subject of this guide.

This step is where the main work is done and it largely determines the final result.
The next step is to run either virt-install or cockpit to merely instantiate the prebuilt image in QEMU/kvm. Post-installation work is limited to checking the functionality and viability of the configuration, and, of course, further customizing the system for individual purposes, as with any system installation.

Overall, the process is very straightforward and efficient.

Select and customize a virtual machine binary – the example of CentOS

This step does the main work.

To get a CentOS virtual machine image, we use the default image repository, which is maintained by the guests-tools project. First, we need a list of available (CentOS) prebuilt machine images.

[…]$ virt-builder --list  |  grep centos

gpg: checking the trustdb
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
...
centos-6                 x86_64     CentOS 6.6
centos-7.0               x86_64     CentOS 7.0
centos-7.1               x86_64     CentOS 7.1
centos-7.2               aarch64    CentOS 7.2 (aarch64)
centos-7.2               x86_64     CentOS 7.2
centos-7.3               x86_64     CentOS 7.3
centos-7.4               x86_64     CentOS 7.4
centos-7.5               x86_64     CentOS 7.5
centos-7.6               x86_64     CentOS 7.6
centos-7.7               x86_64     CentOS 7.7
centos-7.8               x86_64     CentOS 7.8
centos-8.0               x86_64     CentOS 8.0
centos-8.2               x86_64     CentOS 8.2
centosstream-8           x86_64     CentOS Stream 8
centosstream-9           x86_64     CentOS Stream 9

The guestfs-tools project provides a fairly complete set of variants available in recent years. Omitting the grep term reveals an impressive list of of distribution images provided.

We want the latest and greatest CentOS release and would like to get some info about details.

[…]$  virt-builder --notes   centosstream-9
gpg: checking the trustdb
...
CentOS Stream 9

This CentOS Stream image contains only unmodified @Core group packages.

This template was generated by a script in the libguestfs source tree:
    builder/templates/make-template.ml
Associated files used to prepare this template can be found in the
same directory.

Some system administrator may not know off the top of their head what the @core module leaves off. And the abbreviated link provided doesn’t help much either.

Minimal effort customization

Even a quick and experimental setup should take the opportunity to set a number of configurations for the virtual machine very easily right from the start. These include

Either set up or block (recommended) the ROOT account
Setting up another administrative user
Configure the hostname
Configuration of the keyboard layout in case of a non-US user

[…]$ sudo su -
[…]# virt-builder centosstream-9 \
--format qcow2 --output /var/lib/libvirt/images/vm1-el9vb.qcow2 \
--root-password locked:disabled \
--hostname vm1-el9vb.example.com \
--selinux-relabel \
--firstboot-command 'localectl set-keymap de-nodeadkeys' \
--firstboot-command 'useradd -m -G wheel -p "" hostmin ; chage -d 0 hostmin'

Please, adjust the above example as apropriate!

Specifically, US users will omit the 6. line ('--firstboot-command 'localectl…`) of the virt-builder command, other will have to adjust the keyboard layout. On your local Fedora Server run `localectl list-keymaps`to get a list of supported keyboard layouts and their identifiers.

If you really are to install a short term test installation you may omit the third line (--root-password …) of the virt-builder command for connvenience and work directly as root. The app will automatically generate a password and display it. Don’t forget to copy and store it safely.

You get a lot ot output. The process takes some time. Be patient.

gpg: checking the trustdb
...
[   1.4] Downloading: http://builder.libguestfs.org/centosstream-9.xz
[   2.8] Planning how to build this image
...
[  26.2] Setting passwords
[  27.4] SELinux relabelling
[  36.6] Finishing off
                   Output file: /var/lib/libvirt/images/vm1-el9vb.qcow2
                   Output size: 6.0G
                 Output format: qcow2
            Total usable space: 5.4G
                    Free space: 4.3G (80%)

As you see, the virtual disk size is 6.0 G with 4.3 G at your disposition.

Some optional additional customization

As noted above, the maximum disk size is 6 G with about 4 G at your disposition (you may not fill the space 100%). That is not too much and you might want to enlarge it.
```
[…]# qemu-img  info   /var/lib/libvirt/images/vm1-el9vb.qcow2
[…]# qemu-img resize  /var/lib/libvirt/images/vm1-el9vb.qcow2  +10G
[…]# qemu-img  info   /var/lib/libvirt/images/vm1-el9vb.qcow2
```
The example above adds 10 GiB. The maximum virtual disk size of the CentOS image is 16 GiB now. The current (physical) disk size is still about 1 GB, due to the dynamic properties of the qcow2 file format.

You can resize the virtual disk later, too. Therefore, there is no reason to plan too generously in terms of size now.

You should not increase the maximum virtual disk image size by the virt-builder parameter --size. It would increase the physical disk size to the same amount, which you probably don’t want.
Additional information provides the virt-install man page.

Using virt-install CLI to instantiate the VM

Use a terminal window. First, you may check the correct naming for the parameter os-variant.

[…]# virt-install  --osinfo list

Import the virtual disk image.

[…]# virt-install  --name vm1-el9vb \
     --memory 2048  --cpu host --vcpus 2 --graphics none\
     --os-variant centos-stream9\
     --import  \
     --disk /var/lib/libvirt/images/vm1-el9vb.qcow2,format=qcow2,bus=virtio \
     --network type=direct,source=enp0s25,source_mode=bridge,model=virtio \
     --network bridge=virbr0,model=virtio

The parameters are quite descriptive. You will find a more detailed explanation in the appendix.

You see a lot of output:

Starting install...;
Creating domain...;
Running text console command: virsh --connect qemu:///system console vm1-el9vb
Connected to domain 'vm1-el9vb'
Escape character is ^] (Ctrl + ])

[    0.000000] Linux version 5.14.0-71.el9.x86_64 (mockbuild@x86-05.stream.rdu2.redhat.com) &#8230;&#8203;
[    0.000000] The list of certified hardware and cloud instances for Red Hat &#8230;&#8203;
...
[  OK  ] Finished Record Runlevel Change in UTMP.
[  OK  ] Finished Network Manager Wait Online.
[  OK  ] Reached target Network is Online.
         Starting Crash recovery kernel arming&#8230;&#8203;

CentOS Stream 9
Kernel 5.14.0-71.el9.x86_64 on an x86_64

localhost login:

Log in with the administrative user account (hostmin in this example). You can log in without a password, but you must create a (new) password immediately.

Adjust the new VM instance

You did a minimal customization so var and need to do some further adjustments.

Network consolidation

Check the available network connections
```
[…]# ip a
[…]# nmcli con
```
Depending on your runtime environment you may have to consolidate the network configuration. In the above example you get 2 connections, one to the public network and another to the internal server network. For details see Adding Virtualization Support.
Recommended: Adjust the naming

Sometimes a connection gets named something like 'Wired connection 1'. For ease of administration, change the name, e.g. the device name.
```
[…]# nmcli con mod 'Wired connection 1' connection.id enp1s0
```
In case of your internal network (virbr0), which provides DHCP, this step persists the network configuration. So you can assign a specific zone. Otherwise the system would create the connection anew with each boot.

On demand: Adapt the external interface

If the external network does not provide DHCP, you get just a minimal configuration without IP addresses. Adjust as appropriate.

[…]# nmcli con mod enp1s0 ipv6.method manual ipv6.addresses '2a01:xxx:yyy:zzz::uu/88' \
   ipv6.gateway 'fe80::1' \
   ipv6.dns '2a01:xxx:yyy:zzz::uuu:vvv 2a01:xxx:yyy:zzz::uuu:vvv 2a01:xx:yy:zz::uu:vv'

[…]# nmcli con mod enp1s0 ipv4.method manual ipv4.addresses 'xx.yy.zz.ww/vv' ipv4.gateway 'xx.yy.zz.ww' ipv4.dns 'xx.yy.zz.uu xx.yy.zz.vv xx.yy.zz.ww'

On demand: assign a firewall zone

Specifically for the internal interface (usually enp2s0) you might want to assign a specific zone, e.g. trusted or internal. And for the internal interface you might disable IPv6, Adjust as appropriate
```
[…]# nmcli con mod enp2s0 connection.zone 'internal'
[…]# nmcli con mod enp2s0 ipv6.method 'disabled'
```

Finally restart the connections

[…]# nmcli con up enp1s0
[…]# nmcli con up enp2s0

Using Cockpit’s graphical UI to instantiate the VM

Select Virtual Machines in the left navigation bar und click on Import VM. A new form will open up

Cockpit `__Import a virtual machine__` form

Specify a name for the virtual machine to be created. It must be unique at least in the host server’s namespace and at best in the designated domain namespace. Select a connection type, use system for production deployments. Consult the guide Installing a Fedora Server virtual machine using Cockpit for details about connection type.

Fill the remaining fields accordingly. Finally, deactivate Immediately start VM. Otherwise, you will not be able to specify the network environment and the first boot autoconfiguration can not configure the network.

Select Import to create the basic virtual VM definition. It takes a short time and then the Virtual machines page is displayed again. It’s list of virtual machines now shows an entry with the just defined VM in the status "Shut off" and as next action "Run".

Extended definition of the virtual machine

Clicking the virtual machine name in the list opens a detailed configuration page.

Cockpit `__Virtual machin overview form__`

Autostart

Adjust the autostart option in the upper part according to your need.

Network connections

The list of network interfaces shows only one interface, the internal network managed by libvirt. However, you want to have external access, too. The key part of most VMs is to isolate public accesse from the host server for security reasons.

To enable public access, you can either bind a virtual bridge to the interface or use Mac-vlan. The latter adds virtual interfaces to the physical interface, each with its own Mac address and its own IP (and alias IPs if needed). The libvirt toolkit refers to this as direct attachment. It is now the recommended approach. It acts similar to a bridge, but with less system load. The disadvantage is that direct communication between the host and the VMS is not possible, but between the VMs it is. Hosts and VMs can only communicate via the internal, protected network. For administrators of remote, not directly accessible servers, the additional big advantage is that after the initial configuration, there is no need to touch the precious network connection again.

An administrator who sticks to the habit that the first network adapter in the device list establishes the external connection will now edit and rearrange the existing network configuration. Select Edit to access the Configuration form.

Cockpit `__Virtual ethernet configuration form__`

Replace the interface type by Direct attachment and select the external physical interface of the host in the Source field. Leave model and MAC address unchanged.

Next, if you also want an internal network, select Add network interface. A nearly identical form pops up. Select Interface tpye as Virtual network if it is not already preselected and default as Source. Again, leave model (Linux, perf) and MAC address (Generate automatically) unchanged.

Now everything is ready. Select "Run" at the top to complete the import and to start the VM.

The Console window shows the startup process and finally the login prompt. Log in with the administrative user account (hostmin in this example). You can log in without a password, but you must create a (new) password immediately.

Post instantiation tasks

If you look around, you will find

The hostname is already properly defined
If DHCP is on all interfaces available, network connection is working perfectly. However, the interfaces are not configured permanently, but transiently.
Firewall is active. All interfaces belong to the default zone "public".
The system uses a gpt partition table. It contains a 1GB XFS /boot partition, a swap partition, and an XFS root partition that fills the rest.
However, Cockpit is not even installed.

The resemblance of a CentOS server is not perfect, but fairly well done.

Conclusion

Overall, the virt-builder/guestfs-tools-project provides an amazing opportunity to quickly and straightforwardly create a well-crafted virtual machine, instantiating various different distributions. Fortunately, it is an open source project, but unfortunately the "openness" did not receive detailed attention. It is very hard to figure out how to access the source code. So, it is difficult to learn how the virtual machine is built in detail and to check for potential malicious ingredients. The provided information is quite scanty and sometimes misleading. For example, the information on Fedora reads: "Fedora® 35 Server". However, the virtual machine created does not have much to do with "Fedora Server Edition". More appropriate would be something like "Some server based on Fedora RPM’s". Sponsored by Red Hat, a lot of Red Hat engineers are engaged in the project. So you may (and should) pay some leap of faith, anyway.

A system administrator cannot expect to get a differently built, but otherwise identical build of a distribution. This will need a closer assessment and more or less detailed rework.

An alternative is an instantiating using a cloud base image. In the end, both ways produce a similar result that requires some reworking anyway. If available, neither can replace a virtual machine disk image created directly by the distribution itself.

Appendix

Short explanation of the virt-install parameter used

--name VM_NAME: Unique name of the VM to install as shown e.g.in VM list
--memory 3074: Amount of memory to allocate, adjust as appropriate
--cpu host: same cpu type as host
--vcpus 3: number of cpus for VM, adjust as appropriate
--os-variant centos-stream9: Target operating system. Adjust distribution and version as needed
--import: Fixed, skips installation procedure and boots from the first (virtual) disk as specified by the first disk parameter.
--graphics none: Fixed, enforces a redirect of the VM login prompt to the host terminal window for immediate access. Enables to login either via Cockpit terminal window or via host terminal using virsh console <VM_NAME> (you may have to issue one or two additional <enter>)
--disk /var/lib/libvirt/images/VM_NAME.qcow2, format=qcow2,bus=virtio: disk image file, adjust VM_NAME
--network direct,source=enpXsY,source_mode=bridge, model=virtio: specify external netwok (macvlan) first, it will get the name eth0 as usual. Adjust interface name as appropriate.
--network bridge=virbr0,model=virtio: specify the internal network (libvirt generated bridge) second. It will get the name eth1 as usual.

Virt-install in Fedora or CentOS

Critical for customization to work is SELinux relabeling, not only in the case of installing additional software, but for virtually any customization.

Want to help? Learn how to contribute to Fedora Docs ›