Provisioning Fedora CoreOS on VirtualBox

This guide shows how to provision new Fedora CoreOS (FCOS) nodes on the VirtualBox hypervisor.

Prerequisites

Before provisioning an FCOS machine, you must have an Ignition configuration file containing your customizations and host it somewhere (e.g. using python3 -m http.server, we will assume this option is used in the following examples). If you do not already have an Ignition file, see Producing an Ignition File.

Setting up a new VM

You can set up a VirtualBox virtual machine through the GUI or via the vboxmanage CLI. Below is a shell script that does most of what the graphical wizard will do. Note that it is tailored to the installation from live ISO or raw disk image as outlined in the following sections.

Please be aware that for now there is no dedicated Fedora CoreOS image available for VirtualBox and the installation instructions provided are essentially bare metal instructions. A dedicated image may be provided at a later time. Please check the GitHub issues #73 and #144 for more details.

We will rely on the VirtualBox-provided NAT gateway to access the hosts' loopback interface on 10.0.2.2. Check the VirtualBox User Manual for more details.

VM_NAME='fcos-node01'
VM_DISK=$(realpath ~/VirtualBox\ VMs/${VM_NAME})/${VM_NAME}.vdi

vboxmanage createvm --name "${VM_NAME}" --ostype Fedora_64 --register
# Set hardware settings for the VM, including recommended graphics controller
vboxmanage modifyvm "${VM_NAME}" --memory 4096 --cpus 2 --vram 20 --graphicscontroller vmsvga --rtcuseutc on
# Add a storage controller for hard disks
vboxmanage storagectl "${VM_NAME}" --name SATA --add sata --controller IntelAhci --portcount 30 --bootable on
VirtualBox and by extension vboxmanage always expect absolute file paths when specifying locations.

Make sure to create a port forwarding rule that allows access to the guest’s SSH Port with the default NAT setting using e.g. localhost:2222 as target address:

vboxmanage modifyvm "${VM_NAME}" --natpf1 "guestssh,tcp,,2222,,22"

The virtual machine is now ready and FCOS can be installed.

Installing from live ISO

To install FCOS into a VirtualBox virtual machine using the live ISO, follow these steps:

podman run --security-opt label=disable --pull=always --rm -v .:/data -w /data \
    quay.io/coreos/coreos-installer:release download -f iso
  • Create a sufficiently large disk and attach it to the previously created VM:

vboxmanage createmedium --filename "${VM_DISK}" --size 10240 --format VDI
vboxmanage storageattach "${VM_NAME}" --storagectl SATA --type hdd --port 0 --device 0 --medium "${VM_DISK}"

Fedora CoreOS requires the root filesystem to be at least 8 GiB. For practical reasons, disk images for some platforms ship with a smaller root filesystem, which by default automatically expands to fill all available disk space. If you add additional partitions after the root filesystem, you must make sure to explicitly resize the root partition as shown so that it is at least 8 GiB.

Currently, if the root filesystem is smaller than 8 GiB, a warning is emitted on login. Starting from June 2021, if the root filesystem is smaller than 8 GiB and is followed by another partition, Fedora CoreOS will refuse to boot. For more details, see this bug.

  • Attach a DVD drive and the live ISO to the VM:

FCOS_DISK=$(realpath ~/Downloads/fedora-coreos-34.20210711.3.0-live.x86_64.iso)

vboxmanage storagectl "${VM_NAME}" --name IDE --add ide --controller PIIX4 --hostiocache on --portcount 2 --bootable on
vboxmanage storageattach "${VM_NAME}" --storagectl IDE --type dvddrive --port 1 --device 0 --medium "${FCOS_DISK}"

You can now boot the target system you have configured. The ISO is capable of bringing up a fully functioning FCOS system purely from memory (i.e. without using any disk storage). Once booted, you will have access to a bash command prompt.

You can now run coreos-installer inside the VM:

sudo coreos-installer install /dev/sda \
    --insecure-ignition --ignition-url http://10.0.2.2:8000/example.ign
Check out coreos-installer install --help for more options on how to install Fedora CoreOS.

Once the installation is complete, you have to remove the live ISO after shutting down (sudo shutdown -h now) the VM. Once the ISO is removed, you can start the VM anew. The actual first boot process begins now. It is at this time that Ignition ingests the configuration file and provisions the system as specified.

vboxmanage storageattach "${VM_NAME}" --storagectl IDE --type dvddrive --port 1 --device 0 --medium emptydrive
vboxmanage startvm "${VM_NAME}" --type headless
# Once the VM is available, you can log in:
ssh core@localhost -p 2222

Installing from raw bare metal image

The steps below are provided for advanced users, may unexpectedly break in various places with newer Fedora CoreOS images, and require a Unix environment to work. If you encounter issues consider using the live ISO instructions instead.

To install FCOS into a VirtualBox virtual machine using the raw disk image, follow these steps:

podman run --security-opt label=disable --pull=always --rm -v .:/data -w /data \
    quay.io/coreos/coreos-installer:release download -f raw

Before the installation can begin you have to convert the raw disk image to a VirtualBox compatible one:

FCOS_DISK=$(realpath ~/Downloads/fedora-coreos-34.20210808.3.0-metal.x86_64.raw.xz)
UNPACKED_DISK=$(realpath ~/Downloads/fedora-coreos-34.20210808.3.0-metal.x86_64.raw)

# Unpack the raw disk image
unxz "${FCOS_DISK}"

If you want to supply the Ignition configuration file URL statically, you can modify the raw disk before converting it to a VirtualBox disk image.

To correctly mount the boot partition you have to find out the start point of it and the disk’s sector size. Multiply the two values to calculate the mount offset.

TMP_FCOS_BOOT_VOLUME=$(realpath /tmp/fcos_boot_volume)
mkdir "${TMP_FCOS_BOOT_VOLUME}"

fdisk -l "${UNPACKED_DISK}"
sudo mount -o loop,offset="${OFFSET}" "${UNPACKED_DISK}" "${TMP_FCOS_BOOT_VOLUME}"

On macOS you have to replace fdisk -l "${UNPACKED_DISK}" with hdiutil imageinfo -imagekey diskimage-class=CRawDiskImage "${UNPACKED_DISK}". Mounting the partition requires macFUSE with an ext filesystem extension.

The boot config you are looking for is located in /loader/entries/ostree-1-fedora-coreos.conf on the boot partition you just mounted. The file contains a line containing the string ignition.platform.id=metal $ignition_firstboot – add ignition.config.url=http://10.0.2.2:8000/example.ign after the first boot parameter, save your changes and unmount the disk.

sudo vi "${TMP_FCOS_BOOT_VOLUME}/loader/entries/ostree-1-fedora-coreos.conf"
# ...
sudo umount "${TMP_FCOS_BOOT_VOLUME}"

Continue with converting the raw disk image into a VirtualBox disk image:

vboxmanage convertfromraw "${UNPACKED_DISK}" "${VM_DISK}" --format VDI --variant Standard
# Resize the disk to match FCOS requirements
vboxmanage modifymedium disk "${VM_DISK}" --resize 10240
# Add the converted disk to the VM
vboxmanage storageattach "${VM_NAME}" --storagectl SATA --type hdd --port 0 --device 0 --medium "${VM_DISK}"
You can use snapshots and cloning in case you want to reuse the VM as an empty baseline before applying an Ignition configuration to it.

Fedora CoreOS requires the root filesystem to be at least 8 GiB. For practical reasons, disk images for some platforms ship with a smaller root filesystem, which by default automatically expands to fill all available disk space. If you add additional partitions after the root filesystem, you must make sure to explicitly resize the root partition as shown so that it is at least 8 GiB.

Currently, if the root filesystem is smaller than 8 GiB, a warning is emitted on login. Starting from June 2021, if the root filesystem is smaller than 8 GiB and is followed by another partition, Fedora CoreOS will refuse to boot. For more details, see this bug.

You can now boot the target system you previously configured.

If you decided to supply the Ignition configuration URL dynamically or want to override your static URL, press e when the GRUB menu shows up. This allows you to edit the current boot entry and specify the remote Ignition configuration URL to use:

load_video
#...
linux ($root)/ostree/fedora-coros- #...
 ignition.firstboot ignition.config.url=http://10.0.2.2:8000/example.ign
initrd ($root)/ostree/fedora-coreos- #...

The actual first boot process begins now. It is at this time that Ignition ingests the configuration file and provisions the system as specified.

Once the VM finished booting, you can log in from your host:

ssh core@localhost -p 2222

Troubleshooting First-boot Problems

You may encounter problems with your Ignition configuration that require access to the system log which appears during the first boot. To make a copy of the system log you can attach a serial device to the VM before booting.

To attach a serial device simply modify the hardware settings of the powered off VM and add a Serial Port. Select the destination and name of the file to be created. Afterwards power on the VM. When encountering an error, check the file you initially specified – it should contain a copy of the system log.

The serial device can also be added to the VM via vboxmanage as described in the official usage documentation:

VM_LOG=$(realpath .)/${VM_NAME}.log

vboxmanage modifyvm "${VM_NAME}" --uart1 '0x3F8' '4'
vboxmanage modifyvm "${VM_NAME}" --uartmode1 file "${VM_LOG}"