Provisioning Fedora CoreOS on VMware
This guide shows how to provision new Fedora CoreOS (FCOS) nodes on the VMware hypervisor.
| Fedora CoreOS supports VMware ESXi ≥ 7.0, VMware Workstation ≥ 16, and VMware Fusion ≥ 12. It may be possible to modify the metadata of the OVF to run in older VMware products, but compatibility and supportability cannot be guaranteed. |
Prerequisites
Before provisioning an FCOS machine, you must have an Ignition configuration file containing your customizations. If you do not have one, see Producing an Ignition File.
Fedora CoreOS has a default core user that can be used to explore the OS. If you want to use it, finalize its configuration by providing e.g. an SSH key.
|
You also need to have access to a working VMware infrastructure, supporting VMs with at least hardware version 13. The examples below use the govc command-line tool for remote vSphere provisioning and the ovftool for local Workstation or Fusion provisioning.
Downloading the OVA
Fedora CoreOS is designed to be updated automatically, with different schedules per stream. Once you have picked the relevant stream, you can download the latest OVA:
STREAM="stable"
coreos-installer download -s "${STREAM}" -p vmware -f ovaAlternatively, OVA images can be manually downloaded from the download page.
Encoding Ignition configuration
For the vmware provider, Ignition requires two "guestinfo" fields to be present when the VM is first booted:
-
guestinfo.ignition.config.data.encoding: the encoding of the Ignition configuration. -
guestinfo.ignition.config.data: the content of the Ignition configuration, encoded according to the format above.
For maximum compatibility, it is recommended to use base64 encoding and to prepare the Ignition configuration as such:
CONFIG_ENCODING='base64'
CONFIG_ENCODED=$(cat example.ign | base64 -w0 -)An alternative to plain base64 encoding is gzip+base64 as described in the Ignition supported platforms. This is especially useful when submitting the Ignition config via govc as an inline argument. In that case the encoded config is limited to slightly under 128 KiB on Linux, 256 KiB on macOS, and 32 KiB on Windows (8 KiB if using cmd.exe or PowerShell). If your config is larger than that limit, you may be able to submit it inline after compressing it with gzip.
CONFIG_ENCODING='gzip+base64'
CONFIG_ENCODED=$(cat example.ign | gzip -9 | base64 -w0 -)If your generated Ignition configuration is still too large, you will encounter an Argument list too long error or similar. The last resort is to use a configuration file. Instead of setting an environment variable containing your Ignition configuration, create an ovftool compatible configuration file in the directory you are invoking from like so:
printf "extraConfig:guestinfo.ignition.config.data=$(cat example.ign | base64 -w0 -)\n" > ovftool.cfgBooting a new VM on Workstation or Fusion
This section shows how to use Workstation and Fusion facilities to configure and run VMs from the command-line. Some steps can potentially be performed via the graphical UI too.
Importing the OVA
The downloaded OVA has to be imported into the Workstation or Fusion library locally. At the same time the Ignition has to be provided for it to be applied to the VM.
VM_NAME='fcos-node01'
FCOS_OVA='./ova-templates/fedora-coreos-31.20200210.3.0-vmware.x86_64.ova'
LIBRARY="$HOME/Virtual Machines.localized"
ovftool \
--powerOffTarget \
--name="${VM_NAME}" \
--allowExtraConfig \
--extraConfig:guestinfo.ignition.config.data.encoding="${CONFIG_ENCODING}" \
--extraConfig:guestinfo.ignition.config.data="${CONFIG_ENCODED}" \
"${FCOS_OVA}" "${LIBRARY}"Afterwards you can refresh the list of VMs in the Workstation or Fusion UI and the new fcos-node01 VM should appear ready for booting. Its hardware configuration can be further customized at this point, and then powered-up.
If you set up an SSH key for the default core user, you can SSH into the VM and explore the OS:
ssh core@<ip address>Booting a new VM on vSphere
This section shows how to use vSphere facilities to configure and run VMs from the command-line. Similar steps can be performed via the graphical UI too.
Importing the OVA
The downloaded OVA has to be first imported into vSphere library:
FCOS_OVA='./ova-templates/fedora-coreos-31.20200210.3.0-vmware.x86_64.ova'
LIBRARY='fcos-images'
TEMPLATE_NAME='fcos-31.20200210.3.0'
govc session.login -u 'user:password@host'
govc library.create "${LIBRARY}"
govc library.import -n "${TEMPLATE_NAME}" "${LIBRARY}" "${FCOS_OVA}"Setting up a new VM
You can now deploy a new VM, starting from the template in the library and the encoded Ignition configuration:
VM_NAME='fcos-node01'
govc library.deploy "${LIBRARY}/${TEMPLATE_NAME}" "${VM_NAME}"
govc vm.change -vm "${VM_NAME}" -e "guestinfo.ignition.config.data.encoding=${CONFIG_ENCODING}"
govc vm.change -vm "${VM_NAME}" -e "guestinfo.ignition.config.data=${CONFIG_ENCODED}"Note: If the vcenter has multiple datacenters and datastores, you must specify them explicitly:
# Get resource pool using `$ govc find / -type ResourcePool`
RESOURCE_POOL="/Datacenter6.5/host/Cluster6.5/Resources"
DATASTORE="datastore-129"
govc library.deploy -pool=${RESOURCE_POOL} -ds=${DATASTORE} "${LIBRARY}/${TEMPLATE_NAME}" "${VM_NAME}"A new fcos-node01 VM is now available for booting. Its hardware configuration can be further customized at this point, and then powered-up:
govc vm.info -e "${VM_NAME}"
govc vm.power -on "${VM_NAME}"If you set up an SSH key for the default core user, you can SSH into the VM and explore the OS:
ssh core@<ip address>First-boot networking and Ignition
Ignition supports referencing remote content in configuration and fetching it at provisioning time. For this reason, on first-boot FCOS instances try to perform network autoconfiguration via DHCP.
If your VMware setup employs static network configuration instead, you can override this automatic DHCP setup with your own custom configuration. Custom networking command-line ip= parameter can be configured via guestinfo properties as shown below, before booting a VM for the first time.
The provisioning flow follows the usual steps, plus an additional guestinfo.afterburn.initrd.network-kargs entry.
if you are using a provisioning method other than govc, make sure that the guestinfo attribute is provisioned in the VM’s Advanced Configuration Parameters (also known as ExtraConfig). Some management tools may default to a vApp Property instead, which does not work in this scenario.
|
VM_NAME='fcos-node02'
IFACE='ens192'
IPCFG="ip=192.0.2.42::192.0.2.1:255.255.255.0:${VM_NAME}:${IFACE}:off"
govc library.deploy "${LIBRARY}/${TEMPLATE_NAME}" "${VM_NAME}"
govc vm.change -vm "${VM_NAME}" -e "guestinfo.ignition.config.data.encoding=${CONFIG_ENCODING}"
govc vm.change -vm "${VM_NAME}" -e "guestinfo.ignition.config.data=${CONFIG_ENCODED}"
govc vm.change -vm "${VM_NAME}" -e "guestinfo.afterburn.initrd.network-kargs=${IPCFG}"
govc vm.info -e "${VM_NAME}"
govc vm.power -on "${VM_NAME}"The full syntax of the ip= parameter is documented in Dracut manpages.
For further information on first-boot networking, see Afterburn documentation.
Troubleshooting First-boot Problems
You may encounter problems with your Ignition configuration that require access to the system log which appears during first-boot. To make a copy of the system log you can attach a serial device to the VM before booting. vSphere as well as Workstation and Fusion allow this and will save the output to a file of your choice.
To attach a serial device simply modify the hardware settings of the powered off VM and add a Serial Port. Select the destination and name of the file to be created. Afterwards power on the VM. When encountering an error, check the file you initially specified - it should contain a copy of the system log.
The serial device can also be added to the VM via govc as described in the official usage documentation:
VM_NAME='fcos-node01'
govc device.serial.add -vm "${VM_NAME}"
govc device.serial.connect -vm "${VM_NAME}" "[datastore] ${VM_NAME}/console.log"Modifying OVF metadata
| While we provide these instructions for modifying the OVF metadata, we cannot guarantee that any modifications to the OVF metadata will result in a usable guest VM. |
Fedora CoreOS is intended to run on generally supported releases of VMware ESXi, VMware Workstation, and VMware Fusion. Accordingly, the Fedora CoreOS VMware OVA image specifies a virtual hardware version that may not be compatible with older, unsupported VMware products. However, you can modify the image’s OVF metadata to specify an older virtual hardware version.
The VMware OVA is simply a tarball that contains the files disk.vmdk and coreos.ovf. In order to edit the metadata used by FCOS as a guest VM, you should untar the OVA artifact, edit the OVF file, then create a new OVA file.
The example commands below change the OVF hardware version from the preconfigured value to hardware version 13. (Note: the defaults in the OVF are subject to change.)
tar -xvf fedora-coreos-37.20230303.3.0-vmware.x86_64.ova
sed -iE 's/vmx-[0-9]*/vmx-13/' coreos.ovf
tar -H posix -cvf fedora-coreos-37.20230303.3.0-vmware-vmx-13.x86_64.ova coreos.ovf disk.vmdkWant to help? Learn how to contribute to Fedora Docs ›