Installing CoreOS on Bare Metal

This guide provides instructions to install Fedora CoreOS to bare metal. Three options are available:

  • Installing from live ISO

  • Installing from PXE

  • Installing from the container

Prérequis

Before installing FCOS, you must have an Ignition configuration file and host it somewhere (e.g. using python3 -m http.server). If you do not have one, see Producing an Ignition File.

If you have servers with different types and/or number of hard drives, you must create a separate customized Ignition config for each machine (or machine class). A good model is to have the common parts of the configuration factored out into a separate Ignition configuration which can be merged into (via HTTP or inline) the per-machine custom config.

Installing from live ISO

To install FCOS onto bare metal using the live ISO interactively, follow these steps:

podman run --privileged --pull=always --rm -v .:/data -w /data \
    quay.io/coreos/coreos-installer:release download -s stable -p metal -f iso

Note this is just using coreos-installer as a tool to download the ISO.

You can boot the live ISO in either legacy BIOS or UEFI mode, regardless of what mode the OS will use once installed.
  • Burn the ISO to disk. On Linux and macOS, you can use dd. On Windows, you can use Rufus in "DD Image" mode.

  • Boot it on the target system. The ISO is capable of bringing up a fully functioning FCOS system purely from memory (i.e. without using any disk storage). Once booted, you will have access to a bash command prompt.

  • You can now run coreos-installer:

sudo coreos-installer install /dev/sda \
    --ignition-url https://example.com/example.ign

Once the installation is complete, you can simply sudo reboot. After rebooting, the first boot process begins. It is at this time that Ignition ingests the configuration file and provisions the system as specified.

For more advanced ISO installs, including automation, see below.

Check out coreos-installer install --help for more options on how to install Fedora CoreOS.

Installing from PXE

To install from PXE, follow these steps:

  • Download an FCOS PXE kernel, initramfs, and rootfs image:

podman run --privileged --pull=always --rm -v .:/data -w /data \
    quay.io/coreos/coreos-installer:release download -f pxe
Booting the live PXE image requires at least 2 GiB of RAM with the coreos.live.rootfs_url kernel argument, and 3 GiB otherwise. You can install in either legacy boot (BIOS) mode or in UEFI mode, regardless of what mode the OS will use once installed.
  • Follow this example pxelinux.cfg for booting the installer images with PXELINUX:

DEFAULT pxeboot
TIMEOUT 20
PROMPT 0
LABEL pxeboot
    KERNEL fedora-coreos-32.20200726.3.1-live-kernel-x86_64
    APPEND initrd=fedora-coreos-32.20200726.3.1-live-initramfs.x86_64.img,fedora-coreos-32.20200726.3.1-live-rootfs.x86_64.img coreos.inst.install_dev=/dev/sda coreos.inst.ignition_url=http://192.168.1.101:8000/config.ign
IPAPPEND 2

For more details on how to use this information, see this blog post for testing a PXE installation via a local VM and libvirt.

Installing from the container

You can use the coreos-installer container from an existing system to install to an attached block device. For example (substitute docker for podman if needed):

sudo podman run --pull=always --privileged --rm \
    -v /dev:/dev -v /run/udev:/run/udev -v .:/data -w /data \
    quay.io/coreos/coreos-installer:release \
    install /dev/vdb -i config.ign

In this example, coreos-installer will download the latest stable FCOS metal image and install it onto /dev/vdb. It will then inject the Ignition file config.ign in the current directory into the image. Use --help to see all the available options.

Downloading and mirroring the metal image

Sometimes, it’s necessary to download the metal image ahead of time and then have it passed locally to coreos-installer for installation. You can download the metal image directly from the FCOS download page, or you can use coreos-installer download.

When installing via the live ISO or PXE, there is no need to download the metal image. It is already part of those environments.

There are two metal images: one for 512b-sector disks (labeled "Raw" on the download page), and one for 4k-sector native disks (labeled "Raw (4K Native)"). Unless you know to be targeting a 4k native disk, use the 512b one, which is the most common. See this page for more information.

To download the 4kn native metal image with coreos-installer download, use the --format 4k.raw.xz switch.

The metal image uses a hybrid partition layout which supports both BIOS and UEFI booting.

When you’re finally ready to install FCOS, you can point it at your downloaded image using coreos-installer install --image-url <LOCAL_MIRROR> or coreos-install --image-file <PATH>.

Live PXE

In this model, rather than performing a "persistent" installation to disk, you can run directly from RAM. This is useful in e.g. "diskless" scenarios. The steps are similar to the above, just omit the coreos.inst arguments, and instead have the system itself run Ignition:

DEFAULT pxeboot
TIMEOUT 20
PROMPT 0
LABEL pxeboot
    KERNEL fedora-coreos-32.20200726.3.1-live-kernel-x86_64
    APPEND initrd=fedora-coreos-32.20200726.3.1-live-initramfs.x86_64.img,fedora-coreos-32.20200726.3.1-live-rootfs.x86_64.img ignition.firstboot ignition.platform.id=metal ignition.config.url=http://192.168.1.101/config.ign systemd.unified_cgroup_hierarchy=0
IPAPPEND 2

PXE rootfs image

Before August 2020, the Fedora CoreOS PXE image included two components: a kernel image and an initramfs image. Beginning in August, a third rootfs image was added. As of October 6, the rootfs image is mandatory and the live PXE system will not boot without it.

To boot with the rootfs artifact, make one of the following changes to your PXE config:

  • Specify only the initramfs file as the initrd, and pass an HTTP(S) URL for the rootfs using the coreos.live.rootfs_url= kernel argument.

  • Specify both initramfs and rootfs files as initrds in your PXE configuration. This can be done via multiple initrd directives, or using additional initrd= parameters as kernel arguments.

  • Concatenate the initramfs and rootfs files together, and specify the combined file as the initrd.

For more information on this change, see the tracker issue.

Automated ISO/PXE installs with Ignition embedding

The Fedora CoreOS live environment is also CoreOS in the sense that it can boot via Ignition, execute containers, etc. It includes all of the same content.

As noted above for live PXE, the ISO live environment does not have to actually perform a persistent installation. You can boot it from a read-only medium such as a physical CD-ROM/DVD, and do everything you do on any other Fedora CoreOS environment. It also works to boot from a USB stick. Each boot will re-run the Ignition config, and changes will not persist by default.

For the ISO, the mechanism to do this is coreos-installer ignition iso embed, which will create a new .iso file that combines your configuration with the ISO. Similarly, there is coreos-installer pxe ignition wrap for the PXE case.

However, many system administrators will want to perform fully unattended persistent installations instead of running stateless.

Some documentation on this is on the upstream installer site: customizing install.

To emphasize, there are two Ignition configurations here; the first config ("ISO Ignition") will commonly embed a second rendered configuration that runs on the "target" installation.

First, generate target.ign (the file can be named anything) - the configuration that will be passed to coreos-installer. Then, using e.g. butane, embed it as a file in your ISO ignition, and use a custom systemd unit to pass it to coreos-installer:

variant: fcos
version: 1.1.0
storage:
  files:
    - path: /etc/target.ign
      contents:
        inline: |
          Replace this bit with a real butane directive that
          fetches the target Ignition however you like e.g.:
          local: target.ign
      mode: 0644
systemd:
  units:
    - name: my-coreos-installer.service
      enabled: true
      contents: |
        [Unit]
        Description=Run CoreOS Installer
        Requires=coreos-installer-pre.target
        After=coreos-installer-pre.target
        OnFailure=emergency.target
        OnFailureJobMode=replace-irreversibly

        # Can be removed if install doesn't reference remote resources with
        # --stream, --image-url, or --ignition-url
        After=network-online.target
        Wants=network-online.target

        [Service]
        Type=oneshot
        ExecStart=/usr/bin/coreos-installer install -i /etc/target.ign /dev/sda
        ExecStart=/usr/bin/systemctl --no-block reboot
        StandardOutput=kmsg+console
        StandardError=kmsg+console

        [Install]
        RequiredBy=default.target

Note that this configuration is completely independent of the config passed for the ISO/PXE boot, in the sense that e.g. no systemd units and files written from the live Ignition will be preserved when booted into the final installed system, unless you take explicit action to preserve it.

There is however explicit support for copying network configuration with coreos-installer.

A generally useful technique is to add more systemd units that run before or after the systemd unit that invokes coreos-installer. For example, you can run a systemd unit which pulls a container and does hardware validation.

An example post-install action: Some provisioning systems may require a callback to the PXE server to be switched to "boot from local disk" via a HTTP request; this can similarly be done via a systemd unit that is scheduled After=my-coreos-installer.service that uses ExecStart=/usr/bin/curl or pulling a container which makes the HTTP request.

ISO installation on diverse hardware

Commonly bare metal systems will have a diversity of hardware - some systems may have NVMe drives /dev/nvme*, whereas others have /dev/sd* for example. You will almost certainly have to template the value of /dev/sda above.

A useful approach is to script generating a per-machine .iso. If you have a hardware database (whether a text file in git or relational database) then it will work to generate a per-machine target-dell.ign and target-hp.ign for example, embed that with the generic iso.ign to generate fedora-coreos-install-dell.iso and fedora-coreos-install-hp.iso.

Alternatively, instead of generating per-machine ISOs, you can have the ISO Ignition pull a privileged container which inspects the target system, and dynamically invokes coreos-installer.