Migrating from CoreOS Container Linux (CL) to Fedora CoreOS (FCOS)

Fedora CoreOS is the official successor of CoreOS Container Linux, which reached its end of life on May 26, 2020. This page attempts to document the differences between CL and FCOS to ease the transition to FCOS.

Introduction

To migrate from CL to FCOS, you must convert your old Container Linux Configs, Ignition configs, or cloud-config files to a Butane config and adapt the contents for FCOS. Since many of the configuration details have changed, you should reference this page and the CL migration issue on GitHub.

Installation changes

The following changes have been made to the installation process:

  • The coreos-install script has been replaced with coreos-installer. It offers similar functionality.

  • The coreos.autologin kernel command-line parameter is not currently supported in FCOS. For access recovery purposes, there are instructions available here.

  • Certain CL platforms, such as Vagrant, are not yet supported in FCOS. Refer to the Download page to see the available image types.

Software package changes

  • etcd is not included in FCOS. Refer to Running etcd for instructions to run it as a container on FCOS.

  • flannel is not included in FCOS.

  • The Podman container runtime is included in FCOS and is the recommended container runtime. The rkt container runtime is not included.

  • FCOS does not have a recommended mechanism to select the version of docker.

  • Network configuration is now handled by NetworkManager instead of systemd-networkd.

  • For time synchronization, use chronyd rather than ntpd or systemd-timesyncd.

  • Automatic updates are now coordinated by Zincati, as described in the Zincati documentation. The rollback mechanism (via grub) is now provided by rpm-ostree.

  • The functionality of the reboot manager (locksmith) is rolled into Zincati.

  • The update-ssh-keys tool is not provided on FCOS. sshd uses a helper program to read key files directly out of ~/.ssh/authorized_keys.d.

Configuration changes

When writing Butane configs, note the following changes:

  • coreos-metadata is now Afterburn. The prefix of the metadata variable names has changed from COREOS_ to AFTERBURN_, and the following platform names have changed:

  • By default, FCOS does not allow password logins via SSH. We recommend configuring SSH keys instead. If needed, you can enable SSH password authentication.

  • Because usermod is not yet fully-functional on FCOS, there is a docker group in the /etc/group file. This is a stop-gap measure to facilitate a smooth transition to FCOS. The team is working on a more functional usermod, at which time the docker group will no longer be included by default. See the docker group issue.

  • There is no way to create directories below the / directory. Changes are restricted to /etc and /var. Refer to the documentation for the storage node of the Butane config for details about writing directories and files to FCOS.

  • Butane configs no longer have a separate section for network configuration. Use the Butane files section to write a NetworkManager key file instead.

Operator notes

  • FCOS provides best-effort stability, and may occasionally include regressions or breaking changes for some use cases or workloads.

  • CL had three release channels: alpha, beta, and stable. The FCOS production release streams are next, testing, and stable, with somewhat different semantics.

  • In general, SELinux confinement should work the same as in Fedora.

  • To deploy an Ignition config as part of a PXE image (a "custom OEM" in CL terminology), follow the same process as in CL, but place the config.ign file in the root of the archive.

  • In CL, metrics/telemetry data was collected by the update mechanism. In FCOS, nodes are counted (without unique identifiers) via the Count Me mechanism.

  • Cloud CLI clients are not included in FCOS. There is an initiative to create a "tools" container to run on FCOS.

  • When opening an existing file in a sticky directory, the behavior differs from CL. See the relevant systemd commit.

  • CL left Simultaneous Multi-Threading (SMT) enabled but advised users to turn it off if their systems were vulnerable to certain issues such as L1TF or MDS. By default, FCOS automatically disables SMT for vulnerable systems.

  • In general, docker uses the default configuration from Fedora, which is different under many aspects. Notably the logging driver is set to journald and live-restore is enabled.

Implementation notes

  • The default filesystem on CL was ext4. On FCOS, the default is xfs.

  • While CL used systemd socket activation for sshd, FCOS starts sshd at startup by default.

  • CL had an "OEM partition" at /usr/share/oem with a user-customizable GRUB config and some additional tools, but FCOS does not have this.