Configuring Storage

Fedora CoreOS ships with a simple default storage layout: the root partition is the last one and expands to take the full size of the disk. Apart from the boot partition, all data is stored on the root partition. See the Disk layout section for more details.

Below, we provide examples of various ways you can customize this.

Setting up separate /var mounts

Here’s an example FCC file to set up /var on a separate partition on the same primary disk:

Adding a /var partition to the primary disk
variant: fcos
version: 1.1.0
storage:
  disks:
  - # The name of the primary block device. In virtio-based setups, this is
    # likely `/dev/vda`. Elsewhere, it's likely `/dev/sda`.
    device: /dev/vda
    # We do not want to wipe the partition table since this is the primary
    # device.
    wipe_table: false
    partitions:
    - size_mib: 0
      # Start at 5G so that we leave enough space for the root partition.
      # See the important NOTE below about this.
      start_mib: 5000
      # We assign a descriptive label to the partition. This is important
      # for referring to it in a device-agnostic way in other parts of the
      # configuration.
      label: var
  filesystems:
    - path: /var
      device: /dev/disk/by-partlabel/var
      # We can select the filesystem we'd like.
      format: ext4
      # Ask FCCT to generate a mount unit for us so that this filesystem gets
      # mounted in the real root.
      with_mount_unit: true
The start_mib field is very important. In the future, we will make more clear how much space should be reserved for the root filesystem (see https://github.com/coreos/fedora-coreos-tracker/issues/586). For now, make sure to leave at least 5G.

You can of course mount only a subset of /var into a separate partition. For example, to mount /var/lib/containers:

Adding a /var/lib/containers partition to the primary disk
variant: fcos
version: 1.1.0
storage:
  disks:
  - device: /dev/vda
    wipe_table: false
    partitions:
    - size_mib: 0
      # Start at 5G so that we leave enough space for the root partition.
      # See the important NOTE above about this.
      start_mib: 5000
      label: containers
  filesystems:
    - path: /var/lib/containers
      device: /dev/disk/by-partlabel/containers
      format: xfs
      with_mount_unit: true

Alternatively, you can also mount storage from a separate disk. For example, here we mount /var/log from a partition on /dev/vdb:

Adding /var/log from a secondary disk
variant: fcos
version: 1.1.0
storage:
  disks:
  - device: /dev/vdb
    wipe_table: false
    partitions:
    - size_mib: 0
      start_mib: 0
      label: log
  filesystems:
    - path: /var/log
      device: /dev/disk/by-partlabel/log
      format: xfs
      with_mount_unit: true

Reconfiguring the root filesystem

It is possible to reconfigure the root filesystem itself. You can use the path /dev/disk/by-label/root to refer to the original root partition. You must ensure that the new filesystem also has a label of /root.

You must have at least 4G of RAM for root reprovisioning to work.

Here’s an example of moving from xfs to ext4, but reusing the same partition on the primary disk:

Changing the root filesystem to ext4
variant: fcos
version: 1.1.0
storage:
  filesystems:
    - device: /dev/disk/by-partlabel/root
      wipe_filesystem: true
      format: ext4
      label: root

Similarly to the previous section, you can also move the root filesystem entirely. Here, we’re moving root to a RAID1 device:

Moving the root filesystem to RAID1
variant: fcos
version: 1.1.0
storage:
  raid:
    - name: myroot
      level: raid1
      devices:
        - /dev/disk/by-id/virtio-disk1
        - /dev/disk/by-id/virtio-disk2
  filesystems:
    - device: /dev/md/myroot
      format: xfs
      wipe_filesystem: true
      label: root
You don’t need the path or with_mount_unit keys; FCOS knows that the root partition is special and will figure out how to find it and mount it.

Encrypted storage (LUKS)

Here is an example to configure a LUKS device at /var/lib/data.

variant: fcos
version: 1.2.0-experimental
storage:
  luks:
    - name: data
      device: /dev/vdb
  filesystems:
    - path: /var/lib/data
      device: /dev/mapper/data
      format: xfs
      label: DATA
      with_mount_unit: true

The root filesystem can also be moved to LUKS. In the case of the root filesystem the LUKS device must be backed by clevis.

Moving the root filesystem to LUKS
variant: fcos
version: 1.2.0-experimental
storage:
  luks:
    - name: root
      device: /dev/disk/by-partlabel/root
      clevis:
        tpm2: true
      wipe_volume: true
  filesystems:
    - device: /dev/mapper/root
      format: xfs
      wipe_filesystem: true
      label: root
You don’t need the path or with_mount_unit keys; FCOS knows that the root partition is special and will figure out how to find it and mount it.

Disk Layout

All Fedora CoreOS systems start with the same disk image which varies slightly between architectures based on what is needed for bootloading. On first boot the root filesystem is expanded to fill the rest of the disk. The disk image can be customized using Fedora CoreOS configs to repartition the disk and create/reformat filesystems. Bare metal installations are not different; the installer only copies the raw image to the target disk and injects the specified config into /boot for use on first boot.

The boot partition cannot be reformatted, deleted, or moved.
See Reconfiguring the root filesystem for examples regarding the supported changes to the root partition.

Partition Tables

Using partition numbers to refer to specific partitions is discouraged and labels or UUIDs should be used instead. Fedora CoreOS reserves the boot, root, BIOS-BOOT and EFI-SYSTEM labels. Creating partitions or filesystems with those labels is not supported.

x86_64 Partition Table

The x86_64 disk image is GPT formatted with a protective MBR. It supports booting via both BIOS and UEFI (including Secure Boot).

Table 1. Partition Table for x86_64

Number

Label

Description

Partition Type

1

boot

Contains GRUB configuration, kernel/initramfs images

ext4

2

EFI-SYSTEM

Contains EFI GRUB image and Secure Boot shim

FAT32

3

BIOS-BOOT

Contains BIOS GRUB image

raw data

4

root

Contains the root filesystem

xfs

The EFI-SYSTEM partition can be deleted or reformatted when BIOS booting. Similarly, the BIOS-BOOT partition can be deleted or reformatted when EFI booting.

Mounted Filesystems

Fedora CoreOS uses OSTree, which is a system for managing multiple bootable operating system trees that share storage. This is distinct from e.g. Container Linux which used a dual partition system. In Fedora CoreOS each operating system version will be part of the / filesystem. All deployments share the same /var which can be on the same filesystem, or mounted separately.

This shows the default mountpoints for a Fedora CoreOS system installed on a /dev/vda disk:

Default mountpoints on x86_64
$ findmnt --real # Some details are elided
TARGET        SOURCE                                                   FSTYPE  OPTIONS
/             /dev/vda4[/ostree/deploy/fedora-coreos/deploy/$hash]     xfs     rw
|-/sysroot    /dev/vda4                                                xfs     ro
|-/etc        /dev/vda4[/ostree/deploy/fedora-coreos/deploy/$hash/etc] xfs     rw
|-/usr        /dev/vda4[/ostree/deploy/fedora-coreos/deploy/$hash/usr] xfs     ro
|-/var        /dev/vda4[/ostree/deploy/fedora-coreos/deploy/var]       xfs     rw
`-/boot       /dev/vda1                                                ext4    rw
  `-/boot/efi /dev/vda2                                                vfat    rw

Immutable /, read only /usr

As OSTree is used to manage all files belonging to the operating system, the / and /usr mountpoints are not writable. Any changes to the operating system should be applied via rpm-ostree.

Similarly, the /boot and /boot/efi mountpoints are managed by rpm-ostree and changes must not be directly performed by an administrator in those directories. They are not yet mounted as read only but this is expected to change in the future.

Adding top level directories (i.e. /foo) is currently unsupported and disallowed by the immutable attribute.

The real / (as in the root of the filesystem in the root partition) is mounted readonly in /sysroot and must not be accessed or modified directly.

Configuration in /etc and state in /var

The only supported writable locations are /etc and /var. /etc should contain only configuration files and is not expected to store data. All data must be kept under /var and will not be touched by system upgrades. Traditional places that might hold state (e.g. /home, or /srv) are symlinks to directories in /var (e.g. /var/home or /var/srv).

Version selection and bootup

A GRUB menu entry is created for each version of Fedora CoreOS currently available on a system. This menu entry references an ostree deployment which consist of a Linux kernel, an initramfs and a hash linking to an ostree commit (passed via the ostree= kernel argument). During bootup, ostree will read this kernel argument to determine which deployment to use as the root filesystem. Each update or change to the system (package installation, addition of kernel arguments) creates a new deployment. This enables rolling back to a previous deployment if the update causes problems.