Configuring Storage
Fedora CoreOS ships with a simple default storage layout: the root partition is the last one and expands to take the full size of the disk. Apart from the boot partition, all data is stored on the root partition. See the Disk layout section for more details.
Below, we provide examples of various ways you can customize this.
Setting up separate /var mounts
Here’s an example FCC file to set up /var
on a separate partition on the same primary disk:
variant: fcos
version: 1.3.0
storage:
disks:
- # The name of the primary block device. In virtio-based setups, this is
# likely `/dev/vda`. Elsewhere, it's likely `/dev/sda`.
device: /dev/vda
# We do not want to wipe the partition table since this is the primary
# device.
wipe_table: false
partitions:
- size_mib: 0
# Start at 5G so that we leave enough space for the root partition.
# See the important NOTE below about this.
start_mib: 5000
# We assign a descriptive label to the partition. This is important
# for referring to it in a device-agnostic way in other parts of the
# configuration.
label: var
filesystems:
- path: /var
device: /dev/disk/by-partlabel/var
# We can select the filesystem we'd like.
format: ext4
# Ask FCCT to generate a mount unit for us so that this filesystem gets
# mounted in the real root.
with_mount_unit: true
The start_mib field is very important. In the future, we will make more clear how much space should be reserved for the root filesystem (see https://github.com/coreos/fedora-coreos-tracker/issues/586). For now, make sure to leave at least 5G.
|
You can of course mount only a subset of /var
into a separate partition. For example, to mount /var/lib/containers
:
variant: fcos
version: 1.3.0
storage:
disks:
- device: /dev/vda
wipe_table: false
partitions:
- size_mib: 0
# Start at 5G so that we leave enough space for the root partition.
# See the important NOTE above about this.
start_mib: 5000
label: containers
filesystems:
- path: /var/lib/containers
device: /dev/disk/by-partlabel/containers
format: xfs
with_mount_unit: true
Alternatively, you can also mount storage from a separate disk. For example, here we mount /var/log
from a partition on /dev/vdb
:
variant: fcos
version: 1.3.0
storage:
disks:
- device: /dev/vdb
wipe_table: false
partitions:
- size_mib: 0
start_mib: 0
label: log
filesystems:
- path: /var/log
device: /dev/disk/by-partlabel/log
format: xfs
with_mount_unit: true
Reconfiguring the root filesystem
It is possible to reconfigure the root filesystem itself. You can use the path /dev/disk/by-label/root
to refer to the original root partition. You must ensure that the new filesystem also has a label of root
.
You must have at least 4G of RAM for root reprovisioning to work. |
Here’s an example of moving from xfs to ext4, but reusing the same partition on the primary disk:
variant: fcos
version: 1.3.0
storage:
filesystems:
- device: /dev/disk/by-partlabel/root
wipe_filesystem: true
format: ext4
label: root
Similarly to the previous section, you can also move the root filesystem entirely. Here, we’re moving root to a RAID0 device:
variant: fcos
version: 1.3.0
storage:
raid:
- name: myroot
level: raid0
devices:
- /dev/disk/by-id/virtio-disk1
- /dev/disk/by-id/virtio-disk2
filesystems:
- device: /dev/md/myroot
format: xfs
wipe_filesystem: true
label: root
You don’t need the path or with_mount_unit keys; FCOS knows that the root partition is special and will figure out how to find it and mount it.
|
If you want to replicate the boot disk across multiple drives for resiliency to drive failure, you need to mirror all of the default partitions (root, boot, EFI System Partition, and bootloader code). There is special FCC syntax for this:
variant: fcos
version: 1.3.0
boot_device:
mirror:
devices:
- /dev/sda
- /dev/sdb
Encrypted storage (LUKS)
Here is an example to configure a LUKS device at /var/lib/data
.
variant: fcos
version: 1.3.0
storage:
luks:
- name: data
device: /dev/vdb
filesystems:
- path: /var/lib/data
device: /dev/mapper/data
format: xfs
label: DATA
with_mount_unit: true
The root filesystem can also be moved to LUKS. In the case of the root filesystem the LUKS device must be backed by clevis. There is simplified FCC syntax for encrypting the root filesystem; for example:
variant: fcos
version: 1.3.0
boot_device:
luks:
tpm2: true
This is equivalent to the following expanded config:
variant: fcos
version: 1.3.0
storage:
luks:
- name: root
label: luks-root
device: /dev/disk/by-partlabel/root
clevis:
tpm2: true
wipe_volume: true
filesystems:
- device: /dev/mapper/root
format: xfs
wipe_filesystem: true
label: root
The expanded config doesn’t include the path
or with_mount_unit
keys; FCOS knows that the root partition is special and will figure out how to find it and mount it.
Sizing the root partition
If you use Ignition to reconfigure or move the root partition, that partition is not automatically grown on first boot (see related discussions in this issue). In the case of moving the root partition to a new disk (or multiple disks), you should set the desired partition size using the size_mib
field. If reconfiguring the root filesystem in place, as in the LUKS example above, you can resize the existing partition using the resize
field:
variant: fcos
version: 1.3.0
storage:
disks:
- device: /dev/vda
partitions:
- label: root
number: 4
# 0 means to use all available space
size_mib: 0
resize: true
luks:
- name: root
device: /dev/disk/by-partlabel/root
clevis:
tpm2: true
wipe_volume: true
filesystems:
- device: /dev/mapper/root
format: xfs
wipe_filesystem: true
label: root
Adding swap
This example creates a swap partition spanning all of the sdb
device, creates a swap area on it, and creates a systemd swap unit so the swap area is enabled on boot.
Using with_mount_unit: true with format: swap requires FCC spec version 1.4.0-experimental . After spec 1.4.0 is stabilized, version 1.4.0-experimental will no longer be accepted by FCCT and configs that use it will need to be updated.
|
variant: fcos
version: 1.4.0-experimental
storage:
disks:
- device: /dev/sdb
wipe_table: true
partitions:
- number: 1
label: swap
filesystems:
- device: /dev/disk/by-partlabel/swap
format: swap
wipe_filesystem: true
with_mount_unit: true
Advanced examples
This example configures a mirrored boot disk with a TPM2-encrypted root filesystem, overrides the sizes of the automatically-generated root partition replicas, and adds an encrypted mirrored /var
partition which consumes the remainder of the disks.
variant: fcos
version: 1.3.0
boot_device:
luks:
tpm2: true
mirror:
devices:
- /dev/sda
- /dev/sdb
storage:
disks:
- device: /dev/sda
partitions:
# Override size of root partition on first disk, via the label
# generated for boot_device.mirror
- label: root-1
size_mib: 8192
# Add a new partition filling the remainder of the disk
- label: var-1
- device: /dev/sdb
partitions:
# Similarly for second disk
- label: root-2
size_mib: 8192
- label: var-2
raid:
- name: md-var
level: raid1
devices:
- /dev/disk/by-partlabel/var-1
- /dev/disk/by-partlabel/var-2
luks:
- name: var
device: /dev/md/md-var
# No key material is specified, so a random key will be generated
# and stored in the root filesystem
filesystems:
- device: /dev/mapper/var
path: /var
label: var
format: xfs
wipe_filesystem: true
with_mount_unit: true
Disk Layout
All Fedora CoreOS systems start with the same disk image which varies slightly between architectures based on what is needed for bootloading. On first boot the root filesystem is expanded to fill the rest of the disk. The disk image can be customized using Fedora CoreOS Configs to repartition the disk and create/reformat filesystems. Bare metal installations are not different; the installer only copies the raw image to the target disk and injects the specified config into /boot
for use on first boot.
See Reconfiguring the root filesystem for examples regarding the supported changes to the root partition. |
Partition Tables
Using partition numbers to refer to specific partitions is discouraged and labels or UUIDs should be used instead. Fedora CoreOS reserves the boot
, boot-<number>
, root
, root-<number>
, BIOS-BOOT
, bios-<number>
, EFI-SYSTEM
, and esp-<number>
labels, and the md-boot
and md-root
RAID device names. Creating partitions, filesystems, or RAID devices with those labels is not supported.
x86_64 Partition Table
The x86_64 disk image is GPT formatted with a protective MBR. It supports booting via both BIOS and UEFI (including Secure Boot).
The partition table layout has changed over time. The current layout is:
Number |
Label |
Description |
Partition Type |
1 |
BIOS-BOOT |
Contains BIOS GRUB image |
raw data |
2 |
EFI-SYSTEM |
Contains EFI GRUB image and Secure Boot shim |
FAT32 |
3 |
boot |
Contains GRUB configuration, kernel/initramfs images |
ext4 |
4 |
root |
Contains the root filesystem |
xfs |
The EFI-SYSTEM partition can be deleted or reformatted when BIOS booting. Similarly, the BIOS-BOOT partition can be deleted or reformatted when EFI booting.
Mounted Filesystems
Fedora CoreOS uses OSTree, which is a system for managing multiple bootable operating system trees that share storage. This is distinct from e.g. Container Linux which used a dual partition system. In Fedora CoreOS each operating system version is part of the /
filesystem. All deployments share the same /var
which can be on the same filesystem, or mounted separately.
This shows the default mountpoints for a Fedora CoreOS system installed on a /dev/vda
disk:
$ findmnt --real # Some details are elided
TARGET SOURCE FSTYPE OPTIONS
/ /dev/vda4[/ostree/deploy/fedora-coreos/deploy/$hash] xfs rw
|-/sysroot /dev/vda4 xfs ro
|-/etc /dev/vda4[/ostree/deploy/fedora-coreos/deploy/$hash/etc] xfs rw
|-/usr /dev/vda4[/ostree/deploy/fedora-coreos/deploy/$hash/usr] xfs ro
|-/var /dev/vda4[/ostree/deploy/fedora-coreos/deploy/var] xfs rw
`-/boot /dev/vda3 ext4 ro
The EFI System Partition was formerly mounted on /boot/efi
, but this is no longer the case. On systems configured with boot device mirroring, there are independent EFI partitions on each constituent disk.
Immutable /
, read only /usr
As OSTree is used to manage all files belonging to the operating system, the /
and /usr
mountpoints are not writable. Any changes to the operating system should be applied via rpm-ostree
.
Similarly, the /boot
mountpoint is not writable, and the EFI System Partition is not mounted by default. These filesystems are managed by rpm-ostree
and bootupd
, and must not be directly modified by an administrator.
Adding top level directories (i.e. /foo
) is currently unsupported and disallowed by the immutable attribute.
The real /
(as in the root of the filesystem in the root
partition) is mounted readonly in /sysroot
and must not be accessed or modified directly.
Configuration in /etc
and state in /var
The only supported writable locations are /etc
and /var
. /etc
should contain only configuration files and is not expected to store data. All data must be kept under /var
and will not be touched by system upgrades. Traditional places that might hold state (e.g. /home
, or /srv
) are symlinks to directories in /var
(e.g. /var/home
or /var/srv
).
Version selection and bootup
A GRUB menu entry is created for each version of Fedora CoreOS currently available on a system. This menu entry references an ostree
deployment which consist of a Linux kernel, an initramfs and a hash linking to an ostree
commit (passed via the ostree=
kernel argument). During bootup, ostree
will read this kernel argument to determine which deployment to use as the root filesystem. Each update or change to the system (package installation, addition of kernel arguments) creates a new deployment. This enables rolling back to a previous deployment if the update causes problems.