Setting up SSH access and starting containers at boot

Make sure that you have completed the steps described in the initial setup page before starting this tutorial.

In this tutorial, we will set up SSH access and start a container at boot. Fedora CoreOS is focused on running applications/services in containers thus we recommend trying to run containers and avoid modifying the host directly. Running containers and keeping a pristine host layer makes automatic updates more reliable and allows for separation of concerns with the Fedora CoreOS team responsible for the OS and end-user operators/sysadmins responsible for the applications.

As usual, we will set up console autologin, a hostname, systemd pager configuration, but we will also:

  • Add an SSH Key for the core user.

  • Add a systemd service (failure.service) that fails on boot.

  • Add a systemd service that will use a container to bring up a hosted service.

Writing the Butane config and converting to Ignition

Similarly to what we did in the second provisioning scenario, we will write the following Butane config in a file called containers.bu:

Example with automatic serial console login, SSH key, and multiple systemd units
variant: fcos
version: 1.4.0
passwd:
  users:
    - name: core
      ssh_authorized_keys:
        - ssh-rsa AAAA...
systemd:
  units:
    - name: serial-getty@ttyS0.service
      dropins:
      - name: autologin-core.conf
        contents: |
          [Service]
          # Override Execstart in main unit
          ExecStart=
          # Add new Execstart with `-` prefix to ignore failure
          ExecStart=-/usr/sbin/agetty --autologin core --noclear %I $TERM
          TTYVTDisallocate=no
    - name: failure.service
      enabled: true
      contents: |
        [Service]
        Type=oneshot
        ExecStart=/usr/bin/false
        RemainAfterExit=yes

        [Install]
        WantedBy=multi-user.target
    - name: etcd-member.service
      enabled: true
      contents: |
        [Unit]
        Description=Run a single node etcd
        After=network-online.target
        Wants=network-online.target

        [Service]
        ExecStartPre=mkdir -p /var/lib/etcd
        ExecStartPre=-/bin/podman kill etcd
        ExecStartPre=-/bin/podman rm etcd
        ExecStartPre=-/bin/podman pull quay.io/coreos/etcd
        ExecStart=/bin/podman run --name etcd --net=host \
                    --volume /var/lib/etcd:/etcd-data:z  \
                    quay.io/coreos/etcd:latest /usr/local/bin/etcd              \
                            --data-dir /etcd-data --name node1                  \
                            --initial-advertise-peer-urls http://127.0.0.1:2380 \
                            --listen-peer-urls http://127.0.0.1:2380            \
                            --advertise-client-urls http://127.0.0.1:2379       \
                            --listen-client-urls http://127.0.0.1:2379          \
                            --initial-cluster node1=http://127.0.0.1:2380
        ExecStop=/bin/podman stop etcd

        [Install]
        WantedBy=multi-user.target
storage:
  files:
    - path: /etc/hostname
      mode: 0644
      contents:
        inline: |
          tutorial
    - path: /etc/profile.d/systemd-pager.sh
      mode: 0644
      contents:
        inline: |
          # Tell systemd to not use a pager when printing information
          export SYSTEMD_PAGER=cat
Optionally you can replace the SSH pubkey in the yaml file with your own public key so you can log in to the booted instance. If you choose not to do this you’ll still be auto logged in to the serial console.

Run Butane to convert that to an Ignition config:

butane --pretty --strict containers.bu --output containers.ign

Now let’s provision it:

# Setup the correct SELinux label to allow access to the config
chcon --verbose --type svirt_home_t containers.ign

# Start a Fedora CoreOS virtual machine
virt-install --name=fcos --vcpus=2 --ram=2048 --os-variant=fedora-coreos-stable \
    --import --network=bridge=virbr0 --graphics=none \
    --qemu-commandline="-fw_cfg name=opt/com.coreos/config,file=${PWD}/containers.ign" \
    --disk=size=20,backing_store=${PWD}/fedora-coreos.qcow2

On the console you will see:

Fedora CoreOS 36.20220723.3.1
Kernel 5.18.13-200.fc36.x86_64 on an x86_64 (ttyS0)

SSH host key: SHA256:X38nTiEGsp/G+tFz6ojBaeGDoI9a9S350xN8HSNa1oc (ECDSA)
SSH host key: SHA256:gDZoJpgOpLJSCPaLn8OdA1hZQxytI+rdt2XOnLlfPHc (ED25519)
SSH host key: SHA256:H73stdlwb9eJspcVb69wpEOnBEXoF2iBfGnS6cbtBNE (RSA)
enp1s0: 192.168.122.171 fe80::d100:b088:5914:115e
Ignition: ran on 2022/08/21 01:31:56 UTC (this boot)
Ignition: user-provided config was applied
Ignition: wrote ssh authorized keys file for user: core
tutorial login: core (automatic login)

Fedora CoreOS 36.20220723.3.1
[systemd]
Failed Units: 1
  failure.service
[core@tutorial ~]$

If you would like to connect via SSH, disconnect from the serial console by pressing CTRL + ] and then use the reported IP address for the NIC from the serial console to log in using the core user via SSH:

$ ssh core@192.168.122.171
The authenticity of host '192.168.122.171 (192.168.122.171)' can't be established.
ED25519 key fingerprint is SHA256:gDZoJpgOpLJSCPaLn8OdA1hZQxytI+rdt2XOnLlfPHc.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.122.171' (ED25519) to the list of known hosts.
Fedora CoreOS 36.20220723.3.1
Tracker: https://github.com/coreos/fedora-coreos-tracker
Discuss: https://discussion.fedoraproject.org/tag/coreos

Last login: Sun Aug 21 01:32:09 2022
[systemd]
Failed Units: 1
  failure.service

The Failed Units message is coming from the console login helper messages helpers. This particular helper shows us when systemd has services that are in a failed state. In this case we made failure.service with ExecStart=/usr/bin/false, so we intentionally created a service that will always fail in order to illustrate the helper messages.

Now that we’re up and don’t have any real failures we can check out the service that we care about (etcd-member.service):

[core@tutorial ~]$ systemctl status --full etcd-member.service
● etcd-member.service - Run a single node etcd
     Loaded: loaded (/etc/systemd/system/etcd-member.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2022-08-21 01:32:09 UTC; 2min 18s ago
    Process: 1608 ExecStartPre=mkdir -p /var/lib/etcd (code=exited, status=0/SUCCESS)
    Process: 1610 ExecStartPre=/bin/podman kill etcd (code=exited, status=125)
    Process: 1649 ExecStartPre=/bin/podman rm etcd (code=exited, status=1/FAILURE)
    Process: 1657 ExecStartPre=/bin/podman pull quay.io/coreos/etcd (code=exited, status=0/SUCCESS)
   Main PID: 1706 (podman)
      Tasks: 10 (limit: 2254)
     Memory: 91.5M
        CPU: 4.978s
     CGroup: /system.slice/etcd-member.service
             ├─ 1706 /bin/podman run ...
             └─ 1724 /usr/bin/conmon ...

Aug 21 01:32:10 tutorial etcd[1724]: 2022-08-21 01:32:10.719193 N | etcdserver/membership: set the initial cluster version to 3.3
Aug 21 01:32:10 tutorial etcd[1724]: 2022-08-21 01:32:10.719548 I | etcdserver/api: enabled capabilities for version 3.3
Aug 21 01:32:10 tutorial podman[1706]: 2022-08-21 01:32:10.719193 N | etcdserver/membership: set the initial cluster version to 3.3
Aug 21 01:32:10 tutorial podman[1706]: 2022-08-21 01:32:10.719548 I | etcdserver/api: enabled capabilities for version 3.3
Aug 21 01:32:10 tutorial podman[1706]: 2022-08-21 01:32:10.719595 I | etcdserver: published {Name:node1 ClientURLs:[http://127.0.0.1:2379]} to cluster 1c45a069f3a1d796
Aug 21 01:32:10 tutorial podman[1706]: 2022-08-21 01:32:10.719968 I | embed: ready to serve client requests
Aug 21 01:32:10 tutorial etcd[1724]: 2022-08-21 01:32:10.719595 I | etcdserver: published {Name:node1 ClientURLs:[http://127.0.0.1:2379]} to cluster 1c45a069f3a1d796
Aug 21 01:32:10 tutorial etcd[1724]: 2022-08-21 01:32:10.719968 I | embed: ready to serve client requests
Aug 21 01:32:10 tutorial etcd[1724]: 2022-08-21 01:32:10.722332 N | embed: serving insecure client requests on 127.0.0.1:2379, this is strongly discouraged!
Aug 21 01:32:10 tutorial podman[1706]: 2022-08-21 01:32:10.722332 N | embed: serving insecure client requests on 127.0.0.1:2379, this is strongly discouraged!

We can also inspect the state of the container that was run by the systemd service:

[core@tutorial ~]$ sudo podman ps -a
CONTAINER ID  IMAGE                       COMMAND               CREATED         STATUS             PORTS       NAMES
9d854474bba7  quay.io/coreos/etcd:latest  /usr/local/bin/et...  11 minutes ago  Up 11 minutes ago              etcd

And we can set a key/value pair in etcd. For now let’s set the key fedora to the value fun:

[core@tutorial ~]$ curl -L -X PUT http://127.0.0.1:2379/v2/keys/fedora -d value="fun"
{"action":"set","node":{"key":"/fedora","value":"fun","modifiedIndex":4,"createdIndex":4}}
[core@tutorial ~]$ curl -L http://127.0.0.1:2379/v2/keys/ 2>/dev/null | jq .
{
  "action": "get",
  "node": {
    "dir": true,
    "nodes": [
      {
        "key": "/fedora",
        "value": "fun",
        "modifiedIndex": 4,
        "createdIndex": 4
      }
    ]
  }
}

Looks like everything is working!

Cleanup

Now let’s take down the instance for the next test. Disconnect from the serial console by pressing CTRL + ] or from SSH and then destroy the machine:

virsh destroy fcos
virsh undefine --remove-all-storage fcos

You may now proceed with the next tutorial.