Using Kubernetes on Fedora

Bradley G Smith, අනුවාදය F37,F38,F39,rawhide Last review: 2023-12-23

This page discusses third-party software sources not officially affiliated with or endorsed by the Fedora Project. Use them at your own discretion. Fedora recommends the use of free and open source software and avoidance of software encumbered by patents.

Overview

This how-to provides an overview of the Kubernetes (K8s) rpms in the Fedora repositories, how to use them in a few scenarios and a short cluster creation guide using kubeadm on a single Fedora machine. The guide also touches on an alternative source for Kubernetes rpms available in COPR and potential benefits.

What is Kubernetes?

Kubernetes is an "open-source system for automating deployment, scaling, and management of containerized applications" on one or more machines. Kubernetes had its genesis in the concepts and principles used at Google to run container-base workloads at scale and with resilience. Kubernetes is now at the center of a vast ecosystem of products and services (Cloud Native Computing Foundation) that help organizations create, install, run, manage and secure container-based applications and services at any possible scale.

There are numerous ways to install and configure Kubernetes depending on purpose and target environment. Is this for a home lab on a single machine, a small cluster for home or business automation, edge-based services and applications in remote offices or enterprise scale production workloads in the cloud? This guide is narrowly focused on the Kubernetes rpms available from Fedora and using dnf and the command line to install these rpms on Fedora and create a basic cluster using kubeadm.

Terminology

Kubernetes is complex and like many complex systems has its own terminology. The terminology used in this guide are defined here. The Kubernetes teams maintains a comprehensive glossary which is used in the subset below.

cluster	a set of one or more nodes managed as an entity. A cluster has at least one node and one control plane (these can be on the same or separate machines).
control plane	the node or nodes in the cluster hosting the management services for the cluster. At least one node in a cluster has a control plane. A control plane machine can also function as a worker node.
node	a worker machine (either a virtual machine or physical machine) in a Kubernetes cluster that has the services required to run pods. These services include the `kubelet` container runtime and `kube-proxy`.
pods	containerized applications are deployed and managed in Kubernetes as pods. A pod is the base object managed by Kubernetes in a cluster. A pod typically has a single primary container but may include more capabilities including multiple containers.

Kubernetes rpms in Fedora

The number, name, and organization of content in Fedora Kubernetes rpms depends on the Fedora release. Fedora 40 and newer releases (starting with Kubernetes v1.29) have one set of rpms. Fedora 39 and older releases have the legacy set of rpms.

Fedora 39 and older releases

The table below lists the available Kubernetes rpms in Fedora 39 and older releases, what the rpm contains, and notes on purpose and any restrictions or cautions.

වගුව 1. Kubernetes rpms in Fedora 39 (and older)
RPM Name	Contents	Notes
kubernetes	Empty	Also installs kubernetes-node and kubernetes-master.
kubernetes-client	kubectl	Kubernetes command line client. Recommended on any node configured as a control plane as it allows the cluster administrator control over the cluster from an ssh session on the control plane. Install on a machine that can connect to the cluster over the network.
kubernetes-kubeadm	kubeadm	Kubeadm initializes a cluster and joins new nodes to a cluster. This rpm is optional but recommended by the Kubernetes team. Install on every node if used.
kubernetes-master	kube-apiserver, kube-controller-manager, kube-proxy, kube-scheduler	Systemd services for a kubernetes node. Not needed for most installations as kubeadm will install these components as static pods. If used, then install on each node. Also installs kubernetes-client.
kubernetes-node	kubelet	Kubernetes runtime on a node. Required on each node.

Fedora 39 (and older) Installation recommendations

For most modern kubernetes clusters install kubernetes-node, kubernetes-kubeadm, and kubernetes-client on each machine in the cluster. If disk space is a constraint only install kubernetes-client on control-plane machines.

sudo dnf install kubernetes-kubeadm kubernetes-node kubernetes-client

If conducting a manual installation of Kubernetes (see Kubernetes The Hard Way) then install kubernetes-master and kubernetes-kubeadm.

sudo dnf install kubernetes-master kubernetes-kubeadm kubernetes-node kubernetes-client

Fedora 40 and newer releases

Kubernetes rpms have been reorganized starting with Kubernetes version 1.29 in Fedora 40. Rawhide for Fedora 40 initially started with Kubernetes v1.28 and the legacy package organization but these have been superseded by Kubernetes v1.29 starting in late January 2024. The table below lists the available Kubernetes rpms, what the rpm contains, and notes on purpose and any cautions or restrictions.

වගුව 2. Kubernetes rpms in Fedora 40 (and newer)
RPM Name	Contents	Notes
kubernetes	kubelet	Kubelet is the Kubernetes runtime on a node.
kubernetes-kubeadm	kubeadm	Kubeadm initializes a cluster and joins new nodes to a cluster. This rpm is optional but recommended by the Kubernetes team. Install on every node if used.
kubernetes-client	kubectl	Kubernetes command line client. Recommended on any node configured as a control plane as it allows the cluster administrator control over the cluster from an ssh session on the control plane. Install on a machine that can connect to the cluster over the network.
kubernetes-systemd	kube-apiserver, kube-controller-manager, kube-proxy, kube-scheduler	Systemd services for a kubernetes control-plane and/or node. Not needed for most installations as kubeadm will install these components as static pods. If used, then install on all nodes. Use systemctl to enable kube-proxy on all nodes. Enable kube-apiserver, kube-controller-manager, and kube-scheduler on control plane nodes.

Fedora 40 (and newer) installation recommendations

For most modern kubernetes clusters install kubernetes, kubernetes-kubeadm, and kubernetes-client on each machine in the cluster. If disk space is a constraint only install kubernetes-client on control-plane machines.

sudo dnf install kubernetes kubernetes-kubeadm kubernetes-client

If conducting a manual installation of Kubernetes (see Kubernetes The Hard Way) then install all kubernetes rpms except kubernetes-kubeadm.

sudo dnf install kubernetes kubernetes-client kubernetes-systemd

Kubernetes and Fedora version crosswalk

Each Fedora release has a corresponding version of Kubernetes available as listed below. The goal is to provide the most current Kubernetes release available when a Fedora release reaches General Availability (GA). This is not always possible resulting in skipped Kubernetes releases. Skipping a release causes problems for Kubernetes cluster administrators given the Kubernetes cluster upgrade process. Alternative ways to package Kubernetes for Fedora are being explored.

The version of the Go programming language supported for a given Fedora release can also limit the version of Kubernetes available if Kubernetes requires a newer version of Go.

වගුව 3. Kubernetes versions and the corresponding Fedora release
Kubernetes Version	Target Fedora Release	Kubernetes End-of-Life	Kubernetes Golang 'Built-With' Version
1.30	F41	TBD	1.22
1.29	F40	2025.02.28	1.21
1.28	COPR¹	2024.10.28	1.21 (was 1.20)
1.27	F39	2024.06.28	1.21 (was 1.20)
1.26	F38	2024.02.24	1.20 (was 1.19)

¹ Rawhide for Fedora 40 was initialized with Kubernetes v1.28. Kubernetes v1.29 went live while Fedora 40 was still in rawhide and superseded v1.28. Since Fedora 39 has Kubernetes v1.27 and changing to v1.28 would be problematic for existing clusters, Kubernetes v1.28 was moved to a COPR project.

Creating a Kubernetes cluster with kubeadm using Fedora rpms

Below is a guide to creating a functional Kubernetes cluster on a single Fedora machine that is suitable as a learning and exploring environment. This guide is not intended for production environments.

Each Fedora release has a corresponding Kubernetes release as documented at the Fedora Package Sources repository for Kubernetes. Fedora 39, for example, has rpms for Kubernetes 1.27. The cluster initialization is the same for all current Fedora releases. These instructions have been tested on Fedora 38 and Fedora 39 virtual machines and on Raspberry Pi 4 hardware running Fedora 38 and Fedora 39 minimal. The guide below generally follows the Creating a cluster with kubeadm guide.

Update system with DNF. Reboot if necessary, although a reboot can be deferred until after the next step.
```
sudo dnf update
```
Disable swap. Kubernetes is configured to generate an installation error if swap is detected (see this ticket for details). Modern Fedora systems use zram by default. Reboot after disabling swap.
```
sudo systemctl stop swap-create@zram0
sudo dnf remove zram-generator-defaults
sudo reboot now
```
Disable the firewall. Kubeadm will generate an installation warning if the firewall is running. Disabling the firewall removes one source of complexity in a learning environment. Modern Fedora systems use firewalld. See https://devopstales.github.io/kubernetes/k8s-security/#use-firewalld for an alternative solution that retains the firewall and opens necessary ports. The current list of ports and protocols used by a Kubernetes cluster can be found at https://kubernetes.io/docs/reference/networking/ports-and-protocols/.
```
sudo systemctl disable --now firewalld
```
Install iptables and iproute-tc.
```
sudo dnf install iptables iproute-tc
```
Configure IPv4 forwarding and bridge filters. Below copied from https://kubernetes.io/docs/setup/production-environment/container-runtimes/
```
sudo cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
```

Load the overlay and bridge filter modules.

sudo modprobe overlay
sudo modprobe br_netfilter

Add required sysctl parameters and persist.

# sysctl params required by setup, params persist across reboots
sudo cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

Apply sysctl parameters without a reboot.
```
sudo sysctl --system
```

Verify br_filter and overlay modules are loaded.

lsmod | grep br_netfilter
lsmod | grep overlay

Verify that the net.bridge.bridge-nf-call-iptables, net.bridge.bridge-nf-call-ip6tables, and net.ipv4.ip_forward system variables are set to 1 in your sysctl configuration by running the following command:
```
sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward
```
Install a container runtime. CRI-O is installed in this example. Containerd is also an option. Note: If using cri-o, verify that the major:minor version of cri-o is the same as the version of Kubernetes (installed below).
```
sudo dnf install cri-o containernetworking-plugins
```
Install Kubernetes. In this example, all three Kubernetes applications (kubectl, kubelet, and kubeadm) are installed on this single node machine. Please see the notes above on recommended packages for control plane or worker nodes if the cluster will have both types of machines.
```
# fedora 39 and earlier use:
sudo dnf install kubernetes-client kubernetes-node kubernetes-kubeadm

#fedora 40 and later use:
sudo dnf install kubernetes kubernetes-kubeadm kubernetes-client
```
Start and enable cri-o.
```
sudo systemctl enable --now crio
```
Pull needed system container images for Kubernetes.
```
sudo kubeadm config images pull
```
Start and enable kubelet. Kubelet will be in a crash loop until the cluster is initialized in the next step.
```
sudo systemctl enable --now kubelet
```

Initialize the cluster.

sudo kubeadm init --pod-network-cidr=10.244.0.0/16

kubeadm will generate output to the terminal tracking initialization steps. If successful, the output below is displayed. At this point there is a cluster running on this single machine. After kubeadm finishes you should see:

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

The steps listed above allow a non-root user to use kubectl, the Kubernetes command line tool. Run these commands now.

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Allow the control plane machine to also run pods for applications. Otherwise more than one machine is needed in the cluster.
```
kubectl taint nodes --all node-role.kubernetes.io/control-plane-
```
Install flannel into the cluster to provide cluster networking. There are many other networking solutions besides flannel. Flannel is straightforward and suitable for this guide.
```
kubectl apply -f https://github.com/coreos/flannel/raw/master/Documentation/kube-flannel.yml
```
Display list of running pods in the cluster. All pods should display a status of Running. A status of CrashLoopBackOff may show up for the coredns pod. This happens commonly when installing Kubernetes on a virtual machine and the DNS service in the cluster may not select the proper network. Use your favorite internet search engine to find possible solutions.
```
kubectl get pods --all-namespaces
```

At this point there is a single machine in the cluster running the control plane and available for work as a node.

Upgrades to Kubernetes clusters requires care and planning. See Upgrading kubeadm clusters for more information.

The DNF Versionlock plugin is useful in blocking unplanned updates to Kubernetes rpms. Occasionally, the Kubernetes version in a Fedora release reaches end-of-life and a new version of Kubernetes is added to the repositories. Or, an upgrade to Fedora on a cluster machine will also result in a different version of Kubernetes. Once DNF Versionlock is installed, the following command will hold kubernetes rpms and the cri-o rpm at the 1.28 major:minor version but still allow patch updates to occur:

sudo dnf versionlock add kubernetes*-1.28.* cri-o-1.28.*

Kubernetes projects in COPR

There are Kubernetes projects in COPR that might be useful.

Versioned Kubernetes RPMS

The Versioned Kubernetes Packages project is an experimental project exploring the use of versioned Kubernetes packages. A versioned package has a version number as part of the name such as kubernetes1.28 or kubernetes1.28-client. The goal is to have multiple versions of Kubernetes available for a given Fedora release. This uncouples Fedora versions from Kubernetes versions allowing version upgrades to either Fedora or Kubernetes. A cluster manager can update the Fedora machines while maintaining the cluster version constant. Or the cluster manager can update Kubernetes while retaining the same Fedora release.

Kubernetes 1.26 RPMS

The Kubernetes 1.26 project provides Kubernetes 1.26 rpms for all current Fedora releases that provide Go language 1.20 or newer. This includes Fedora 39 and Fedora 40 (rawhide). Kubernetes 1.26 is directly available in Fedora 38.

Kubernetes 1.27 RPMS

The Kubernetes 1.27 project provides Kubernetes 1.27 rpms for all current Fedora releases that provide Go language 1.20 or newer. This includes Fedora 38 and Fedora 40 (rawhide). Kubernetes 1.27 is directly available in Fedora 39.

Kubernetes 1.28 RPMS

The Kubernetes 1.28 project provides Kubernetes 1.28 rpms for all current Fedora releases that provide Go language 1.20 or newer. This includes Fedora 38, Fedora 39 and Fedora 40 (rawhide).i Kubernetes 1.28 is not otherwise available.

Kubernetes 1.29 RPMS

The Kubernetes 1.29 project provides Kubernetes 1.29 rpms in the new package structure. Kubernetes v1.29 requires Go language 1.21 or newer which is only available in Fedora 40. This project was initialized for early access to the new package structure and will transition over time as the source for v1.29 in subsequent Fedora releases.

References

Want to help? Learn how to contribute to Fedora Docs ›