Compose generation

A Fedora compose is a single run of Pungi that produces a complete set of repositories, installation trees, and images for a given point in the distribution’s development. This page covers how composes are generated for both Rawhide and branched releases.

For where composes fit in the broader release lifecycle, see Fedora release process.

Overview

Pungi is the compose tool Fedora uses to assemble distribution trees. Fedora-specific configuration lives in pungi-fedora; upstream Pungi development is on Pagure.

A typical compose run proceeds through several phases:

  1. Init : load variant definitions, fetch comps files, prepare work directories

  2. Pkgset : query Koji for the latest builds in the configured tags, resolve dependencies, and determine the package set for each arch and variant

  3. Gather : assemble package repositories (createrepo) for each variant

  4. Buildinstall / image phases : submit Koji jobs for installer trees (Lorax), live/cloud images (Kiwi), OSTree commits, and container images

  5. Createiso : produce ISO and disk images from completed Koji tasks

  6. Extra files and checksums : generate metadata, CHECKSUM files, and compose reports

Most image build work happens in Koji, not on the compose host directly. Pungi submits tasks to Koji builders and waits for them to complete. Koji job templates for Kiwi images are defined in spin-kickstarts and fedora-kiwi-descriptions. Lorax handles network-install and some legacy installer media. Newer atomic and container-native images may use bootc or image-mode tooling alongside the traditional Kiwi and OSTree paths.

The compose driver on Fedora systems is pungi-koji, which integrates Pungi with Koji task submission and fedora-messaging notifications.

If you want to understand what Pungi is doing at any moment, start with logs/global/pungi.global.log in the compose directory. It records every phase transition and most Koji task submissions.

Rawhide composes

Trigger and schedule

Rawhide composes run automatically every day at 05:15 UTC on compose-rawhide01.rdu3.fedoraproject.org. The cron entry is defined in roles/releng/templates/rawhide.j2 in the Infra Ansible repo:

15 5 * * * root touch /tmp/fedora-compose-rawhide && TMPDIR=`mktemp -d /tmp/rawhide.XXXXXX` && \
  chmod 755 $TMPDIR && cd $TMPDIR && \
  git clone https://forge.fedoraproject.org/releng/pungi-fedora.git && \
  cd pungi-fedora && \
  /usr/local/bin/lock-wrapper rawhide-compose "LANG=en_US.UTF-8 ./nightly.sh" && \
  sudo -u ftpsync /usr/local/bin/update-fullfiletimelist ...

nightly.sh clones releng tooling, invokes pungi-koji with fedora.conf and the --nightly flag, then rsyncs the result to /pub/fedora/linux/development/rawhide, sends fedora-messaging events, and emails a compose report.

Output location

Composes are stored on koji storage and published at:

Each compose is a directory named Fedora-Rawhide-YYYYMMDD.n.R where n means nightly and R is the respin number (.0 for the first run that day, .1 for a same-day respin).

The latest-Fedora-Rawhide symlink updates when a new compose starts, so it always points at the current or most recent run.

Reading the global log

The file pungi.global.log is the primary diagnostic log. A successful run shows a sequence of [BEGIN] and [DONE] markers for each phase:

2026-06-08 05:15:03 [INFO    ] [BEGIN] ---------- PHASE: INIT ----------
2026-06-08 05:15:18 [INFO    ] [DONE ] ---------- PHASE: INIT ----------
2026-06-08 05:15:18 [INFO    ] [BEGIN] ---------- PHASE: PKGSET ----------
...

To trace Koji tasks, search the global log (and per-variant logs under logs/) for lines containing task IDs or Koji URLs. Pungi logs task submission during buildinstall, createiso, live_media, ostree, and related phases. Open the task in the Koji web UI at https://koji.fedoraproject.org/koji/taskinfo?taskID=<ID>; to inspect build logs and failure reasons.

Per-arch and per-variant logs live under logs/<arch>/ and logs/<variant>/. The logs/depcheck output lists broken dependencies found during the compose. logs/*verbose files are included in the nightly email report.

Compose STATUS values

Every compose directory contains a STATUS file written by Pungi when the run completes. The possible values are:

STATUS Meaning

STARTED

Compose is in progress; STATUS is written at kickoff

FINISHED

All phases completed successfully

FINISHED_INCOMPLETE

Compose finished but one or more optional variants or arches failed; inspect logs for the failing phase

FAILED

A required phase failed; the compose did not produce a complete tree

Check STATUS before assuming a compose is usable for testing. A FINISHED_INCOMPLETE Rawhide compose may still be rsynced to development mirrors if the primary arches succeeded, but the email report and fedora-messaging events will reflect the partial failure.

Common failure scenarios

Broken dependencies in pkgset

The pkgset phase cannot resolve all required packages. Check logs/depcheck and the global log for the specific missing or conflicting RPMs. Fix the offending Koji builds or comps entries, then wait for the next nightly or trigger a respin.

Koji image build failure

A Kiwi or Lorax task failed on a Koji builder. Find the task ID in pungi.global.log or the variant-specific log, then open the task in Koji for the builder log. Common causes include kickstart errors, missing packages in the compose tree, or builder resource limits.

Unsigned builds

Pungi may skip or fail on packages that have not been signed yet. Check whether robosignatory is running and whether the build is stuck in a -signing-pending tag. See Sign the packages.

OSTree signing wait

Rawhide composes use the pungi-wait-for-signed-ostree-handler notification script. Atomic variants (Silverblue, Kinoite, and related) may block until OSTree commits are signed. Check logs/ for ostree phase output and Sigul/robosignatory status.

Compose host issues

Disk space, git clone failures, or lock contention (lock-wrapper rawhide-compose) can prevent a compose from starting. Check /tmp/fedora-compose-rawhide on the compose host and releng cron mail (releng-cron@lists.fedoraproject.org).

To resume a failed compose in debug mode, set COMPOSE_ID in nightly.sh to the existing compose directory and re-run with --debug-mode --compose-dir=…​. This is commented in the script for operator use.

Release composes

How release composes differ from Rawhide

Branched and milestone composes use the same Pungi machinery but differ in several ways:

  • Freeze enforcement : compose tags draw from locked Koji tags; only approved builds enter

  • Milestone labels : Beta and Final RC composes carry labels like Beta-1.5 or RC-1.3

  • Manual trigger : RC composes are requested via RelEng tickets, not cron

  • Compose tag snapshot : specific builds are tagged into f*-compose before the run

  • Configuration : branch-specific configs (fedora-branched.conf, fedora-beta.conf, fedora-final.conf)

Branched development composes follow a schedule defined in Ansible cron templates for the release branch, similar to Rawhide. Milestone RC composes are one-off runs coordinated with QE.

Running Pungi for a branched release

For day-to-day branched composes, the branch-specific script and config in pungi-fedora replace nightly.sh / fedora.conf. The general pungi-koji invocation pattern is:

cd pungi-fedora/
pungi-koji \
  --notification-script=/usr/bin/pungi-fedmsg-notification \
  --notification-script=pungi-wait-for-signed-ostree-handler \
  --config=fedora-branched.conf \
  --old-composes=/mnt/koji/compose/{branched} \
  --target-dir=/mnt/koji/compose/{branched}

For Beta or Final Release Candidates, RelEng first generates a candidate config:

./create-candidate-configs.py --minor=5 beta

Then reviews the f{branched}-compose tag, tags requested builds, and runs:

./release-candidate.sh

See Beta RC Compose SOP and Final RC Compose SOP for the full tagging and review steps.

Output location

Release composes are published under:

https://kojipkgs.fedoraproject.org/compose/<BRANCH>/

where <BRANCH> is the release number (for example 44 for Fedora 44). Within that directory, compose IDs follow the pattern Fedora-<BRANCH>-YYYYMMDD.n.R for development composes or carry milestone labels for RC composes.

As with Rawhide, a latest-Fedora-<name> symlink points at the current compose, and each directory contains COMPOSE_ID, STATUS, compose/, logs/, and work/ subdirectories.

Composes and the Go/No-Go process

QE validates a specific RC compose against the release validation test plan. The nominated compose ID is recorded in the RelEng ticket and the Go/No-Go meeting agenda.

If the meeting votes Go, RelEng stages that exact compose for mirror publication. If the vote is No-Go, builds are fixed, a new RC compose is produced with an incremented global_release minor version, and validation runs again.

See Go/No-Go process for the full lifecycle from compose to mirror publication.

Do not stage a compose for public release until Go/No-Go has approved the specific compose ID. Staging the wrong compose means re-running signing and mirror steps.

Signing

All official Fedora compose artifacts are GPG-signed before they reach public mirrors.

RPM signing

RPMs are signed automatically as they pass through Koji tags via robosignatory. Pungi expects signed builds in the compose tags; unsigned builds can cause compose failures or delays in the OSTree signing handler.

Image and CHECKSUM signing

After a compose is approved for release, RelEng runs staging scripts from releng/tooling:

release-process/staging/stage_release_checksum_sign.sh {branched}_Beta Fedora-{branched}-YYYYMMDD.0 fedora-{branched} 1
release-process/staging/stage_release.sh {branched}_Beta Fedora-{branched}-YYYYMMDD.0 fedora-{branched} 1

stage_release_checksum_sign.sh signs all *CHECKSUM files in the compose tree. stage_release.sh copies the signed tree to the staging area on mirror master.

If signing fails, do not push the compose to mirrors. Fix the signing issue, re-run the staging scripts, and verify signatures before the mirror bit-flip. Unsigned CHECKSUM files or images will break mirror consistency and user verification.

Verification

Confirm signatures on CHECKSUM files before notifying mirror admins. After the bit-flip, verify that dl.fedoraproject.org serves the signed tree and that MirrorManager reports the new release correctly.

For day-to-day signing troubleshooting, see Sign the packages.

Want to help? Learn how to contribute to Fedora Docs ›