Onboarding of a CI System

Testing in Fedora

It’s relatively easy to start testing Fedora artifacts (Koji builds, Bodhi updates, etc.) and to contribute test results back so that they can be later used for gating — i.e. deciding whether the tested artifact should be promoted or not.

Here we describe necessary steps to add a new CI system.

What is a (suitable) CI System?

In order to provide a good workflow and user experience, here are some aspects of CI systems that have proven to be successful:

  • Tests are reliable (low false negative and false positive rates) and cover important stories

  • Tests can be contributed to (open source model) and are ideally similar to other tests, e.g. by using established frameworks / languages

  • Results / notifications are easy to understand and help identify errors quickly

  • Tests are reproducible, if necessary artifacts from test runs are stored for users to consume, such as virtual machine images

In short, test results should be directly actionable! As a developer, I need to quickly decide whether the test is broken or the code, and then fix the issue.

Testing and Gating Builds

Gating Workflow

On the highest level, the gating workflow consists of the following steps:

  • Submit build(s) of one or more package(s) (Koji), and an update containing those builds (Bodhi)

  • Trigger CI systems to run tests (Fedora CI, Your CI, etc.)

  • Collect results from the CI systems (ResultsDB)

  • Make a decision (Greenwave)

  • If the decision is "pass", then let the build pass the gate to the main repository (Bodhi)

Gating Messages

Gating process is implemented as a set of services which interact with each other via message bus (Fedora Messaging).

Therefore to add a piece (for example a CI system) to the process, you essentially need to start receiving and sending messages via FedoraMessaging.

How to add a CI System

CI systems in Fedora are autonomous entities that typically need to handle following things:

  • triggering when certain events occur

  • actual testing

  • publishing test results

Triggering and Testing

Services in Fedora publish messages when various events occur and thus CI systems can trigger testing when for example a Bodhi update is created, or the builds in it change.

When either of those things happen, a new "koji-build-group.build.complete" message is published on the org.fedoraproject.prod.bodhi.update.status.testing.koji-build-group.build.complete topic.

The schema of these "koji-build-group.build.complete" messages is defined in the CI Messages specification.

You can design your CI system to trigger your tests in response to these messages. Some systems may have RabbitMQ listener code available for you to use (the current Fedora Messaging system is based on RabbitMQ). If you are using Jenkins you can try to follow the way the Fedora CI triggers work, e.g. the rpmdeplint trigger. If you can write the trigger code in Python, you can follow the fedora-messaging consumer documentation.

It’s of course possible to trigger testing on other types of events, not just Bodhi updates. You can find more Fedora message topics in the fedora-messaging documentation. Beware though, the list is incomplete.

Sharing Test Results

CI systems should publish results to ResultsDB. This is required if you want the results to appear in the Bodhi web UI and for it to be possible to gate Bodhi updates on the results. Results should usually follow the format used by established systems like Fedora CI and openQA. In particular, for Bodhi gating to work, the 'item' for your result must be either a package NVR (in which case its 'type' must be 'koji_build') or an update ID like 'FEDORA-2026-be33882b5c' (in which case its type must be 'bodhi_update').

If your reporting code is in Python, you may find the resultsdb_conventions library a convenient helper for generating results in the expected formats for Fedora package, update or compose tests.

As well as final "results", CI systems can and likely should publish "results" that indicate test execution progress. The special 'outcomes' QUEUED and RUNNING exist for this purpose. Bodhi understands these results and displays them appropriately so people can see the current status of queued and running tests.

Publishing results to the production ResultsDB instance requires authentication. You will need to ask the Infrastructure team for credentials for this. It is a good idea to test your reporting code against a local ResultsDB instance first.

CI systems may also publish standardized CI messages so the progress of the testing can be observed and the results can be acted upon by other services in the Fedora infrastructure. There is a system that automatically forwards CI messages in certain formats to ResultsDB, so you may be able to avoid having explicit result reporting code. Note there is currently (as of 2026-07) a plan to decommission that system and stop publishing CI messages as a matter of course.

There are four types of messages that CI systems should be sending:

  • test.queued - when there is an artifact (a Bodhi update for example) in a queue waiting to be tested

  • test.running - when testing is in progress

  • test.complete - when testing is finished

  • test.error - when testing couldn’t start or couldn’t finish due to an outside circumstances (typically an infrastructure error)

These messages have well-defined schemas. The schemas are part of the CI Messages specification.

For convenience, here are links to schemas for simple koji-build artifacts and fedora-update artifacts:

Test Identifiers

When you send a "koji-build.test." or "fedora-update.test." message to the message bus, a result should be forwarded to ResultsDB by ci-resultsdb-listener, if the message contained the fields it expects (and the system is not explicitly excluded, as e.g. openQA is, because it does its own reporting). Later on you can refer to the stored test result in a Greenwave policy: "if test XYZ passed, let the build through the gate". Again, note this system may be dropped in future in favor of expecting CI systems to report directly to ResultsDB.

Therefore you need unique identifiers for test results.

Test identifiers are built from three parts: test namespace, test category and test type.

In the koji-build and fedora-update message schemas, these variables are represented by namespace, category and type fields respectively.

Namespace is always an ID of your CI system with the name of the artifact type, so for example: fedora-ci.koji-build, or osci.pull-request.

Category you can choose from a predefined list:

  • static-analysis

  • functional

  • integration

  • validation

Type is an arbitrary string, which you can define based on the specifics of your CI system.

It is your responsibility as an owner of the CI system to maintain consistent naming of tests under your namespace.

The final test identifier may look like this: fedora-ci.koji-build.tier0.functional

Useful Links

Want to help? Learn how to contribute to Fedora Docs ›