Architecture

Everything FDWG does starts with the Fedora message bus. Almost every action across the Fedora Project - a build, a package update, a new account - emits a message. FDWG’s job is to capture it, store it, and make it available for analysis.

Two systems do this work:

Datanommer is the archiver. It listens to the message bus, ingests every message, and stores it in a database FDWG tooling then exports that data as Apache Parquet files - a columnar format that is efficient to query and easy to move around.

Hatlas is the data platform. It hosts the exported data and provides the shared infrastructure - storage, access control, and community tooling - that lets analysts and data scientists actually work with it, whether through in-platform queries and dashboards or by pulling the raw Parquet files down to a local machine.

The result feeds decision support for CommOps through community health analytics.

At a high level, data flows in one direction - from the message bus, through Datanommer, into Hatlas, and out to the people who use it: