Working with Fedora Infrastructure

This document explains how to efficiently work with the Fedora Infrastructure team. Your close attention to this document will help both you and us do the work you are asking us to do.

Our Workflow

Security related issues

Is your issue/problem related to security of an application or service we run?

Emergency/Authentication issues

Is your issue/problem urgent? (An important service is down, you need a change asap) or is your issue/problem such that you cannot file a ticket (authentication, no account, ticketing system down)

  • Login to a matrix account. join the #admin:fedoraproject.org channel and explain your issue there. The infrastructure team monitors the channel regularly, but responses may not be immediate.

  • If no one is available there:

Ticket tracking

By default, the infrastructure team tracks its work in tickets at: https://forge.fedoraproject.org/infra/tickets/issues/. If you need something from us, please open a new ticket with as much information as you think is needed to process this request.

Once created your ticket will follow the following flow:

750
Figure 2: Daily Process Ticket Flow

A few notes:

  • Make sure to note if there is a deadline or if this issue blocks you.

  • We review tickets during the two stand ups we hold Monday through Thursday (one more Europe timezone friendly and one more US timezone friendly).

  • There is no need to ping team members or notify us about the newly filed ticket.

  • Your ticket will be triaged by a team member and moved to a new state:

    • A Gain and Pain levels will be added to the ticket, these are used by the team member to prioritize their work. (You can find the definition of each level in the glossary.)

    • If it’s moved to Waiting on asignee it’s waiting for a team member to start working on it.

    • If it’s moved to Waiting on reporter it means that you need to answer questions posed in the ticket before it can be worked on.

    • If the ticket is closed with initiative, see New Initiative Workflow.

    • If the ticket is otherwise closed, it will be with a explanation from a team member.

  • If you have an update to your issue/task or want to know when it might be worked on:

    • comment in the ticket adding that information or asking for time frame.

  • When someone is available, your ticket will be assigned to someone to work on.

    • Watch for progress reports/ticket being marked done.

  • If the work is not fully completed as required, please re-open the ticket and indicate this.

    • Go back to the previous step for additional work.

Meetings of the Infrastructure Team

The Infrastructure Team holds two types of meetings. Meetings are held on Matrix, regular meetings in the #meeting:fedoraproject.org room, triage meetings in #meeting-3:fedoraproject.org.

Regular meeting

This meeting is held every Thursday at 08:00 US Pacific Time. You can use the following shell snippet to convert this to your local time:

pdx2local() {
    local PDX_TIME="${1:-now}"
    local EPOCH=$(TZ=America/Los_Angeles date -d "$PDX_TIME" +%s)
    date -d @$EPOCH +"%Y-%m-%d %H:%M:%S %Z"
}

pdx2local now
pdx2local "September 23, 2025 08:00"

Meetings will typically be an hour in length but can end earlier if discussions related to agenda items end early.

If you have an item for the agenda, please join the meeting and let us know at the start. We will call on you once we reach that point in the discussion.

Infrastructure triage meeting

This meeting is held every working day (apart from Friday). This meeting can be a maximally 30 minutes long and the infrastructure team uses it for planning upcoming work, sharing what has been done and reviewing tickets.

Meeting logs

Every meeting is logged by our friend meetbot, you can search through the logs on https://meetbot.fedoraproject.org/.

  • search for infrastructure for regular meeting logs

  • search for fedora infrastructure ops daily standup for the standup meeting logs

Decommissioned: "Oncall" Role in Our Team

The Oncall role has been decommissioned. The infrastructure team held a discussion and decided that the role doesn’t get used much at all after the move to Matrix.

This part of the documentation is being kept here for historical reasons. If you need urgent help, please look here.

Initiatives

All tasks involving new applications, major deployments, major development work or the like will be asked to follow the New Initiative Workflow. It will then be scoped and prioritized from there.

General Ticket Considerations

Please provide as much information as you can in your ticket to avoid back and forth for information. If you know your issue is going to cause a lot of discussion, start a mailing list or discussion thread for that.

Make sure your ticket:

  • Explains the problem or issue you are having, with URLs where possible to the services or applications involved.

  • Tells us how important or urgent this is to you.

  • Includes any error messages or output you see.

It is your responsibility as ticket reporter to follow your ticket, provide information that is asked for, and keep us aware of any urgency you may have. Do not simply file and forget your ticket.

Your ticket may take a while to process, depending on the current workload of the team has and how important we think it is. If your ticket is blocking you, make sure you note that in the ticket, but keep in mind that we may already be working on tickets that are blocking more people.

Every now and then, we will go through our old tickets. When this happen we may ask you to check if the issue still exists (it could be that a complimentary change fixed it, or that was just an intermittent issue or simply that it got fixed without us knowing). In those situation, we kindly ask that you reply to our question/ping within two weeks, otherwise we reserve the right to close the ticket (knowing that you can always re-open it or open a new one if the issue persists or re-appeared).

Matrix

Matrix is a great way to communicate, but please do not ping team members directly. Instead, update your ticket with any new information you have and when the team member(s) working on that issue have time/availability, they may contact you on matrix for more interactive debugging/testing.

Direct emails

E-mail is also a great communication method, but if you mail work items or information to one person directly, they cannot easily hand off the issue, you must wait for them to have time to address the issue (when others could perhaps have already solved it, etc). So, please avoid direct emails and instead update tickets with any information you want to add.

RFC 1149

Pigeons are too slow for most work items, and require facilities (e.g. dovecots) that most team members do not have. Even if the oncall member does have a free dovecot, feed, and is trained in handling carrier pigeons, sending a pigeon to a single team member has the same problems as using matrix or email for the same purpose, which means tickets are still the correct way to report problems.

In other words, please don’t send us any birds.

Want to help? Learn how to contribute to Fedora Docs ›