Processing Livestreams: A Standardized Workflow

Adrian Edwards, Justin W. Wheeler, Fedora community contributors

This guide outlines a streamlined procedure for processing Fedora livestreams (e.g., Flock, release parties, Fedora Week of Diversity), making it easier to edit and share recordings of these events. This standardized workflow utilizes tools and automation to minimize manual effort and ensure consistency.

Tools You’ll Need

This workflow assumes you have basic familiarity with command-line tools and video editing concepts.

Step 1: Prepare Talk Metadata

This process relies on a CSV file (comma-separated values) containing information about each talk (title, speakers, identifier). This data is used for automation and consistency.

1.1 Automatic Metadata from Pretalx

If your event uses Pretalx, you can automatically generate the CSV file using a Python script:

  1. Clone the repository: git clone https://gitlab.com/fedora/commops/home.git

  2. Navigate to the script: cd commops/scripts/conference

  3. Run the script: python main.py <Pretalx Schedule URL>

Ensure the Pretalx schedule accurately reflects the livestream order.

The script can also generate YouTube descriptions based on a template file (description.template.txt).

1.2 Manual Metadata Preparation

If you’re not using Pretalx, create a CSV file with these columns:

  • Title: The title of the talk.

  • Speakers: The speaker(s) of the talk.

  • Identifier: A unique identifier for the talk (lowercase, no spaces or special characters).

Populate the CSV with information for each talk.

Step 2: Tag Chapters and Breaks

Use Sponsorblock to tag intermissions and chapters in the YouTube video. This helps automate splitting the video later.

  1. Install the Sponsorblock extension for your browser.

  2. Open the YouTube video.

  3. Use the Sponsorblock controls to tag intermissions (usually "Intermission/Intro Animation") and optionally tag individual talks as "Chapter."

  4. Submit your tags to the Sponsorblock database.

Step 3: Generate Chapter Timestamps

YouTube uses a specific format for chapters. You can automate this formatting using a Python script:

  1. Navigate to the script: cd commops/scripts/timestamps

  2. Run the script: python main.py <YouTube video URL> [<stream CSV file>]

The script will generate the timestamps. If you provide the stream CSV, it will also fill in the talk titles.

Alternatively, you can manually edit the timestamps from Sponsorblock to match the YouTube format.

Step 4: Download and Split the Video

Use yt-dlp to download the video:

yt-dlp <YouTube video URL>

Then, use LosslessCut to split the video based on the Sponsorblock tags:

  1. Open LosslessCut and drag the downloaded video into it.

  2. Import the losslesscut.csv file (if generated) to load the segments.

  3. Verify the segments align with the talks.

  4. Click the export button and configure the settings:

    • Output folder

    • Output file names: ${SEG_LABEL}${EXT}

    • Enable "Smart cut (experimental)"

  5. Click Export to split the video.

Step 5: Create Thumbnails

Use Inkscape with the Next Generator extension to create thumbnails for each talk:

  1. Design your thumbnail template in Inkscape, using variables like %VAR_TalkName% and %VAR_PresenterNames% to represent data from your CSV.

  2. Go to Extensions > Export > NextGenerator.

  3. Configure the export settings:

    • CSV file path

    • File format

    • File name pattern (e.g., %VAR_Identifier%)

  4. Click Export to generate the thumbnails.

Step 6: (Optional) Add Thumbnail Intros

You can add a short intro with the thumbnail at the beginning of each talk using a Python script:

  1. Navigate to the script: cd commops/scripts/transcode

  2. Prepare a CSV file with talk data (if not already generated).

  3. Configure the config.ini file with settings for intro duration, etc.

  4. Run the script according to the instructions in the README.

Areas for Further Automation

While this workflow incorporates significant automation, some areas can be further improved:

  • Chapter Creation: Streamline the process of creating YouTube chapters.

  • Recombining Clips: Automate the handling of interruptions and inconsistencies in the livestream.

  • Video Reformatting: Automatically convert videos to optimal formats for processing.

  • Captions: Explore automated captioning and translation solutions.

  • Audio Processing: Implement loudness normalization and dynamic range compression.

  • End Cards: Automate the addition of end cards to videos.

These improvements would further enhance the efficiency and robustness of the livestream processing pipeline.

Want to help? Learn how to contribute to Fedora Docs ›