Generating OpenH264 RPMs and sending them to Cisco

Description

OpenH264 RPMs are a special case and we need to handle it with caution.

The process is as follows:

  1. We generate and sign the RPMs using koji dist-repo.

  2. We send the RPMs to Cisco for hosting.

  3. Once we get confirmation from Cisco that they published the RPMs on their CDN we publish the repodata on our side.

Legal Background

OpenH264 is a free software library for real-time encoding and decoding video streams in the H.264/MPEG-4 AVC format.

In 2013, Cisco released both binaries and source code, and pays all royalties for its use to MPEG LA themselves for any software projects that use Cisco’s precompiled binaries (thus making Cisco’s OpenH264 binaries free to use); any software projects that use Cisco’s source code instead of its binaries would be legally responsible for paying all royalties to MPEG LA themselves, however.

The workaround is that the Cisco binaries are built in Fedora’s infrastructure but distributed by Cisco.

The RPMs must never be distributed via our build system or websites.

Prerequisites

  1. Valid Kerberos token (kinit).

  2. Koji admin permission (set within Koji itself).

  3. sysadmin-releng FAS group membership.

Actions

Generating the RPMs

Once a request comes in through the Release Engineering issue tracker to update OpenH264 to a new version (or to generate new RPMs for the new Rawhide after a mass branching), we will first need to generate the RPMs and repodata. We do this using the koji dist-repo utility.

The command below is merely an example and some arguments passed to it (tag, signing key) will have to be changed depending on the situation.

Run a command such as this:

$ koji dist-repo f46-openh264 91211fce --noinherit --split-debuginfo --with-src -a aarch64 -a ppc64le -a s390x -a x86_64

In the case of the above command:

  1. f46-openh264 is the tag the builds are tagged into.

  2. 91211fce is the signing key for the RPMs. The keys for various versions can be found here.

  3. --noinherit causes only packages tagged into f46-openh264 to be included, without inheritance from the parent tag (e.g. f46).

  4. --split-debuginfo causes the debug packages to be placed in a separate debug repo.

  5. --with-src causes the source RPM to be included.

  6. -a <ARG> options are the respective architectures.

After the command finishes the directory should appear here.

The RPMs are visible through the web frontend, but we have a rewrite rule that redirects any attempt to download them to a wiki page saying why that is not possible.

In order to get to the RPMs themselves we must do it from a host on which Koji is mounted where they will be in the /mnt/koji/repos-dist/f46-openh264 directory.

For example:

$ ssh username@compose-branched01.rdu3.fedoraproject.org

After you verify that everything needed is there you may clone the directory to your local machine:

$ rsync -avhHP username@compose-branched01.rdu3.fedoraproject.org:/mnt/koji/repos-dist/f46-openh264/6556232 /home/username/openh264/2.6.0/f46

It may be a good idea to copy the directory to your home directory on batcave01.rdu3.fedoraproject.org as well.

Sending the RPMs to Cisco

Preparing the Tarball

We now have the necessary files on our local machine. Now we need to package the files into a tarball and deliver it to Cisco.

  1. Create a new directory into which we can copy the RPMs and out of which we will create the tarball.

    $ mkdir /home/username/openh264/2.6.0/f46/f46_2.6.0_rpms

  2. Copy the RPMs (including the source RPM) from the original directory to the new directory.

    $ cd /home/username/openh264/2.6.0/f46/
$ cp -rv 6556232/*/*/*/*rpm f46_2.6.0_rpms

  3. Create a list of the RPMs that will be sent to Cisco along with the tarball.

    $ ls f46_2.6.0_rpms > f46_2.6.0_rpms_list.txt

  4. Create the tarball.

    $ tar -cJvf f46_2.6.0_rpms.tar.xz f46_2.6.0_rpms

Uploading the Tarball

We now have the tarball and need to get it to Cisco.

  1. Upload the tarball through Cisco’s upload link. The link can be found in Bitwarden.

  2. Send an email to openh264-publishing@cisco.com to let Cisco know the tarball has been uploaded and the RPMs are ready. Ask them to publish the RPMs and notify you once done.

  3. Wait for a reply from Cisco.

Publishing the Repodata

Cisco confirmed the RPMs are published. Now it is time to publish the repodata on Fedora’s infrastructure.

  1. Verify the RPMs are available using curl.

    $ curl -I http://ciscobinary.openh264.org/openh264-2.6.0-4.fc46.x86_64.rpm

  2. Navigate to the relevant directory on your local machine and sync the dist-repo directory to your home directory on sundries01.

    $ rsync -avhHP ./6556232 username@sundries01.rdu3.fedoraproject.org:/home/fedora/username/openh264/2.6.0/46

  3. Connect to sundries01.

    $ ssh username@sundries01.rdu3.fedoraproject.org

  4. Navigate to the relevant directory.

    $ pwd
/home/fedora/username/openh264/2.6.0/46

  5. Sync the contents of the directory to /srv/web/codecs.fedoraproject.org.

    $ sudo rsync -avhHP 6556232/. /srv/web/codecs.fedoraproject.org/openh264/46

  6. Make sure the directory is owned by root.

    $ sudo chown -R root:root /srv/web/codecs.fedoraproject.org/openh264/46

(Optional) Updating MirrorManager

At this point everything is done and MirrorManager will update automatically within a day. If you need it to update sooner, you can force a new cronjob run in OpenShift:

$ oc create job --from=cronjob/primary-mirror-codecs primary-mirror-codecs-username-YYYYMMDD.1 -n mirrormanager

The job name must be unique. Use your username and today’s date as shown above.

Resources

  1. OpenH264 dist-git repository

Want to help? Learn how to contribute to Fedora Docs ›