Golang Packaging Guidelines
This document details best practices for packaging Golang packages. Most of it is automated by an extensive use of macros.
go2rpm
go2rpm is tool that automates many of these steps. It is advisable to try go2rpm import_path
first before attempting to write a SPEC by hand.
Import Path
In Golang, packages are referenced by full URLs. Since this URL is referenced in several places throughout the rpmspec, set the base import path as a global define at the top of the spec file
%global goipath github.com/kr/pretty
All macros, including package name, source URL, will be computed from this value.
Take the time to identify it accurately. Changing it later will be inconvenient.
If upstream confused itself after multiple forks and renamings, you will need to fix references to past names in the Go source files, unit tests included. Perform this fixing in |
Nomenclatura
Source packages (src.rpm)
-
Golang source packages dedicated to providing code MUST be named after their main import path. This process is automated by the
%{goname}
macro. This macro will remove any capitalization, "go" keywords, and any duplication in the import path.For example:
-
the import path
github.com/kr/pretty
will becomegolang-github-kr-pretty
-
the import path
github.com/DATA-DOG/go-txdb
will becomegolang-github-data-dog-txdb
-
the import path
github.com/gopherjs/gopherjs
will becomegolang-github-gopherjs
The filename of spec MUST match the name of the package.
If you’re not sure what will the name processed by the
%{goname}
macro be, simply build the SRPM with:fedpkg --release f31 srpm
The SRPM filename will be the one to use in your rpmspec and spec filename.
-
-
Source packages that provide a well-known application such as
etcd
MUST be named after the application. End users do not care about the language their applications are written in. But do not name packages after an obscure utility binary that happens to be built by the package.
Implementation: %{gorpmname}
%gometa
uses the %{gorpmname}
macro to compute the main %{goname}
from %{goipath}
.
|
Go code packages: %{goname}-devel
In a source package dedicated to providing Go code
Packages that ship Go code in %{goipath}
should be named %{goname}-devel
. If your source package is already named %{goname}
then:
%package devel
[…]
%description devel
[…]
%files devel -f devel.file-list
This has been automated by the %{gopkg}
and %{gopdevelkg}
macros described in the Package metadata section below.
In a another kind of source package
If your source package is named something other than %{goname}
, you SHOULD use:
%package -n %{goname}-devel
[…]
%description -n %{goname}-devel
[…]
%files -n %{goname}-devel -f devel.file-list
Separate code packages
And, finally, if you wish to split the project Go code in multiple packages, you can compute the corresponding names with:
%global goname1 %gorpmname importpath1
[…]
%package -n %{goname1}-devel
[…]
%description -n %{goname1}-devel
[…]
%files -n %{goname1}-devel -f %{goname1}.file-list
See also the Dealing with cyclic dependencies chapter.
Do remember that for Go, each directory is a package. Never separate the .go files contained in a single directory in different packages.
Import path compatibility packages: %{compat-%{oldgoname}-devel}
When a project can be referenced under multiple import paths, due to forks, renamings, rehostings, organizational changes, or confusing project documentation, it is possible to generate compatibility sub-packages to connect code that uses one of the other import paths to the canonical one.
The canonical import path SHOULD always be the one referenced in the project documentation. However some projects do not document import path changes, and rely on HTTPS redirections (for example https://github.com/docker/docker → https://github.com/moby/moby). Such a redirection is a sufficient indicator the canonical import path has changed (but please make sure with upstream).
The new import path SHOULD be reflected in %{goipath}
and compatibility import paths MUST be declared with the goaltipaths
macro:
# A space-separated list of import paths to simulate.
%global goaltipaths
For example, the Go library github.com/Sirupsen/logrus
was renamed github.com/sirupsen/logrus
and the documentation reflects this new import path. The packager SHOULD thus request a renaming of his package with a new import path and a compatibility import path:
%global goipath github.com/sirupsen/logrus
%global goaltipaths github.com/Sirupsen/logrus
If a project has gone through multiple rename, multiple compatibility import paths can be specified as well.
Never defer renamings Packagers MUST NOT use import path compatibility sub-packages to alias the canonical import path to one of the previous namings. Packagers MUST apply upstream renaming choices to the main |
Go binary packages
The binaries produced by your rpmspec SHOULD generally be listed in the main package. However, if you want a more appropriate name or split binaries among different packages, you can create additional binary subpackage. Of course these package MUST NOT be noarch.
For example we can create the package bbolt that will contain the binary of the same name:
%package -n bbolt
[…]
%description -n bbolt
[…]
%files -n bbolt
%license LICENSE
%{_bindir}/bbolt
Versioning
Many Go libraries do not use package versions or have regular releases and are instead maintained in public version control. In this case, follow the standard Fedora version conventions. This means that often Go packages will have a version number of 0
and a release number like 0.10.20190319git27435c6
.
Most of this process is automated by macros so that you don’t have to specify the release number yourself.
You first specify either a Version, tag or commit in the header.
Version:
%global tag
%global commit
Then use the %gometa
macro:
%gometa
The %gometa
macro will automatically process the %{?dist}
tag of the Release
field to take into account the commit if any.
Commits vs releases You SHOULD package releases in priority. Please reward the projects that make an effort to identify stable code states. Only fall back to commits when the project does not release, when the release is more than six months old, or if you absolutely need one of the changes of a later commit. In the later cases please inform the project politely of the reason you needed to give up on their official releases. Promoting releases is a way to limit incompatible commit hell. |
Go Language Architectures
To compile on various architectures, golang and gcc-go compilers are available. The golang compiler currently supports x86, x86_64, ppc64le, ppc64 (partially, see upstream issue#13192), s390x, armv7hl and aarch64.
Binaries SHOULD set ExclusiveArch so that we only attempt to build packages on those arches. This is now automatically added by the %gometa
macro by leveraging the %{golang_arches}
macro. Packagers can exclude %ix86
(see Changes/EncourageI686LeafRemoval) by passing -f
to the %gometa
macro. The -f
flag tells %gometa
to set ExclusiveArch: %{golang_arches_future}
instead of ExclusiveArch: %{golang_arches}
. %{golang_arches_future}
includes the same architectures as {golang_arches}
sans %ix86
.
Dependências
Packages MUST have BuildRequires: go-rpm-macros
.
This is automated by the %gometa
macro.
Automatic Dependency Generation
Most of the golang-*
packages are source code only. The *-devel
sub-package that includes the source code should explicitly have provides for the golang imports that it includes. These provides are automatically deduced from import paths.
Binary builds that include these imports will use them in BuildRequires, for example:
BuildRequires: golang(github.com/gorilla/context)
Bundled or unbundled
At the moment golang projects packaged in Fedora SHOULD be unbundled by default. It means projects are built from dependencies packaged in Fedora.
For some project it can be reasonable to build from bundled dependencies. Every bundling needs a proper justification.
BuildRequires
The BuildRequires of the project contains the dependencies needed by unit tests and binaries.
export GOPATH=/home/user/go
export GO111MODULE=off
export goipath="github.com/sirupsen/logrus"
go get $goipath
(sort -u | xargs -I{} echo "BuildRequires: golang({})") <<< "$(
golist --imported --package-path $goipath --skip-self
golist --imported --package-path $goipath --skip-self --tests
)"
outputs:
BuildRequires: golang(github.com/stretchr/testify/assert)
BuildRequires: golang(github.com/stretchr/testify/require)
BuildRequires: golang(golang.org/x/crypto/ssh/terminal)
If automatic buildrequires are available on your build target, you can use the %go_generate_buildrequires
macro in %generate_buildrequires
:
%generate_buildrequires
%go_generate_buildrequires
This macro leverages golist
to gather build dependencies and tests dependencies from the package source.
Testing
You MUST run unit tests. Due to the nature of Go development, especially the lack of use of semantic versioning, API breakages are frequent.These need to be detected early and dealt with upstream.
Some tests may be disabled, especially the following kinds of unit tests are incompatible with a secure build environment such as mock:
-
tests that call a remote server or API over the Internet,
-
tests that attempt to reconfigure the system,
-
tests that rely on a specific app running on the system, like a database or syslog server.
If a test is broken for some other reason, you can disable it the same way. However, you SHOULD also report the problem upstream. Remember to trace in a comment why each check was disabled, with links to eventual upstream problem reports.
Walkthrough
This chapter will present a typical Go spec file step by step, with comments and explanations.
Spec preamble: %{goipath}
, %{forgeurl}
and %gometa
Usual case
A Go package is identified by its import path. A Go spec file will therefore start with the %{goipath}
declaration. Don’t get it wrong, it will control the behaviour of the rest of the spec file.
%global goipath google.golang.org/api
If your package is hosted on a forge like GitHub, GitLab, Bitbucket or Pagure, the hosting of the Go package will be automatically deduced from this variable (typically by prefixing it with https://). If that is not the case, you need to declare explicitly the hosting URL with the %{forgeurl}
macro
%global forgeurl https://github.com/google/google-api-go-client
The %{forgeurl}
declaration is followed by either Version
, commit
or tag
. Use the combination that matches your use-case.
%global commit 2dc3ad4d67ba9a37200c5702d36687a940df1111
You can also define |
Now that we have all the required variables, the %gometa
macro can be run
%gometa
It will compute and set the following variables if they are not already set by the packager:
- goname
-
an rpm-compatible package name derived from goipath
- gosource
-
a URL that can be used as SourceX: value
- gourl
-
a URL that can be used as URL: value
It will delegate processing to the %forgemeta
macro for:
- forgesource
-
a URL that can be used as SourceX: value
- forgesetupargs
-
the correct arguments to pass to
%setup
for this source used by%forgesetup
and%forgeautosetup
- archivename
-
the source archive filename, without extensions
- archiveext
-
the source archive filename extensions, without leading dot
- archiveurl
-
the URL that can be used to download the source archive, without renaming
- topdir
-
the source archive top directory (can be empty)
- extractdir
-
the source directory created inside
%{_builddir}
after using%forgesetup
,%forgeautosetup
or%{forgesetupargs}
- repo
-
the repository name
- owner
-
the repository owner (if used by another computed variable)
- shortcommit
-
the commit hash clamping used by the forge, if any
- scm
-
the scm type, when packaging code snapshots: commits or tags
- distprefix
-
the prefix that needs adding to dist to trace non-release packaging
Most of the computed variables are both overridable and optional.
Now we can add the remaining elements of the preamble. First, we can define a multiline description block shared between subpackages:
%global common_description %{expand:
cmux is a generic Go library to multiplex connections based on their payload.
Using cmux, you can serve gRPC, SSH, HTTPS, HTTP, Go RPC, and pretty much any
other protocol on the same TCP listener.}
This description MUST stay within 80 characters per line.
If you need specific devel summary and description, you can also define:
|
Then we MUST specify the license files that will be added to the devel subpackages:
%global golicenses
: A space-separated list of shell globs matching the project license files.
And the possible documentation that SHOULD be included:
%global godocs
: A space-separated list of shell globs matching the project documentation files. The Go rpm macros will pick up “.md” files by default without this.
You can also exclude files from
For example you might want to exclude |
Source package metadata: %{goname}
, %{gourl}
and %{gosource}
We can declare the usual rpm headers, using the values computed by %gometa
:
Name: %{goname}
# If not set before
Version:
Release: 1%{?dist}
Summary:
License:
URL: %{gourl}
Source: %{gosource}
You can replace them with manual definitions. For example, replace %{gourl}
with the project homepage if it exists separately from the repository URL. Be careful to only replace %{go*}
variables when it adds value to the specfile and you understand the consequences. Otherwise you will just add maintenance-intensive discrepancies in the distribution.
BuildRequires
If they are not automatically generated, you can now add the dependencies needed for package building and unit testings:
BuildRequires: golang(github.com/stretchr/testify/assert)
BuildRequires: golang(github.com/stretchr/testify/require)
BuildRequires: golang(golang.org/x/crypto/ssh/terminal)
See this section on how to get the BuildRequires list manually.
Package metadata
The process of declaring Go code packages has been automated with the %{gopkg}
macro. It will automatically generate a %package
and %description
for all primary import paths and compatibility ones.
%gopkg
If you only need to generate Go devel subpackages without compat, use:
%godevelpkg
If necessary, you can then define one or multiple Go packages that will contain the binaries being built. See the previous Go binary packages section.
%prep: %goprep
, removing vendoring
%goprep
unpacks the Go source archives and creates the project “GOPATH” tree used in the rest of the spec file. It removes vendored (bundled) code: use the -k
flag if you wish to keep this vendored code, and deal with the consequences in the rest of the spec.
%goprep
%goprep
only performs basic vendoring detection. It will miss inventive ways to vendor code. Remove manually missed vendor code after the %goprep
line.
%goprep
will not fix upstream sources for you. Since all the macro calls that follow %goprep
assume clean problem-free sources, you need to correct them just after the %goprep
line:
-
replace calls to deprecated import paths with their correct value
-
patch code problems
-
remove dead code (some upstreams deliberately ship broken source code in the hope someone will get around to fix it)
You SHOULD send fixes and problem reports upstream. Any patch used SHOULD contain a link to an upstream bug report or merge request. At minimum, a comment SHOULD be added to explain the patch rationale. |
When you package an import path, that participates in a dependency loop, you need bootstrapping to manage the initial builds: https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping For Go code, that means your bootstrap section should:
-
remove unit tests that import other parts of the loop
-
remove code that imports other parts of the loop
Sometimes one can resolve dependency loops just by splitting specific subdirectories in a separate -devel subpackage. See also Dealing with cyclic dependencies.
Automatic BuildRequires
If BuildRequires generator are supported, you can now add them to your build:
%generate_buildrequires
%go_generate_buildrequires
Packaging a binary: the %build section
If your package is a source package only, you can skip this %build
section entirely.
Otherwise, you first need to identify manually the project parts that can be built, and how to name the result. Practically, it’s any directory containing a main() Go section. Nice projects put those in cmd
subdirectories named after the command that will be built, which is what we will document here, but it is not a general rule. Sometimes the whole %goipath
builds as a single binary.
for cmd in cmd/* ; do
%gobuild -o %{gobuilddir}/bin/$(basename $cmd) %{goipath}/$cmd
done
Installing the packages
Source package installation
If you package a library, %gopkginstall
will perform installation steps for all primary import paths and compatibility ones.
%gopkginstall
If you only need to install Go devel subpackages without compat, use:
%godevelinstall
Binary package installation
For binaries, we simply create the %{_bindir}
directory in the buildroot and install the commands as executable in it:
install -m 0755 -vd %{buildroot}%{_bindir}
install -m 0755 -vp %{gobuilddir}/bin/* %{buildroot}%{_bindir}/
Be wary of command names which might already exist in Fedora. If you have any doubt, you can check if the command is already provided in Fedora:
If this is the case, use your best judgment to rename the command. |
Running the unit tests: %gocheck
As said before, you MUST run unit tests in %check
:
%gocheck
However it is often necessary to disable some of them. You have 3 exclusion flags to do so:
-
-d <directory>
: exclude the files contained in<directory>
non-recursively (subdirectories are not excluded) -
-t <tree root>
: exclude the files contained in<tree root>
recursively (subdirectories are excluded) -
-r <regexp>
: exclude files matching<regexp>
Remember to document why a test has been disabled.
Examples
Simple source package
# https://github.com/stretchr/testify
%global goipath github.com/stretchr/testify
Version: 1.2.2
%gometa
%global common_description %{expand:
Golang set of packages that provide many tools for testifying
that your code will behave as you intend.
Features include:
- Easy assertions
- Mocking
- Testing suite interfaces and functions}
%global golicenses LICENSE
Name: %{goname}
Release: 1%{?dist}
Summary: Tools for testifying that your code will behave as you intend
License: MIT
URL: %{gourl}
Source: %{gosource}
BuildRequires: golang(github.com/davecgh/go-spew/spew)
BuildRequires: golang(github.com/pmezard/go-difflib/difflib)
BuildRequires: golang(github.com/stretchr/objx)
%description
%{common_description}
%gopkg
%prep
%goprep
%install
%gopkginstall
%check
%gocheck
%gopkgfiles
%changelog
* Thu Mar 21 22:20:22 CET 2019 Robert-André Mauchin <zebob.m@gmail.com> - 1.2.2-1
- First package for Fedora
Handling package renames
# https://github.com/sirupsen/logrus
%global goipath github.com/sirupsen/logrus
Version: 1.4.0
%gometa
%global goaltipaths github.com/Sirupsen/logrus
%global common_description %{expand:
Logrus is a structured logger for Go (golang), completely API compatible with
the standard library logger.}
%global golicenses LICENSE
%global godocs *.md
Name: %{goname}
Release: 1%{?dist}
Summary: Structured logger for Go
License: MIT
URL: %{gourl}
Source: %{gosource}
BuildRequires: golang(golang.org/x/crypto/ssh/terminal)
BuildRequires: golang(github.com/stretchr/testify/assert)
%description
%{common_description}
%gopkg
%prep
%goprep
%install
%gopkginstall
%check
%gocheck
%gopkgfiles
%changelog
* Wed Oct 31 2018 Robert-André Mauchin <zebob.m@gmail.com> - 1.4.0-1
- First package for Fedora
Simple binary package
# https://github.com/square/go-jose
%global goipath gopkg.in/square/go-jose.v2
%global forgeurl https://github.com/square/go-jose
Version: 2.1.9
%gometa
%global common_description %{expand:
Package jose aims to provide an implementation of the Javascript Object
Signing and Encryption set of standards. This includes support for JSON Web
Encryption, JSON Web Signature, and JSON Web Token standards.}
%global golicenses LICENSE
%global godocs *.md
%global godevelheader %{expand:
# The devel package will usually benefit from corresponding project binaries.
Requires: %{name} = %{version}-%{release}
}
Name: %{goname}
Release: 1%{?dist}
Summary: An implementation of JOSE standards (JWE, JWS, JWT) in Go
# Detected licences
# - *No copyright* Apache License (v2.0) at 'LICENSE'
# json/ is BSD-3-Clause
License: Apache-2.0 AND BSD-3-Clause
URL: %{gourl}
Source: %{gosource}
BuildRequires: golang(golang.org/x/crypto/ed25519)
BuildRequires: golang(golang.org/x/crypto/pbkdf2)
BuildRequires: golang(github.com/stretchr/testify/assert)
BuildRequires: golang(gopkg.in/alecthomas/kingpin.v2)
%description
%{common_description}
%gopkg
%prep
%goprep
%build
for cmd in jose-util jwk-keygen; do
%gobuild -o %{gobuilddir}/bin/$(basename $cmd) %{goipath}/$cmd
done
%install
%gopkginstall
install -m 0755 -vd %{buildroot}%{_bindir}
install -m 0755 -vp %{gobuilddir}/bin/* %{buildroot}%{_bindir}/
%check
%gocheck
%files
%license %{golicenses}
%doc %{godocs}
%{_bindir}/*
%gopkgfiles
%changelog
* Thu Mar 21 21:59:10 CET 2019 Robert-André Mauchin <zebob.m@gmail.com> - 2.1.9-1
- First package for Fedora
Want to help? Learn how to contribute to Fedora Docs ›