Python Packaging Guidelines
This version of Python Packaging Guidelines is in effect since 2021 and represents a major rewrite and paradigm shift. Not all packages are updated to reflect this. Older guidelines are still being kept up to date, and existing packages MAY use them instead of this document:
-
“201x-era” Python packaging guidelines (Packages using these usually use the
%py3_install
or%py3_install_wheel
macro or callsetup.py install
.) -
Python 2 appendix Note that Python 2 packages require a FESCo exception.
These guidelines only support current Fedora releases. For older releases (such as in EPEL 8), consult the 201x-era guidelines. |
The two Distro-wide guidelines below apply to all software in Fedora that uses Python at build- or run-time.
The rest of the Guidelines apply to packages that ship code
that can be imported with Python’s import
statement.
Specifically, that is all packages
that install files under /usr/lib*/python*/
.
Except for the two “Distro-wide guidelines”, these Guidelines do not apply to simple one-file scripts or utilities, especially if these are included with software not written in Python. However, if an application (e.g. CLI tool, script or GUI app) needs a more complex Python library, the library SHOULD be packaged as an importable library under these guidelines.
A major goal for Python packaging in Fedora is to harmonize with the wider Python ecosystem, that is, the Python Packaging Authority (PyPA) standards and the Python Package Index (PyPI). Packagers SHOULD be prepared to get involved with upstream projects to establish best practices as outlined here. We wish to improve both Fedora and the wider Python ecosystem.
Some build tools (like CMake or autotools)
may not work with the latest PyPA standards yet.
(For example, they might generate .egg-info directories
rather than .dist-info .)
While this document’s normative points (MUST/SHOULD) are tool-agnostic,
many of the practical tips and helper macros will not be applicable.
If this affects you, consider contacting the
Python SIG
for guidance
and/or following the older guidelines
for the time being.
|
Fedora’s Python SIG not only develops these guidelines, but it’s also involved in PyPA standards and Python packaging best practices. Check out the wiki or mailing list if you need help or wish to help out. |
Distro-wide guidelines
Build-time dependency on python3-devel
Every package that uses Python (at run-time and/or build-time)
and/or installs Python modules
MUST have a build-time dependency on python3-devel
,
even if Python is not actually invoked during build-time.
Such a package MUST use one of the following in its .spec
file:
-
%pyproject_buildrequires
in the%generate_buildrequires
section -
BuildRequires: python3-devel
Only having a transitive build-time dependency on python3-devel
is not sufficient.
If the package uses an alternate Python interpreter instead of python3
(e.g. pypy
, jython
, python2.7
),
it MAY instead require the corresponding *-devel
package.
The *-devel
package brings in relevant RPM macros.
It may also enable automated or manual checks: for example,
Python maintainers use this requirement to list packages
that use Python in some way
and might be affected by planned changes.
Mandatory macros
The following macros MUST be used where applicable.
The expansions in parentheses are provided only as reference/examples.
The macros are defined for you in all supported Fedora and EPEL versions.
-
%{python3}
(/usr/bin/python3
): The Python interpreter. For example, this macro should be used for invoking Python from aspec
file script, passed toconfigure
scripts to select a Python executable, or used as%{python3} -m pip
to run a Python-based tool.If the packaged software invokes Python at run time (as opposed to running Python to build/test it), it might be necessary to pass flags to
%{python3}
to isolate it from user-installed packages. See Shebangs for details. -
%{python3_version}
(e.g.3.9
,3.10
): Version of the Python interpreter. -
%{python3_version_nodots}
(e.g.39
,310
): Version of the Python interpreter without the dot. -
%{python3_sitelib}
(e.g./usr/lib/python3.9/site-packages
): Where pure-Python modules are installed. -
%{python3_sitearch}
(e.g./usr/lib64/python3.9/site-packages
): Where Python extension modules (native code, e.g. compiled from C) are installed.
The rest of this document uses these macros,
along with %{_bindir}
(/usr/bin/
),
instead of the raw path names.
Python implementation support
Fedora primarily targets CPython, the reference implementation of the Python language. We generally use “Python” to mean CPython.
Alternate implementations like pypy
are available,
but currently lack comprehensive tooling and guidelines for packaging.
When targetting these, there are no hard rules
(except the general Fedora packaging guidelines).
But please try to abide by the spirit of these guidelines.
When in doubt, consider consulting the Python SIG.
Python version support
Fedora packages MUST NOT depend on other versions
of the CPython interpreter
than the current python3
.
In Fedora, Python libraries are packaged for a single version of Python,
called python3
.
For example, in Fedora 32, python3
is Python 3.8.
In the past, there were multiple Python stacks,
e.g. python3.7
and python2.7
,
installable together on the same machine.
That is also the case in some projects that build on top of Fedora,
like RHEL, EPEL and CentOS.
Fedora might re-introduce parallell-installable stacks in the future
(for example if a switch to a new Python version needs a transition period,
or if enough interested maintainers somehow appear).
Fedora does include alternate interpreter versions,
e.g. python2.7
or python3.5
,
but these are meant only for developers that need to test upstream code.
Bug and security fixes for these interpreters only cover this use case.
Packages such as pip
or tox
,
which enable setting up isolated environments
and installing third-party packages into them,
MAY, as an exception to the rule above, use these interpreters
as long as this is coordinated with the maintainers
of the relevant Python interpreter.
Naming
Python packages have several different names, which should be kept in sync but will sometimes differ for historical or practical reasons. They are:
Some examples (both good and worse):
Fedora component | Built RPM | Project name | Importable module |
---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Elsewhere in this text,
the metavariables SRPMNAME
, RPMNAME
, PROJECTNAME
, MODNAME
refer to these names, respectively.
Canonical project name
Most of these names are case-sensitive machine-friendly identifiers,
but the project name has human-friendly semantics:
it is case-insensitive
and treats some sets of characters (like ._-
) specially.
For automated use,
it needs to be normalized to a canonical format
used by Python tools and services such as setuptools, pip and PyPI.
For example, the canonical name of the Django
project
is django
(in lowercase).
This normalization is defined in
PEP 503,
and the %{py_dist_name}
macro implements it
for Fedora packaging.
The canonical name is obtained by switching the project name to lower case
and converting all runs of non-alphanumeric characters to single “-” characters.
Example: “The $$$ Tree” becomes “the-tree”.
Elsewhere in this text,
the metavariable DISTNAME
refers to the canonical form of the project name.
Note that in some places, the original,
non-normalized project name must be used.
For example,
the %pypi_source
macro and the %autosetup
macro
need Django
, not django
.
Name limitations
The character +
in names of built (i.e. non-SRPM) packages
that include .dist-info
or .egg-info
directories
is reserved for Extras and MUST NOT be used for any other purpose.
As an exception, +
characters MAY appear at the end of such names.
The +
character triggers
the automatic dependency generator for extras.
Replace any +
signs in the upstream name with -
.
Omit +
signs on the beginning of the name.
Consider adding Provides
for the original name with +
characters
to make the package easier to find for users.
Library naming
A built (i.e. non-SRPM) package for a Python library
MUST be named with the prefix python3-
.
A source package containing primarily a Python library
MUST be named with the prefix python-
.
The Fedora package’s name SHOULD contain
the Canonical project name.
If possible, the project name SHOULD be the same
as the name of the main importable module,
in lowercase,
with underscores (_
) replaced by dashes (-
).
If the importable module name and the project name do not match, users frequently end up confused. In this case, packagers SHOULD ensure that upstream is aware of the problem and (especially for new packages where renaming is feasible) strive to get the package renamed. The Python SIG is available for assistance.
A Python library is a package meant to be imported in Python,
such as with import requests
.
Tools like Ansible or IDLE, whose code is importable
but not primarily meant to be imported from other software,
are not considered libraries in this sense.
So, this section does not apply for them.
(See the
general Libraries and Applications guidelines
for general guidance.)
The Fedora component (source package) name for a library
should be formed by taking the canonical project name
and prepending python-
if it does not already start with python-
.
This may leads to conflicts
(e.g. between bugzilla
and python-bugzilla).
In that case, ensure upstream is aware of the potentially confusing naming
and apply best judgment.
Application naming
Packages that primarily provide applications, services
or any kind of executables SHOULD be named
according to the general Fedora naming guidelines
(e.g. ansible
).
Consider adding a virtual provide according to Library naming above
(e.g. python3-PROJECTNAME
),
if it would help users find the package.
Files to include
Source files and bytecode cache
Packages MUST include the source file (*.py
)
AND the bytecode cache (*.pyc
) for each pure-Python importable module.
The source files MUST be included in the same package as the bytecode cache.
Scripts that are not importable
(typically ones in %{_bindir}
or %{_libexecdir}
)
SHOULD NOT be byte-compiled.
The cache files are found in a __pycache__
directory
and have an interpreter-dependent suffix like .cpython-39.pyc
.
The cache is not necessary to run the software, but if it is not found, Python will try to create it when a module is imported. If this succeeds, the file is not tracked by RPM and it will linger on the system after uninstallation. If it does not succeed, users can get spurious SELinux AVC denials in the logs.
Normally, byte compilation (generating the cache files)
is done for you by the brp-python-bytecompile
BRP script,
which runs automatically
after the %install
section of the spec file has been processed.
It byte-compiles any .py
files that it finds
in %{python3_sitelib}
or %{python3_sitearch}
.
You must include these files of your package
(i.e. in the %files
section).
If the code is in a subdirectory (importable package), include the entire directory:
%files
%{python3_sitelib}/foo/
Adding the trailing slash is best practice for directories.
However, this cannot be used for top-level modules (those directly in
e.g. %{python3_sitelib}
), because both %{python3_sitelib}
and
%{python3_sitelib}/__pycache__/
are owned by Python itself. Here, the
%pycached
macro can help. It expands to the given *.py
source file
and its corresponding cache file(s). For example:
%files
%pycached %{python3_sitelib}/foo.py
expands roughly to:
%files
%{python3_sitelib}/foo.py
%{python3_sitelib}/__pycache__/foo.cpython-3X{,.opt-?}.pyc
Manual byte compilation
If you need to bytecompile stuff
outside of %{python3_sitelib}
/%{python3_sitearch}
,
use the %py_byte_compile
macro.
For example,
if your software adds %{_datadir}/mypackage
to Python’s import path
and imports package foo
from there, you will need to compile foo
with:
%py_byte_compile %{python3} %{buildroot}%{_datadir}/mypackage/foo/
Dist-info metadata
Each Python package MUST include Package Distribution Metadata conforming to PyPA specifications (specifically, Recording installed projects).
The metadata SHOULD be included in the same subpackage as the main importable module, if there is one.
This applies to libraries (e.g. python-requests
)
as well as tools (e.g. ansible
).
When software is split into several subpackages, it is OK to only ship metadata in one built RPM. In this case, consider working with upstream to also split the upstream project.
The metadata takes the form of a .dist-info
directory
installed in %{python3_sitelib}
or %{python3_sitearch}
,
and contains information that tools like
importlib.metadata
use to introspect installed libraries.
For example, a project named MyLib
with importable package mylib
could be packaged with:
%files -p python3-mylib
%{python3_sitelib}/mylib/
%{python3_sitelib}/MyLib-%{version}.dist-info/
%doc README.md
%license LICENSE.txt
Note that some older tools instead put metadata in an .egg-info
directory,
or even a single file.
This won’t happen if you use the %pyproject_wheel
macro.
If your package uses a build system
that generates an .egg-info
directory or file,
please contact Python SIG.
As an exception, the Python standard library MAY ship without this metadata.
Explicit lists
Packages MUST NOT own shared directories owned by Python itself,
such as the top-level __pycache__
directories
(%{python3_sitelib}/__pycache__
, %{python3_sitearch}/__pycache__
).
Similarly to the general rule, packagers SHOULD NOT simply glob everything under a shared directory.
In addition to the general list,
the following SHOULD NOT be used in %files
:
-
%{python3_sitelib}/*
-
%{python3_sitearch}/*
-
%{python_sitelib}/*
-
%{python_sitearch}/*
-
%pyproject_save_files '*'
-
%pyproject_save_files +auto
This rule serves as a check against common mistakes which are otherwise hard to detect. It does limit some possibilities for automation.
The most common mistakes this rule prevents are:
-
installing a test suite system-wide as an importable module named
test
, which would then conflict with other such packages, and -
upstream adding new unexpected importable modules – you should always check such changes for conflicts, and keep the list of such files explicit and auditable.
PyPI parity
Every Python package in Fedora SHOULD also be available on the Python Package Index (PyPI).
The command pip install PROJECTNAME
MUST
install the same package (possibly in a different version),
install nothing,
or fail with a reasonable error message.
If this is not the case, the packager SHOULD contact upstream about this. The goal is to get the project name registered or blocked on PyPI, or to otherwise ensure the rule is followed.
If your package is not or cannot be published on PyPI, you can:
-
Ask upstream to publish it
-
If you wish: publish it to PyPI yourself and maintain it
-
Ask Python SIG to block the name on PyPI for you
-
Email PyPI admins to block the name for you, giving the project name and explaining the situation (for example: the package cannot currently be installed via
pip
). You can ask questions and discuss the process at the Python Discourse.
Project names that were in Fedora but not on PyPI
when these guidelines were proposed are blocked from being uploaded to PyPI.
This prevents potential trolls from taking them,
but it also blocks legitimate owners.
If your package is affected, contact the Python SIG or
file a PyPA issue
and mention @encukou .
|
If your package’s project name conflicts with a different package on PyPI, change the project name. As painful as it is, we need to use a single global namespace across the Python ecosystem. Software that is not written specifically for Fedora already expects that project names use the PyPI namespace: for example, if a third-party library identifies a dependency by name, we don’t want that dependency satisfied by an unrelated Fedora package.
As always, specific exceptions can be granted by the Packaging Committee.
Provides and requirements
Provides for importable modules
For any module intended to be used in Python 3 with import MODNAME
,
the package that includes it SHOULD provide python3-MODNAME
,
with underscores (_
) replaced by dashes (-
).
This is of course always the case if the package is named python3-MODNAME
.
If the subpackage has some other name,
then add %py_provides python3-MODNAME
explicitly.
See the following section to learn about %py_provides
.
Automatic python- and python3.X- provides
For any FOO
,
a package that provides python3-FOO
SHOULD use %py_provides
or an automatic generator
to also provide python-FOO
and python3.X-FOO
, where X
is the minor version of the interpreter.
The provide SHOULD NOT be added manually:
if a generator or macro is not used,
do not add the python-FOO
/ python3.X-FOO
provides at all.
This is done automatically for package names by a generator.
If absolutely necessary, the generator can be disabled
by undefining the %__pythonname_provides
macro.
For provides that aren’t package names,
or (for technical reasons) for packages without files,
the generator will not work.
For these cases, the following invocation will provide python3-FOO
,
python-FOO
and python3.X-FOO
:
%py_provides python3-FOO
Using the generator or macro is important, because the specific form of the provide may change in the future.
Machine-readable provides
Every Python package MUST provide python3dist(DISTNAME)
and python3.Xdist(DISTNAME)
,
where X
is the minor version of the interpreter
and DISTNAME
is the Canonical project name
corresponding to the Dist-info metadata.
For example, python3-django
would provide
python3dist(django)
and python3.9dist(django)
.
This is generated automatically from the dist-info metadata. The provide SHOULD NOT be added manually: if the generator fails to add it, the metadata MUST be fixed.
These Provides are used for automatically generated Requires.
If absolutely necessary,
the automatic generator can be disabled by undefining the
%{?__pythondist_provides}
macro.
Consider discussing your use case with the Python SIG if you need to do this.
Dependencies
As mentioned above,
each Python package MUST explicitly BuildRequire python3-devel
.
Packages MUST NOT have dependencies (either build-time or runtime)
with the unversioned prefix python-
if the corresponding python3-
dependency can be used instead.
Packages SHOULD NOT have explicit dependencies
(either build-time or runtime)
with a minor-version prefix such as python3.8-
or python3.8dist(
.
Such dependencies SHOULD instead be automatically generated
or a macro should be used to get the version.
Packages SHOULD NOT have an explicit runtime dependency on python3
.
Instead of depending on python3
,
packages have an automatic dependency on python(abi) = 3.X
when they install files to %{python3_sitelib}
or %{python3_sitearch}
,
or they have an automatic dependency on /usr/bin/python3
if they have executable Python scripts,
or they have an automatic dependency on libpython3.X.so.1.0()
if they embed Python.
These rules help ensure a smooth upgrade path
when python3
is updated in new versions of Fedora.
Automatically generated dependencies
Packages MUST use the automatic Python run-time dependency generator.
Packages SHOULD use the opt-in build-dependency generator if possible.
The packager MUST inspect the generated requires for correctness. All dependencies MUST be resolvable within the targeted Fedora version.
Any necessary changes MUST be done by patches
or modifying the source (e.g. with sed
),
rather than disabling the generator.
The resulting change SHOULD be offered to upstream.
As an exception, filtering
MAY be used for temporary workarounds
and bootstrapping.
Dependencies covered by the generators SHOULD NOT
be repeated in the .spec
file.
(For example, if the generator finds a requests
dependency,
then Requires: python3-requests
is redundant.)
The automatically generated requirements
are in the form python3.Xdist(DISTNAME)
,
potentially augmented with version requirements or combined together
with rich dependencies.
Any .0
suffixes are removed from version numbers
to match the behavior of Python tools.
(PEP 440 specifies
that X.Y
and X.Y.0
are treated as equal.)
Note that the generators only cover Python packages.
Other dependencies, often C libraries like openssl-devel
,
must be specified in the .spec
file manually.
Where the requirements are specified in the source
depends on each project’s build system and preferences.
Common locations are pyproject.toml
, setup.py
, setup.cfg
,
config.toml
.
Run-time dependency generator
The automatic runtime dependency generator uses package metadata
(as recorded in installed *.dist-info
directories)
to determine what the package depends on.
In an emergency, you can opt-out from running the requires generator by
adding %{?python_disable_dependency_generator}
to the package (usually, just before the main package’s %description
).
Build-time dependency generator
The opt-in (but strongly recommended) build-time dependency generator
gathers information from
pyproject.toml
build-system information
(with fallback to setuptools
) plus a standardized
build-system hook
to gather further requirements.
See the %pyproject_buildrequires
macro
for more details.
Note that without the -R
flag, the generator will include run-time
requirements in BuildRequires. This is useful for running tests and for
checking that the dependencies are available in Fedora.
Test dependencies
See the Tests section.
Extras
Python extras are a way for Python projects to declare that extra dependencies are required for additional functionality.
For example,
requests
has several standard dependencies (e.g. urllib3
).
But it also declares an extra named requests[security]
,
which lists additional dependencies (e.g. cryptography
).
Unlike RPM subpackages, extras can only specify additional dependencies,
not additional files.
The main package will work if the optional dependency is not installed,
but it might have limited functionality.
Python tools treat extras as virtual packages.
For example, if a user runs pip install 'requests[security]'
,
or installs a project that depends on requests[security]
,
both requests
and cryptography
will be installed.
In Fedora, extras are usually provided by packages with no files.
Instead of square brackets,
Fedora package names conventionally use the +
character
(which is valid in RPM package names,
but not in Python canonical project names nor in extras identifiers).
Handling extras
Python packages SHOULD have Provides for all extras the upstream project specifies, except:
-
those that are not useful for other packages (for example build/development requirements, commonly named
dev
,doc
ortest
), and -
those that have requirements that are not packaged in Fedora.
A package that provides a Python extra
MUST provide python3dist(DISTNAME[EXTRA])
and python3.Xdist(DISTNAME[EXTRA])
,
where X
is the minor version of the interpreter,
DISTNAME
is the Canonical project name,
and EXTRA
is the name of a single extra.
For example, python3.9dist(requests[security])
.
These requirements SHOULD be generated
using the automatic dependency generator.
A package that provides a Python extra MUST require the extra’s main package with exact NEVR.
A subpackage that primarily provides one Python extra SHOULD be named
by appending +
and the extra name to the main package name.
For example, python3-requests+security
.
The most straightforward way to provide an extra
is with a dedicated subpackage containing no files (a “metapackage”).
This case can be automated with
the %pyproject_extras_subpkg
macro
or the %python_extras_subpkg
macro.
This is not the only way: when some extra is always useful in a distro, it can be provided by the main package; when several extras are related, they may be provided by a single subpackage. However, having one dedicated subpackage per extra allows you to use the automatic dependency generator to ensure that the extras’ requirements will stay in sync with upstream. If you create a dedicated subpackage and want it to be always/usually installed, you can Require/Recommend/Suggest it from the main package.
The dependency generator for extras activates if the following holds:
-
The package name must end with
+EXTRA
(whereEXTRA
is the extra name). -
The package must contain the
.dist-info
directory, usually as%ghost
.
Example and convenience macros
The extra subpackage for setuptools_scm[toml]
can be specified
using the %pyproject_extras_subpkg
convenience macro as follows.
The macro takes the main package name and name(s) of the extra(s):
%pyproject_extras_subpkg -n python3-setuptools_scm toml
If not using %pyproject_install
,
you will instead need to use %python_extras_subpkg
and pass a path to the dist-info
directory:
%python_extras_subpkg -n python3-setuptools_scm -i %{python3_sitelib}/*.dist-info toml
For this case,
the extras dependency generator will read upstream metadata
from the .dist-info
directory.
If it finds that the extra requires on toml
,
it will generate Requires: python3.Xdist(toml)
and Provides: python3dist(setuptools-scm[toml])
(and the corresponding python3.Xdist
provide).
If you need additional features
that the *_extras_subpkg
macros do not cover,
you will need to write the subpackage sections manually.
Such features can be, for example:
-
Obsoleting/providing other names (e.g. obsoleted extras packages)
-
Manual strong or weak dependencies on other (possibly non-Python) packages
As an example of what you need to write in these cases,
both of the *_extras_subpkg
macro invocations above
expand to the following:
%package -n python3-setuptools_scm+toml
Summary: Metapackage for python3-setuptools_scm: toml extra
Requires: python3-setuptools_scm = %{?epoch:%{epoch}:}%{version}-%{release}
%description -n python3-setuptools_scm+toml
This is a metapackage bringing in toml extra requires for python3-setuptools_scm.
It contains no code, just makes sure the dependencies are installed.
%files -n python3-setuptools_scm+toml
%ghost %{python3_sitelib}/*.dist-info
Note that the dependency generator
does not add a dependency on the main package
(the Requires: python3-setuptools_scm = ...
above).
If you are not using the %python_extras_subpkg
macro,
you need to add it manually.
Removing extras
If an existing extra is removed from an upstream project, the Fedora maintainer SHOULD try to convince upstream to re-introduce it (with an empty list of dependencies). If that fails, the extra SHOULD be Obsoleted from either the main package or another extras subpackage.
Note that removing extras is discouraged in setuptools documentation (see the Tip box near the end of the Optional dependencies section).
Automatic Requires for extras
The automatic Run-time dependency generator
will generate Requires on python3.Xdist(DISTNAME[EXTRA])
from upstream Requires-Dist
metadata.
If the required package does not yet provide metadata for the extra, contact the Fedora maintainer to add it.
In an emergency, you can define
the %_python_no_extras_requires
macro
to avoid automatically generating all extras requirements.
Interpreter invocation
Shebangs
Shebang lines to invoke Python MUST use %{python3}
as the interpreter.
Shebang lines to invoke Python SHOULD be #!%{python3} -%{py3_shebang_flags}
and they MAY include extra flags.
If (some of) the default flags from
the %{py3_shebang_flags}
macro are not desirable,
packages SHOULD explicitly redefine the macro to remove them by undefining
the relevant %{_py3_shebang_...}
macro.
Using #!%{python3}
(#!/usr/bin/python3
)
rather than e.g. #!/usr/bin/env python
ensures that the system-wide Python interpreter is used to run the code,
even if the user modifies $PATH
(e.g. by activating a virtual environment).
By default, -%{py3_shebang_flags}
expands to -sP
(or just -s
on Python version lower than 3.11 and Fedora Linux older than 37).
The -s
flag,
stored in the %{_py3_shebang_s}
macro,
means don’t add user site directory to sys.path
.
That ensures the user’s Python packages
(e.g. installed by pip install --user
,
or just placed in the current directory)
don’t interfere with the RPM installed software.
Sometimes, such content is desirable, such as with plugins.
The -P
flag,
stored in the %{_py3_shebang_P}
macro,
means don’t add the script’s directory to sys.path
.
Sometimes, adding the script’s directory to sys.path
is desirable,
such as with executable Python scripts installed in a custom directory,
importing each other.
Removing the undesired flag(s) from the %{py3_shebang_flags}
macro
rather than not using the macro at all,
ensures that existing or future automation won’t add the flag.
# Remove -s from Python shebang - ensure that extensions installed with pip
# to user locations are seen and properly loaded
%undefine _py3_shebang_s
# Don't add -P to Python shebangs
# The executable Python scripts in /usr/share/opt-viewer/ import each other
%undefine _py3_shebang_P
The %pyproject_install
macro
automatically changes all Python shebangs
in %{buildroot}%{_bindir}/*
to use %{python3}
and add contents of the %{py3_shebang_flags}
macro
to the existing flags.
If you’re not using that macro
or you need to change a shebang in a different directory,
you can use the %py3_shebang_fix
macro as follows:
%py3_shebang_fix SCRIPTNAME …
Invokable Python modules
Every executable TOOL
for which the current version of Python matters
SHOULD also be invokable by python3 -m TOOL
.
If the software doesn’t provide this functionality, packagers SHOULD ask the upstream to add it.
This applies to tools that modify the current Python environment (like installing or querying packages), use Python for configuration, or use Python to run plugins. It does not apply to tools like GIMP or Bash which support plugins in multiple languages and/or have other means to specify the interpreter.
For example, pip
can be invoked as python3 -m pip
.
This allows users to accurately specify
the Python version used to run the software.
This convention works across different environments
that might not always set $PATH
or install scripts consistently.
Using Cython
Tightening the
general Fedora policy,
packages MUST NOT use files pre-generated by Cython.
These MUST be deleted in %prep
and regenerated during the build.
As an exception, these sources MAY be used temporarily to prevent build time circular dependencies by following the bootstrapping guidelines.
Generated files (the ones that must be deleted)
have a generic .c
or .cpp
extension.
Cython source files (which should stay)
usually have the .pyx
or .pxd
extension.
Cython is a popular tool for writing extension modules for Python. If compiles a Python-like language to C, which is then fed to the C compiler. Historically, Cython was hard to use upstream as a build-time dependency. Many projects include pre-generated C files in source distributions to avoid users from needing to install the tool.
Cython uses CPython’s fast-changing internal API for performance reasons. For a new release of Python, Cython generally needs to be updated and the C files regenerated. In Fedora, this is frequently needed before upstreams release re-generated sources (e.g. for Alpha versins of Python). Since we do not have a problem with build-time dependencies, we always want to run the Cython step.
For example, PyYAML
removes a generated C file with:
rm -rf ext/_yaml.c
For another example, in python-lxml
all C files are generated with Cython,
which allows removing them with:
# Remove pregenerated Cython C sources
find -type f -name '*.c' -print -delete
Some upstreams mix generated and hand-written C files.
In such cases a grep like this one from scipy
helps
(but might not be entirely future proof):
# Remove pregenerated Cython C sources
rm $(grep -rl '/\* Generated by Cython')
Tests
Running tests
If a test suite exists upstream,
it SHOULD be run in the %check
section.
If that is not possible with reasonable effort,
at least a basic smoke test (such as importing the packaged module)
MUST be run in %check
.
You MAY exclude specific failing tests. You MUST NOT disable the entire testsuite or ignore its result to solve a build failure.
As an exception,
you MAY disable tests with an appropriate %if
conditional
(e.g. bcond)
when bootstrapping.
Most errors in Python happen at run-time, so tests are extremely important to root out issues, especially when mass rebuilds are required.
Common reasons for skipping tests in %check
include requiring
network access,
dependencies not packaged in Fedora,
and/or specialized hardware or resources.
In these cases,
you can use the %pyproject_check_import
or the %py3_check_import
macro
to test that installed modules are importable.
Tox
A popular testing tool, and one which is well integrated in Fedora,
is tox
.
Upstream, it is commonly used to test against multiple Python versions.
In a Fedora package, BuildRequire test dependencies via %pyproject_buildrequires -t
or -e
(see Test dependencies below)
and run tox
with:
%tox
This sets up the environment
($PATH
, $PYTHONPATH
, $TOX_TESTENV_PASSENV
)
and instructs tox
to use the current environment rather than create new ones.
For more options, see Build macros.
pytest
When upstream doesn’t use tox
,
the tests need to be run directly
depending on upstream choice of a test runner.
A popular runner is pytest
, which can be invoked using %pytest
.
Use positional arguments to specify the test directory.
See python3 -m pytest --help
for how to select tests.
For example, if network-related tests are marked “network”,
you might use -m
to deselect them:
%pytest -m "not network"
The %pytest
macro sets several environment variables
appropriate for %check
:
-
Locations in the buildroot are added to
$PATH
and$PYTHONPATH
. -
$PYTHONDONTWRITEBYTECODE
is set to avoid writing pytest-specific cache files to buildroot -
$PYTEST_XDIST_AUTO_NUM_WORKERS
is set to%{_smp_build_ncpus}
-
If unset,
$CFLAGS
and$LDFLAGS
are set to match the build flags
Other test runners
If upstream doesn’t use tox
or pytest
,
other test runners can be invoked with the %{py3_test_envvars}
macro,
available since Fedora Linux 38.
This macro sets several environment variables similarly to %pytest
,
but requires the actual test runner to be invoked after the macro, for example:
%{py3_test_envvars} %{python3} -m unittest
Or:
%{py3_test_envvars} %{python3} tests/run_tests.py
Test dependencies
One part of the Python packaging ecosystem that is still not standardized is specifying test dependencies (and development dependencies in general).
A good, common way for upstreams to specify test dependencies
is using an extra like [test]
, [testing]
or [dev]
.
In this case, upstream’s instructions to install test dependencies
might look like $ pip install -e.[test]
.
Projects using tox
usually specify test dependencies
in a tox
-specific format:
a requires
key in the configuration.
These two forms are handled by
the %pyproject_buildrequires
macro.
If upstream does not use either form,
list test dependencies as manual BuildRequires in the spec
file,
for example:
# Test dependencies:
BuildRequires: python3dist(pytest)
If you need to do this,
consider asking upstream to add a [test]
extra.
Linters
In %check
, packages SHOULD NOT run “linters”:
code style checkers,
test coverage checkers
and other tools that check code quality rather than functionality.
Tools like black
, pylint
, flake8
, or mypy
are often “opinionated” and their “opinions” change frequently enough
that they are nuisance in Fedora,
where the linter is not pinned to an exact version.
Furthermore, some of these tools take a long time
to adapt to new Python versions,
preventing early testing with Alpha and Beta releases of Python.
And they are just not needed: wrongly formatted code is not important enough
for the Fedora packager to bug the upstream about it.
Making such an issue break a package build is entirely unreasonable.
Linters do make sense in upstream CI. But not in Fedora.
If a linter is used, disable it and remove the dependency on it.
If that is not easy, talk to upstream about making it easy
(for example with a configuration option or a separate tox
environment).
For packages that contain such linters, use them at runtime or extend them,
you will usually need to run the linter in %check
.
Run it to test functionality, not code quality of the packaged software.
Source files from PyPI
Packages MAY use sources from PyPI.
However, packages SHOULD NOT use an archive that omits test suites, licenses and/or documentation present in other source archives.
For example, as of this writing pip
provides a
source tarball (“sdist”)
which omits the relatively large tests
and docs
directories
present in the source on GitHub.
In this case, the tarball from GitHub should be used.
(See the Git tags section
of Fedora SourceURL guidelines.)
When using sources from PyPI,
you can use the the %pypi_source
macro
to generate the proper URL.
Some Python packages use metadata from git
(or a similar version control system)
to construct their version string,
for example via setuptools_scm.
When publishing a package to PyPI,
this version metadata is usually stored and included in a file,
so the version control history is no longer needed to construct it.
However, when using tarballs from a git forge directly,
this version information is missing
and must be manually provided by the packager.
For example, the SETUPTOOLS_SCM_PRETEND_VERSION environment variable can be set
to the desired value in the %generate_buildrequires and %build scripts in the spec file
for packages that use setuptools_scm for this purpose.
|
Example spec file
The following is a viable spec file
for a Python library called Pello
that follows packaging best practices.
Note that the project name Pello
normalizes
to the lowercase pello
.
The example spec shows where each variant is typically used.
The project has an extra color
,
which enables colorized output when installed.
Since the required dependency is quite minimal
and color improves the user experience,
the extra is Recommended from the main package.
Name: python-pello
Version: 1.0.4
Release: 1%{?dist}
Summary: Example Python library
License: MIT-0
URL: https://github.com/fedora-python/Pello
Source: %{url}/archive/v%{version}/Pello-%{version}.tar.gz
BuildArch: noarch
BuildRequires: python3-devel
%global _description %{expand:
A python module which provides a convenient example.
This description provides some details.}
%description %_description
%package -n python3-pello
Summary: %{summary}
Recommends: python3-pello+color
%description -n python3-pello %_description
%pyproject_extras_subpkg -n python3-pello color
%prep
%autosetup -p1 -n Pello-%{version}
%generate_buildrequires
%pyproject_buildrequires -t
%build
%pyproject_wheel
%install
%pyproject_install
# Here, "pello" is the name of the importable module.
%pyproject_save_files -l pello
%check
%tox
# Note that there is no %%files section for
# the unversioned python module, python-pello.
# For python3-pello, %%{pyproject_files} handles code files and %%license,
# but executables and documentation must be listed in the spec file:
%files -n python3-pello -f %{pyproject_files}
%doc README.md
%{_bindir}/pello_greeting
%changelog
Empty spec file
The following is an unfinished spec file template to copy, paste and edit.
Name: python-...
Version: ...
Release: 0%{?dist}
Summary: ...
License: ...
URL: https://...
Source: %{url}/archive/v%{version}/...-%{version}.tar.gz / %{pypi_source ...}
BuildArch: noarch / BuildRequires: gcc
BuildRequires: python3-devel
%global _description %{expand:
...}
%description %_description
%package -n python3-...
Summary: %{summary}
%description -n python3-... %_description
%prep
%autosetup -p1 -n ...-%{version}
%generate_buildrequires
%pyproject_buildrequires -x... / -t
%build
%pyproject_wheel
%install
%pyproject_install
%pyproject_save_files ...
%check
%tox / %pytest / %pyproject_check_import ...
%files -n python3-... -f %{pyproject_files}
%doc README.*
%{_bindir}/...
%changelog
Macro Reference
This section documents macros that are available to help with Python packaging. The expansions in parentheses are provided only as reference/examples.
See the Mandatory macros section above for:
-
%{python3}
(/usr/bin/python3
) -
%{python3_version}
(e.g.3.9
) -
%{python3_version_nodots}
(e.g.39
) -
%{python3_sitelib}
(e.g./usr/lib/python3.9/site-packages
) -
%{python3_sitearch}
(e.g./usr/lib64/python3.9/site-packages
)
Shebang macros
-
%{py3_shebang_flags}
(sP
ors
before Fedora Linux 37)Flags for
%{python3}
to use in shebangs. See Shebangs for details. Includes flags from several%{_py3_shebang_...}
macros listed here.
-
%{_py3_shebang_s}
(s
)Undefine this macro to drop
s
from%{py3_shebang_flags}
.
-
%{_py3_shebang_P}
(P
)Undefine this macro to drop
P
from%{py3_shebang_flags}
. Introduced in Fedora Linux 37.
-
%py3_shebang_fix PATHS
(pathfix.py ... PATHS
)A macro to fix shebangs in specified
PATHS
. Only shebangs that already havepython
in them are changed. If a directory is given, all.py
files in it are fixed, recursively. (So, if you need to fix shebangs in files not named*.py
, you need to list each file separately or use a Shell glob, such as%{buildroot}%{_libexecdir}/mytool/*
.) Existing flags are preserved and%{py3_shebang_flags}
are added.For example,
#! /usr/bin/env python
will be changed to#! /usr/bin/python3 -s
and#! /usr/bin/python -u
will be changed to#! /usr/bin/python3 -su
.This macro is called automatically by
%pyproject_install
on%{buildroot}%{_bindir}/*
.
Convenience macros
-
%{pypi_source PROJECTNAME [VERSION [EXT]]}
(e.g.https://.../Django-3.0.5.tar.gz
)Evaluates to the appropriate URL for source archive hosted on PyPI. Accepts the project name and up to two optional arguments:
-
The version of the PyPI project. Defaults to
%version
(the package version) with any~
removed. -
The file extension to use. Defaults to
tar.gz
.
In most cases it is not necessary to specify those two arguments.
For backward compatibility, the first argument is technically optional as well, but omitting it is deprecated. (It defaults to
%srcname
if defined, or to%pypi_name
if defined, or to%name
.) -
-
%{python3_platform}
(e.g.linux-x86_64
)The platform name. Used in some Python build systems. This corresponds to
sysconfig.get_platform()
.
-
%{python3_ext_suffix}
(e.g..cpython-39-x86_64-linux-gnu.so
)Filename extension for Python extension modules. This corresponds to the
EXT_SUFFIX
sysconfig variable.
-
%{python3_platform_triplet}
(e.g.x86_64-linux-gnu
)A string identifying the architecture/platform. This corresponds to the
MULTIARCH
sysconfig variable.
-
%{python3_cache_tag}
(e.g.cpython-311
)Part of the bytecode cache filename that identifies the interpreter. This corresponds to the
sys.implementation.cache_tag
value.
Build macros
The “pyproject macros” are most useful
for packaging Python projects that use the pyproject.toml
file
defined in PEP 518
and PEP 517,
which specifies the package’s build dependencies
(including the build system, such as setuptools
, flit
or poetry
).
If pyproject.toml
is not found,
the macros automatically fall backs to using setuptools
with configuration in setup.cfg
/setup.py
.
A full tutorial and discussion for the macros is available in the macros’ README.
-
%pyproject_buildrequires
Generate BuildRequires for the package. Used in the
%generate_buildrequires
section of thespec
file. The macro has these options:-
-R
: Don’t include run-time requirements (e.g. if the build backend does not support this). -
-r
: Include run-time requirements (this flag is not needed and exists for backward-compatibility reasons only, run-time requirements are included by default). -
-x EXTRA
: Include dependencies given by the given extra. Cannot be used with-R
. -
+-p
: Read run-time dependencies from pyproject.toml [project] table. This reads also the [optional-dependencies] for the given extra. Cannot be used with-R
. -
-t
: Include dependencies for the default tox environment. Cannot be used with-R
. -
-e ENV
: Include dependencies for the given tox environment, and save theENV
name as%{toxenv}
. Cannot be used with-R
. Multiple comma separated values can be given, for example:%pyproject_buildrequires -e %{toxenv}-unit,%{toxenv}-integration
-
-
%pyproject_wheel
Build the package. Commonly, this is the only macro needed in the
%build
section.This macro needs BuildRequires generated by
%pyproject_buildrequires
.
-
%pyproject_install
Install the package built by
%pyproject_wheel
. Calls%py3_shebang_fix %{_buildroot}%{_bindir}/*
.This macro needs BuildRequires generated by
%pyproject_buildrequires
.
-
%pyproject_save_files MODNAME …
Generate a list of files corresponding to the given importable module(s) and save it as
%{pyproject_files}
.Note that README file is not included. The LICENSE file is included when it is specified in the metadata. Also, while the macro allows including executable and other files (using the
+auto
flag), this feature MUST NOT be used in Fedora.The
MODNAME
may be a glob pattern, which should be specific to your package. To prevent Shell from expanding the globs, put them in''
, e.g.%pyproject_save_files '*pytest'
. As mentioned in the Explicit lists section, expressions like%pyproject_save_files '*'
are not acceptable.The macro has these options:
-
-l
: Declare that a missing license should terminate the build. Packagers are encouraged to use this flag when the%license file
is not manually listed in%files
to avoid accidentally losing the file in a future version. -
-L
: Explicitly disable the check for a missing license file. When the%license
file is manually listed in%files
, packagers can use this flag to ensure future compatibility in case the-l
behavior eventually becomes a default.
-
-
%{pyproject_files}
Path of the file written by
%pyproject_save_files
, to be used as:%files -n python3-DISTNAME -f %{pyproject_files}
Test macros
-
%tox
Run tests using
tox
.This macro needs BuildRequires generated by the
-t
or-e
option of the%pyproject_buildrequires
macro.Different environments may be specified with
-e
, for example:%check %tox %{?with_integration_tests:-e %{toxenv},%{toxenv}-integration}
Flags for the
tox
command can be specified after--
:%tox -- --parallel 0
Additional arguments for the test runner may be specified after another
--
:%tox -- --parallel 0 -- --verbose tests/*
-
%{toxenv}
The tox environment(s) used by the
%tox
macro. Multiple environments are separated by commas. Can be overridden manually or with%pyproject_buildrequires -t ENV1,ENV2
.
-
%{default_toxenv}
(e.g.py39
)The system-wide default value of
%{toxenv}
.
-
%pytest
Run
%__pytest
with environment variables appropriate for tests in%check
. See Running tests for details.
-
%__pytest
(/usr/bin/pytest
)The command that
%pytest
uses. May be redefined.
-
%py3_test_envvars
(PATH=... PYTHONPATH=... PYTHONDONTWRITEBYTECODE=1 ...
)The environment variables used by
%pytest
and%tox
. It may be used to invoke custom test runners in%check
. See Other test runners for details. Introduced in Fedora Linux 38.
-
%py3_check_import
Imports all provided modules. If running an upstream test suite is not feasible, use this macro in
%check
to test that public Python modules are importable.Takes these arguments:
-
-f
: path to file containing qualified module names (separated by newlines). Optional, can be used multiple times. -
-e
: glob to exclude the matching module names. Optional, can be used multiple times. -
-t
: if set, import only top-level module names -
Positional arguments (separated by spaces or commas) specify the module name(s) to check.
The macro sets various environment variables such as
PATH
andPYTHONPATH
to ensure the packaged versions of modules are imported. -
-
%pyproject_check_import
Imports all public modules found by the
%pyproject_save_files
macro whose names match any of the providedMODNAME
globs.This macro needs to be used with
%pyproject_save_files
(use%py3_check_import
in other cases).The macro takes
-e
/-t
as well as positional arguments for%py3_check_import
above.
Extras macros
-
%pyproject_extras_subpkg
Generates a simple subpackage for a Python extra. See Extras for more information.
This macro needs to be used with
%pyproject_install
(use%python_extras_subpkg
in other cases).Required arguments:
-
-n
: name of the “base” package (e.g.python3-requests
) -
Positional arguments (separated by spaces or commas): the extra name(s). Multiple metapackages are generated when multiple names are provided.
The macro also takes
-i
/-f
/-F
arguments for%python_extras_subpkg
below, but if they are not given, a filelist written by%pyproject_install
is used.Similarly, the
-a
/-A
flags are passed to%python_extras_subpkg
.This macro generates all the subpackage definition sections (
%package
including theSummary
andRequires
on the base package,%description
and, by default,%files
). Hence, it cannot be extended with custom Provides/Obsoletes/Requires/etc. This macro is designed to fit only the most common uses. For more complicated uses, construct the subpackage manually as shown in the Extras section.The
%files
section is last. It can be continued to add files that only make sense with the extra and the base package does not fail without them. For example, the following macro will package the extracli
for the projecta-cool-tool
and include ana-cool-tool
command:%pyproject_extras_subpkg -n a-cool-tool cli %{_bindir}/a-cool-tool
Due to technical limitations, the macro never generates requirements on the arched
BASE_PACKAGE%{?_isa} = %{?epoch:%{epoch}:}%{version}-%{release}
. It only addsRequires: BASE_PACKAGE = %{?epoch:%{epoch}:}%{version}-%{release})
because a macro cannot reliably detect if the subpackage is arched or not. So far, this has not been a problem in practice. -
-
%python_extras_subpkg
Generates a simple subpackage for a Python extra. See Extras for more information. Takes these arguments:
-
-n
: name of the “base” package (e.g.python3-requests
) -
-i
: the%files %ghost
path (glob) to the.dist-info
directory -
Positional arguments (separated by spaces or commas) specify the extra name(s) — multiple metapackages are generated when multiple names are provided.
-
-f
: Relative path to the filelist for this metapackage (which should contain the%files %ghost
path (glob) to the the metadata directory). Conflicts with-i
and-F
. -
-F
: Skip the %files section entirely (if the packager wants to construct it manually). Conflicts with-i
and-f
. -
-a
: IncludeBuildArch: noarch
in the package definition, to be used only when the package is archful, but the “base” package passed to-n
is not. -
-A
: Explicitly disables-a
(does nothing at the moment).
As with
%pyproject_extras_subpkg
:-
This macro generates all the subpackage definition sections, with only
%files
being customizable. For more complicated uses, construct the subpackage manually as shown in the Extras section. -
It never generates requirements on the arched
BASE_PACKAGE%{?_isa} = %{?epoch:%{epoch}:}%{version}-%{release}
.
-
Manual generation
The following macros are available for cases where automatic generation is turned off. They can also be useful for handling files in non-standard locations where the generators don’t look.
-
%pycached MODNAME.py
Given a Python file, lists the file and the files with its bytecode cache. See Source files and bytecode cache for more information.
-
%py_provides python3-MODNAME
Generates
Provides
forpython3-MODNAME
,python3.X-MODNAME
andpython-MODNAME
. See Automatic python- and python3.X- provides for more details.
-
%py_byte_compile INTERPRETER PATH
Byte-compile a Python file into a
__pycache__/*.pyc
.If the
PATH
argument is a directory, the macro will recursively byte compile all*.py
files in the directory. (So, if you need to compile files not named*.py
, you need to use the macro on each file separately.)The
INTERPRETER
determines the compiled file name’s suffix and the magic number embedded in the file. These muct match the interpreter that will import the file. Usually, theINTERPRETER
should be set to%{python3}
. If you are compiling for a non-default interpreter, use that interpreter instead and add aBuildRequires
line for it.
-
%{py_dist_name PROJECTNAME}
Given a project name (e.g.
PyYAML
) it will convert it to the canonical format (e.g.pyyaml
). See Canonical project name for more information.
-
%{py3_dist PROJECTNAME …}
Given one or more project names, it will convert them to the canonical format and evaluate to
python3dist(DISTNAME)
, which is useful when listing dependencies. See Machine-readable provides for more information.
System Settings
The following macros can be redefined for special use cases.
-
%{__python}
(errors by default if not redefined)Defining this macro sets the meaning of all “unversioned” Python macros such as
%{python}
or%{python_sitelib}
. Don’t use these macros without redefining%{__python}
.
-
%{__python3}
(/usr/bin/python3
)The python 3 interpreter. Redefining this macro changes all the
%{python3...}
macros, e.g.%{python3}
or%{python3_sitelib}
.
-
%{python3_pkgversion}
(3
)Distro-wide Python version, i.e. the
3
inpython3
. Projects that build on top of Fedora might define it to e.g.3.9
to try allowing multiple Python stacks installable in parallel. Packages in Fedora MAY use it (e.g. in package names:python%{python3_pkgversion}-requests
), but MUST NOT redefine it.
Comparing Python versions
When comparing Python versions
(e.g. to ask: is %{python3_version}
greater than 3.8?),
using naïve %if %{python3_version} > 3.8
or %if "%{python3_version}" > "3.8"
is not possible,
because the comparison is performed alphabetically on strings.
Hence it is true that "3.10" < "3.8"
(which is not desired).
It is possible to explicitly compare version literals by using the v
prefix,
similar to the Python string prefixes:
%if v"0%{?python3_version}" > v"3.8"
...
%endif
As a workaround for compatibility with RPM releases before 4.16 (Fedora 33),
This will work with Python 3.10 (310 > 39), but eventually break with Python 4.0 (40 < 310). |
Disabling automation
The following macros can turn off Python-specific automation.
Consider contacting the Python SIG if you need to do this.
-
%{?python_disable_dependency_generator}
Disables the automatic dependency generator. See Automatically generated dependencies for details.
-
%undefine __pythonname_provides
Disables automatic generation of unversioned/versioned provides for package names, e.g.
python-FOO
andpython3.9-FOO
forpython3-foo
. See Automatic python- and python3.X- provides for more details.
-
%undefine __pythondist_provides
Disables automatic generation of machine-readable Provides, e.g.
python3dist(foo)
. See Machine-readable provides for more details.
-
%global _python_no_extras_requires 1
If defined, Automatic Requires for extras will not be generated.
-
%global _python_dist_allow_version_zero 1
From Fedora Linux 38 on, it is no longer possible to build a Python package with version 0 to prevent an accidental loss of the actual version information. If defined, the macro will allow to build such package.
Deprecated Macros
The following macros are deprecated. See the 201x-era Python Packaging guidelines for how some of them were used.
Want to help? Learn how to contribute to Fedora Docs ›