Upgrade Paths in Modularity
Since modularity consists of modules and packages, there two levels of upgrade.
When DNF have multiple module builds of a stream available in the repositories, it will first separate the builds by a module context. Then in each context DNF will sort the builds by a version. Finally DNF will pick as an upgrade the highest version of a context which matches the currently installed context.
There are two algorithms for separating modules by a context depending on a type of the context:
Module builds with
static_context: true keyword are called static-context.
A value of
context field is the only factor considered when establishing an upgrade path.
For instance, having installed
foo:stream:0:A which requires
bar:x and the following four static-context modules in a repository:
Upgrade path A Upgrade path B ──────────────── ───────────────── foo:stream:2:A foo:stream:2:B ╮ requires bar:y requires: bar:z │ in foo:stream:1:A <┐ foo:stream:1:B │ a repository requires bar:x │ requires: bar:z ╯ foo:stream:0:A ┘ ╮ already requires bar:x ╯ installed
DNF will create two lineages
foo:stream is already installed with a static context
foo:stream:2:A cannot be installed because it conflicts with an already enabled stream
and will upgrade to
foo:stream:1:A because version
1 is the highest number which has met run-time dependencies.
Module builds without
static_context: true keyword are called dynamic-context (or nonstatic).
Establishing an upgrade path actually does not consider the context value, but run-time dependencies of the modules.
This algorithm is deprecated because it’s cumbersome, does not support changing run-time dependencies, and adding and removing contexts.
For instance, having installed
foo:stream:0:Z which requires
bar:x and the following four dynamic-context modules in a repository:
Upgrade path 1 Upgrade path 2 ──────────────── ──────────────── foo:stream:2:A <┐ foo:stream:2:C ╮ requires bar:x │ requires bar:y │ in foo:stream:1:B │ foo:stream:1:D │ a repository requires bar:x │ requires bar:y ╯ foo:stream:0:Z ┘ ╮ already requires bar:x ╯ installed
DNF will create two lineages
2, and will upgrade to
This is because
foo:stream:2:A has the highest version in a lineage whose modules require
bar:x as the already installed
Observe that with the dynamic context a module maintainer cannot express that
foo:stream:1:B should upgrade to
foo:stream:2:C if the maintainer decided to change a dependency from
bar:y and were unable to build
foo:stream:2:A (e.g. an upstream rewrote the application from Ruby 2.4 to Ruby 2.6).
Once DNF identifies the latest module, all modular metadata defined in that module build will apply and replace modular metadata occurring in the older module versions.
That includes a module description shown with
dnf module list command, a list of packages installed when installing a profile, or a list of demodularized packages.
The only data which are used from older versions of (a context of) the stream are RPM packages. That enables users to downgrade a package in case the newer module build contained a bug.
For the reasons explained above, if you want to add a new context (e.g. you want to add a support for a new Perl 5.32 to your perl-DBI module), you need to use a static context:
Upgrade path A Upgrade path B ──────────────────── ──────────────────── perl-DBI:stream:2:A ↑ perl-DBI:stream:2:B ↑ requires perl:5.30 ↑ requires perl:5.32 ↑ perl-DBI:stream:1:A ↑ requires perl:5.30 ↑ ╰──────────────────╯ ╰──────────────────╯ Old context New context
It’s not possible to achieve this by standard means with a dynamic context. DNF would report a warning or an error. If you ever needed to achieve this with nonstatic contexts, the key point how to indulge DNF is fill a matrix of contexts and versions with artificial modules which have never been built (i.e. manually writing the modulemd documents):
Upgrade path 1 Upgrade path 2 ──────────────────── ──────────────────── perl-DBI:stream:2:A ↑ perl-DBI:stream:2:B ↑ requires perl:5.30 ↑ requires perl:5.32 ↑ ↑ ┌────────────────────┐ perl-DBI:stream:1:A ↑ │perl-DBI:stream:1:B │ An artificial module build requires perl:5.30 ↑ │ requires perl:5.32│ filling a gap in this matrix └────────────────────┘
On the other hand, with a static context DNF will understand that modules from context
B should not mix with module from a context
A. When installing
perl-DBI:stream for the first time, DNF will pick the right context based on an already installed
perl stream. When upgrading
perl-DBI:stream, DNF will preserve the context based on an already installed
perl-DBI:stream context. Thus using static-context modules is recommended.
The same constrains of dynamic context and a strong recommendation for static context apply here. With static context you can stop building modules for that context whenever you want. With dynamic context you cannot and need to maintain the full matrix forever.
With a static context it’s possible to use a
modulemd-obsoletes document to switch to a completely different stream.
This is useful when you want to stop supporting an existing stream, or a context, and you provide a replacement.
When user has installed an obsolete stream, or a context, DNF will offer switching to the replacement.
Upgrade path A Upgrade path B ──────────────────────── ────────────── ┌> perl:5.32:1:B perl:5.30 │ obsoleted by perl:5.32 │ perl:5.30:1:A ┘
This feature is currently experimental and needs to be enabled in DNF
On the package level, upgrades of a modular system work the same way as on a traditional system — using NEVRA comparison to determine which packages are the newest. There is, however, one additional step right before the NEVRA comparison that needs to happen on a modular system — limiting which packages are going to be part of the comparison based on what modules are enabled — and that is the key difference.
There are up to three classes of RPM packages available to a modular system:
Nonmodular packages — packages not being part of a module. In Fedora, these are coming from the Everything repository.
Modular packages — packages being part of a module. In Fedora, these are coming from the Modular, Modular Updates, and Modular Test Updates repositories.
Hotfixes — nonmodular packages created on-demand by users or vendors meant to fix a critical issue before an official upgrade comes from the distribution. These need to be provided in a separate repository with a hotfix flag set. Fedora doesn’t provide such packages.
To determine the limited set of packages for the NEVRA comparison, the following algorithm is used:
For all enabled and default streams:
Collect all modular packages, i.e. NEVRAs, that are part of an active context of that stream and add them to a pile of available packages for DNF. To do this, look at
artifactssection of all module versions belonging to the same context of the stream. That includes a currently installed version, new versions coming into Modular Updates repository, as well as historical versions already presented in the Modular repository.
Add names of these packages onto a list of modularly filtered names with an exception of names recorded as removed from a module in the latest version of the module. This list of modularly filtered names will be used for preferring modular packages over nonmodular ones.
Add all nonmodular packages whose name is not on the list to the pile of available packages. This step ensures that modular packages have always a higher priority than nonmodular ones. In other words, nonmodular packages can never upgrade modular packages unless explicitly unlisted by the stream.
Add all hotfix packages to the pile. These are just added, exempted from the modular filtering. That means hotfixes can potentially upgrade both traditional and modular packages.
The next step is to take this pile of packages, and run a NEVRA comparison to determine the highest version of each package. The highest versions are then installed as a part of the upgrade process.
A system has enabled streams
loo:1; a repository contains:
Package From a module Requires a module ────────────────────────────────────────── foo-1 ← a nonmodular package foo-2 bar:1:2022:a loo:1 ╮ foo-3 bar:1:2023:a loo:1 │ modular foo-4 bar:1:2023:b loo:2 │ packages foo-5 bar:2:2023:a ╯ loo:1:2000:c ╮ empty loo:2:2000:c ╯ modules
dnf install foo command will install
A reasoning is following: Active contexts are
(It is not
loo:1 is enabled.)
Hence packages available to DNF are
(It is not
foo-1 because it’s nonmodular and has the same name as modular
It is not
bar:1::b has an unmet modular dependency.
It is not
bar:2 is not enabled.)
Highest NEVRA among the two available packages is
foo-3. Therefore DNF will
Next day a new update appears:
Package From a module Requires a module Unlisted package ──────────────────────────────────────────────────────────── bar:1:2024:a loo:1 ╮ empty bar:1:2024:a foo ╯ modules
dnf upgrade foo command will not upgrade anything, reporting
foo-3 is already latest and installed.
A reasoning is that the pile of available packages contains
foo-3 is already installed.
Please observe that
foo-1 package became available because
foo is unlisted name now.
However, unlisting that name does not disappear the modular packages
Modular and and nonmodular
foo packages are now handled equally.
Another day a next update appears:
Package From a module ────────────────────── foo-6 ← a nonmodular package
dnf upgrade foo command will upgrade to
A reasoning is that
foo-6 is the highest one from a pile of available packages among
This demonstrates that a successful demodularization of a package requires both unlisting that package name and building that package, this time out of any module, with a new, highest version.