Java Specifics in Fedora for Packagers

Directory Layout

This section describes most of directories used for Java packaging. Each directory is named in RPM macro form, which shows how it should be used in RPM spec files. Symbolic name is followed by usual macro expansion (i.e. physical directory location in the file system) and short description.

Directories commonly used by regular packages
%{_javadir} — /usr/share/java

Directory that holds all JAR files that do not contain or use native code and do not depend on a particular Java standard version. JAR files can either be placed directly in this directory or one of its subdirectories. Often packages create their own subdirectories there, in this case subdirectory name should match package name.

%{_jnidir} — /usr/lib/java

Directory where architecture-specific JAR files are installed. In particular, JAR files containing or using native code (Java Native Interface, JNI) should be installed there.

%{_javadocdir} — /usr/share/javadoc

Root directory where all Java API documentation (Javadoc) is installed. Each source package usually creates a single subdirectory containing aggregated Javadocs for all binary packages it produces.

%{_mavenpomdir} — /usr/share/maven-poms

Directory where Project Object Model (POM) files used by Apache Maven are installed. Each POM must have name that strictly corresponds to JAR file in %{_javadir} or %{_jnidir}.

%{_ivyxmldir} — /usr/share/ivy-xmls

Directory where ivy.xml files used by Apache Ivy are installed. Each XML must have name that strictly corresponds to JAR file in %{_javadir} or %{_jnidir}.

Other directories
%{_jvmdir} — /usr/lib/jvm

Root directory where different Java Virtual Machines (JVM) are installed. Each JVM creates a subdirectory, possibly with several alternative names implemented with symbolic links. Directories prefixed with java contain Java Development Kit (JDK), while directories which names start with jre hold Java Runtime Environment (JRE).

%{_jvmsysconfdir} — /etc/jvm
%{_jvmcommonsysconfdir} — /etc/jvm-common

Directories containing configuration files for Java Virtual Machines (JVM).

%{_jvmprivdir} — /usr/lib/jvm-private
%{_jvmlibdir} — /usr/lib/jvm
%{_jvmdatadir} — /usr/share/jvm
%{_jvmcommonlibdir} — /usr/lib/jvm-common
%{_jvmcommondatadir} — /usr/share/jvm-common

Directories containing implementation files of Java Virtual Machines (JVM). Describing them in detail is out of scope for this document. Purpose of each directory is commented briefly in macros.jpackage file in /etc/rpm. More detailed description can be found in JPackage policy.

%{_javaconfdir} — /etc/java

Directory containing Java configuration files. In particular it contains main Java configuration file — java.conf.

JAR File Identification

Complex Java applications usually consist of multiple components. Each component can have multiple implementations, called artifacts. Artifacts in Java context are usually JAR files, but can also be WAR files or any other kind of file.

There are multiple incompatible ways of identifying (naming) Java artifacts and each build system often encourages usage of specific naming scheme. This means that Linux distributions also need to allow each artifact to be located using several different identifiers, possible using different schemes. On the other hand it is virtually impossible to every naming scheme, so there are some simplifications.

This chapter describes artifact different ways to identify and locate artifacts in system repository.

Relative paths

JAR artifacts are installed in one of standard directory trees. Usually this is either %{_javadir} (/usr/share/java) or %{_jnidir} (/usr/lib/java).

The simplest way of identifying artifacts is using their relative path from one of standard locations. All artifact can be identified this way because each artifacts has a unique file name. Each path identifying artifact will be called artifact path in this document.

To keep artifact paths simpler and more readable, extension can be omitted if it is equal to jar. For non-JAR artifacts extension cannot be omitted and must be retained.

Additionally, if artifact path points to a directory then it represents all artifacts contained in this directory. This allows a whole set of related artifacts to be referenced easily by specifying directory name containing all of them.

If the same artifact path has valid expansions in two different root directories then it is unspecified which artifacts will be located.

Artifact specification

As noted in previous section, every artifact can be uniquely identified by its file path. However this is not always the preferred way of artifact identification.

Modern Java build systems provide a way of identifying artifacts with an abstract identifier, or more often, a pair of identifiers. The first if usually called group ID or organization ID while the second is just artifact ID. This pair of identifiers will be called artifact coordinates in this document. Besides group ID and artifact ID, artifact coordinates may also include other optional information about artifact, such as extension, classifier and version.

In Linux distributions it is important to stay close to upstreams providing software being packaged, so the ability to identify artifacts in the same way as upstream does is very important from the packaging point of view. Every artifact can optionally be identified by artifact coordinates assigned during package build. Packages built with Maven automatically use this feature, but all other packages, even these built with pure javac, can use this feature too (see description of %mvn_artifact and %add_maven_depmap macros).

Aliases

Aliases working in two ways:

  • Symlinks for paths

  • Additional mappings for artifact specifications

In the real world the same project can appear under different names as it was evolving or released differently. Therefore other projects may refer to those alternative names instead of using the name currently prefered by upstream.

Artifact aliases

XMvn provides a way to attach multiple artifact coordinates to a single artifact. Dependent projects that use alternative coordinates can then be built without the need to patch their POMs or alter the build by other means. It will also generate virtual provides for the alias, so it can be also used in Requires and BuildRequires. Creating an alias is achieved by %mvn_alias macro.

Example invocation
# com.example.foo:bar (the actual artifact existing in the project) will also
# be available as com.example.foo:bar-all
%mvn_alias com.example.foo:bar com.example.foo:bar-all

# You don't need to repeat the part of coordinates that stays the same
# (groupID in this case)
%mvn_alias com.example.foo:bar :bar-all

# You can specify multiple aliases at once
%mvn_alias com.example.foo:bar :bar-all :bar-lib

# The macro supports several shortcuts to generate multiple alisaes.
# Braces - {} - capture their content, which can then be referenced in the
# alias part with @N, where N is the index of the capture group.
# * acts as a wildcard (matching anything)
# The following generates aliases ending with shaded for all artifacts in the
# project
%mvn_alias 'com.example.foo:{*}' :@1-shaded

Compatibility versions

Handling of compatibility packages, versioned jars etc.

In Fedora we prefer to always have only the latest version of a given project. Unfortunately, this is not always possible as some projects change too much and it would be too hard to port dependent packages to the current version. It is not possible to just update the package and keep the old version around as the names, file paths and dependency provides would clash. The recommended practice is to update the current package to the new version and create new package representing the old version (called compat package). The compat package needs to have the version number (usually only the major number, unless further distinction is necessary) appended to the name, thus effectivelly having different name from RPM’s point of view. Such compat package needs to perform some additional steps to ensure that it can be installed and used along the non-compat one.

You should always evaluate whether creating a compat package is really necessary. Porting dependent projects to new versions of dependencies may be a complicated task, but your effort would be appreciated and it is likely that the patch will be accepted upstream at some point in time. If the upstream is already inactive and the package is not required by anything, you should also consider retiring it.

Maven Compat Versions

XMvn supports marking particular artifact as compat, performing the necessary steps to avoid clashes with the non-compat version. An artifact can be marked as compat by %mvn_compat_version. It accepts an artifact argument which will determine which artifact will be compat. The format for specifying artifact coordinates is the same as with %mvn_alias. In the common case you will want to mark all artifacts as compat. You can specify multiple compat versions at a time.

Dependency resolution of compat artifacts

When XMvn performs dependency resolution for a dependency artifact in a project, it checks the dependency version and compares it against all versions of the artifact installed in the buildroot. If none of the compat artifacts matches it will resolve the artifact to the non-compat one. This has a few implications:

  • The versions are compared for exact match. The compat package should provide all applicable versions that are present in packages that are supposed to be used with this version.

  • The dependent packages need to have correct BuildRequires on the compat package as the virtual provides is also different (see below).

File names and virtual provides

In order to prevent file name clashes, compat artifacts have the first specified compat version appended to the filename. Virtual provides for compat artifacts also contain the version as the last part of the coordinates. There are multiple provides for each specified compat version. Non-compat artifact do not have any version in the virtual provides.

Example invocation of %mvn_compat_version
# Assuming the package has name bar and version 3
# Sets the compat version of foo:bar artifact to 3
%mvn_compat_version foo:bar 3
# The installed artifact file (assuming it's jar and there were no
# %mvn_file calls) will be at %{_javadir}/bar/bar-3.jar
# The generated provides for foo:bar will be
# mvn(foo:bar:3) = 3
# mvn(foo:bar:pom:3) = 3

# Sets the compat versions of all artifacts in the build to 3 and 3.2
%mvn_compat_version : 3 3.2

Dependency Handling

RPM has multiple types of metadata to describe dependency relationships between packages. The two basic types are Requires and Provides. Requires denote that a package needs something to be present at runtime to work correctly and the package manager is supposed to ensure that requires are met. A single Requires item can specify a package or a virtual provide. RPM Provides are a way to express that a package provides certain capability that other packages might need. In case of Maven packages, the Provides are used to denote that a package contains certain Maven artifact. They add more flexibility to the dependency management as single package can have any number of provides, and they can be moved across different packages without breaking other packages' requires. Provides are usually generated by automatic tools based on the information from the built binaries or package source.

Dependency handling for Maven packages

The Java packaging tooling on Fedora provides automatic Requires and Provides generation for packages built using XMvn. The Provides are based on Maven artifact coordinates of artifacts that were installed by the currently being built. They are generated for each subpackage separately. They follow a general format mvn(groupId:artifactId:extension:classifier:version), where the extension is omitted if its jar and classifier is omitted if empty. Version is present only for compat artifacts, but the trailing colon has to be present unless it is a Jar artifact with no classifier.

# Example provide for Jar artifact
mvn(org.eclipse.jetty:jetty-server)
# Example provide for POM artifact
mvn(org.eclipse.jetty:jetty-parent:pom:)
# Example provide for Jar artifact with classifier
mvn(org.sonatype.sisu:sisu-guice::no_aop:)

The generated Requires are based on dependencies specified in Maven POMs in the project. Only dependencies with scope set to either compile, runtime or not set at all are used for Requires generation. Requires do not rely on package names and instead always use virtual provides that were described above, in exactly the same format, in order to be satisfiable by the already existing provides. For packages consisting of multiple subpackages, Requires are generated separately for each subpackage. Additionally, Requires that point to an artifact in a different subpackage of the same source package are generated with exact versions to prevent version mismatches between artifacts belonging to the same project.

The requires generator also always generates Requires on java-headless and javapackages-tools.

Dependency handling for non-Maven packages that ship POM files

If the package is built built using different tool than Apache Maven, but still ships Maven POM(s), the you will still get automatic provides generation if you install the POM using %mvn_artifact and %mvn_install. The requires generation will also be executed but the outcome largely depends on whether the POM contains accurate dependency insformation. If it contains dependency information, you should double-check that it is correct and up-to-date. Otherwise you need to add Requires tags manually as described in the next section.

Dependency handling for non-Maven packages that don’t ship POM files

For packages without POMs it is necessary to specify Requires tags manually. In order to build the package you needed to specify BuildRequires tags. Your Requires tags will therefore likely be a subset of your BuildRequires, excluding build tools and test only dependencies.

Querying Requires and Provides of built packages

The generated Requires and Provides of built packages can be queried using rpm:

rpm -qp --provides path/to/example-1.0.noarch.rpm
rpm -qp --requires path/to/example-1.0.noarch.rpm
Generating BuildRequires

While Requires and Provides generation is automated for Maven projects, BuildRequires still remains a manual task. However, there are tools to simplify it to some extent. XMvn ships a script xmvn-builddep that takes a build.log output from mock and prints Maven-style BuildRequires on artifacts that were actually used during the build. It does not help you to figure out what the BuildRequires are before you actually build it, but it may help you to have a minimal set of BuildRequires that are less likely to break, as they do not rely on transitive dependencies.

Javadoc packages

Javadoc subpackages in Fedora provide automatically generated API documentation for Java libraries and applications. Java Development Kit comes with tool called javadoc. This tool can be used for generating the documentation from specially formated comments in Java source files. Output of this tool, together with license files, usually represents only content of javadoc subpackages. Note javadoc invocation is typically handled by build system and package maintainer does not need to deal with it directly.

Javadoc subpackage shouldn’t depend on its base package and vice versa. The rationale behind this rule is that documentation can usually be used independently from application / library and therefore base package does not need to be always installed. Users are given an option to install application and documentation separately.

You can learn more about javadoc from official documentation.

Core Java packages

JVM

Fedora allows multiple Java Virtual Machines (JVMs) to be packaged independently. Java packages should not directly depend on any particulat JVM, but instead require one of three virtual JVM packages depending of what Java funtionality is required.

java-headless

This package provides a working Java Runtime Environment (JRE) with some functionality disabled. Graphics and audio support may be unavailable in this case. java-headless provides functionality that is enough for most of packages and avoids pulling in a number of graphics and audio libraries as dependencies. Requirement on java-headless is appropriate for most of Java packages.

java

Includes the same base functionality as java-headless, but also implements audio and graphics subsystems. Packages should require java if they need some functionality from these subsystems, for example creating GUI using AWT library.

java-devel

Provides full Java Development Kit (JDK). In most cases only packages related to Java development should have runtime dependencies on java-devel. Runtime packages should require java-headless or java. Some packages not strictly related to java development need access to libraries included with JDK, but not with JRE (for example tools.jar). That is one of few cases where requiring java-devel may be necessary.

Packages that require minimal Java standard version can add versioned dependencies on one of virtual packages providing Java environment. For example if packages depending on functionality of JDK 8 can require java-headless >= 1:1.8.0.

Epoch in versions of JVM packages

For compatibility with JPackage project packages providing Java 1.6.0 or later use epoch equal to 1. This was necessary because package java-1.5.0-ibm from JPackage project had epoch 1 for some reason. Therefore packages providing other implementations of JVM also had to use non-zero epoch in order to keep version ordering correct.

Java Packages Tools

Java Packages Tools are packaged as several binary RPM packages:

maven-local

This package provides a complete environment which is required to build Java packages using Apache Maven build system. This includes a default system version of Java Development Kit (JDK), Maven, a number of Maven plugins commonly used to build packages, various macros and utlilty tools. maven-local is usually declared as build dependency of Maven packages.

ivy-local

Analogously to maven-local, this package provides an environment required to build Java packages using Apache Ivy as dependency manager.

javapackages-local

Package providing a basic environment necessary to geterate and install metadata for system artifact repository.

javapackages-tools

Package owning basic Java directories and providing runtime support for Java packages. The great majority of Java packages depend on javapackages-tools.