On Thu, Oct 7, 2021 at 11:08 AM Mat Booth fedora@matbooth.co.uk wrote:

On Thu, 7 Oct 2021 at 07:34, Mikolaj Izdebski mizdebsk@redhat.com wrote:

...

Hello,

I want to propose updating Java Packaging Guidelines [1] to allow installation of JNI shared objects in %{_prefix}/lib/%{name} or in %{_libdir}/%{name}, according to packager preference.

%{_libdir} expands to either /usr/lib or /usr/lib64 (depending on system architecture), while %{_prefix}/lib always expands to /usr/lib. To see what %{_libdir} expands to on each platform, you can run: grep %_libdir -r /usr/lib/rpm/platform/

Currently Java Packaging Guidelines require all packages to install JNI shared objects in %{_libdir}/%{name}, which causes issues for noarch packages that want to refer to such shared objects. For example Maven needs to install a symlink to libjansi.so. Since Maven is a noarch package, the symlink must technically point to one fixed location, but libjansi.so ought to be installed in different locations on different architectures. Moving libjansi.so to one location across all architectures solves this problem. An alternative solution involves usage of a ghost symlink manipulated by scriplets, which in my opinion is an inferior solution.

Java has been granted multilib exemption by FESCo [2], therefore I would like to allow installation of JNI shared objects in %{_prefix}/lib/%{name} or %{_libdir}/%{name}.

Does anyone have any objections or feedback before I go with this request to FPC?

[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/Java/ [2] https://pagure.io/fesco/issue/961

Could it not be argued that JNI objects should *always* be in %{_prefix}/lib because Java itself is not multilib-aware?.

Indeed, myself I would always put them in %{_prefix}/lib in most (if not all) cases, but there may be reasons for putting them in %{_libdir}, for example when JNI code is only part of otherwise multilib-aware package and you want to avoid file conflicts.

-- Mikolaj

-- Mat Booth http://fedoraproject.org/get-fedora

On Fri, Oct 8, 2021 at 1:23 PM Florian Weimer fweimer@redhat.com wrote:

• Mikolaj Izdebski:

...

I want to propose updating Java Packaging Guidelines [1] to allow installation of JNI shared objects in %{_prefix}/lib/%{name} or in %{_libdir}/%{name}, according to packager preference.

%{_libdir} expands to either /usr/lib or /usr/lib64 (depending on system architecture), while %{_prefix}/lib always expands to /usr/lib. To see what %{_libdir} expands to on each platform, you can run: grep %_libdir -r /usr/lib/rpm/platform/

Isn't this a consequence of using absolute paths for JNI loading?

The issue goes away if the system default or OpenJDK default paths are used. Neither of these paths include %{name}, of course.

JNI shared objects are internal DSOs, which are expected to be loaded and called only by corresponding Java bytecode. As such they should be placed outside of dynamic linker search path, in a subdirectory called %{name}. OpenJDK does not have code to load such DSOs. The Java counterpart of particular JNI DSO typically contains specific code to search for the DSO and load it.

A typical Java application is installed into a single directory tree containing everything needed to run that application - Java bytecode, native libraries, configuration files, data etc. Java applications packaged as Fedora RPM packages install each of these into different system directories - bytecode in /usr/share/java, native libraries in /usr/lib or /usr/lib64, configuration files in /etc, data in /var and so on. The single upstream directory is then created in /usr/share/%{name}, but instead of regular files there are symbolic links pointing to the actual location where the files are installed.

...

Since Maven is a noarch package, the symlink must technically point to one fixed location, but libjansi.so ought to be installed in different locations on different architectures.

This also suggests a different problem: Why does Maven as a noarch package even have to know that libjansi.so exists? A future version of jansi could very well be noarch. I suspect it's related to dependencies bubbling up to consumers (which is related to the Class-Path: attribute ban).

This is the way Maven is distributed and installed in binary form - with all dependencies bundled in a single directory tree. Fedora maven package needs to recreate this directory structure, but separate native code from Java bytecode. Symbolic links are used to achieve this split. And indeed, the latest Maven 3.8.3 (not yet in Fedora) does not include libjansi for Linux, only for Windows.

-- Mikolaj Izdebski

On Mon, Oct 11, 2021 at 8:50 AM Florian Weimer fw@deneb.enyo.de wrote:

• Mikolaj Izdebski:

...

On Fri, Oct 8, 2021 at 1:23 PM Florian Weimer fweimer@redhat.com wrote:

...

• Mikolaj Izdebski:

...

I want to propose updating Java Packaging Guidelines [1] to allow installation of JNI shared objects in %{_prefix}/lib/%{name} or in %{_libdir}/%{name}, according to packager preference.

%{_libdir} expands to either /usr/lib or /usr/lib64 (depending on system architecture), while %{_prefix}/lib always expands to /usr/lib. To see what %{_libdir} expands to on each platform, you can run: grep %_libdir -r /usr/lib/rpm/platform/

Isn't this a consequence of using absolute paths for JNI loading?

The issue goes away if the system default or OpenJDK default paths are used. Neither of these paths include %{name}, of course.

JNI shared objects are internal DSOs, which are expected to be loaded and called only by corresponding Java bytecode. As such they should be placed outside of dynamic linker search path, in a subdirectory called %{name}. OpenJDK does not have code to load such DSOs.

I believe OpenJDK as packaged for Fedora tries to load from /usr/java/packages/lib. Unfortunately, this path has been somewhat unstable (1.8.0 used /usr/java/packages/lib/amd64, it seems, and I've also seen /usr/lib/jni and other variants), but that should not matter for current Fedora Java packaging because 11 and 17 use the same path.

Furthermore, in the current glibc implementation, private shared objects do not really exist because the dynamic loader tries too hard to avoid loading objects with the same soname. So even if the JVM tries to partition the C symbol namespace with RTLD_LOCAL on a per-class-loader basis, I doubt that this enables conflicting JNI libraries.

...

...

...

Since Maven is a noarch package, the symlink must technically point to one fixed location, but libjansi.so ought to be installed in different locations on different architectures.

This also suggests a different problem: Why does Maven as a noarch package even have to know that libjansi.so exists? A future version of jansi could very well be noarch. I suspect it's related to dependencies bubbling up to consumers (which is related to the Class-Path: attribute ban).

This is the way Maven is distributed and installed in binary form - with all dependencies bundled in a single directory tree.

That makes sense for installing things from a ZIP file, without a package manager.

...

Fedora maven package needs to recreate this directory structure, but separate native code from Java bytecode. Symbolic links are used to achieve this split. And indeed, the latest Maven 3.8.3 (not yet in Fedora) does not include libjansi for Linux, only for Windows.

Why do Fedora packages have to do this? I thought that Maven does not come with a module system.

Because Maven directory structure can be considered as part of its API, which other software relies upon. For example, users can configure Jenkins or Eclipse to use Maven by pointing at Maven home directory (/usr/share/maven). These tools may rely on directory layout to do various things, such as getting version of Maven by opening certain JAR files and reading embedded resources.

I think it should be possible to use consistent JAR names, do away with the symbolic links, and rely on the Class-Path attribute for linking. Doesn't Debian already do this?

That's not impossible, but upstream should make this change, not downstream Fedora maintainers. Like Fedora, Debian maven package also has /usr/share/maven directory with symlinks.

-- Mikolaj Izdebski