Can't package Quarkus

Help
1

I’m trying to package Quarkus. So I tried:

# This method is the simplest, but not the cleanest (download the whole maven dependency in one output)
# https://nixos.org/manual/nixpkgs/stable/#maven
{ stdenv, maven, lib, fetchFromGitHub }:
stdenv.mkDerivation rec {
  name = "quarkus-${version}";
  version = "2.10.1.Final";

  src = fetchFromGitHub {
    owner = "quarkusio";
    repo = "quarkus";
    rev = "${version}";
    sha256 = "sha256-OUxVkKFqAEwWQiwvVJfetHC0nEPARvYXpeu8bKSVu7M=";
  };

  buildInputs = [ maven ];

  buildPhase = ''
    export MAVEN_OPTS="-Xmx4g"
    mvn package -Dmaven.repo.local=$out -Dquickly # also tried with -Dmaven.test.skip=true -DskipTests
  '';
  
  # keep only *.{pom,jar,sha1,nbm} and delete all ephemeral files with lastModified timestamps inside
  installPhase = ''
    find $out -type f \
      -name \*.lastUpdated -or \
      -name resolver-status.properties -or \
      -name _remote.repositories \
      -delete
  '';

  # don't do any fixup
  dontFixup = true;
  outputHashAlgo = "sha256";
  outputHashMode = "recursive";
  # replace this with the correct SHA256
  outputHash = lib.fakeSha256;
}

Unfortunately the compilation fails with No such file or directory on a binary file… which makes me wonder if NixOs has not again bitten me with its purity:

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Quarkus - Parent pom 2.10.1.Final:
[INFO] 
[INFO] Quarkus - IDE/Tools - Config ....................... SUCCESS [ 59.572 s]
[INFO] Quarkus - Revapi Configuration ..................... SUCCESS [  2.980 s]
[INFO] ArC - Parent pom ................................... SUCCESS [ 16.616 s]
[INFO] ArC - Runtime ...................................... SUCCESS [  5.279 s]
[INFO] ArC - Processor .................................... SUCCESS [  5.247 s]
[INFO] ArC - Tests ........................................ SUCCESS [  1.983 s]
[INFO] Quarkus - Bootstrap - Parent ....................... SUCCESS [  0.019 s]
[INFO] Quarkus - Bootstrap - BOM .......................... SUCCESS [  0.015 s]
[INFO] Quarkus - Bootstrap - Test BOM ..................... SUCCESS [  0.014 s]
[INFO] Quarkus - Bootstrap - App Model .................... SUCCESS [  1.176 s]
[INFO] Quarkus - Bootstrap - Maven Resolver ............... SUCCESS [  4.765 s]
[INFO] Quarkus - Bootstrap - Gradle Resolver .............. SUCCESS [  8.947 s]
[INFO] Quarkus - Bootstrap - Core ......................... SUCCESS [  2.002 s]
[...]
[INFO] Quarkus - REST Client Config - Runtime ............. SUCCESS [  1.146 s]
[INFO] Quarkus - REST Client .............................. SUCCESS [  0.029 s]
[INFO] Quarkus - REST Client - Runtime .................... SUCCESS [  1.942 s]
[INFO] Quarkus - Undertow ................................. SUCCESS [  0.016 s]
[INFO] Quarkus - Undertow - Runtime ....................... SUCCESS [  1.153 s]
[INFO] Quarkus - gRPC parent .............................. SUCCESS [  0.020 s]
[INFO] Quarkus - gRPC - API ............................... SUCCESS [  0.569 s]
[INFO] Quarkus - gRPC Common .............................. SUCCESS [  0.028 s]
[INFO] Quarkus - gRPC Common - Runtime .................... SUCCESS [  1.933 s]
[INFO] Quarkus - SmallRye Stork Integration ............... SUCCESS [  0.018 s]
[INFO] Quarkus - SmallRye Stork - Runtime ................. SUCCESS [  0.728 s]
[INFO] Quarkus - gRPC - Protoc Plugin ..................... SUCCESS [  2.126 s]
[INFO] Quarkus - gRPC - Code Gen .......................... SUCCESS [  7.716 s]
[INFO] Quarkus - gRPC - Stubs for health and reflection ... FAILURE [  4.464 s]
[INFO] Quarkus - gRPC - Runtime ........................... SKIPPED
[INFO] Quarkus - Micrometer ............................... SKIPPED
[INFO] Quarkus - Micrometer - Runtime ..................... SKIPPED
[INFO] Quarkus - Micrometer Registry - Prometheus - Parent  SKIPPED
[INFO] Quarkus - Micrometer Registry - Prometheus Runtime . SKIPPED
[INFO] Quarkus - RESTEasy - Common - SPI .................. SKIPPED
[INFO] Quarkus - RESTEasy Reactive - SPI .................. SKIPPED
[INFO] Quarkus - Undertow - SPI ........................... SKIPPED
[INFO] Quarkus - JSON-P - Deployment ...................... SKIPPED
[INFO] Quarkus - RESTEasy - Common - Deployment ........... SKIPPED
[INFO] Quarkus - JAX-RS SPI - Parent ...................... SKIPPED
[INFO] Quarkus - JAX-RS - SPI - Deployment ................ SKIPPED
[...]
[INFO] Quarkus - Integration Tests - gRPC - Hibernate ..... SKIPPED
[INFO] Quarkus - Integration Tests - gRPC - Hibernate Reactive SKIPPED
[INFO] Quarkus - Integration Tests - Helper project containing packaging a proto file for grpc-external-proto-test SKIPPED
[INFO] Quarkus - Integration Tests - gRPC - External proto Test SKIPPED
[INFO] Quarkus - Integration Tests - gRPC with Stork - gathering response times SKIPPED
[INFO] Quarkus - Integration Tests - Google Cloud Functions HTTP SKIPPED
[INFO] Quarkus - Integration Tests - Google Cloud Function  SKIPPED
[INFO] Quarkus - Documentation ............................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  07:13 min
[INFO] Finished at: 2022-07-02T07:51:53Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.xolstice.maven.plugins:protobuf-maven-plugin:0.6.1:compile (compile) on project quarkus-grpc-stubs: An error occurred while invoking protoc: Error while executing process.: Cannot run program "/build/source/extensions/grpc/stubs/target/protoc-plugins/protoc-3.19.3-linux-x86_64.exe": error=2, No such file or directory -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :quarkus-grpc-stubs

any idea how to make this work?

2

If you ldd that file you will likely find that it is a pre-built version of protobuf, downloaded by maven at build time, and that it’s missing some dependencies.

Mild rant:

Using prebuilt binaries in your compilation is the absolute worst way to do builds for a variety of reasons, it breaks in pretty much all scenarios that aren’t the simplest on-dev-laptop one. It’s a shame it’s so common.

You’ll find that debian doesn’t even allow packaging stuff like this, it’s frowned upon by any system integrator for security and provenance reasons. You just notice it less because you can break debian without putting conscious effort into it.

Rant aside, on NixOS this breaks because the binary will have been built on some flavor of ubuntu, and therefore its dynamic linker interpreter path points to /usr/lib/ld-linux.so or something. NixOS’ dynamic linker is somewhere in /nix/store. Linux isn’t smart enough to untangle this, so you get a “file not found” error, because it can’t find the interpreter (at that hardcoded location).

As I said earlier, ldd will show you exactly what paths the binary is expecting (and will try to give you guesses at where those paths are on your system, but they won’t always be right or used correctly).

The dynamic linker interpreter, of course, is the binary that is responsible for figuring out where it should load dependency shared libraries from. All non-statically compiled binaries need this to run, so it’s kind of a big deal when it’s missing. It’s almost the same as if you didn’t have python installed and are trying to run a python script.

In other words, we need to either change where the binary is looking for the interpreter, or make the interpreter available where the binary is trying to find it. Or replace the binary with one that was built by us outright, sidestepping the issue and a lot of others while we’re at it. There are a couple of options:

  • Read and understand the build script. Maybe it only downloads the binary if the host doesn’t already have protoc, or maybe there is an option to use the host protoc, in which case adding protobuf to your buildInputs, and flipping whatever configuration is necessary, may be enough.
    • This is probably the best option. If no such option exists, adding one upstream is the clean, tech debt free way of solving this issue.
  • You could replace the binary with the one from the package using cp.
    • This is the best option if the above isn’t possible.
  • See Packaging/Binaries - NixOS Wiki, we can just patch the binary that is randomly distributed with your source code before it is executed.
  • Another alternative is to use buildFhsUserEnv, which uses either chroot or bubblewrap to make your system behave as if your ld-linux.so is where the binary expects it.
  • Some people add a symlink to the one currently used by NixOS in /usr/lib/ld-linux.so. This is a bad idea and can completely bork your system. The other options aren’t really hard enough that I would recommend resorting to this.

In exchange, assuming you use one of the two good options, your build will work fine if you try to do cross compilation or network builds, and you won’t randomly be missing a library if you try to distribute your package. You also won’t be vulnerable to quite as many supply chain attacks.

Small note from me: Java build systems are a house of cards. You’ll run into this kind of problem a lot with Java software. It’s surprising, really, given their byte code should be interpretable anywhere, but maybe this makes people care less about the build time.

3 Likes
3

Thanks a lot for your answer. I do understand the problem, but I’m not sure how I would implement the solutions 2 & 3 (that I would prefer over buildFhsUserEnv as it makes it more complicated to use). I can easily see how to patching or replacing the binary (this replacing idea is neat actually) in a “normal” scenario where I first get the file before running the compilation. Here, mvn seems to both download and compile at the same time, so I don’t know how to “hook” into it (btw, for people interested to package directly the pre-built .jar, the solution is here).

So do you know how I could hook into mvn to do that? As you say, this should be fairly frequent in java, so it would be great to provide a way to do that patching automatically (or at least provide an option to do it automatically). Meanwhile, I created an issue here.

4

I’m not sure how maven works in this instance :slight_smile: You’re using stdenv.mkDerivation, and I don’t see anything disabling the sandbox, so maven should not be able to download anything.

Maybe see if you can access that file in the patchPhase?

5

As far as I understand maven can download everything thanks to outputHash, this counts as a fixed derivation output and it seems to allow downloading stuff (at least it’s apparently how fetchurl is implemented, see Advanced Attributes - Nix Reference Manual). So I guess that the patch phase would only see the github repo without the maven dependencies,) .

6

Ah, of course, I’m blind. The way I’ve done this with other package managers is to do it in two steps; make an FOD (with outputHash) that only downloads the dependencies (usually in a let block if downstream, separate file called e.g. quarkus-raw if I care for others being able to override my package), and then have the build step as a separate derivation.

In other words, you’d want a separate derivation that does mvn dependency:go-offline (apparently), and then have either a fixupPhase that runs after the dependency download or a patchPhase before the build, probably depending on whether you patch the binary or replace it.

The issue becomes figuring out where maven expects these files to be, and how to store and then make them available again without upsetting it. I’ve found this very challenging with gradle, for example.

This is also nice because it means that the build itself can be done in a real derivation, which makes you a bit less prone to issues caused by what is most likely a non-reproducible build.

Others will have done similar things in nixpkgs, it’s worth looking for maven in there.

7

There is also maven-fod even though I was never able to make it work… Whenever you explicitly set the mvnFetchExtraArgs the build fails with a can not coerce set to a string because the parameter is not expected by stdenv.mkDerivation. No idea how that is supposed to work… If somebody could explain that I would be very grateful.

In the case of protobuf, I simply added protobuf as nix input and put the protoc command in the pom.xml, see here. Maybe you can do something similar in a patch.

8

you probably setting mvnFetchExtraArgs = "string";, while it should be mvnFetchExtraArgs = { attribute1 = "value1"; attribute2 = 2; };

the string one is mvnParameters

edit: nevermind, i’m having the same issue as you even setting that as an actual set

error: cannot coerce a set to a string

       at /nix/store/zd8n8s2m9yz21xsk0q3vahp4mmq5wbnn-source/pkgs/applications/science/engineering/modelio/default.nix:51:3:

           50|
           51|   mvnFetchExtraArgs = {
             |   ^
           52|     postPatch = ''

       … while evaluating the attribute 'mvnFetchExtraArgs' of the derivation 'modelio-5.1.0'

       at /nix/store/zd8n8s2m9yz21xsk0q3vahp4mmq5wbnn-source/pkgs/stdenv/generic/make-derivation.nix:270:7:

          269|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          270|       name =
             |       ^
          271|         let

i don’t think i like the pattern used in this mavenfod. why is that like that? i’d prefer let in

9

Ok, thanks a lot everyone for your help! I will give it a try!

10

Can you double check if you really want to package quarkus itself and not quarkus-cli? Do you want to to have quarkus as a nix dependency somehow or what’s the goal?

11

So my initial goal was to load quarkus-cli directly from shell.nix (not sure about the difference between quarkus and quarkus-cli, I thought that quarkus was providing also the quarkus-cli) to have reproducible builds while developing an app. My first attempt was to try to package it and it fails, but meanwhile I found other solutions by directly downloading the .jar or using jbang as explained here. This solution is good enough for my need (it’s even better than packaging from source as I don’t need to recompile quarkus), however I’m “theoretically” annoyed by the fact that I can’t build quarkus from source using Nix, and this post aims to find solutions/document how to deal with the above errors I got (as I’m sure I’m not alone).

More generally, I think that NixOS is definitely a great tool for regular users and managing servers (and it would certainly be hard for me to use another distro later), but my experience of using NixOs for developing is more mitigated. I find myself always fighting with build tools. This post is one example I can show, but for instance I still don’t know 4 months later how to compile an electron app! Given how javascript and java are widely used, I’m afraid that this kind of issue will make NixOS unpopular among developers. That’s why I think that it would be really important to provide robust documentations and solutions (potentially automatized where the users just specifies the language that they are coding with) to help developers to code on NixOS and with nix tools.

1 Like
12

When you want to use quarkus-cli you don’t need to compile the entire framework. On the other hand you could just use maven to create a quarkus project.

Java does somehow works ok for development. The main issue is when you build downloads and executes binaries - Which I normally don’t see as good practice at all. But I don’t know the use case for doing this.

13

But if quarkus fails to build, then it means that potentially some parts of the framework won’t compile in NixOS, and that therefore I can’t use it in my project? Sorry if what I say does not make sense, I’m definitely not a java person.

Regarding build downloading binaries, that’s certainly not a good practice, but in the real world™ not everybody follows good practice, especially when it saves time… And I guess that Nix should be more resilient to these bad practices as developpers don’t want to lose 2 days just because one small dependency did not followed the good practices…

14

This will not help you but I just wanted to add a note

In the Nix ecosystem, some ecosystems like Java (which is pretty popular as you notice) are very badly supported. https://www.youtube.com/watch?v=gQstiX7H8MQ

Besides I feel that Nixpkgs just accepting pre-built src inputs such as JARs when source is available hinders the need to develop better ways to build Java (Maven, Gradle, …) from source. Unfortunately, I see little interest from the community in developing the JVM ecosystem on Nix.

15

Yeah, that is a fair assessment. I think there are some development workflows that are supported pretty well (rust, haskell, npm (not yarn!), lisp, python, …), but also some that very much aren’t.

I think it reflects the tastes of the community, as popular as Java is, that’s likely more for legacy reasons. I don’t know many polyglot developers who would go out of their way to use it - rather the opposite.

Even the well-supported environments pretty much require jumping through figuring out shell.nix, which in the eyes of many here is a feature (virtualenvs, but less of a mess, and cross-language!), but it can be very annoying to work with certain projects that really expect you to follow their exact development instructions.

I’ve personally fallen back to using fedora/ubuntu VMs with home-manager while developing on bazel (java), maptool (java), parcel (yarn, which due to suspicious metric collection I don’t run outside VMs anyway, screw you facebook) and a yocto project that shall remain unnamed. GNOME Boxes is very good at spinning up straightforward VMs: gnome.gnome-boxes.

Annoying, for sure. But NixOS is very different, and projects aren’t usually prepared to be developed on in very different environments.

If you’re interested in improving java development in the nix world, consider contributing to GitHub - nix-community/dream2nix: Simplified nix packaging for various programming language ecosystems [maintainer=@DavHau].

1 Like
17

But if quarkus fails to build, then it means that potentially some parts of the framework won’t compile in NixOS

Quarkus is heavliy modularized. So if one module cannot be compiled for what reasons so ever it does not mean you cannot use quarkus on NixOS.

That’s most likely cause by the design choice of NixOS and tbh I cannot blame when (especially java builds fail) because you use non java binaries for building.

I’m working mostly with java and for normal workflows I did not face major problems. The main issue is having different jvms, maven, etc. versions and everything needs to be packages by nix. But that’s the downside you need to accept when using NixOS.

If you need easier handling: spin up a container or a vm.

Besides I feel that Nixpkgs just accepting pre-built src inputs such as JARs when source is available hinders the need to develop better ways to build Java (Maven, Gradle, …) from source.

At least for maven I don’t see any better way of having more appropriate tooling. Indeed there are too many project just using binaries instead of compiling the packages on their own.