The JDK needs better support in nixpkgs. The JDK ecosystem is large and critical, and improved support would benefit not only the Java ecosystem but also many others, such as Kotlin, Scala, Clojure, and Groovy.
I am grateful for the past efforts of contributors to the JDK infrastructure. But my impression is that many, if not all, have moved on to other projects, leaving the JDK ecosystem with very limited support. I also believe that currently, no one with commit access is particularly interested in the JDK. Some of the issues I consider urgent include:
JDK versions are not updated automatically. There is no updateScript for JDK derivations.
JDK versions are updated slowly. For example, the current JDK LTS is 21. We are on version 21.0.0, while the latest release is 21.0.3. Nixpkgs unstable is three versions behind.
JDK-related PRs, even minor ones, take a long time to be reviewed and merged. For instance, I submitted a PR to update the default JDK from version 19 to version 21. Version 19 was already end-of-life and marked as insecure at the time, but it took approximately three months to be merged.
We have too many JDK flavors in nixpkgs. Some of these are likely unnecessary and could be removed. For example, AdoptOpenJDK is a deprecated project replaced by Adoptium (which is also available in nixpkgs).
There are many more areas for improvement, but in my opinion, we should first address the points mentioned above.
We need to put more effort into the JDK infrastructure and seek the help of contributors with commit access to unblock existing PRs and incentivize new ones.
Next steps and call to action
First and foremost, Iād like to identify anyone interested in helping improve the JDK infrastructure on nixpkgs. My goal is to achieve incremental improvements and get changes merged efficiently. To achieve this, even just getting some reviewers and testers for PRs would be extremely helpful. We can discuss the technical details in a separate thread.
Who is interested in contributing to a better JDK infrastructure on nixpkgs? Is anyone with commit access to nixpkgs interested in lending a hand with this effort?
On my end, Iāll continue trying to improve the JDK ecosystem on nixpkgs. For now, Iām putting together a list of ideas for what to do next, and Iāll share it once Iāve cleaned it up a bit.
Iām interested because I regularly rely on the JDK ecosystem a lot, and I do have commit access - but also Iām already drowning in projects so I donāt plan on adding any new significant ones .
As a positive aside, practically speaking Iāve been mostly very happy with the Java support in nixpkgs.
I think part of the stagnation might be caused by a fear of breaking things. I think it would be reasonable to explicitly say, since the JVM ecosystem is low on maintainers, we should move a bit faster on the shared infrastructure, and if that breaks downstream derivations, that might be an incentive to either remove those derivations from nixpkgs, or encourage their maintainers to help out with the shared infrastructure more, or encourage more users to help out with maintenance.
Awesome!
Iām a little concerned we can generate ideas much faster than we can implement them, but good to collect them somewhere. Maybe we should turn them into GitHub issues with topic:java and kind:enhancement tags when theyāve been cleaned up, so we can discuss them there?
feel free to tag me for reviews - I am no Java expert and donāt use it day-to-day but we need better support in NixOS for it IMO Iād hope @fzakaria would be involved. I know heās done some work with Java and NixOS
I bet some of this could be simplified with a separate repo overlay people can adopt ?
I am lately into radical simplification.
Part of the challenge with Nixpkgs (to me) is it accepts too much.
I saw recently a PR to cleanup some JDK stuff which I looked at it and commented Iād be in favor of going further and just consolidating on OpenJKD.
(Anything else should be an overlay outside nixpkgs)
I totally endorse this effort, and would like to allocate some of my time to it (if that would help even a little)!
I think that the problem behind poor support for JVM languages in Nixpkgs is not only due to technical difficulties, but also because of a certain stigma that the whole Java-adjacent ecosystem has garnered over the last decade or so. I firmly believe that the JVM is a fine piece of software that still has much to give, if used correctly, together with the astonishing amount of well-written and feature-complete libraries that build upon the JDK (for which suitable replacements do not exist at all, sometimes).
I can totally see the reasoning behind this. Maybe we should just maintain a solid, accessible and trivial building infrastructure within Nixpkgs, and then delegate something like a full-fledged Maven builder to external overlays/flakes? If that is the case, we should strive for maximum āpluggabilityā of the Nixpkgs-side of things, I guessā¦
I too support this effort, and would like to be pinged for reviews.
Itās important to note also, that (in my experience) many jdk depending packages are not source based, and are just a makeWrapper packages that wrap a .jar file downloaded from a GitHub release page. Thatās a problem because we donāt have any way to tell if such a packages break when we update JDK, and thatās why people are afraid to support merging major version updates of JDK packages, which a red 10.rebuild-linux: PR label.
I think we should add a --version test in an installCheckPhase or something like that, for all of those .jar based packages. This would help nixpkgs-review catch breakages when we perform a major JDK update. It may very well be, that many of these packages havenāt been updated in years even by their upstream developers (usually games in my experience), and hence we should not make any effort to fix these.
A quick git grep showed me there are 76 such packages:
I am using Java daily on nixos and darwin targets, and so far I found the support to be really good.
I donāt have commit access, but I have been looking for meaningful ways to contribute back to the community. Happy to help with code reviews and testing (especially on aarch64 darwin) if needed. Iād also be happy to work on PRs if I manage to get up to speed.
As the author of the big bump PR that seems to have kicked all of this off, Iām glad to see people excited to discuss it! As mentioned, please take a look at the PRs linked in the umbrella issue! The big bump, which should address a lot of the outdated packages, has been ready to merge for quite a while now, and I hope it gets merged sometime in the next 3 months.
Once I get a bit more time, Iāll try and put down my thoughts on the various sub-issues linked in the umbrella issue.
I think we should add a --version test in an installCheckPhase or something like that
testers.testVersion?
There is no updateScript for JDK derivations.
I do not believe this is āpossibleā, since not every Java program builds on all JDK.
Also, we have a much more serious problem in my modest opinion: Java bootstrap depends on open source abandonware.
I do not know how much Nixpkgs care about bootstrappability, but I believe this is far from a serious priority.
I am a maintainer of two such packages: jquake and protege-distribution.
While the first one is a proprietary application, I packaged the second like that because, at the time, I couldnāt figure out a way of building complex Maven applications with an acceptable level of maintenance burden on the Nixpkgs side of things.
If we manage to get a good solution for Maven, then maybe I could deprecate that package in favor of a source-built one.
As the maintainer of maptool, I would also be very happy to move to a source-built package if there was a better story for managing Gradle dependencies.
The difference, is that installCheckPhase makes the build fail if --version | grep ${version} fails, whereas testers.testVersion is not being run by nixpkgs-review.
Iād also encourage you to add a some kind of an installCheckPhase for the proprietary package, so weāll be able to know via nixpkgs-review if it gets broken without running it ourselves.
Even with that PR, or something alike merged, people wonāt get used to add the --tests flag to their nixpkgs-review runs, so you might still be missing breakages. I donāt understand this insistence on passthru.tests.testVersion where an installCheckPhase is a much stronger and simpler compliance that a package can fulfill.
Thanks for quoting Nixpkgsā manual . This was discussed a bit at #119731 but people gave their attention in the discussions there on topics other then this. Someone there did raise my opinion, but nobody commented on this opinionā¦
I noted myself to discuss this at Nixpkgs at some point when Iāll have time.
.
. This was discussed a bit at