As a Nix user for almost one year, I think it is now a good time for me to start contributing back. I have been adding custom packages to my personal overlay and the process has been smooth and efficient (one of the best features of Nix I would say). Every now and then there can be packages that I feel it would be great to have in nixpkgs.
To find out how to do that, I find the quick start in the manual, and I understand that I can fork nixpkgs, create commits and make PRs. However there are still some question unanswered before I can confidently try to contribute:
How to thoroughly test the package before I submit the PR? I can probably make sure it works by adding the package to my personal overlay. However, this only tests it in a specific environment (my NixOS), on a specific box (my workstation).
Is it considered a good practice to run NixOS on my fork of nixpkgs instead of the official nixpkgs channel? In that case, if I added the commit to my fork of nixpkgs first and try the package, it better resembles what the other users will see if the commit gets merged.
Is there any other rules or steps to take before making the PR, in order to save the time of the reviewer? Where can I find the set of rules?
Sorry if my hunt for such documentation is not exhaustive. And thanks in advance for the help!
the contributing.md serves as a foundation for all contributions. But specifics for how to do packaging can be found on the nixpkgs manual.
If you prefer learning through a video, i made this video demonstrating how to do a PR requests (the first ~20 mins is me packaging it, you can skip most of it if you don’t care)
Ideally you would have some test suite, but many packages don’t. If you do include tests, make sure they realtively quick to run (hard to review packages if it takes 2 mins to build, but 2 hours to run the tests).
I also usually like to run the application $out/bin/prog -- help. This may seem trivial, but it ensures that it’s not trivially broken due to linking or certain env assumptions.
If by run NixOS you mean, use that as a way to facilitate nixos-rebuild, then that’s up to you. But if you mean package and run the nixpkg I created, then yes, I would use a fork.
Yes, one of the benefits of nixpkgs as a mono-repo. The changes you’ve done to your fork should reflect exactly (assuming same architecture and platform) as other users.
The PR template has many of these hints. Following CONTRIBUTING.md and running nixpkgs-review is really big. I would run nixpkgs-review pr <pr number>, it will catch many unintended errors. For example, python packages may parameterize the python interpreter for you, so you may have built the package with python3.8 in mind, but also included a failing python2.7 or python3.7 package. nixpkgs-review would catch this and save a “round trip” from a maintainer.
Personal criteria I look for (not standard across all maintainers):
Is the upstream package still maintained (why add unmainted/deprecated packages?)
Does the PR cause any regressions/breakages? (shouldn’t break something unless there’s a good reasons such as security)
Package should appeal to many users (There’s compute and storage overhead associated with each new package, there should be a good use case with each package)
If your package may be applicable to others, but not many, then the NUR may be a good location to store your derivation. Similar to the AUR
Is there any way to run the tests for the package on different platforms (Darwin, aarch64) prior to submitting the PR? I added some packages in the past and when I tested them on a Mac I was regularly finding additional issues with the tests.
Thanks @austin, @dR3b and @jonringer for the great explanations and references! That is really helpful and I hope this would help others who read the posts.
It looks like the packages in my mind actually belongs to NUR rather than the main nixpkgs repository. However I will still learn the processes and documentation for contributing to nixpkgs as well.
For what it’s worth, as long as you are committed to maintain the packages I think you can/should add them to nixpkgs.
“Wide usage” is not a very good inclusion criterion in my opinion. Maybe “some usage”, but anything beyond that is
hard to judge
inherently dynamic
possibly part of a feedback cycle; maybe its not widely used because its not packaged in nixpkgs.
Discovery in the NUR is not great. If I want to use those packages in the future, I’ll probably check if they are available in nixpkgs and package them myself otherwise. That leads to duplicate work.
What we really need in my opinion is dedicated maintainers. We should not discourage them to get involved in nixpkgs. If compute & space are problems, I’d prefer to fix them within nixpkgs by using support tiers and telling hydra to ignore anything below a certain tier.
Its probably not very helpful to @breakds to hear two different guidelines from two different people, but unfortunately this is still an area of active discussion without clear consensus. I’d say if you’re invested in those packages and would like to see them in nixpkgs, do try to create a PR.
No it’s not. But I’ve added a few packages ( python3Packages.dotnetcore2 ), which really only gets used if you use some of the Azure Machine Learning SDK, which isn’t available in nixpkgs because of many reasons.
On one hand, I’ve been good about maintaining it, and only I have had to spend time on it.
On the other hand, I’m likely the only user of that package, but it’s hard to know what to prioritize when doing large package rebuilds.
Thanks Timo and on the contrary, I think it is great to hear about different opinions. Everything comes with benefits and costs and hearing different voices helps me avoid seeing only one side of it.
Slightly off-topic, I’d assume the packages being maintained in nixpkgs will only grow more rapidly in the future. Knowing little about how the official hydra is working, but my guess would be it has great scalability as long as there is enough computing resources. More packages might marginally improve the popularity, which hopefully leads to more contribution (time and $) from the users and therefore more dedicated computing resources. I might be too idealistic.
Another issue I have bumped into with Nix package sets (including my own ) is that they sometimes only work with the latest release version xor unstable.
Things just change over time (e.g. how cargoSha256 is computed has changed twice (?), sometimes newer versions are needed than what is in the latest release branch, etc.). There are solutions to that, such as pinning nixpkgs. But nixpkgs has the benefit of being a coherent package set, so packages are more likely to build.
Only because people take the time to address failing builds. This also makes it harder to do needed changes without breaking a lot of builds within the repo. Pretty sure this is what edolstra means by nixpkgs is bloated.
It’s great that nixpkgs can have a huge number of packages, but I think we need more (as in number, not severity) of maintainers for the large package set to work. Darwin, for example, is constantly being broken because very few maintainers actually use it.
) is that they sometimes only work with the latest release version xor unstable.