When I was porting OpenMSX to Nixpkgs, I was paralyzed by the custom build system they use (namely, a Python script embedded in a Makefile).
As I have low programming skills in order to understand and modify the package by myself, I decided to talk directly to OpenMSX developers via IRC.
I quickly explained a bit about Nixpkgs, and how its idiosyncrasies prevented a successful detection of TCL/TK toolkit. Basically, they were trying to detect tcl script in a default location like /usr/local. Then, they made a small patch in their build system in order to accomodate unusual locations for TCL/TK.
Now, my question is:
How should be our policy of relationship with the upstream about that subject of submitting Nixpkgs-specific patches?
I say this because the unorthodox way Nixpkgs works exposes some “flaws” in their building scripts, as hard-coded paths and tacit dependency assumptions like the ubiquitous which.
In the case above, they patched the source in order to handle our specific distro. I think it would be better if we sent some patches to upstream because maintaining them on our side is a nonsense burden, while those patches could help these upstream developers.
But we need to do it in a way we don’t appear like arrogant ad invasive to them.
I’ve often had positive results in submitting upstream, as long as the fixes help generally and don’t special case Nix. Sometimes it helps to show that a patch has been in use in Nixpkgs for a while to show that it works/is stable/etc.
My experience is that it can take a bit of convincing but generally people are happy when their build system becomes more flexible. Most of the patches are related to source binaries from PATH instead of fixed locations, which also helps with other use-cases such as BSD installations and various sandboxed environments.
Yeah working with upstream is hard, but I think its the single most important thing for the continued health and growth of Nixpkgs.
I spent some time working with Meson to improve their cross compilation support, for example—I found this is a nice shortcut because while individual packages may not care about weird situations, build systems are incentivized to to attract users who might be tempted just to hand-roll something simpler but that just works for their exact use-case. Build systems also are used with many upstream packages, so it’s a change to amplify the effect of ones efforts.
Ultimately, I think we’ll need to leverage things like recursive Nix to directly appeal to developers. The idea is to get people to want to use Nix for there own CI and even development builds, and easy Nixpkgs packaging is just a side benefit they don’t even need to be aware of.
Upstream projects usually appreciate if you make things more correct.
You can explain how de-hardcoding makes things more correct because it will not also help for Nix, but also:
“Make the code more portable” (there are many Linux distributions and upstream devs usually like when you help to make things more portable).
It is standard even in very old build systems (e.g. autoconf) to make things very configurable, e.g. the --prefix where the software shall be installed. People usually agree that modern build systems should not be worse than autoconf.
It helps with cross-compilation (something everybody who’s ever built something on Raspberry Pis or for phones will immediately appreciate).
It is almost universally agreed that undeclared dependencies are bad. If something requires which, PR this to the corresponding README section on dependencies.