Building and Debugging Upstream Packages

Help
1

I would like to build an upstream package and perform some debugging and troubleshooting. It’s not clear to me how to go about this.

I’ve cloned nixpgs and I’ve “built” against my local repo. Of course, all this does is retrieve from the binary cache since the hash hasn’t changed. I realize I can make a change to the derivation to trigger an actual build, but what I actually want to do is manipulate the source of the upstream code itself. I want to make change to the code, build, and try things.

The reason I want to do this is that there are bugs in one or more NixOS packages. The upstream code is fine and works under other distros. I would like to determine the problem and possibly PR patches to fix the package.

This would seem to me to be a very common scenario, particularly for package maintainers. Can someone walk me through the workflow for this?

2

A clumsy way that has occurred to me is perhaps to edit the derivation to forgo the fetchurl step and somehow use a local, untarred copy of the package. I think that (or something like it) should be possible but it feels hacky and I’d rather learn the conventional way of approaching this.

3

So building against the local copy seems to be working. Pushing on.

I would still welcome any assistance.

4

No, this isn’t accomplishing what I want.

I’d like to build a particular nixpkg package (a library, in this case), build it, presumably with nix-shell and install it, so that clients will link against it.

5

When I tempered around with some derivations, I just used nix build -f ./default.nix hello from within the cloned repository root after changing the derivations code.

Alternatively, you can create an overlay locally on your system, that includes the libraries changes as override or overrideAttributes.

Once you figured that the applied changes in the overlay work correctly, then you can apply them directly to the derivation in the nixpkgs.

1 Like
6

Maybe you could have a look here:
https://gricad.github.io/calcul/nix/tuto/2017/07/04/nix-tutorial.html#how-to-add-a-package-to-nixpkgs

1 Like
7

When I try this, I get:

error: cannot auto-call a function that has an argument without a default value ('stdenv')

8

For me the command from above works perfectly.

$ pwd
/home/nmelzer/Projects/nixpkgs
$ nix build -f ./default.nix hello
$ readlink -f result
/nix/store/77gczr7dc287sqdc13gi2zcpwl7960yz-hello-2.10

(Build itself produces no output, as hello is probably always there for me, I do use it about everywhere to test things…)

1 Like
9

Ah, so you run this at the root of your nixpkgs repo. I’ve been running it in the subdir of the particular package. I’ll give this a try.

In the meantime, I’ve taken to trying this:

nixos-rebuild build -I nixpkgs=/home2/nixos/nixpkgs

10

That rebuilds your system configuration using your copy of the repository, not something I’d do without really knowing what I’m doing. It does not switch though, I’d still be careful. Also it does not even require that the derivation you are fiddling with will get build at all, as your config might not need it.

11

Hmm, so I see. I was apparently working in the 20.09 branch (my system’s at 19.09). I did switch and it did build the package in question. Well, easy enough recover from – yay NixOS!

Anyway, rebuilding my system was intentional. The thing I’m looking at is the jack2 client library. It’s used by a server (part of my system config) and by client apps. IOW, I need to wind up with a consistent environment that includes both of those things. I agree that your suggestion is a more conservative approach, but can I achieve this that way?

12

Well, I am able to rebuild my system, including the jack2 library. However, when I switch, it doesn’t seem to activate the new library build. I assume it’s because I’ve neglected to provide a new hash. I’m kind of stumped.

Are there any library maintainers out there that can set me straight?

13

If the hash is missing or wrong, then nothing should have been built, as the buildsystem should complain about the wrong hash.

How do you know that your changed library isn’t picked up?

Remember that, depending on the nature of your software restarting the affected programs and services might not be enough and log-out/log-in or even a reboot might be necessary.

In such scenarios I’d probably use nixos-rebuild build-vm hoping that the VM emulates necessary hardware correctly…

But yes, sometimes one needs to get into the trouble of rebuilding the channel into its own system configuration…

14

Well, two things led me to be believe that.

  • The path of the output didn’t seem to be the same as the one in my env.
  • I didn’t see the behavior change I expected

However, both of those points are moot now, for reasons I’ll elaborate below.

I’ve been executing:

nixos-rebuild switch -p test -I nixpkgs=/home2/nixos/nixpkgs

followed by a reboot into the test profile.

Thanks for the VM suggestion – another thing to try.

Anyway…

I woke up this morning realizing that my thinking around this was flawed. The code I was changing was not the server code but the client library code. To see my change, I would also need to rebuild the client(s) I want to observe in my testing. And it may not be important that I build my entire system, but I may only need to build the client library and the app that links against it. So I may be back to trying nix build or nix-shell.

Does it make sense to build with nix-shell for the library followed by a nested nix-shell for the app?

I really appreciate all the help BTW!

15

nixos-rebuild does use the nixos channel AFAIR…

16

I think the -I nixpkgs=/home2/nixos/nixpkgs overrides that though. That’s where my doctored derivation lives and it is triggering a build of the upstream package.

17

SOLVED

I managed to install my doctored libjack2 , but I was never able to get ecasound to use it.

Because I was running unstable in my user profile and stable in my system.

All I had to do is move the “broken” jack clients from my profile to my system config and, so far, they all just work.

It was all just a massive user error on my part. :man_facepalming:

Thanks @NobbZ for all the help. It wasn’t a pointless exercise; I’m just a little more familiar with Nix and NixOS.