Nix-build-phases: run nix build phases interactively

Announcements
1

I wrote small, but very handy nix-build-phases.bash script which allows to run nix package build phases interactively. It is very useful when developing or debugging packages. It also allows to run single build phase multiple times which can save a tons of time.

Example:

mkdir dev; cd dev

nix develop nixpkgs#geos

. ../nix-build-phases.bash
>>> Phase:    **unpackPhase** patchPhase configurePhase buildPhase checkPhase installPhase fixupPhase installCheckPhase distPhase
>>> Command:  phases=unpackPhase genericBuild
>>> Press ENTER to run, CTRL-C to exit

unpacking sources
unpacking source archive /nix/store/bps3riq6avqb05z55jvwg6i8234fr63v-geos-3.11.2.tar.bz2
source root is geos-3.11.2
setting SOURCE_DATE_EPOCH to timestamp 1679011021 of file geos-3.11.2/Version.txt

>>> Phase:    unpackPhase **patchPhase** configurePhase buildPhase checkPhase installPhase fixupPhase installCheckPhase distPhase
>>> Command:  phases=patchPhase genericBuild
>>> Press ENTER to run, CTRL-C to exit

patching sources

>>> Phase:    unpackPhase patchPhase **configurePhase** buildPhase checkPhase installPhase fixupPhase installCheckPhase distPhase
>>> Command:  phases=configurePhase genericBuild
>>> Press ENTER to run, CTRL-C to exit

configuring
fixing cmake files...
cmake flags: -DCMAKE_FIND_USE_SYSTEM_PACKAGE_REGISTRY=OFF ....

Before running each phase, it prints information how to run the phase manually if needed (for example to run buildPhase: phases=buildPhase genericBuild).

Enjoy !

29 Likes
2

Looks very useful! Does nix itself designed to be hard to use so that we have to use a helper scritp on top of it?

3

No, it just needs some more love and work and everything will be awesome.

4

You dont have to go through genericBuild anymore, you can just runPhase x

https://github.com/NixOS/nixpkgs/pull/230874/files

You can submit this to nixpkgs for review if you can integrate it in to setup.sh

Splitting the phases= in to a function from genericBuild would be good for reusability

2 Likes
5

Thank you very much for pointing me to this change. I’ll update my script and will think about integration to setup.sh.

Actually, I need to wait until this change is present in all supported nixpkgs/NixOS versions (currently it is not in 23.05 which I still use).

6

That’s quite practical. I have a small question: if you quit your script (like restart computer), how would you restart it from where you left? Notably, do you need first to manually go to the source that were extracted and skip the unpack phases?

7

For now, you need to go to correct directory and run next phases manually.

1 Like
8

this is awesome, this needs putting directly into the mainline nixpkgs… TODAY.

if we have gdb , this is the npdb (nix package debugger).

All hail NPDB.

3 Likes
9

The stdenv builder is a part of nixpkgs, not nix.

10

you know what i mean… nix is nothing without nixpkgs :slight_smile: .

Who’s idea was it to called every thing nix anyway ;-).

LOL!..

nix that.

11

I would also encourage you to try to sprint this into mainline (nixpkgs), while the context is still fresh. We may ask the Nixpkgs Architecture Team to consider it somewhat timely for review to keep your churn low.

1 Like
12

make it so number 1.

13

@blaggacao , @nixinator thank you very much for feedback. I am already thinking about integration to nixpkgs stdenv , but I am not sure about correct design.

1 Like
14

I would go with the NIX_DEBUG flag or something similar that is already understood by the stdenv. Or why not use NIX_DEBUG_INTERACTIVE = true?

However you start off, the real relevant feedback will come on the draft PR, anyways. Good luck! Keep us posted!

15

I renamed the script to nix-develop-interactive .

1 Like
16

Note that the manual for nix3-develop has nix develop starts a bash shell that provides an interactive build environment in its description.

17

I am using a similar approach to develop some packages (and to troubleshoot existing ones), just by

NIX_BUILD_TOP=/home/user/ch/78 nix-shell -E "$nixcode" --run 'unpackPhase'
NIX_BUILD_TOP=/home/user/ch/78 nix-shell -E "$nixcode" --run 'cd source; patchPhase'
NIX_BUILD_TOP=/home/user/ch/78 nix-shell -E "$nixcode" --run 'cd source; eval "$configurePhase" && eval "$buildPhase"'

There are 2 disadvantages:

  1. Some packages need configurePhase while other eval "$configurePhase" depending on whether the phase defined explicitly or use default, so general approach should analyze .drv file to invoke the phase correctly.
  2. Nix phases are not written with idempotency in mind. You cannot run patchPhase for the second time without error. Often you cannot run buildPhase for the second time too (because it might have e.g. sed; or second patchelf might corrupt executables), so as the prerequisite one have to fix idempotency of the phase which supposes to be repeated.
1 Like
18

For 1 isn’t phases="configurePhase" genericBuild supposed to deal with both cases? Otherwise can’t we check the type of configure phase?

For 2 I was thinking to save the source/environment variables/bash functions in a file, and directly load that file to skip non-idempotent phases. Or for non-idempotent phases, to restart from the previous saved step.

19

For 1 theres runPhase

1 Like
20

How do I use it? It’s not exposed as a function inside a nix-shell.