Help with python flake build from local source

[iFreilicht]

@firecat53

2. I’m struggling with the development workflow in this setup. I understand that nix can’t be used similar to a pip install -e . to be able to make a code change and then immediately run the app because the source is copied to the nix store.

Yes, you can’t build easily directly inside the working directory, but nix can do something pretty close:

@firecat53

Just a simple hatch shell seems pretty straightforward compared to the nix flow in this case

How about nix shell?

I’ll try to demonstrate what it does:

$ which python
/usr/bin/python
$ nix develop
$ which python
/nix/store/1r6n7v2wam7gkr18gxccpg7p5ywgw551-python3-3.10.12/bin/python
$ which keepmenu
keepmenu not found
$ nix shell path:.
$ which keepmenu
/nix/store/mh6mlwfw7y0m8cl59rzgr8k64w448if7-keepmenu/bin/keepmenu

So basically, nix develop puts you in a shell where all the dependencies are present that are needed for building your default package (most probably packages.x86_64-linux.default). You don’t even need the devShells output for this! Try it, if you remove devShells, you can still run nix develop and it will still work!
Basically, it puts you in the same environment that the scripts run by nix build will be in by default. specifying devShells will override that behaviour so you can add additional stuff that is needed for development but not building, for example.

And nix shell builds the package and then puts you in a shell where the default package is available.In this specific case (PATH diff abridged):

$ env > env-no-nix
$ nix shell path:.
$ diff env-no-nix <(env)
71c71
< SHLVL=1
---
> SHLVL=2
92c92
< PATH=/home/felix/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/home/felix/.nix-profile/bin
---
> PATH=/home/felix/.nix-profile/bin:/nix/store/xd0ykfrrlqwi0wgb6lc64hshyn99cf1k-keepmenu/bin:/nix/var/nix/profiles/default/bin:/home/felix/.nix-profile/bin
106c106
< OLDPWD=/home/felix
---
> OLDPWD=/home/felix/repos/keepmenu

There’s actually some discussion about potentially renaming nix shell because this distinction is somewhat confusing, though I personally don’t agree with that.

@firecat53

I have found plenty of posts around setting up the development environment, but I can’t find anyone showing how to actually install and iteratively develop their apps!

The reason is probably that this wildly differs between projects. nix shell is nice and all, but it requires rebuilding the whole package from scratch every time, and so Nix itself has dedicated documentation about how to build and re-build and how to run tests individually inside a development shell instead of running nix shell every time.

But this brings us to your specific problem again:

@firecat53

I added pip to the mkShell command. So with this workflow:

[…]

ModuleNotFoundError: No module named 'keepmenu'

So the issue here is that pip expects that it can just add stuff to your import path and that this will be picked up by the next python invocation, but Nix tries hard to avoid side-effects like that, and does so successfully. I couldn’t find out what pip actually does to achieve this, but the bottom line is, it doesn’t work, we have to manually add the repository’s directory to the PYTHONPATH so python actually finds it:

$ nix develop
$ pip install -e . --prefix $TMPDIR/
$ export PYTHONPATH=$PYTHONPATH:$PWD
$ python $TMPDIR/bin/keepmenu

And voilá, now it runs as expected, using exactly the dependencies locked by your flake.

Though I’m curious how often you’d want to do that when you can just

$ nix develop
$ python -m keepmenu

instead.

@firecat53

However, this works:

❯ python

When running python in interactive mode, the current working directory is always searched for modules in addition to PYTHONPATH, basically replacing the step we added to the workflow above.

@firecat53

There’s direnv, devshell, nix-direnv, etc etc, but I haven’t really seen how those would help, other than making some of the boilerplate switching between environments easier.

Yeah I agree. For me direnv is just a convenience feature so I don’t have to type nix develop manually, which is pretty cool, but not a game-changer.

@firecat53

1. Can the path:. argument to nix build be made default just for that flake? That way you can just run nix build or nix run instead of having to add the path:. argument.

No, unfortunately. I tried overriding the self input, but that only gets evaluated after Nix already determined what type of flake it’s working with, and there’s no builtin option for this.

If you want to make things more convenient, you could use a command-runner like just, and add that to the development shell so people only have to run nix develop once and just build, not worrying about the details underneath.

You could also use this to automate the workflow above. just run, just install or just test have a nice ring to them.

@firecat53

(thanks so much for your patience!)

Thank you for all your work! I’ve used keepmenu and other tools of yours in the past and am thrilled to give something back