Lost on how to get a python project running on NixOS

Help
1

I’d like to run the python package beancount, along with some additional related python packages on NixOS. I’ve got a flake-based configuration for my system up and working but am finding the wiki entry on python particularly inscrutable as there seem to be many different ways of approaching this.

On my Arch machine, I installed the latest python package in the arch repository, created a virtual environment using venv in my project directory, and then ran pip commands to install a requirements.txt list of packages. Each time I enter the directory I activate the virtual environment and my beancount programs run as expected.

Can anyone recommend an opinionated simple approach to get my beancount project working on NixOS? I guess I’d prefer to do it the “nix way” as much as possible, theoretically declaring everything somewhere in my system repo (my system’s flake.nix? a shell.nix file in my beancount directory? somewhere else?), but my brainpower started to stall out when I got to the " Using a Python package not in Nixpkgs" part in the wiki. I thought I could try easy mode by running nix-shell to install python and pip, but that only works up to the point that I try to subsequently install the beancount programs using pip, at which point I run into errors related to the nix-store.

Here is my config repo: GitHub - dc-bond/nixos-configs: configuration files for NixOS

My requirements.txt file for my beancount project looks like this:
beancount==2.3.6
beancount-black==0.2.1
beancount_reds_importers==0.8.0
smart_importer==v0.4

2

I personally use nix development shells. You can find my python/jupyter notebook shell here.

I start it with nix develop and it just works every time, haven’t used it in a while tho…

I use them as a per-project shell, so every project has its own flake.nix with a unique dev shell. You could also install globally, but I prefer this since not on all my devices I work with the same projects. And anyone else who might need to run it is independent from your PC config too.

3

This is somewhat ancillary to my original question, but when I created a test directory and ran this flake, it added a bunch of extensions to vscodium that I don’t have defined in my vscodium system module and I can’t get them to uninstall (besides manually uninstalling from within vscodium). I tried nix-garbage-collect, tried logging out of my display manager, and even tried rebooting. I’m very confused - isn’t the purpose of a devShell to create an isolated disposable environment? What else did this install on my system that is still around after a reboot?

Regardless, thanks for the idea but I think I’m looking for something a little simplier that doesn’t involve jupyter notebooks, vscodium, etc. and some of the beancount python packages are not packaged in nixpkgs so I can’t just write them into the python3.withPackages part of the flake…

4

You can absolutely use pip, but you need to use a virtual environment (which you should do anyway when using Python). Roughly:

$ nix-shell -p 'python3.withPackages (ps: [ ps.pip ])'
$ python3 -m venv .venv # First time creation of venv, only needed once
$ source .venv/bin/activate
$ pip -r install requirements.txt

If your Python dependencies depend on some system (C) libraries, you will still need these in your shell via Nix.

5

I ended up getting this working (I think) by putting a flake.nix and shell.nix in the project directory, and then running nix develop while in the directory:

flake.nix:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  };

  outputs = { nixpkgs, ... }:
    let
      forAllSystems = nixpkgs.lib.genAttrs [ "aarch64-linux" "x86_64-linux" ];
    in
    {
      devShells = forAllSystems (system: {
        default = nixpkgs.legacyPackages.${system}.callPackage ./shell.nix { };
      });
    };
}

shell.nix:

{ 
  pkgs, 
  lib, 
  stdenv, 
  ... 
}:

let
  pythonPackages = pkgs.python3Packages;
in
pkgs.mkShell {
  buildInputs = with pkgs; [
    pythonPackages.python
    pythonPackages.venvShellHook
  ];
  venvDir = "./.venv";
  postVenvCreation = ''
    unset SOURCE_DATE_EPOCH
  '';
  postShellHook = ''
    unset SOURCE_DATE_EPOCH
    export LD_LIBRARY_PATH=${lib.makeLibraryPath [stdenv.cc.cc]}
    pip install -r requirements.txt
    exit
  '';
}

This also seems to fix the issue with the shell environment python not pulling the correct shared libraries because of how the nix store is setup. I don’t know exactly how this is working but I think it has something to do with the shell hooks?

6

Sometimes I will take a crack at packaging the python module and then using my package along with what is in nixpkgs.

If it is just for local development your approach should work

7

You might want to look into direnv as well.
With it you don’t even have to type nix develop when you enter the directory.

2 Likes