Almost every time I update my nixos, Python needs to be recompiled in every development shell

Help
1

Hello

I am quite new to nixos, I’ve been running it for ~ 3 months.

Almost every time I update my nixos with sudo nixos-rebuild switch --flake ".#default" and afterwards use nix-shell command in a directory that has a shell.nix I need to recompile Python which takes some time.

I’ve found another topic in discourse talking about it might have todo with overlays. (https://discourse.nixos.org/t/why-do-i-need-to-compile-python-and-packgages-every-time-there-is-an-update/23280/6). I have some overlays, but I don’t understand why this is happening. I want to use stable and unstable nixos repo, so I use it as an overlay, is that why I need to recompile Python every time?
The reason why I use unstable and stable is because stable has some ‘outdated’ packages like neovim, so for this I want to use unstable, and some packages are broken on unstable so that is why I want to use stable.

for example a shell.nix file:

let
  pkgs = import <nixpkgs> {};
in pkgs.mkShell {
  packages = with pkgs; [
    python311
    (poetry.override { python3 = python311; })
    (python311.withPackages (p: with p; [
        pip
    ]))
  ];
  LD_LIBRARY_PATH = "${pkgs.stdenv.cc.cc.lib}/lib";
}

my flake.nix with overlay:
Github: https://github.com/sevaho/dotFiles/blob/929238a28f9afd54768435017710896e966eabd9/flake.nix#L28

File:

{
  description = "Nixos config flake";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    nixpkgs-stable.url = "github:nixos/nixpkgs/nixos-24.05";

    home-manager = {
      url = "github:nix-community/home-manager";
      inputs.nixpkgs.follows = "nixpkgs";
    };

  };

  outputs = { self, nixpkgs, nixpkgs-stable, home-manager, ... }@inputs: {
    nixosConfigurations.default = nixpkgs.lib.nixosSystem {
        modules = [
          #  This is required so we can use both stable and unstable nixpkgs
          {
            nixpkgs.overlays = [
              (final: prev: {
                stable = nixpkgs-stable.legacyPackages.${prev.system};
              })
            ];
          }
          ./nixos/configuration.nix
          home-manager.nixosModules.home-manager
          {
            home-manager.useGlobalPkgs = true;
            home-manager.useUserPackages = true;
            home-manager.users.sevaho = import ./home-manager/home.nix;
          }
        ];
    };
  };
}

Anyone could help me with this problem?

1 Like
2

Firstly please pin your nixpkgs for projects this will probably massively help your issue.
Secondly you are using a different python than the default python 3 in nixpkgs,
see the anwser of your linked post Why do I need to compile Python (and packgages) every time there is an update? - #15 by NobbZ .
packages are not built for that. so everytime any package or any transitive dependencies changes if a package you are using is affected you will need to recompile it.
i highly recommend switching to python3 which currently is python 3.12

3

Hi @eldritch_cookie thanks for your response! With pinning you mean using a flake.nix and flake.lock file? I have the same issue with this as well.

I need to use different Python version for different projects, so atm. I am using 3.9 to 3.13. The alternative is using uv / rye / pyenv but I prefer a nix shell.

Greetings

4

Flakes are a form of pinning, you also could use niv or the newer npins, you just can’t use because this means using your system’s nixpkgs which in the worst case means recompiling on every update.
That being said if you use flakes your system config should be irrelevant to your dev shells.

I don’t know much about python, do you really need all those versions? would you need all those versions with the alternatives? if you need 5 python versions regardless of whether you use nix or not you will need to compile your projects at least 5 times