Help understand what is happening with my development environment flake

I am a newbie trying to understand flakes and what is happening behind the scenes, so sorry if this is a silly question.

I am experimenting with a Python development environment based on a flake in essence similar to the onein this link: https://github.com/kemalmao19/devShell/blob/50c33eb8421898e1f4b156a029f17c0749dde6d0/python/python311/flake.nix

When building with Python311 (as in the flake linked) and just trying to build the Python data science package Polars (listing “polars” under the packages) everything builds ok when I do “nix develop” and I enter the development shell - however I am only able to get the Polars version 0.19.12 while as per the NixOS package search the latest version should be 0.20.0.

Initially I suspected that it has something to do with how i define the input, but experimenting with different input sources for nixpkgs - ending with “/nixos-unstable”, “/nixpkgs-unstable”, “/master” and just “nixpkgs” - my flake.lock file updates but still only able to get Polars in 0.19.12.

Any hints on what could likely be the cause? Thanks.

Generally speaking having inputs.nixpkgs.url = "github:NixOS/nixpkgs/master"; set and update your lockfile should give you the unstable packages (I would hope :slight_smile: ). Hence you should be able to get polars above 0.19.12.

1 Like

Continuing the discussion from Help understand what is happening with my development environment flake:

Yes that is what I was hoping as well :slight_smile: , but I can not get the 0.20.0 version. My lockfile updates to:

“‘github:NixOS/nixpkgs/53928007cdaad3c333c7b4647b8d31ae6f99050a’ (2023-12-30)”

so it looks ok but still stuck on 0.19.12. Could I be missing something when it comes to Polars 0.20.0 actually being available at all in nixpkgs yet?

Nope, Polars 0.20.0 is clearly in the unstable nixpkgs. So that should be available for both Python 3.11 and 3.10. The update has been done roughly 2 weeks ago.

I should say I also cannot reproduce your problem because your flake as it is on GitHub right now, doesn’t work at all for me. I get some ImportError in scikit-learn-extra-0.3.0.

I used the linked repo flake only as inspiration - the below is my more minimal flake for testing this. Doing a nix develop with this flake still gives me the version 0.19.12 (as shown by running the Python REPL , doing

import polars
print(polars.__version__)
{
  description = "Python dev: Data Science Libraries Edition";

  # Flake inputs
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/master"; # also valid: "nixpkgs"
  };

  # Flake outputs
  outputs = { self, nixpkgs, }:
    let
      # Systems supported
      allSystems = [
        "x86_64-linux" # 64-bit Intel/AMD Linux
        #"aarch64-linux" # 64-bit ARM Linux
        #"x86_64-darwin" # 64-bit Intel macOS
        #"aarch64-darwin" # 64-bit ARM macOS
      ];

      # Helper to provide system-specific attributes
      forAllSystems = f:
        nixpkgs.lib.genAttrs allSystems
        (system: f { pkgs = import nixpkgs { inherit system; }; });
    in {
      # Development environment output
      devShells = forAllSystems ({ pkgs }: {
        default = let
          # Use Python 3.11
          python = pkgs.python311;

        in pkgs.mkShell {
          # The Nix packages provided in the environment
          packages = [
            # Python plus helper tools
            (python.withPackages (ps:
              with ps; [
                
                # notebook
                #jupyter
                #jupyterlab
                #ipykernel
                #ipython

                polars
              
                # Visualization
                #matplotlib # plot & graphs

                # Formatting
                #black

                # external api
                #openai
                
              ]))
          ];
        };
      });
    };
}

Yes, now I can reproduce it:

>>> import polars
>>> polars.__version__
'0.19.12'

I will maybe play a bit with it after New Year. But it is clear that it isn’t just your config or something. I will try set this up in another flake as well. Right now it is at least not evident to me what the issue is.

Maybe someone else comes by in the meantime.

Thanks for checking - good to know that I am hopefully just not missing something simple :slight_smile:

I found a log file in the nixpkgs repo for what I believe is the update to 0.20.0 (this link: Log viewer loading...) . I am not that adapt to reading these logs but to me it kind of looks like it is building from 0.19.12 wheel file:

[...]
Running phase: installPhase
Executing pypaInstallPhase
Successfully installed polars-0.19.12-cp38-abi3-linux_x86_64.whl
Finished executing pypaInstallPhase
Running phase: pythonOutputDistPhase
Executing pythonOutputDistPhase
Finished executing pythonOutputDistPhase
[...]