Installing pandas (and other python libraries)

Help
1

I’m brand new to nixos. I had no trouble installing it. I’ve added software to the configuration.nix file and rebuilt it, no worries. I installed python3 within my user profile in configuration.nix. That also worked fine. But, how/where do I install a library (specifically pandas at this point)?? I’ve tried the following 2 options, but I’m not sure if I’m getting it completely correct and in the right place in the config file (from the nixos wiki):

To install Python packages you have to declare them when installing Python:
let
my-python-packages = ps: with ps; [
pandas
requests
# other python packages
];
in
environment.systemPackages = [
(pkgs.python3.withPackages my-python-packages)
];

If this does not work, try:

environment.systemPackages = with pkgs; [
(python38.withPackages(ps: with ps; [ pandas requests]))
];
}

Neither one worked. Now, with that said, I tried the above with python311 instead of python38 and a host of other versions. The configuration.nix file fails to build with a variety of errors. Does anyone have or can anyone provide a configuration.nix file entry that shows a specific version of python being installed and includes the pandas library. The only successful pandas install I’ve had is from within a terminal window using the following:
nix-shell -p python311Packages.pandas
I was able to start python3 and import the pandas library within that terminal window. I was unable to import pandas within Thonny in the same user session. Also, this install of pandas was not persistent to my next login…meaning I would have to do this again on every reboot, unless I scripted this as part of the login process.

I’ve read many forum posts about python environments in nixos and because of my lack of understanding of different python environments and nixos they just don’t give the complete picture from start to finish, there are always details missing and I’m reluctant to just read and read and try and fail a thousand times to get something simple like the pandas library installed.

I will say this though, I love nixos, the way it looks (gnome), the efficiency, and the overall concept and would like to keep going forward with it. I’m just looking for some python library install hints so I can give up this frustration. Thanks in advance.

2

what’s the build error?

3

Thanks for responding.

I get syntax errors with both Wiki suggestions. Part of the problem for me is it says to try “this” and if “this” doesn’t work, then try "that’. But there aren’t many specifics here as to where those code snippets belong in the configuration.nix file, which is why I was asking for someone to post a working version that I could look at. Using the first solution from the nixos wiki python entry above that starts with “let” (not that it says where the code actually goes within the configuration.nix file, and not that I could guess where it should go):

error:
       … while evaluating the attribute 'config'

         at /nix/store/xc3h7hmrs5mc2rajsb070p6i4vrhnnnp-nixos-23.11.2217.d02d818f22c7/nixos/lib/modules.nix:320:9:

          319|         options = checked options;
          320|         config = checked (removeAttrs config [ "_module" ]);
             |         ^
          321|         _module = checked (config._module);

       … while calling the 'seq' builtin

         at /nix/store/xc3h7hmrs5mc2rajsb070p6i4vrhnnnp-nixos-23.11.2217.d02d818f22c7/nixos/lib/modules.nix:320:18:

          319|         options = checked options;
          320|         config = checked (removeAttrs config [ "_module" ]);
             |                  ^
          321|         _module = checked (config._module);

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: syntax error, unexpected ID, expecting '{'

       at /etc/nixos/configuration.nix:98:11:

           97|       freecad
           98|       let my-python-packages = ps: with ps; [pandas];
             |           ^
           99|       in environment.systemPackages = [(pkgs.python311.withPackages my-python-packages)];
building Nix...
error:
       … while evaluating the attribute 'config'

         at /nix/store/xc3h7hmrs5mc2rajsb070p6i4vrhnnnp-nixos-23.11.2217.d02d818f22c7/nixos/lib/modules.nix:320:9:

          319|         options = checked options;
          320|         config = checked (removeAttrs config [ "_module" ]);
             |         ^
          321|         _module = checked (config._module);

       … while calling the 'seq' builtin

         at /nix/store/xc3h7hmrs5mc2rajsb070p6i4vrhnnnp-nixos-23.11.2217.d02d818f22c7/nixos/lib/modules.nix:320:18:

          319|         options = checked options;
          320|         config = checked (removeAttrs config [ "_module" ]);
             |                  ^
          321|         _module = checked (config._module);

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: syntax error, unexpected ID, expecting '{'

       at /etc/nixos/configuration.nix:98:11:

           97|       freecad
           98|       let my-python-packages = ps: with ps; [pandas];
             |           ^
           99|       in environment.systemPackages = [(pkgs.python311.withPackages my-python-packages)];
building the system configuration...
error:
       … while evaluating the attribute 'config.system.build.toplevel'

         at /nix/store/xc3h7hmrs5mc2rajsb070p6i4vrhnnnp-nixos-23.11.2217.d02d818f22c7/nixos/lib/modules.nix:320:9:

          319|         options = checked options;
          320|         config = checked (removeAttrs config [ "_module" ]);
             |         ^
          321|         _module = checked (config._module);

       … while calling the 'seq' builtin

         at /nix/store/xc3h7hmrs5mc2rajsb070p6i4vrhnnnp-nixos-23.11.2217.d02d818f22c7/nixos/lib/modules.nix:320:18:

          319|         options = checked options;
          320|         config = checked (removeAttrs config [ "_module" ]);
             |                  ^
          321|         _module = checked (config._module);

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: syntax error, unexpected ID, expecting '{'

       at /etc/nixos/configuration.nix:98:11:

           97|       freecad
           98|       let my-python-packages = ps: with ps; [pandas];
             |           ^
           99|       in environment.systemPackages = [(pkgs.python311.withPackages my-python-packages)];

Here is the result (similar) of trying is the other way (the second way) from the wiki:
error:
… while evaluating the attribute ‘config’

     at /nix/store/xc3h7hmrs5mc2rajsb070p6i4vrhnnnp-nixos-23.11.2217.d02d818f22c7/nixos/lib/modules.nix:320:9:

      319|         options = checked options;
      320|         config = checked (removeAttrs config [ "_module" ]);
         |         ^
      321|         _module = checked (config._module);

   … while calling the 'seq' builtin

     at /nix/store/xc3h7hmrs5mc2rajsb070p6i4vrhnnnp-nixos-23.11.2217.d02d818f22c7/nixos/lib/modules.nix:320:18:

      319|         options = checked options;
      320|         config = checked (removeAttrs config [ "_module" ]);
         |                  ^
      321|         _module = checked (config._module);

   (stack trace truncated; use '--show-trace' to show the full trace)

   error: syntax error, unexpected '='

   at /etc/nixos/configuration.nix:98:34:

       97|       freecad
       98|       environment.systemPackages = with pkgs; [
         |                                  ^
       99|       (python38.withPackages(ps: with ps; [ pandas requests])) ];

building Nix…
error:
… while evaluating the attribute ‘config’

     at /nix/store/xc3h7hmrs5mc2rajsb070p6i4vrhnnnp-nixos-23.11.2217.d02d818f22c7/nixos/lib/modules.nix:320:9:

      319|         options = checked options;
      320|         config = checked (removeAttrs config [ "_module" ]);
         |         ^
      321|         _module = checked (config._module);

   … while calling the 'seq' builtin

     at /nix/store/xc3h7hmrs5mc2rajsb070p6i4vrhnnnp-nixos-23.11.2217.d02d818f22c7/nixos/lib/modules.nix:320:18:

      319|         options = checked options;
      320|         config = checked (removeAttrs config [ "_module" ]);
         |                  ^
      321|         _module = checked (config._module);

   (stack trace truncated; use '--show-trace' to show the full trace)

   error: syntax error, unexpected '='

   at /etc/nixos/configuration.nix:98:34:

       97|       freecad
       98|       environment.systemPackages = with pkgs; [
         |                                  ^
       99|       (python38.withPackages(ps: with ps; [ pandas requests])) ];

building the system configuration…
error:
… while evaluating the attribute ‘config.system.build.toplevel’

     at /nix/store/xc3h7hmrs5mc2rajsb070p6i4vrhnnnp-nixos-23.11.2217.d02d818f22c7/nixos/lib/modules.nix:320:9:

      319|         options = checked options;
      320|         config = checked (removeAttrs config [ "_module" ]);
         |         ^
      321|         _module = checked (config._module);

   … while calling the 'seq' builtin

     at /nix/store/xc3h7hmrs5mc2rajsb070p6i4vrhnnnp-nixos-23.11.2217.d02d818f22c7/nixos/lib/modules.nix:320:18:

      319|         options = checked options;
      320|         config = checked (removeAttrs config [ "_module" ]);
         |                  ^
      321|         _module = checked (config._module);

   (stack trace truncated; use '--show-trace' to show the full trace)

   error: syntax error, unexpected '='

   at /etc/nixos/configuration.nix:98:34:

       97|       freecad
       98|       environment.systemPackages = with pkgs; [
         |                                  ^
       99|       (python38.withPackages(ps: with ps; [ pandas requests])) ];

Again, another syntax error, probably because I have no idea if this is the full snippet needed and no idea where to put it.

Thanks again for offering help.

4

Placing the second suggestion in the place below, actually builds, and I can start the python interpreter, but pandas still isn’t available.

  environment.systemPackages = with pkgs; [
  #  vim # Do not forget to add an editor to edit configuration.nix! The Nano editor is also installed by default.
  #  wget
  python311
  (python311.withPackages(ps: with ps; [ pandas requests]))
  gnome-extension-manager
  gnomeExtensions.dash-to-dock
  ];
5

Well, now you have two conflicting Python derivations installed – one without the modules and one with:

python3.withPackages creates a self-contained Python derivation, the modules will not be available outside "${python3.withPackages (…)}/bin/python3".

And, the /run/current-system/sw/bin/python3 symbolic link created by environment.systemPackages can only point to a single derivation’s $out/bin/python3. And, as a result, only one of them can be available on PATH under the python3 name. If the collision resolves to the one without modules, then you will not have the modules.

Finally, other modules than your configuration.nix can be adding python3 to environment.systemPackages too. Or it might even be on PATH due to some other source (e.g. you might have installed it with nix-env and forgotten about it). Try removing it from your NixOS configuration and verifying it disappears from PATH to be sure.

6

Thanks for explaining that.

I removed the first python311 and rebuilt and I can now start python3 and pandas is available. When reading the wiki instructions (and not being familiar with nixos syntax), I read the part about installing libraries as an additional line to add to get libraries installed, rather than as a line that installs python311 and the specified libraries.
With that said, I installed Thonny as a python IDE and it still isn’t seeing system installed pandas as an available library, however, I believe that Thonny maintains it’s own library environment/options rather than referencing the system install libraries. I can easily install pandas within Thonny (through it’s pypi tools), so no issues there.

Thank you for your time and help jtojnar, I’m glad to have gotten past this issue. So, for anyone else struggling with this, this is what worked for me and where I installed it in my configuration.nix file (in the system profile):

  # List packages installed in system profile. To search, run:
  # $ nix search wget
  environment.systemPackages = with pkgs; [
  #  vim # Do not forget to add an editor to edit configuration.nix! The Nano editor is also installed by default.
  #  wget
  #python311
  (python311.withPackages(ps: with ps; [ pandas requests]))
  gnome-extension-manager
  gnomeExtensions.dash-to-dock
  ];

It also works here in the user profile:

  # Define a user account. Don't forget to set a password with ‘passwd’.
  users.users.bryan = {
    isNormalUser = true;
    description = "Bryan";
    extraGroups = [ "networkmanager" "wheel" ];
    packages = with pkgs; [
      firefox
      cura
      #python311
      #(python311.withPackages(ps: with ps; [ pandas requests]))
      thonny
      libreoffice-fresh
      freecad
      ];
    #  thunderbird
  };

Aside from being user-wide vs system-wide, I’m still learning what the implications of the different choices are.

Thanks again jtojnar!