Nix develop python package fails test

Help
1

I have a flake that brings in various Python packages and works with nix develop to give me the development environment I like. But when I added the redshift-connector package, it stopped working.

I tried to use an overlay to set doCheck to false for the package, by following How to override a Python package using overlays.

This is the result (with the full list of packages removed for some brevity):

{
  description = "Python Data Science Redshift Environment";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem
      (system:
        let
          overlay = self: super: {
            python = super.python.override {
              packageOverrides = python-self: python-super: {
                redshift-connector = python-super.redshift-connector.overridePythonAttrs (oldAttrs: {
                  doCheck = false;
                });
              };
            };
          };

          pkgs = import nixpkgs {
            inherit system;
          };
        in
        with pkgs;
        {
          devShells.default = mkShell {
            name = "py-redshift";
            packages = [
              python311Packages.pandas
              python311Packages.redshift-connector
            ];
          };
        }
      );
}

However the tests still run, and fail.

$ nix develop
error: builder for '/nix/store/zi0hjw4d90yjbx7p9asgd3sxshd1yw64-python3.11-redshift-connector-2.0.914.drv' failed with exit code 2;
       last 10 log lines:
       >     csv_str = "\col1\n" + "1\n" * max_params + "1"  # 32768 rows
       >
       > -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
       > =========================== short test summary info ============================
       > ERROR test/performance/bulk_insert_performance.py - configparser.NoSectionError: No section: 'ci-cluster'
       > ERROR test/performance/performance.py - configparser.NoSectionError: No section: 'ci-cluster'
       > ERROR test/performance/protocol_performance.py - redshift_connector.error.InterfaceError: ('communication error', Connection...
       > !!!!!!!!!!!!!!!!!!! Interrupted: 3 errors during collection !!!!!!!!!!!!!!!!!!!!
       > ======================== 28 warnings, 3 errors in 0.86s ========================
       > /nix/store/bbxdw4rgwwl3gnajri82yidr1nlsfskf-stdenv-linux/setup: line 1596: pop_var_context: head of shell_variables not a function context
       For full logs, run 'nix log /nix/store/zi0hjw4d90yjbx7p9asgd3sxshd1yw64-python3.11-redshift-connector-2.0.914.drv'.
error: 1 dependencies of derivation '/nix/store/6avnk7775cpggn4z9i3lnsc1yn3wv498-py-redshift-env.drv' failed to build

I have a sense that I am not overriding the correct thing, but I am struggling to figure it out. What am I missing? (Edit: looking at this having typed it, I am also suspicious of the overlay being inside the eachDefaultSystem without system being referenced anywhere inside it…

2

greetings !

i haven’t tried to get it working using overlay but this works

{
  description = "Python Data Science Redshift Environment";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem
      (system:
        let
          python = pkgs.python3.override {
            self = python;
            packageOverrides = python-self: python-super: {
              redshift-connector = python-super.redshift-connector.overridePythonAttrs (oldAttrs: {
                doCheck = false;
              });
            };
          };

          pkgs = import nixpkgs {
            inherit system;
          };
        in
        with pkgs;
        {
          devShells.default = mkShell {
            name = "py-redshift";
            packages = [
              (python.withPackages (p: with p; [
                pandas
                redshift-connector
              ]))
            ];
          };
        }
      );
}
1 Like
3

It does work, thank you very much @kirillrdy !

I understand what you have done. I don’t entirely know how I could have figured it out myself. I am at the stage of the learning curve where I understand some generalities, but I don’t know enough to arrive at idioms and conventions from first principles. This pattern should get me working, at least.