Python/ansible shell not using the chosen python interpreter in the path

I’m using a dev shell for ansible like this, but I noticed the python interpreter that is in the path (or at least has a higher priority) is not the same as the one for which ansible is installed:

{ pkgs ? import <nixpkgs> {} }:

let
  py = pkgs.python310;
  pypkgs = [ py (py.withPackages (p: with p; [ pexpect ansible-core jmespath ]))];
  ansibleCollectionPath = pkgs.callPackage ./ansible-collections.nix {} pkgs.ansible {
    "containers-podman" = {
        version = "1.9.3";
        sha256 = "sha256:1vjsm7696fp9av7708h05zjjdil7gwcqiv6mrz7vzmnnwdnqshp7";
    };
  };
in

pkgs.mkShell {
  buildInputs = with pkgs; [
    ansible-lint # for vscode plugin (rest of vscode is in user env)
  ] ++ pypkgs;
  shellHook = ''
    export ANSIBLE_COLLECTIONS_PATHS="${ansibleCollectionPath}"
    export SSH_AUTH_SOCK=/run/user/$UID/keyring/ssh
    export GIT_SSL_CAINFO=/etc/ssl/certs/ca-certificates.crt
    export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
  '';
}

I’m using this shell.nix with nix-direnv, generally the env is loaded ok (I do have e.g. ansible-playbook in my vscode terminal), but:

> which python
/nix/store/rdm31z7s8m6aaydnq949g2zql4wixsgd-python3-3.9.13/bin/python
> which ansible-playbook
/nix/store/463z4qfsqhx3bfrribqbb0v8pzm2q39q-python3.9-ansible-core-2.13.0/bin/ansible-playbook

Note that’s python-3.9 whereas I explicitly listed python 3.10 in the buildInputs.

I know the shell is impure, but unloading direnv proves that I have not installed anything related user/system-wide:

> which python3
python3 not found
> which ansible-playbook
ansible-playbook not found

Any ideas?

Check your PATH. Likely you will see your Python 3.10, it’s just not the first Python on PATH.

Yes, 3.10 it is there, but the question is how python-3.9 got there in the first place (with the above expression).

Other than that, shouldn’t a shell (even if impure) prefer (i.e. through the PATH order) the buildInputs that it contains? Otherwise impure shells would be pretty much unusable?

So that should mean my shell.nix is somehow not doing what I think it should be doing? (I’m not doing much else, just to make sure, the ansible-collections.nix that’s called only does this:

{ stdenv, lib, pkgs }: ansible: collections:

let

  installCollections =
    lib.concatStringsSep "\n" (lib.mapAttrsToList installCollection collections);

  installCollection = name: versionAndSha256:
    "${ansible}/bin/ansible-galaxy collection install ${collection name versionAndSha256}/collection.tar.gz";

  collection = name: { version, sha256 }:
    stdenv.mkDerivation {
      pname = name;
      version = version;

      src = builtins.fetchurl {
        name = name;
        url = "https://galaxy.ansible.com/download/${name}-${version}.tar.gz";
        sha256 = sha256;
      };

      phases = [ "installPhase" ];

      installPhase = ''
        mkdir -p $out
        cp $src $out/collection.tar.gz
      '';
    };

in
pkgs.runCommand "ansible-collections" {} ''
  mkdir -p $out
  export HOME=./
  export ANSIBLE_COLLECTIONS_PATH=$out
  ${installCollections}
''

It probably comes from ansible-lint which propagates the interpreter.

nix-shell with mkShell is just a bad solution as it gives a build environment. A much better method is to put your items of interest in a buildEnv and add that to buildInputs.

1 Like

Thanks! That was not only entirely correct, but also highly informative!

So I finally ended up with this shell, which seems to implement your suggestion if I’m not mistaken?

{ pkgs ? import <nixpkgs> {} }:

let
  myPkgs = with pkgs; (python310.withPackages (p: with p; [ pexpect ansible-core ansible-lint jmespath ]));
  myEnv = pkgs.buildEnv {
    name = "ansible-env";
    paths = [ myPkgs ];
  };
  ansibleCollectionPath = pkgs.callPackage ./ansible-collections.nix {} myPkgs.pkgs.ansible-core {
    "containers-podman" = {
        version = "1.9.3";
        sha256 = "sha256:1vjsm7696fp9av7708h05zjjdil7gwcqiv6mrz7vzmnnwdnqshp7";
    };
  };
in

pkgs.mkShell {
  buildInputs = [ myEnv ];
  shellHook = ''
    export ANSIBLE_COLLECTIONS_PATHS="${ansibleCollectionPath}"
    export SSH_AUTH_SOCK=/run/user/$UID/keyring/ssh
    export GIT_SSL_CAINFO=/etc/ssl/certs/ca-certificates.crt
    export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
  '';
}

Indeed it appears that now all dependencies are consistently under the same storage path!
I was struggling a bit with how to refer to the same pkg attribute for ansible-core as is in the attr list for the env to pass to the ansible collection function, but it seems that’s also ok now.

Indeed this should be the default solution to use for build and dev envs.

1 Like

what does this technically mean (e.g. where are the issues with this combination)?
Where is this documented?

As I now think I understood, going the buildEnv way has two advantages:

  • it makes it more likely you define all your buildInputs in one list, limiting the chance that you’ll have different instances of the same package being pulled in as respective dependencies
  • it produces one derivation of which all parts (all under the store path with the chosen name of the buildEnv) are sure to refer to eachother as necessary.

I’d still be interested in a documentation link too, I found no info on buildEnv in the context of dev shells.