Ansible: How to manage collections with nix?

Development
1

Starting with ansible version 2.10, ansible and all its modules are no longer provided in a single repository, but split into

  • an ansible-base package, which includes some base modules and a collection manager (ansible-galaxy), and
  • collections, which are installed with ansible-galaxy (into ~/.ansible, by default).

See https://github.com/ansible-collections/overview/blob/b7a2e4a3d0ccfb7a9bad9c9c2b41a8337aa2ee16/README.rst#previously-ansible-29-and-earlier for more information on that. With nixos 20.09, nixpkgs provides ansible 2.10 as the default ansible version, which now no longer includes modules that were part of ansible 2.9 and earlier.

Originally, it was enough to enter a nix-shell providing ansible to be able to run ansible playbooks. Now you need to

  1. manually install the collections your ansible playbooks depend on, or
  2. get the necessary collections as nix packages, and research how to point ansible-specific environment variables to the correct locations in the nix-store.

2 is obviously the way you’d like it to work with nix, but nixpkgs currently provides no framework for dealing with collections, so 1 is the current way to do it if you need to get work done. What kind of user interface would nixpkgs even need to have for this to be actually usable and consistent with how other non-reproducible package managers are lifted into nixpkgs? I imagine this could be similar to how python packages are handled, but I don’t know how much effort it is to maintain such a package set.

Any input on how to approach lifting such a package manager into nixpkgs?

9 Likes
2

Managing ansible collections is about:

  1. downloading collections,
  2. then installing the collections with ansible-galaxy collection install <collection.tar.gz>.

The following function automates this inside the nix store:

{ stdenv, lib, pkgs }: ansible: collections:

/* Install ansible collections
 *
 * Collections are downloaded, then each collection is installed with:
 * $ ansible-galaxy collection install <collection.tar.gz>.
 *
 * The resulting derivation is the ansible collections path.
 *
 * Example:
 *
 * ANSIBLE_COLLECTIONS_PATH = callPackage ./ansible-collections.nix {} ansible_2_10 {
 *   "community-postgresql" = {
 *     version = "1.4.0";
 *     sha256 = "14izkq5knrch2xv4jx7jvxnq2lwhhdlyck7i7ga9mny3skrz9qfp";
 *   };
 * };
 */

let

  installCollections =
    lib.concatStringsSep "\n" (lib.mapAttrsToList installCollection collections);

  installCollection = name: versionAndSha256:
    "${ansible}/bin/ansible-galaxy collection install ${collection name versionAndSha256}/collection.tar.gz";

  collection = name: { version, sha256 }:
    stdenv.mkDerivation {
      pname = name;
      version = version;

      src = builtins.fetchurl {
        name = name;
        url = "https://galaxy.ansible.com/download/${name}-${version}.tar.gz";
        sha256 = sha256;
      };

      phases = [ "installPhase" ];

      installPhase = ''
        mkdir -p $out
        cp $src $out/collection.tar.gz
      '';
    };

in
pkgs.runCommand "ansible-collections" {} ''
  mkdir -p $out
  export HOME=./
  export ANSIBLE_COLLECTIONS_PATH=$out
  ${installCollections}
''
2 Likes
3

TL;DR

the env var needed by ansible is ANSIBLE_COLLECTIONS_PATHS (note the trailing “S”), which leads to e.g. the following shell.nix to make a nice ansible env:

{ pkgs ? import <nixpkgs> {} }:

let
    ansibleCollectionPath = pkgs.callPackage ./ansible-collections.nix {} pkgs.ansible {
        "containers-podman" = {
            version = "1.9.3";
            sha256 = "sha256:1vjsm7696fp9av7708h05zjjdil7gwcqiv6mrz7vzmnnwdnqshp7";
        };
    };
in
# we make an fhs to make easier portable playbook execution, which assumes #!/bin/sh for script commands
(pkgs.buildFHSUserEnv {
  name = "ansiblenv";
  targetPkgs = pkgs: with pkgs; [
      zsh
      (python39.withPackages (p: with p; [ pexpect ansible jmespath ]))
  ];
  runScript = ''
    ${pkgs.zsh}/bin/zsh
  '';
  profile = ''
    export ANSIBLE_COLLECTIONS_PATHS="${ansibleCollectionPath}"
    export SSH_AUTH_SOCK=/run/user/$UID/keyring/ssh
    export GIT_SSL_CAINFO=/etc/ssl/certs/ca-certificates.crt
    export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
  '';
}).env
4

at some point I got fed up with ansible-galaxy brokenness (forced to use it) and adapted your code at GitHub - teto/ansible2nix . HAven’t used it since then but could be a starting point for someone motivated

1 Like
5

Hi all,
NIX starter here.

I am trying to get the above scripts to run but during the ansible-galaxy install command in ansible-collections.nix I get the following error.

building '/nix/store/yychd5gmdsh5lgwy04c0277b41119yq6-ansible-collections.drv'...
No config file found; using defaults
Starting galaxy collection install process
Process install dependency map
[WARNING]: Skipping Galaxy server https://galaxy.ansible.com. Got an unexpected
error when getting available versions of collection ansible.posix: Unknown
error when attempting to call Galaxy at 'https://galaxy.ansible.com/api/':
<urlopen error [Errno -3] Temporary failure in name resolution>
ERROR! Unknown error when attempting to call Galaxy at 'https://galaxy.ansible.com/api/': <urlopen error [Errno -3] Temporary failure in name resolution>

I have the feeling the I am missing some env settings. Do you have any clues?