The presentation took place yesterday.
I have spent a good part of the last 2 decades standing in front of rooms full of people, explaining technical material in highly interactive ways. Inviting continuous questions and observing the audience’s body language were both a fundamental part of my approach. This was a remote presentation where everyone had cameras and microphones muted throughout, until the Q&A session at the end. So, compared to what I am accustomed to, this left me with absolutely no idea how it went!
Very brief summary of the presentation:
Software packaging/configuration is a nightmare: Nix solves some of the major problems.
Brief outline of major problems, along with non-technical description of how they are absent with Nix.
Brief, high-level description of underlying Nix theory
- pure functions
- content addressing
Briefly link theory to aforementioned problems and their solutions.
Emphasize unprecedented reproducibility: important for science.
Why we shouldn’t move to Nix:
- biggest problem: HPC clusters
- complexity for the ‘Nix-admins’ of the project
- Our Nix Bus number is currently 1
Why we should stick with Conda: ease of installation on HPC clusters.
Value proposition: If you install Nix on a machine, I can guarantee that
you can forget about config woes on Nix-using projects.
Live demo, width direnv support, of:
cd project-with-complex-installation-procedure: dependencies magically appear!
git checkout commit-with-different-dependency-versions: dependencies magically update!
Ephemeral package installation with
nix-shell -p (I insist on avoiding
Edit declaration of required packages in project’s
shell.nix: package magically installed!
cd dependencies magically disappear, leaving an unadulterated environment.
I toyed with the idea of installing Nix and one of our projects, from scratch in a VM, in the demo. In the end I decided against it because:
- there’s a lot of stuff to download, and it would take too long,
- I didn’t want to risk the Nix bandwidth throttling the conference connection.
As I said earlier, I have no idea how this was received, because, throughout the presentation I had as much audience feedback as I’d get from a brick wall: for all I know, they all went for a coffee while I was talking.
The Q&A session afterwards didn’t suggest any obvious opposition.
From what little feedback I have, I (would like to?) think that the value proposition and the demo were convincing, and what went before gave some reason to believe that they weren’t just empty salesman’s claims.
Much of the discussion was devoted to understanding our HPC situation. It seems that Singularity is installed almost everywhere we care about. We have some machines which we control or where we have strong influence, which run CentOS 6, precluding a multi-user install of Nix. On some of these we’ll try Singularity. On the one we control fully, we’ll probably upgrade OS, but not for a while, as it is currently involved in important production runs.
MacOS was also discussed, in the shadow of the bigger HPC issue. I guess it will take centre stage once we get over the HPC hurdle. The specific project in which I am proposing to replace existing Conda-reliance with Nix, seems to work OK in terms of macOS-on-Nix. Further down the line, the biggest problem is probably going to be Geant4. As a non-Mac-user, I have no idea where to start on this. I am utterly confused by the status/need of XCode in darwin support in nixpkgs.
We will, at leisure, try running Nix in Singularity and take it from there.
(There will be second-order issues to address down the line, such as macOS, MKL, CUDA, AVX, etc. but I think that getting over the first hurdle of Nix acceptance will be a big leap forward for the project, which will pay dividends in the long term.)
Thank you all for your input on this topic.
At the moment it looks like Singularity might be a life-saver! As I have zero experience with it, I have to reserve judgement, but if it works out then it will have simplified the task significantly, and may well turn out to be the single factor which makes Nix adoption possible in our case.