Usability study session #6

toraritte · September 4, 2022, 1:21am

1. Bio / Persona

Software developer. Has a BSc in in Computer Science, and did systems programming in college.
Self-taught functional programmer, and interested in the paradigm as a way to prove stuff about programs. Jumped to Haskell to get a grasp on type theory. Read some chapters of Programming in Haskell and Type-driven Development with Idris.

Not a user who reads documentation.

2. Prior Nix experience

Found Nix when needed to set up an AWS application using different programming languages. Wanted to test locally, but it was a pain, and learned to use Nix out of anger.

Using MacOS as much as possible with Nix. When setting up Clear Linux or Amazon Linux in the cloud, the first thing done is to install Nix.

Also used Nix for Haskell (via haskell.nix); likes that it provides the flexibility needed (e.g., download packages from Hackage and create lock files), but doesn’t want to use the Nix infrastructure for Haskell.

Packaged a lot of Python applications with Nix.

6.3 Approaches to learning Nix

Wants to do stuff with the latest versions of software, so jumped right at the bleeding edge, starting with flakes immediately. Found out about flakes with no help, and finds them a better abstraction for packages instead of functions:

clear where inputs come from
callPackage convention is not visible

6.4 Using Nix

Built a machine learning environment previously that is somewhat working, but entirely unsatisfactory as it is mostly a bunch of Docker images glued together. Doesn’t like the Conda package manager, and would love to have NVIDIA CUDA tooling with Nix instead.

The goal is to create a Nix flake that provides an environment that allows to build and hack on Python files, and this session starts with trying to write it from scratch.

Immediately sets inputs.nixpkgs = nixpkgs-unstable
Does not remember Nix syntax well, especially flakes, so usually copy-pastes flakes around.
Needs pytorch and a few other dependencies:
Goes to search.nixos.org to grab package names to put them into buildInputs

Does python310 have withPackage?

$ nix repl

nix-repl> :lf nixpkgs/nixpkgs-unstable

Tab-mashing ensues to go through available attributes, and withPackages is there.

INFO The `nix-reply` help prompt for reference:

nix-repl> :?
The following commands are available:

 <expr>        Evaluate and print expression
 <x> = <expr>  Bind expression to variable
 :a <expr>     Add attributes from resulting set to scope
 :b <expr>     Build a derivation
 :bl <expr>    Build a derivation, creating GC roots in the working directory
 :e <expr>     Open package or function in $EDITOR
 :i <expr>     Build derivation, then install result into current profile
 :l <path>     Load Nix expression and add it to scope
 :lf <ref>     Load Nix flake and add it to scope
 :p <expr>     Evaluate and print expression recursively
 :q            Exit nix-repl
 :r            Reload all files
 :sh <expr>    Build dependencies of derivation, then start nix-shell
 :t <expr>     Describe result of evaluation
 :u <expr>     Build derivation, then start nix-shell
 :doc <expr>   Show documentation of a builtin function
 :log <expr>   Show logs for a derivation
 :te [bool]    Enable, disable or toggle showing traces for errors

Refactors using python310.withPackages
Stops here as the RAPIDS package is not available on Nixpkgs.

Thinks about figuring out if there is some way to build it from source. Goes to NGC catalog, starts reading about how a Docker container is prepared, but feels inexperienced using Nix with Docker. Estimates a couple of full days to make this work.

“Having to do stuff on my own makes me wanting to do it less, and contributing slows down the process.” Already wrote an own z3 package, and wanted to test arbitrary versions, but the Nixpkgs flake did not allow overriding versions.

Interviewer:
What made it impossible to override versions for z3?
Participant:
I did not want to override the “environment”, i.e. the surrounding Nixpkgs collection and stdenv.

Interviewer:
Do you know about overrides and overlays?
Participant:
Yes, using overlays for the “sourced Nixpkgs” for different versions of z3. Wrote the overrideAttr expression after many iterations with help from colleagues that took about a week of trying and a month of on-and-off trying.

QUESTION: “sourced Nixpkgs” → pinned Nixpkgs?

Interviewer:
What made this task so difficult and time-consuming?
Participant:
Finding the right functions to use and how to use them is really hard. Not having types in the language made it feel like being blind.

Switching to another task: Nixify a TypeScript project. Done this before when setting up a visualization tool in TypeScript for Liquid Haskell. Want to use Deno but ran into some packaging problems with it before.

Starts out with copy-pasted flake template, and replaces Deno in buildInputs. Would like to override Deno to use a newer version, but needs to find its Nix expression first:

Searches for deno on search.nixos.org
Goes to the source of Deno’s Nix expression in Nixpkgs
Find version, src, sha... attrributes to override:

Types deno.override then stops: there are several flavors (override, overrideAttrs, etc.) to consider. Once settled on one, fills in exact same data as found in the original Deno Nix expression. Doesn’t know whether the syntax is right so keeps adding parentheses until nix develop stops complaining. New error crops up after the syntax is fixed:

undefined variable `pname`

Copy-pastes pname, and re-run nix develop. Misses that the attribute set is recursive then fixes it, and finally it works.

Looks online to find the latest version of Deno, copies the Git commit hash, and inserts it. Doesn’t know what the cargoSha256 should be, so keeps it empty, and hopes that Nix will tell the right one.

A ton of issues pop up, and one of them is a suggestion to use zeroes for the cargoSha256, but still errors out. Looks into the log, but nothing helpful is there.

The session ends here.

^archive