Feedback on how I structure the nix code for my Thesis repo

Help
1

I’m writing my thesis for university at the moment and I’m managing everything connected to it in a monorepo. This means I have Latex documents to build, Typst documents and also further assets requiring a diverse set of build envs.

So far, I am managing the builds with nix. However, as I’m new to nix, I don’t know how to structure my nix files and would like to get feedback on my current structure.

With my flake interface, I am happy. I am exposing each individual file to be build (pdf document, asset file, etc.) as a package of the flake.

As a matter of fact, I have 3 subdirectories called notes, expose and assets.
At the moment, each of these subdirectories gets a packages.nix file which lists all of the packages which exist in that directory and its subdirectories.
For example, the file /assets/packages.nix looks like this:

{ libreoffice, typst, stdenv, lib, inkscape, latex, time-schedule, typix, system }:
let
  make = import ../nix/make-libreoffice.nix { inherit stdenv libreoffice lib; };
  listSources = import ../nix/list-sources.nix lib.fileset;

  expose = stdenv.mkDerivation {
    name = "Bachelor Thesis Latex";
    src = listSources [ ./expose.tex ../works.bib ];
    nativeBuildInputs = [ latex inkscape ];
    buildPhase = ''
      ${import ../nix/setup-links-script.nix {inherit lib;} {"build/assets/time-schedule.svg" = time-schedule;}}
      export HOME=$(mktemp -d)
      latexmk -shell-escape -lualatex artifacts/expose.tex
      mv expose.pdf $out
    '';
  };
  expose-presentation = typix.lib.${system}.buildTypstProject {
    name = "Expose Presentation";
    src = listSources [ ./expose-presentation.typ ../works.bib ../notes/lib.typ ../notes/defs.typ ../assets/equi-consistency-diagram.svg ./res ];
    typstSource = "artifacts/expose-presentation.typ";
    typstOpts = { root = ".."; };
    virtualPaths = [
      {
        dest = "build/assets/time-schedule.svg";
        src = time-schedule;
      }
    ];
    unstable_typstPackages = import ../nix/typst-packages.nix;
  };
in
{
  inherit expose expose-presentation;
  recap-for-romain = make ./misc/recap-for-romain.odp;
}

listing one latex, one typst and one office document as packages. They are wrapped into a function declaring the dependencies.

Then in my flake.nix, I am inserting these packages into the flake’s package list like so:

inherit (callPackages ./artifacts/packages.nix { }) expose expose-presentation recap-for-romain;

.
As you can see, I am using the callPackages function.
However, I am defining my own version of that so I can inject all of the flake packages themselves into dependency resolution:

callPackages = pkgs.lib.callPackagesWith (pkgs // packages // { inherit latex callPackages typix; });

Furthermore, when I have duplicate code like for building office files (which is needed in different subdirectories), I am placing them in the directory /nix/….

I hope this was enough to understand the structure of my nix code. I would be very glad for any feedback.

The thing I am most curious about is whether I should use callPackages with a function per subdirectory returning an attribute set like at the moment, or whether I should have an attribute set of functions per subdirectory.
The latter would allow defining dependencies more granularily but would add a little blote to the nix files.

Thank you very much!

2 Likes
2

It’s up to you to decide what your natural unit of ‘package’ is—is assets a package? If so, having one package.nix for the assets folder is good. If not, making a Nix file per thing in the assets folder would be better. Things like rendered documents are unconventional fits for the ‘package’ model, but that doesn’t matter too much—all you’re really using from the package model is the concept of dependencies. So if you want to go granular with your dependencies, do so; if not, don’t.

Speaking of dependencies, though, the feedback I’d give is that things like make-libreoffice.nix should also be dependencies injected via callPackage(s), instead of imported and having their own dependencies passed in at the use sites. So, assuming your assets are still all one package, I’d want to write it like this:

{
  buildTypstProject,
  listSources,
  makeLibreOffice,
  setupLinksScript,
  stdenv,
  time-schedule,
}:
{
  expose = stdenv.mkDerivation { ... };
  expose-presentation = buildTypstProject { ... };
  recap-for-romain = makeLibreOffice misc/recap-for-romain.odp;
}
1 Like
3

This is great, thank you very much!

I noted injecting the utility functions as function parameters. Is the benefit of that just looser coupling?

As for the packaging granularity, what I do not want to compromise is cache ability. So what i don’t want to have is for example one derivation which builds all assets each time something changes in one of them. But I suppose what I could do is use buildEnv to combine multiple derivations into one package?
As you mentioned, rendering documents is unconventional. If I want to have one package for assets for example, I would need the derivation for that package to either provide multiple outputs or to resolve to a folder containing the different files. Do you know what I mean? Is one version to be preferred? I assume it doesn’t really matter, right?

Otherwise, I would probably go ahead and make one file per package as you said.

4

That, and also less repetition. Each of those functions has the same dependencies wherever it’s used. Every instance of import ../nix/make-libreoffice.nix { inherit stdenv libreoffice lib; } is the same. Why repeat yourself like that when you can instead use callPackage nix/libreoffice.nix { } once in your top-level scope attrset, and let the dependency injection do its thing?

Ah, don’t confuse derivations (the things that get built and placed in the Nix store) with package files (the things you invoke with callPackage)! Derivation builds are cached individually regardless of how you package them up. A derivation will only be rebuilt if its inputs change.

Both of those would be ways to put assets into one derivation. But what you have right now splits each asset into its own derivation, and I think that’s smart. You’ve put them all in one package file, which may or may not be smart depending on how you want to think about it. The dependencies of a package are the function parameters at the top of the file; the inputs of a derivation are all the values that make it into the arguments of the derivation-creating function. In most cases, there’s a pretty tight alignment of the two, but you’re in a corner of the design space where there isn’t really a need to do that.

In other words, making your package files granular is part of how you organize your Nix code without affecting the final result, whereas making your derivations granular actually affects the final result and the process of how that result is assembled. You can keep them in sync if you want by making one package per derivation—that’s probably the conventional thing, or as close to conventional as your use case permits. You can also continue to do what you’re doing for simplicity, since you’re just writing scaffolding for your thesis, not building a long-lived collaborative software package repository!

1 Like
5

Nice, I understand it all now!
Special thanks to the elaborate and kind explanation, but also the tip for the utility functions!
<3