Properly wrapping a package to satisfy requireFile

Help
1

I’m trying to build a flake that takes user input (specifically, some authentication values) and wraps another package using overrideAttrs in order to inject a prePhase script that uses said input to fetch the relevant source.

In this specific case, nix-foundryvtt requires users to have already installed FoundryVTT from the website. It cannot otherwise be fetched declaratively, because of license restrictions (no permission to redistribute the software) and because downloading it requires authentication and CSRF validation.

The way I’ve structured this flake is that the package override can be configured to take a user’s authentication information, as well as the Foundry build information (because this doesn’t appear to be easily exposed from the base package provided by nix-foundryvtt, and is important for the downloaded filename).

Here is the general structure of the override function that my flake provides. You can find the full source here.

foundryPkg:
{
  username ? "",
  passwordFile ? "",
  # etc.
  ...
}:
foundryPkg.overrideAttrs (old:
  {
    prePhases = old.prePhases or [] ++ [''
      # Relevant bash script
    ''];
  }
)

Here’s an example configuration using said flake:

let
  basePackage = inputs.foundryvtt.packages.${pkgs.system}.foundryvtt_13;
  foundryvtt = inputs.fetch-foundryvtt.pkgOverride basePackage {
    username = "kolastor";
    passwordFile = config.sops.secrets.foundryvtt-password.path;
  };
in
{
  services.foundryvtt.package = foundryvtt;
}

But no matter what I do, the underlying derivation still complains about requireFile not being met.

How can I effectively preempt the resolution of mkDerivation.src, to run that script and ensure the requisite file is downloaded beforehand?

2

Can’t do it easily (if at all). This requires too many impure operations that aren’t easy to force through Nix. What’s the problem with doing it the usual way (i.e. download from URL, add to store, continue)?

1 Like
3

What’s the problem with doing it the usual way (i.e. download from URL, add to store, continue)?

It’s functional, but a bit inconvenient. I’d prefer to keep my nixos deployments more hands-off if possible, especially so when deploying remote servers.

Plus, fully declarative feels more like the nix way. I’d like to minimize imperative installations where I can (though, obviously, unfree software is going to be the biggest area where I can’t).

4

It’s still fully declarative. Nothing about the system deployment is done imperatively.

What you’re doing is just adding an FOD to the nix store by hand. It’s a source, and inherently an input to the build, it’s inherently impure and lies outside of what can be described by your code; the source hash is however still declared there.

Downloading it should also only be necessary once for each version, and therefore a very minor overhead; create a gcroot on your build machine so it doesn’t get collected and you’ll never have to do this by hand again.

If you use remote deployment the workflow also stays really convenient; make sure you’re not missing the fact that nix can do that. It sounds like you’re ssh-ing to a remote machine and doing a bunch of stuff manually on it.

You’ve almost certainly already spent more time trying to work around this than you ever would have just downloading new foundry versions, they don’t release new ones that often.

1 Like
5

Fair play on the mixed terminology. I guess I was confusing the initial installation with the actual deployment. The point was more to minimize those manual installation steps.

I definitely have been more concerned with whether I can, than whether I should, with this project. It’s served as a way for me to learn more about derivations in a bit more of a hands-on fashion, but it does sound like this particular work is more trouble than it’s worth.

I’ve not actually done remote machine deployment with nix yet, I’ve been purely working locally and was more looking to the future where I plan on configuring remote machines. Do you have any pointers for where to look when it comes to nix’s strengths in that respect?

6

Won’t your script need access to the network? It looks like it should be in a fixed-output derivation, which is then used to replace src

7

Nix can send closures over ssh using copy-closure, which lets you send fully-built system closures to remote machines. After that you just need to run the activation script on the remote, which basically all deployment tools support doing for you.

It also supports remote builds, which lets you invoke nix on a remote machine, which lets it perform a specified build (and download its own files, or substitute them, as well as get any missing files supplied with copy-closure).

This lets you deploy NixOS configurations in many different ways, depending on your needs, without giving up any of the convenience of doing stuff with your local dev machine. You should never want to log into a shell on your remotes.

E.g. in this case, you’ll download the foundry tarball on your dev machine, and create a gcroot for it. You can either build your remote system locally, and send the fully-built foundry over the network, or you can ask nix to build stuff on the remote, in which case it will grab the foundry tarball from your local store so the remote can build foundry. In either case, you never have to manually download the tarball on the remote - nix handles all of this for you.

I think the most user-friendly entrypoint into that is this project: GitHub - serokell/deploy-rs: A simple multi-profile Nix-flake deploy tool.

But remote deployments are fully supported by nixos-rebuild as well, it’s just a bit more finnicky to have to learn about the CLI flags and what options you even have: nixos-rebuild - Official NixOS Wiki

I also really recommend setting up PAM to use SSH agent authentication for sudo, since that fixes a few bugs with this stuff, and is anyway neat: NixOS Search

1 Like