Replacing builtins.fetchTarball with pkgs.fetchFromGitHub

Help
1

The NixOS Wiki has a nice snippet describing how to use home-manager as a NixOS module.

I would like to have a config that does not need to download things during evaluation time, so I thought I’d replace

home-manager = builtins.fetchTarball "https://github.com/nix-community/home-manager/archive/master.tar.gz";

by

  home-manager = pkgs.fetchFromGithub {
    owner = "nix-community";
    repo = "home-manager";
    rev = "888eac32bd657bfe0d024c8770130d80d1c02cd3";
    sha256 = "0kj49bdl67d1yf5wvqfcrlhf13jmqgvrl33k2bscw1crinab9na9";
  };

So I end up with

{ config, pkgs, ... }:
let
  home-manager = pkgs.fetchFromGitHub {
    owner = "nix-community";
    repo = "home-manager";
    rev = "888eac32bd657bfe0d024c8770130d80d1c02cd3";
    sha256 = "0kj49bdl67d1yf5wvqfcrlhf13jmqgvrl33k2bscw1crinab9na9";
  };
in
{
  imports = [
    (import "${home-manager}/nixos")
  ];

  home-manager.users.jan = {
    home.packages = [ pkgs.jq ];
  };
  users.users.jan = {
    isNormalUser = true;
  };
}

Trying to build this results in:

, nixos-generate --configuration example_config.nix --format lxc
error: infinite recursion encountered, at /nix/var/nix/profiles/per-user/root/channels/nixos/lib/modules.nix:365:28
(use '--show-trace' to show detailed location information)

Running it with --show-trace suggests that pkgs is the problem. It works If I replace the file contents with the following:

{ config, ... }:
let
  pkgs = import <nixpkgs> {};
  home-manager = pkgs.fetchFromGitHub {
    owner = "nix-community";
    repo = "home-manager";
    rev = "888eac32bd657bfe0d024c8770130d80d1c02cd3";
    sha256 = "0kj49bdl67d1yf5wvqfcrlhf13jmqgvrl33k2bscw1crinab9na9";
  };
in
{
  imports = [
    (import "${home-manager}/nixos")
  ];

  home-manager.users.jan = {
    home.packages = [ pkgs.jq ];
  };
  users.users.jan = {
    isNormalUser = true;
  };
}

While that makes it work, I am confused why. If I write a very minimal config that included pkgs.jq that works without my workaround, so the pkgs passed to the expression doesn’t seem to be completely wrong…?

4
  1. Problem: this uses important from derivation, which is slow as it requires at least one additional evaluation.
  2. Problem: you import HM as a module, which might change pkgs, which would cause a re import of nixpkgs, which might cause a rebuild which causes a reimport which causes change which causes import which causes infinite recursion.

Possible solutions:

  1. Use a home manager channel
  2. Use a wrapper that prefetches an locks HM
  3. Use flakes
  4. Use the ugly additional import (which even might be a different nixpkgs than the one you use everywhere else)
1 Like
5

Problem: you import HM as a module, which might change pkgs , which would cause a re import of nixpkgs, which might cause a rebuild which causes a reimport which causes change which causes import which causes infinite recursion.

I see. I guess the original solution from the NixOS wiki doesn’t suffer from this as pkgs is not involved in importing the home-manager module.

I settled for solution one for now, although when I would install that to a new computer, I would first have to apply a config without home-manager, then add the home-manager channel, and then continue with the final config including home-manager.

6

You can use builtins.fetchTarball with a specific URL and sha256 to achieve the same effect as fetchFromGitHub (another alternative would be builtins.fetchGit with a rev):

builtins.fetchTarball {
  url = "https://github.com/nix-community/home-manager/archive/888eac32bd657bfe0d024c8770130d80d1c02cd3.tar.gz";
  sha256 = "0kj49bdl67d1yf5wvqfcrlhf13jmqgvrl33k2bscw1crinab9na9";
}

I’d recommend this approach as the builtin fetchers are basically made for fetching eval time dependencies and avoid the double evaluation of nixpkgs you have in your third example.

2 Likes
7

Will the nixpkgs downloaded by builtins.fetchTarball be persistent? I somehow believed that the builtin fetchers cache stuff for 2 hours our so, and afterwards have to download things again. That was my main motivation for trying the pkgs.fetchFromGitHub approach.

8

That’s only if you don’t specify a sha256 argument. Without a fixed hash, the builtin fetchers don’t assume that the content at the URL won’t change.

1 Like
9

To solve that problem you have to do even more preparations than just using pkgs.fetch*, even though this creates a derivation and will create an entry in the store, it will be ephemeral unless you consider countermeasures.

You need to create a “fixed” reference in your configuration that holds on a GC-lock for the downloaded store path.

That should be considered by a content sha’d built-in as well.

1 Like
10

Nice, then the builtin with a sha256 is exactly what I want!

11

Yes, preventing things from being garbage-collected is a problem that I haven’t solved at all, especially for nix-shells. So far I just don’t garbage-collect too often :wink: I’d love some garbage-collection based on age mechanism. Just searched for it, there is actually a Github issue: Enhancement: garbage collect based on age · Issue #1680 · NixOS/nix · GitHub

12

It won’t expire after TARBALL_TTL, but it may be collected by nix-store --gc unless you gcroot it which is also the case for eval time only pkgs.fetchFromGitHub though.

One possibility would be to hack in a thing in your system which creates a symlink to the path of the fetchTarball result.

1 Like