Local flake-based `nix search`, `nix run` and `nix shell`

On a flake-based Nix setup, nix search, nix shell or nix run a nixpkgs package often require downloading the nixpkgs tarball. This would take at least 10 seconds, 20 MiB of download consumption for each search and doesn’t work without network connection.

A solution would a small nix-search-local, nix-shell-local, and nix-run-local script in your profile that finds/execute the specified attributes in the flake you want that is already in the /nix/store.

Here’s what I do. I use nixos-20.09, install extra packages from nixos-unstable, and want to be able to search and run both of them locally.

For flake-based home-manager,

  • In flake.nix
    {
      inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-20.09";
      inputs.nixpkgs-nixos-unstable.url = "github:NixOS/nixpkgs/nixos-unstable";
      inputs.home-manager = "github:nix-community/home-manager/release-20.09";
      inputs.home-manager.inputs.nixpkgs.follows = "nixpkgs";
    
      let
        system = "x86_64-linux";
        username = "shamrock";
      in outputs = inputs@{ self, home-manager, ... }:
        homeManagerConfiguration.${username} = home-manager.lib.homeManagerConfiguration {
            configuration = import ./home.nix;
            inherit system username;
            homeDirectory = "/home/${username}";
            extraSpecialArgs = {
              # Inject the flakes here
              inherit (inputs) nixpkgs nixpkgs-nixos-unstable system-config-flake;
            };
          };
      };
    }
    
  • In home.nix
    args@{config, lib, pkgs, ... }:
    {
      # ...
      home.packages = with pkgs; [
        # ...
        (writeShellScriptBin "nix-search-local" ''
          nix search "${nixpkgs}" "$@"
        '')
        (writeShellScriptBin "nix-shell-local" ''
          if [ "$#" -lt 1 ]; then
            echo "Requires an argument" >&2
            exit 1
          fi
          ATTRIBUTE="$1"
          shift
          nix shell "${nixpkgs}#$ATTRIBUTE" "$@"
        '')
        (writeShellScriptBin "nix-run-local" ''
          if [ "$#" -lt 1 ]; then
            echo "Requires an argument" >&2
            exit 1
          fi
          ATTRIBUTE="$1"
          shift
          nix run "${nixpkgs}#$ATTRIBUTE" "$@"
        '')
        (writeShellScriptBin "nix-search-local-unstable" ''
          nix search "${nixpkgs-nixos-unstable}" "$@"
        '')
        (writeShellScriptBin "nix-shell-local-unstable" ''
          if [ "$#" -lt 1 ]; then
            echo "Requires an argument" >&2
            exit 1
          fi
          ATTRIBUTE="$1"
          shift
          nix shell "${nixpkgs-nixos-unstable}#$ATTRIBUTE" "$@"
        '')
        (writeShellScriptBin "nix-run-local-unstable" ''
          if [ "$#" -lt 1 ]; then
            echo "Requires an argument" >&2
            exit 1
          fi
          ATTRIBUTE="$1"
          shift
          nix run "${nixpkgs-nixos-unstable}#$ATTRIBUTE" "$@"
        '')
      ];
      # ...
    }
    

Similar setup can be put in /etc/configuration.nix environment.systemPackages

It may also written in a plain flake.nix and install with nix profile install, but I don’t know how and haven’t tried before.

If you want to write something like nix-search-system that search for the nixpkgs used by the NixOS system configuration flake, you can specify

{
  # ...
  inputs.system-config-flake.url = "/etc/nixos"
  # ...
}

Pass it to homeManagerConfiguration and use
"${system-config-flake.inputs.nixpkgs}" as the flake uri.

5 Likes

You can also use the nix registry command or the nix.registry option to overwrite the nixpkgs entry and make nix run nixpkgs#foo use your local thing.

1 Like

That sounds interesting. But reading nix registry add --help I could not yet figure out what the URI for the nixpkgs input of my currently running system configuration flake is.

Both of these did not work:

nix registry add sys /etc/nixos#nixpkgs
nix registry add sys /etc/nixos#inputs.nixpkgs

Do you have an idea how I could add a local entry to the registry to search the already downloaded version of nixpkgs?

1 Like

I found a way to set the registry in my flake.nix:

{
  inputs.nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
  outputs = { self, nixpkgs, ... }@inputs: {
    nixosConfigurations.foo = nixpkgs.lib.nixosSystem {
      modules = [
        ({ pkgs, ... }: {
          nix.registry.sys = {
            from = { type = "indirect"; id = "sys"; };
            flake = nixpkgs;
          };
        })
        # ...
      ];
    };
  };
}

Now I can nix search sys foo :slight_smile:

7 Likes

That sounds like a much cleaner and declarative way to evaluate system/local flakes.

Thanks a lot!

1 Like

As an extension, you can also set $NIX_PATH to the same version of nixpkgs so legacy tools like nix-shell will use the version of nixpkgs that your system config uses (and not the channels that you might not update any longer once you use flakes):

{
  inputs.nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
  outputs = { self, nixpkgs, ... }: {
    nixosConfigurations.foo = nixpkgs.lib.nixosSystem {
      modules = [ {
        nix.nixPath = [ "nixpkgs=${nixpkgs}" ];
      } ];
    };
  };
}

See Introduction - Nix Reference Manual and man configuration for docs.

2 Likes

Is there a Home Manager equivalence?

1 Like

Sorry, I don’t know,I don’t use HM.

I’ve not tried it, but the option documented here will probably do the trick.

Will need some reading into what actual configuration the above nix settings compile down to, but it should otherwise be pretty trivial :slight_smile:

I found some bug in the flake.nix I posted.
Here’s the correct one:
flake.nix

{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-21.05";
  inputs.nixpkgs-nixos-unstable.url = "github:NixOS/nixpkgs/nixos-unstable";
  inputs.home-manager.url = "github:nix-community/home-manager/release-21.05";
  inputs.home-manager.inputs.nixpkgs.follows = "nixpkgs";

  outputs = inputs@{ self, home-manager, nixpkgs, ... }:
    let
      system = "x86_64-linux";
      username = "shamrock";
      hostname = "nixos-202104";
    in {
      homeConfigurations.${username} = home-manager.lib.homeManagerConfiguration {
        configuration = import ./home.nix;
        inherit system username;
	homeDirectory = "/home/${username}";
        stateVersion = "21.05";
        extraSpecialArgs = {
          inherit (inputs) nixpkgs nixpkgs-nixos-unstable;
          inherit hostname;
        };
      };
  };
}

I also experience the issue where too often I have to download nixpkgs’ tarball for commands like nix search, and it’s a bit annoying. I noticed that the tarball-ttl nix configuration option controls how many hours nixpkgs’ tarball is considered updated. I set it to 432000 for 12 hours cache time. Do that while also remembering that you might need to remove files from ~/.cache/nix/tarballs/.

Relevant Nix issues:

Update:
After recent update of the Nix package in the NixOS stable version (21.05), nix search on an absolute path to the unpacked source requires requires the path: schema to be presented.

  • nix-search-local should now be defined as
    (writeShellScriptBin "nix-search-local" ''
      nix search "path:${nixpkgs}" "$@"
    '')
    
    and nix-search-local-unstable
    (writeShellScriptBin "nix-search-local-unstable" ''
      nix search "path:${nixpkgs-nixos-unstable}" "$@"
    '')
    

A more declarative way (through custom registry definition) can be achieved with Home Manager after the nix module is ported.

The port can be started after the Nixpkgs PR 139075 that shift the nixos/nix module to the structural settings pattern. It would be great if someone can take a look.

For anyone getting here for some reason in the future:
home-manager now merged a module to configure nix-registry

having a line in the home-manager nix module
nix.registry.nixpkgs.flake = nixpkgs;
seems to works well, by creating local registry in .config/nix/registry.json

3 Likes