How to distribute package with flake or derivation?

Help
1

I have managed to create a derivation and then recently a flake to build 2 Rust packages. Using it locally in a nix-shell or nix develop shell works like a charm. Pushing it to cachix and fetching it from there works as well.

There are two more goals I want to achieve:

  1. Make the nix shell available for others (through a Github repo)
  2. Provide an option to include a specific version of the package

In relation to 1. I’ve uploaded the flake to a Github repo. When I reference that from a local flake that creates a shell, it works.

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    holochain.url = "github:jost-s/holochain-nix";
  };

  outputs = { self, nixpkgs, holochain }:
    let
      system = "aarch64-darwin";
      pkgs = import nixpkgs { inherit system; };
    in {
      devShells.${system}.default = pkgs.mkShell {
        packages = [
          holochain.packages.${system}.holochain
          holochain.packages.${system}.lair
        ];
      };
    };
}

When combining derivation and devShell in the same flake, I get an error when running

$ nix develop github:jost-s/holochain-nix
error: cannot write modified lock file of flake 'github:jost-s/holochain-nix' (use '--no-write-lock-file' to ignore)
(use '--show-trace' to show detailed location information)

My main questions are:

  • Is this the way how to distribute a derivation (without submitting it to nix packages)?
  • Can shell and derivation be combined in one flake? Or should I break up the flake into separate ones for each package?
2

I don’t see a devshell in your flake on github, but just a package. So the goal is to allow others to consume the package in a devshell? This should work. Do you just want the package available in the shell, or do you want the development environment for the package itself? For the latter, you can just call nix develop on the package derivation, the former is what your snippet provides here.

As far as the error, it looks like the lockfile on your github flake just needs some updates. You can try nix flake lock, or even nix flake update to update all the inputs.

1 Like
3

Do you just want the package available in the shell, or do you want the development environment for the package itself?

I want the packages available in the shell.

I don’t see a devshell in your flake on github, but just a package.

It was there but I had buildInputs instead of packages in it. I’ve corrected that and put it back in. Also added the flake.lock file.

When I run nix develop github:jost-s/holochain-nix now, it goes about compiling the packages. They should be pulled from cachix, but that’s not happening.

4

one nice feature of a flake is that you can set up any substituters that might be useful to the binaries it outputs right in the flake for a user to optionally pick up automatically.

Just add a section like this to the flake.nix:

{
  nixConfig = {
    extra-subtituters = [ /* list of substituters */ ];
    extra-trusted-public-keys = [ /* list of the pub keys for the substituters */ ];
  };
}

In general, any values which are valid in a regular nix.conf are valid in nixConfig. By default the user will be prompted on first use to accept these values.

1 Like
5

Ah, right! Thank you for your hints.

I’ve added the section to the nix flake like this:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    rust-overlay.url = "github:oxalica/rust-overlay";
  };

  nixConfig = {
    extra-substituters = [ "https://cache.nixos.org/" ];
    extra-trusted-public-keys =
      [ "jost-s.cachix.org-1:MJaFoUZA8dZ+v4zO8dLQd9D154zUWBwOUtEw0W26GL8=" ];
  };
...

It’s still wanting to compile when entering the dev shell. The above is following the format found here Binary Cache - NixOS Wiki. Does it need to be different?

Another question I have is, how can I implement this same scenario with a derivation? Using nix-build and nix-shell only?

6

This line is your problem, you are just setting the official nixos cache (which is probably already set by default, and thus redundant), not your cachix. If your public cachix key is an indication the proper url should be "https://jost-s.cachix.org"

7

Mmh, for some reason it still compiles instead of fetching from the cache. I’ve tried both https://jost-s.cachix.org and https://app.cachix.org/cache/jost-s which it resolves to.

Can you point me to how to achieve this

Another question I have is, how can I implement this same scenario with non-experimental features?

8

Have you already uploaded the binary to your cache? And if so, are you sure you haven’t made any changes that would have invalidated the package you have cached?

As far as “non-experimental” features, you could just keep the flake and use flake-compat to eval it with a Nix without flakes enabled.

9

Yes, I’ve had the packages built on the CI prior to trying again. But I’ll build and upload them manually just to be sure, and then delete them from my store and run nix develop github:jost-s/holochain-nix again.

As far as “non-experimental” features, you could just keep the flake and use flake-compat to eval it with a Nix without flakes enabled.

That’s even nicer, thanks!

And out of curiosity, is it possible to specify a git repo as an “import” or the like in a nix derivation?

10

You can import any repo with the various fetch functions (pkgs.fetchFromGitHub), or builtins.fetchTree. The latter understands the same fetch types as flake inputs themselves, e.g. builtins.fetchTree { type = "github"; repo = "someRepo"; owner = "someOwnwer"; narHash = "";}.

The if the repo contains valid Nix code, you can just import it. A lot of projects have a default.nix at the project root, so in that case, you could just call import directly on the repository source and Nix will import it.

11

Amazing, it works! I’ve deleted the packages from the store and entered the dev shell from with the github flake without compiling.

12

Brilliant, thanks for the explanation. For all your help!

13

I’ve been experimenting a while now and fetching from the binary cache still doesn’t work as I’d like it to. That is neither with the flake nor with the derivations.

I’ve added derivations to the repo for the two packages that I want to make available in nix shells, holochain.nix and lair-keystore.nix. I saw your pointer to the hybrid usage of nix flakes, but I wanted to use derivations as an exercise.

Both packages have a version. My plan is to update both packages’ refs whenever there’s a new release of the underlying repos, then push the updated packages where a CI builds the derivations and pushes them to the binary cache.

So far so good. The derivations build fine locally and also on the CI. However, when I enter a dev shell with this shell.nix, the packages are not being pulled from the cache, but being built:

let
  pkgs = import <nixpkgs> { };
  holochain = import (builtins.fetchTree {
    type = "github";
    owner = "jost-s";
    repo = "holochain-nix";
    ref = "0.0.146";
  });
in
pkgs.mkShell {
  inputsFrom = [ holochain ];
}

My questions are:

  • Why do the packages that the CI built have different hashes than when I run nix-shell with that shell.nix?
  • Is that the way the default.nix and the shell.nix should be written?
  • How can I keep both packages in one derivation file?

I’d be thankful for any responses.

14

Because their inputs from <nixpkgs> are probably different.

1 Like
15

:bulb: Of course, that’s where pinning the input will help. Thanks!