Reference a local file from flake.nix

Help
1

Hey :wave:

I am trying to wrap my brains around flakes and how they can be useful for building a project

I have a repo with a flake.nix that is supposed to build a rust wasm example. In order to build, it refers to cargoLock.lockFile = ./Cargo.lock;. However, when running nix build it errors out due to a mistake in the path

error: path '/nix/store/szb5p2s1j4kqv4qrk72rhmgnd0r0mz8s-source/Cargo.lock' does not exist

  1. How do I refer to a local file? According to the official nix reference manual, paths starting with ./ should be relative to the flake (in this case the project directory, I assume). The syntax I used is taken from an example on how to build a rust project with makeRustPlatform where they also have cargoLock.lockFile = ./Cargo.lock;

  2. I believe the flake is supposed to build the Cargo.lock if it doesn’t already exist. If it requires Cargo.lock to build normally, should I be building the project through the dev shell to generate the Cargo.lock ? Is flake.nix only useful to make reproducible builds and not actually building incremental changes?

├── Cargo.lock
├── Cargo.toml
├── flake.nix
├── rust-toolchain.toml
├── src
│   ├── lib.rs
│   └── utils.rs
└── tests
    └── web.rs

# flake.nix
{
  description = "Rust with WebAssembly";

  inputs = {
    fenix = {
      url = "github:nix-community/fenix";
      inputs.nixpkgs.follows = "nixpkgs";
    };

    nixpkgs.url = "nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, fenix, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
        f = fenix.packages.${system};
        target = "wasm32-unknown-unknown";
        toolchain = with fenix.packages.${system}; combine [
          minimal.cargo
          minimal.rustc
          targets.${target}.latest.rust-std
        ];
      in {  
        packages.default = (pkgs.makeRustPlatform {
          cargo = toolchain;
          rustc = toolchain;
        }).buildRustPackage {
          pname = "rust-wasm-game-of-life";
          version = "0.1.0";
          src = ./.;
          cargoLock.lockFile = ./Cargo.lock;
        };

        devShells.default = pkgs.mkShell {
          name = "rust-wasm-first-attempt";

          packages = with pkgs; [
            f.stable.toolchain
            nodejs
            wasm-pack
          ];
        };
      }
  );
}
2

git add the file. Flakes will only “see” tracked files because nix insists on copying your entire repo to /nix/store… with all the useability, disk use, write, and performance implications. It uses git tracking as a heuristic for which files are needed.

It’s probably the most major flaw in the whole flakes concept, and a large part of why I don’t really think it should be considered ready. The new lazy trees feature helps a bit since it means only files needed for the evaluation are copied (and not, e.g., random build caches or, say, text files with secrets), but it doesn’t address the underlying issue that we’re constantly copying around entire repositories’ worth of files.

Kinda. flake.nix doesn’t do anything, it’s just sugar for invoking nix commands, which ultimately do exactly what the traditional nix-build does with a specified file (or default.nix). Flakes are not at all relevant here.

But the nix builder you invoke is only designed to build full repos, not do incremental builds of anything. It gets a source tree and produces all the binaries in the workspace, and doesn’t keep any other files it produces incidentally.

There are third party builders (e.g. crate2nix) which can do more granular builds, primarily for dependency crates, but in general nix will not keep build caches (since that would be an inherently impure build, and goes against everything nix stands for).

Nix is ultimately a metabuild/integration tool, it’s not a cargo replacement.

For development, you should build a nix shell with specific tool versions to use (and perhaps even a pre-built crate cache with some of the fancy builders) and run cargo manually.

2 Likes
3

Thank you very much!

I was under the impression that the flakes were evaluated locally in my work dir. Thank you for clearing that up and for the great explanations

For development, you should build a nix shell with specific tool versions to use

For curiosity, would you say that devShells (as seen in my flake) is equivalent to having a shell.nix ?

devShells.default = pkgs.mkShell {
  name = "rust-wasm-first-attempt";

  packages = with pkgs; [
    f.stable.toolchain
    nodejs
    wasm-pack
  ];
};

Cheers.

4

Pretty much. Both are just a way to tell nix about a build environment created with e.g. pkgs.mkShell. I refer to this concept as a “nix shell” regardless of whether you index it with a flake or use an explicit shell.nix file.

The ultimate work nix does does not meaningfully differ whether you use a flake.nix file or if you spread your nix expressions into the traditional “standard” file tree.

Flakes’ raison d’être is precisely that there was no good standard for how to structure projects using nix - the flake.nix file is a rough standard for that.

There are some subtle differences, the nix develop command invokes the shell slightly differently (leading to some limitations), and since flakes enforce input locking your shells never update with your system - which can be both good and bad, depending on your use case. But it’s pretty much the same idea save for little details.

Flakes also add a better dependency resolution system than channels (though this can be solved without language additions, see e.g. npins) and the silly project copy thing to assert build purity in the least efficient way imaginable.

1 Like
5

I’ve been bitten by this and my reaction was pretty much “ok I’m not going to use flakes then, unless I’m working on something related to nix itself.”

But does your comment imply some alternative solution in the future?

6

It says that there is an alternative solution to dependency resolution you can use today, npins.

As for evaluation purity, pure evaluation can be used without flakes, but AFAIK it still copies evaluated files into /nix/store.

To my knowledge the only work in the direction of solving that problem currently is the lazy trees feature, which doesn’t really solve very much since any self reference breaks that feature and IME it’s very hard to write a flake that doesn’t use self. It’s also currently only implemented in dix, and AIUI there it is implemented in such a way that ironically it risks build purity, which is why the implementation isn’t making it into nix.

I can see a world in which it is solved with cgroups. My guess for why that route wasn’t taken is that that risks adequate MacOS support. There’s anyway not too much interest in evaluation purity besides flakes, which in theory need it for evaluation caching, but that’s not even strictly true, so I don’t really see anyone besides detsys picking up this work.

IMO it’s largely a useless feature, at least as far as most users are concerned. Its only real effect is to make newcomers confused about why their uncommitted files are invisible. Everything else it offers would be more adequately done with a linter.

7

Thanks for the insight. I suppose macos compat is important. I wonder if compatibility with non-git workflows is on the wish list.

One can imagine pluggable “builder” backends for flakes where copying the file tree is a choice, linux containers are another, and third – well, macos containers?

Yeah no doubt #3 especially is a lot of work but a door would also be open to a 4th option: do nothing and evaluate impurely.

If I had to speculate blindly: evaluation purity is of greatest importance on build servers and while developing a flake (on your macbook) the occasional check would suffice?

8

Not using git has and will be always an option, though if your flakes resides within a git repository, you have to be explicit that you do not want to use git support, which comes with some extra downsides, like really copying everything, including your .git.

1 Like