Introducing Crane: Composable and Cacheable Builds with Cargo

Crane is a Nix library for building cargo projects.

A more detailed blog post can be found here, but here’s a quick overview:

  • heavily inspired by Naersk
  • automatic source fetching using Cargo.lock
  • incrementally cached builds: make your CI run fast without having to recompile dependencies over and over
  • composable configuration: easily run extra tests in your CI which reuse cached artifacts without having to run them for other from-source builds
  • easily run any cargo command: cargo {clippy,fmt,tarpaulin} are already supported out of the box, but you can bring your own invocation as well

Git dependencies and private cargo registries are not supported out of the box just yet, but I’m hoping to add support for them in the near future.

Please check it out if you are interested and share any feedback you might have!

12 Likes

I have a dumb question.

Any way to use the cached builds with normal cargo build?

2 Likes

What’s the advantage over naersk, does it cache each crate build individually?

2 Likes

It’s very similar to Naersk in the regard that it will build the entire workspace in one go. The unit of caching is the entire derivation itself, just like in Naersk.

The advantage over Naersk is that the nix API makes it much easier to compose different cargo invocations as completely separate derivations (you could do this with Naersk but you really have to understand all the internals to wire it all up right).

Here’s a quick flake example illustrating what you could do:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    crane.url = "github:ipetkov/crane";
    crane.inputs.nixpkgs.follows = "nixpkgs";
    flake-utils.url = "github:numtide/flake-utils";
    flake-utils.inputs.nixpkgs.follows = "nixpkgs";
  };

  outputs = { self, nixpkgs, crane, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs {
          inherit system;
        };

        craneLib = crane.lib.${system};
        src = ./.;

        # Build *just* the cargo dependencies, so we can reuse
        # all of that work (e.g. via cachix) when running in CI
        cargoArtifacts = craneLib.buildDepsOnly {
          inherit src;
        };

        # Run clippy (and deny all warnings) on the crate source,
        # resuing the dependency artifacts (e.g. from build scripts or
        # proc-macros) from above.
        #
        # Note that this is done as a separate derivation so it
        # does not impact building just the crate by itself.
        my-crate-clippy = craneLib.cargoClippy {
          inherit cargoArtifacts src;
          cargoClippyExtraArgs = "-- --deny warnings";
        };

        # Build the actual crate itself, reusing the dependency
        # artifacts from above.
        my-crate = craneLib.buildPackage {
          inherit cargoArtifacts src;
        };

        # Also run the crate tests under cargo-tarpaulin so that we can keep
        # track of code coverage
        my-crate-coverage = craneLib.cargoTarpaulin {
          inherit cargoArtifacts src;
        };
      in
      {
        defaultPackage = my-crate;
        checks = {
         inherit
           # Build the crate as part of `nix flake check` for convenience
           my-crate
           my-crate-clippy
           my-crate-coverage;
        };
      });
}
1 Like

It’s possible to add a devshell hook which will copy the artifacts from the store to the target directory, yes, and then keep developing locally with cargo build outside.

Note that by default Crane will build everything in release mode, so if you copy the artifacts over but invoke cargo build, cargo may decide to rebuild everything with debug flags on, etc.

1 Like

Would it make sense to also cache the crates’ sources and copy them to ~/.cargo/registry with a devshell hook?

Crate sources do get put in the store at some point in order to prepare a vendored directory so cargo can use it during sandboxed builds.

If you really wanted to you could copy them over to cargo’s home directory and avoid having cargo touch the network. I don’t think cargo’s internal file layout is documented as stable, so YMMV if any breakage happens there :slight_smile:

(FWIW sources are usually cheap to download from crates.io, but the artifact caching that nix can provide is the real winner over the default cargo experience in my view)

1 Like

I got this:

❯ nix build
error: access to canonical path '/nix/store/5mvgi56jp7gjm88whxp0h22i8yx4065s-nix-shell' is forbidden in restricted mode
(use '--show-trace' to show detailed location information)

Do I need to use --impure? Maybe I did something wrong. I used the “custom toolchain” example to use oxalica/rust-overlay.

EDIT: I think I did something wrong, I have the same problem with nix flake check.

I’m not sure, are you using flakes at all?

If you’re willing to share the configuration I could try taking a look at what seems wrong

Yeah, my whole nix-config uses flakes.

I was also using direnv.

I wonder if I broke something while doing incremental changes to the flake.nix file.

I disabled direnv, removed result, copied the old flake.nix and started over. now it works fine. even with direnv.

Here’s my flake.nix file:

{
  description = "Build a cargo project with a custom toolchain";

  inputs = {
    nixpkgs.url = "nixpkgs/nixpkgs-unstable";

    crane = {
      url = "github:ipetkov/crane";
      inputs.nixpkgs.follows = "nixpkgs";
    };

    flake-utils = {
      url = "github:numtide/flake-utils";
      inputs.nixpkgs.follows = "nixpkgs";
    };

    rust-overlay.url = "github:oxalica/rust-overlay";
  };

  outputs = { self, nixpkgs, crane, flake-utils, rust-overlay, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs {
          inherit system;
          overlays = [ (import rust-overlay) ];
        };

        src = ./.;

        rustWithWasiTarget = pkgs.rust-bin.stable.latest.default.override {
          #targets = [ "wasm32-wasi" ];
        };

        # NB: we don't need to overlay our custom toolchain for the *entire*
        # pkgs (which would require rebuidling anything else which uses rust).
        # Instead, we just want to update the scope that crane will use by appending
        # our specific toolchain there.
        craneLib = (crane.mkLib pkgs).overrideScope' (final: prev: {
          rustc = rustWithWasiTarget;
          cargo = rustWithWasiTarget;
          rustfmt = rustWithWasiTarget;
        });

        # Build *just* the cargo dependencies, so we can reuse
        # all of that work (e.g. via cachix) when running in CI
        cargoArtifacts = craneLib.buildDepsOnly {
          inherit src;

          nativeBuildInputs = with pkgs; [
            pkg-config
          ];

          buildInputs = with pkgs; [
            openssl
          ];
        };

        my-crate = craneLib.buildPackage {
          inherit src cargoArtifacts;
          # src = ./.;

          #cargoExtraArgs = "--target wasm32-wasi";

          nativeBuildInputs = with pkgs; [
            pkg-config
          ];

          buildInputs = with pkgs; [
            openssl
          ];

          # Tests currently need to be run via `cargo wasi` which
          # isn't packaged in nixpkgs yet...
          doCheck = false;
        };
      in
      {
        checks = {
          inherit my-crate;
        };

        defaultPackage = my-crate;
        packages.my-crate = my-crate;

        apps.my-app = flake-utils.lib.mkApp {
          drv = my-crate;
        };
        defaultApp = self.apps.${system}.my-app;

        devShell = pkgs.mkShell {
          inputsFrom = builtins.attrValues self.checks;

          # Extra inputs can be added here
          nativeBuildInputs = with pkgs; [
            rustWithWasiTarget
          ];

          buildInputs = with pkgs; [
            clang
            openssl
            pkg-config
          ];
        };
      });
}
1 Like

Nice work! It’s always exciting to see folk aiming at improving the Rust dev workflow, especially w.r.t. improving caching.

Q: What’s the advantage over naersk, does it cache each crate build individually?


A: It’s very similar to Naersk in the regard that it will build the entire workspace in one go. The unit of caching is the entire derivation itself, just like in Naersk.

The fact that you’ve managed to separate the dependencies from the src of the crate itself already sounds like a big win for the common caching needs!

That said, I’d love to hear your thoughts on the possibility of creating derivations per crate. Is this something that you considered but turned down due to difficulty of implementation? Or perhaps the state of cargo, its arbitrary-rust-code build scripts and macros make this simply not worth it?

One of my most major issues with the Rust dev process is the need to build every crate from scratch when starting new projects or updating the Rust version, even if you’ve just built the same list of dependencies for 100 other similar projects. The lack of user/system-wide caching of build artifacts is painful, results in huge initial compilation times and bloated target directories.

It seems like Nix is uniquely positioned to have a chance at solving this with its flexible-yet-hermetic build system. I think the cargo2nix project is having a go at this:

Design

This Nixpkgs overlay builds your Rust crates and binaries by first pulling the dependencies apart, building them individually as separate Nix derivations and linking them together. This is achieved by passing custom linker flags to the cargo invocations and the underlying rustc and rustdoc invocations.

Last time I tried I remember running into a lot of odd build errors, but that must have been a year ago now - perhaps it’s worth trying again…

4 Likes

I will happily give credit to Naersk for this technique before me, it really goes a long way!

My approach with Crane was not in eschewing cargo but embracing it completely, only doing the necessary work to make cargo work within Nix’s sandbox, and then let it take over the hard work.

The crate2nix project takes a very interesting approach to effectively removing cargo altogether and therefore gaining better caching at the crate level itself (but more on this later). There’s two main reasons I did not want to take a similar path:

  1. although I find what crate2nix is doing as laudable and inspiring, I could not hope to be able to re-implement all of cargo’s behavior within Nix itself (nor do I want to find myself playing catch up with cargo’s features and behavior)
  2. removing cargo means losing the ability to use cargo specific tools (clippy and wasm-pack are some which come to mind, others may be needed for things like generating models/types from schemas, etc.)

On the topic of per-crate caching: changing which feature flags are enabled requires rebuilding the crate, so the best possible global caching you could get is one-copy-per-crate-per-feature-flag-combination. My hope is that the broader Rust community can solve this problem in a more universal way and then find ways to integrate that with Nix!

3 Likes

My view is cargo2nix is very important to show upstream Cargo what we would like to do. Then we can figure out together what is missing in Cargo itself — namely the “planning” vs “execution” isn’t separated enough.

Without going the full derivation-per-crate route, they will not be able to understand where Cargo fall short as well, because there is no demonstration of what the missing functionality can do.

5 Likes

Crane v0.2.0 now supports alternative cargo registries!

Here is an example on how to get started. More details can be found in the API docs.

2 Likes

My view is cargo2nix is very important to show upstream Cargo what we would like to do.

I agree with this - long term we want language build tools to make available their build plan so we can have Nix execute it.
But short term the approach of trying to emulate the build tool’s behavior is of limited viability, as it creates only a second rate experience with truly incredible amounts of effort. @nmattia tried to do this for Haskell with Snack (I tried helping out a tiny bit) and we both came to that conclusion.

1 Like

Crane v0.3.0 now supports automatically handling git dependencies!

There is no example this time because there is nothing to configure, it should Just Work :tm: :wink:
It’s also worth noting that cargo retains the flexibility to only use these dependencies when they are actually needed, without forcing an override for the entire workspace (for example, a pinned git dependency only used in a testing crate doesn’t end up being used for building the actual production binaries).

I think I’ve covered most of the edge cases, but if something doesn’t seem to work please open an issue!

4 Likes
Hosted by Flying Circus.