Rust build-speed improvements

Announcements
1

Introduction

It has already been reported in the past[1] that Rust builds using buildRustPackage tend to be rather slow since they usually compile the sources in both the release and debug mode. The latter is needed to run the tests in the checkPhase.

Since this significantly slows down the builds and I also think that it’s better to actually run the artifacts that are used after that (i.e. the release-builds), I filed a PR against nixpkgs to fix this: rustPlatform: increase build-speed of `checkPhase` for rust-packages by Ma27 · Pull Request #82342 · NixOS/nixpkgs · GitHub (including a corresponding Hydra jobset: https://hydra.nixos.org/jobset/nixpkgs/pr-82342 - I recommend to use the Compare to-feature of Hydra to compare against staging-next to only see new failing builds).

I decided to publish this here to share the details of it and to request further feedback from the Rust users and maintainers in this community before merging this.

Implementation

In theory, it would’ve been sufficient to run cargo test --release in the checkPhase (or to be more specific, the checkPhase now uses the profile defined in the buildType-argument), however a few more changes were needed to get this working:

  • Right now the installPhase searches for libraries and executables in target/release. The script which searches for those files has been moved to the buildPhase (after the postBuild-hook) as it needs to be done before the checkPhase. Otherwise, the test-binaries (e.g. rg-edd1cb3b1163a7ff) would be installed as well.

  • Some packages however expect the debug-profile for their tests (e.g. if debug_assert! is used). In those cases it’s possible to explicitly define the debug-profile in checkPhase using buildRustPackage { /* ... */ checkType = "debug"; }.

  • Internally, the builder defines a slightly different directory-structure during cargo build --target (instead of target/<profile> it is target/<arch>/<profile>). The structure will be changed back to the default one, however this has to be done after all cargo-related steps (since this is expected by cargo --target) and is therefore done in installPhase now.

    While this seems like a minor change, it actually has implications for several packages:

    • If a test-suite depends on the default directory-structure, the test-suite needs to be patched accordingly (an example for this is the ripasso-cursive package in my PR).

    • Some packages have their own install-script (mostly based on a Makefile). When running make install in a build-environment that was created by the default buildPhase, this will probably fail due to the different structure in target/. In my experience, this can be usually fixed by defining a custom build phase (or to leave it empty to use the make-approach like in uutils-coreutils).

As you can see there are several (minor) breaking changes because of this. While the packages in nixpkgs should be fixed now, it may be possible that some of your personal Rust-packages may break because of that.

What’s next?

I’d like to get some feedback on this from a few more people. After that, I’d take care of the following things:

  • Write some release-notes for 20.09 about the breaking changes.
  • Extend the docs for Rust-builds to cover the new changes.
  • Merge into staging :tada:

[1] Speeding up Rust app packaging - #6 by bhipple, buildRustPackage uses different compilation options for tests · Issue #63730 · NixOS/nixpkgs · GitHub

8 Likes