Build Rust app using Cargos 'build-std' feature (with naersk) fails because rust-src is missing

Help
1

Hi,
I’m new to Nix and try to build a Rust app for the target “x86_x64-unknown-uefi”. In order to achieve this, I need the build-std-feature of Cargo. This means, that Cargo cross compiles Rusts own libraries (core, alloc) itself. To achieve this, Cargo fetches data from ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/.

I use naersk (GitHub - nix-community/naersk: Build Rust projects in Nix - no configuration, no code generation, no IFD, sandbox friendly. [maintainer=@AxelSilverdew]) to build my Rust app with Nix. During the build inside a Nix shell, it complains that /nix/store/751qp25wf6iblrjprpxsq6h8hlgxd8c2-rust-1.54.0-nightly-2021-05-09-ca82264ec/lib/rustlib/src is not available (only src, the last component) is missing.

How/where can I specify that “rust-src” must be included? The following code example is the template I used, therefore my Nix files are almost identical to it: https://github.com/nasirhm/hello-rust-nix/blob/b23bc8701fb5fab21997f8aa2b7c95437cf4e598/helloworld.nix

My repository GitHub - phip1611/rust-hello-world-uefi-nix: A hello world UEFI-Application written in Rust that gets packaged with Nix.
(the relevant part is only that it should build a Rust binary for the target x86_64-unknown-uefi.)

I’m looking forward to your help

Kind regards
Philipp

2 Likes
2

Check out the “Shell.nix example” section in the wiki here: Rust - NixOS Wiki. Setting that environment variable should be all you need!

3

It’s not that easy, because even if you get the rust source code installed, building std still requires crates from crates.io. My current workaround is to pull these in via [dev-dependencies] in Cargo.toml, but this is unsatisfactory, because these dependencies interfere with the normal dependencies of the app that I’m trying to build…

I’ll post a full example later today.

4

So I have a working example using Naersk now. The main problem is that the build-std feature of cargo needs dependencies from crates.io to build the standard library. Naersk only provides sources for everything in Cargo.toml, so this fails.

My workaround is to add everything to [dev-dependencies] in Cargo.toml. This is pretty ugly, but it works:

[dev-dependencies]
compiler_builtins = "0.1.43"
cfg-if = "0.1.10"
getopts = "0.2.21"
libc = "=0.2.93"
hashbrown = "=0.11.0"
rustc-demangle = "=0.1.18"
hermit-abi = "=0.1.17"
dlmalloc = "0.2.1"
wasi = "=0.9.0"
fortanix-sgx-abi = "0.3.3"
cc = "1.0.67"

I’m currently not sure where this needs to be fixed. It feels like Cargo should add all dependencies it uses to compile std to Cargo.lock.

In the meantime, it would be nice to get a hand from Naersk here and be able to specify additional crates that need to be downloaded from crates.io.

Maybe @nmattia has an opinion here?

5

I’d check out the “cross compiling” section of of rust.section.md. It shows how to use a custom --target and compiles a custom sysroot for it. Relevant PR is #95542 (by me).

1 Like
6

You should check out GitHub - rust-lang/wg-cargo-std-aware: Repo for working on "std aware cargo", the working group for standard-aware cargo.
build-std is the first step towards that goal, but there’s a lot of questions open about how Cargo should treat programs that need to build the std library.

One issue that might be interesting is this one Possibly publish a synthetic `Cargo.toml`. · Issue #27 · rust-lang/wg-cargo-std-aware · GitHub, which proposes a synthetic Cargo.toml that could be very useful to tools like Naersk.

Progress is slow though because build_std users are still a niche subset of Rust users.
I’d imagine making Nix tooling like Naersk aware of build-std would be quicker than waiting for Cargo to make it easier.

1 Like
7

@aaronjanse I’ve tried using the buildRustPackage tooling, but so far without success. It seems the wrong linker is used or the linker is missing the correct emulation. See below for a log.

You can reproduce this with:

% nix-build https://github.com/blitz/nix-uefi-rust/archive/refs/heads/master.tar.gz -A hello-nixpkgs

The relevant Nix expression is here

error: linking with `/nix/store/ndggid6v98wzv5spbmp7pjm8jc23imfq-x86_64-linux-stage-final-gcc-wrapper-10.3.0/bin/x86_64-linux-cc` failed: exit status: 1
  |
  = note: "/nix/store/ndggid6v98wzv5spbmp7pjm8jc23imfq-x86_64-linux-stage-final-gcc-wrapper-10.3.0/bin/x86_64-linux-cc" "-flavor" "link" "/NOLOGO" "/entry:efi_main" "/subsystem:efi_application" "/NXCOMPAT" "/LIBPATH:/nix/store/kp30n5mp97psqf03158cicjgsdg1pjng-rust-default-1.54.0-nightly-2021-05-19/lib/rustlib/x86_64-unknown-uefi/lib" "/build/hello/target/x86_64-unknown-uefi/release/deps/hello-7a1d1a4ed2d69282.hello.5ita5hbe-cgu.0.rcgu.o" "/build/hello/target/x86_64-unknown-uefi/release/deps/hello-7a1d1a4ed2d69282.hello.5ita5hbe-cgu.1.rcgu.o" "/build/hello/target/x86_64-unknown-uefi/release/deps/hello-7a1d1a4ed2d69282.hello.5ita5hbe-cgu.2.rcgu.o" "/build/hello/target/x86_64-unknown-uefi/release/deps/hello-7a1d1a4ed2d69282.hello.5ita5hbe-cgu.3.rcgu.o" "/build/hello/target/x86_64-unknown-uefi/release/deps/hello-7a1d1a4ed2d69282.hello.5ita5hbe-cgu.4.rcgu.o" "/build/hello/target/x86_64-unknown-uefi/release/deps/hello-7a1d1a4ed2d69282.hello.5ita5hbe-cgu.5.rcgu.o" "/OUT:/build/hello/target/x86_64-unknown-uefi/release/deps/hello-7a1d1a4ed2d69282.efi" "/OPT:REF,ICF" "/DEBUG" "/NODEFAULTLIB" "/LIBPATH:/build/hello/target/x86_64-unknown-uefi/release/deps" "/LIBPATH:/build/hello/target/release/deps" "/LIBPATH:/nix/store/kp30n5mp97psqf03158cicjgsdg1pjng-rust-default-1.54.0-nightly-2021-05-19/lib/rustlib/x86_64-unknown-uefi/lib" "/build/hello/target/x86_64-unknown-uefi/release/deps/libuefi_services-09f50bd46e64d8a4.rlib" "/build/hello/target/x86_64-unknown-uefi/release/deps/libqemu_exit-da81c41e8c1b4286.rlib" "/build/hello/target/x86_64-unknown-uefi/release/deps/libuefi-a78fb86ee459121e.rlib" "/build/hello/target/x86_64-unknown-uefi/release/deps/libbitflags-03534ab58822e87d.rlib" "/build/hello/target/x86_64-unknown-uefi/release/deps/libucs2-de341727e2f0ed40.rlib" "/build/hello/target/x86_64-unknown-uefi/release/deps/libbit_field-efaa8b3fe6b221a9.rlib" "/build/hello/target/x86_64-unknown-uefi/release/deps/liblog-30885f03c80ded7d.rlib" "/build/hello/target/x86_64-unknown-uefi/release/deps/libcfg_if-d9e021cd093c0367.rlib" "/build/hello/target/x86_64-unknown-uefi/release/deps/librustc_std_workspace_core-d52e875c54594f48.rlib" "/build/hello/target/x86_64-unknown-uefi/release/deps/libcore-cfc999e34fb786da.rlib" "/build/hello/target/x86_64-unknown-uefi/release/deps/libcompiler_builtins-734cc574a9f2f56d.rlib"
  = note: x86_64-linux-gcc: error: link: No such file or directory
          x86_64-linux-gcc: error: /NOLOGO: No such file or directory
          x86_64-linux-gcc: error: /entry:efi_main: No such file or directory
          x86_64-linux-gcc: error: /subsystem:efi_application: No such file or directory
          x86_64-linux-gcc: error: /NXCOMPAT: No such file or directory
          x86_64-linux-gcc: error: /LIBPATH:/nix/store/kp30n5mp97psqf03158cicjgsdg1pjng-rust-default-1.54.0-nightly-2021-05-19/lib/rustlib/x86_64-unknown-uefi/lib: No such file or directory
          x86_64-linux-gcc: error: /OUT:/build/hello/target/x86_64-unknown-uefi/release/deps/hello-7a1d1a4ed2d69282.efi: No such file or directory
          x86_64-linux-gcc: error: /OPT:REF,ICF: No such file or directory
          x86_64-linux-gcc: error: /DEBUG: No such file or directory
          x86_64-linux-gcc: error: /NODEFAULTLIB: No such file or directory
          x86_64-linux-gcc: error: /LIBPATH:/build/hello/target/x86_64-unknown-uefi/release/deps: No such file or directory
          x86_64-linux-gcc: error: /LIBPATH:/build/hello/target/release/deps: No such file or directory
          x86_64-linux-gcc: error: /LIBPATH:/nix/store/kp30n5mp97psqf03158cicjgsdg1pjng-rust-default-1.54.0-nightly-2021-05-19/lib/rustlib/x86_64-unknown-uefi/lib: No such file or directory
          x86_64-linux-gcc: error: unrecognized command-line option '-flavor'
8

@bythebook These issues look indeed very interesting. Thanks for the pointers!

9

@aaronjanse I think the linking problem stems from pkgs/development/compiers/rust/rustc.nix:

    "${setBuild}.linker=${ccForBuild}"
    "${setHost}.linker=${ccForHost}"
    "${setTarget}.linker=${ccForTarget}"

I think these need to point to lld, if crossSystem has useLLVM = true; (or even without it?).

Update: I tried to play with the linker settings, but to no avail. I always end up with failing builds. If someone has insights into how to get the Rust toolchain use the lld linker, this would be greatly appreciated. :slight_smile:

1 Like
10

I proposed some related improvements that require reviews:
https://github.com/NixOS/nixpkgs/pull/215408

11

I’ve created a PR for naersk that allows for specifying an additional Cargo.lock: Add additionalCargoLock argument by bouk · Pull Request #328 · nix-community/naersk · GitHub

This makes it possible to add the rust Cargo.lock, which will ensure the stdlib dependencies are downloaded without having to modify your own Cargo.toml and deal with potential dependency conflicts

1 Like