Bare metal development on NixOS

Help
1

I’m trying to follow A Minimal Rust Kernel | Writing an OS in Rust to get started on a project. Once the author has configured unstable.build-std, it builds for him, but I get:

> cargo build --target target.json
   Compiling compiler_builtins v0.1.160 (/home/alice/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/compiler-builtins/compiler-builtins)
error: linking with `cc` failed: exit status: 1
  |
  = note:  "cc" "-m64" "/home/alice/os/target/debug/build/compiler_builtins-7d15b2f6d23c3780/rustcfCZrSx/symbols.o" "<3 object files omitted>" "-Wl,--as-needed" "-Wl,-Bstatic" "<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib/{libstd-*,libpanic_unwind-*,libobject-*,libmemchr-*,libaddr2line-*,libgimli-*,libcfg_if-*,librustc_demangle-*,libstd_detect-*,libhashbrown-*,librustc_std_workspace_alloc-*,libminiz_oxide-*,libadler2-*,libunwind-*,liblibc-*,librustc_std_workspace_core-*,liballoc-*,libcore-*,libcompiler_builtins-*}.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-L" "/home/alice/os/target/debug/build/compiler_builtins-7d15b2f6d23c3780/rustcfCZrSx/raw-dylibs" "-B<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld" "-fuse-ld=lld" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "<sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/alice/os/target/debug/build/compiler_builtins-7d15b2f6d23c3780/build_script_build-7d15b2f6d23c3780" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: /home/alice/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld/ld.lld: line 5: /nix/store/9q0ah902348jm3y4v4m975sia92lmb8h-rustup-1.28.2/nix-support/ld-wrapper.sh: No such file or directory
          collect2: error: ld returned 127 exit status
          

error: could not compile `compiler_builtins` (build script) due to 1 previous error

I can’t find any information online about what ld-wrapper.sh or its purpose is, so I don’t really know what to do here. Does anybody else know? Maybe it’s a simple case of a missing package. The only relevant packages I have are pkgs.clang and pkgs.rustup. I still get the error if I install separately pkgs.lld or pkgs.llvmPackages.bintools, or if I replace pkgs.clang with pkgs.clang.cc.

2

Nixpkgs patches rustup such that rustup patches programs it downloads for compatibility with nix. What’s happened here is that rustlib was installed by an older rustup which has since been garbage collected so the patched programs no longer refer to extant files.

rustup update should fix the issues, you might need to specify the toolchain.

3

The generally recommended way to develop with Rust in Nix these days is rust-overlay:

3 Likes
4

Oh, yeah, it works. Surprisingly simple!

5

Okay, I’ll take a look at rust-overlay.

6

I prefer Fenix myself, I’ve had better luck with it in the past:

What are the reasons to use rust-overlay instead of Fenix BTW? Seem mostly just like being able to pick more arbitrary Rust versions without generating hashes, is that correct?

I also use crate2nix when building Rust projects within Nix most of the time, although that’s less relevant here (especially as OP is compiling for bare-metal).

7

Yep, that’s pretty much it, I think. There might be other benefits but I’m not aware of them.

1 Like
8

Better compatibility with makeRustPlatform is also part of it. fenix’ toolchains are organized a bit weirdly. Ever since rust-overlay stopped being an overlay it’s been slightly nicer than fenix.

I’d like to also elaborate on why these projects are recommended, because it’s not as straightforward as “always use rust-overlay”.

The built-in rustPlatform lacks rust-src inclusion, which you want for rust-analyzer.

If you’re not going to use rust-analyzer, and don’t need some exotic rust toolchain, which you mostly don’t since async has long since stabilized, there’s absolutely no reason to use rust-overlay or fenix. You should prefer rustPlatform wherever possible for deduplication purposes - sadly for actual development you probably need a rust-src.

1 Like
9

@TLATER thank you, I hadn’t thought about there being situations where the builtin rustPlatform might be preferred! So would you say that e.g. for flakes that don’t customize the toolchain and just use pkgs.rust-bin.stable.latest.default, it’d be a good idea to use rust-overlay for devShells but not for packages?

Can you elaborate? I don’t understand what you mean by this.

10

Sorry for the aside – even as much as rust-overlay helps, I’ve found the bare-metal process to be pretty tough if build-std (e.g. build-std = "core") is required, as you frequently end up requiring conflicting versions of the same crate for build-std vs your crate’s dependencies (How to deal with conflicting requirements for build-std? · Issue #245 · oxalica/rust-overlay · GitHub) and no simple way to get them both into Cargo.lock. The only workaround I’ve identified is to litter your Cargo.toml with the extra deps in different sections (dev deps, build deps, etc.). Does Fenix offer a better approach here?

11

Ehh, that’s up to the downstream’s choice I’d say. You’d end up using different toolchains in dev and for production builds, which isn’t nice.

My opinion is that downstream flakes should not optimize for NixOS use cases, and should have separate upstream nixpkgs packages instead. Seems like duplicated work, but the purposes of the expressions are very different, and users should generally prefer distro packages because those can be integrated properly. That way you can have a stable release in nixpkgs which deduplicates toolchains correctly, while still using the same toolchain for dev and bleeding-edge builds.

I would recommend using rustPlatform.buildRustPackage for third party package repos, though (i.e., if you add some downstream packages to your personal config, or NUR), since those are inherently intended to be used with NixOS, and as such will want the deduplication benefits, and generally don’t care about developing on the packages they package.

But this is the anicent old story of devs vs distro maintainers. Devs will recommend you run their bleeding edge stuff, all the time, disk space and integration be damned, because they don’t care about the rest of the distro integrating with them - which makes sense, when wearing my dev hat I also prefer not getting support requests for old commits.

oxalica/rust-overlay is a fork of the mozilla overlay (which sucks for various reasons, don’t use it), and both of those projects predate flakes. They used to be applied to nixpkgs using nixpkgs.overlays = [ <overlay> ];, and provide their utilities that way - that was generally how third party nix-related projects did stuff.

Ever since flakes however we’ve slowly moved away from adding stuff to a NixOS deployment by adding it to an overlay on top of pkgs (I honestly don’t understand why people started doing that in the first place, global namespacing seems like an obviously bad idea; though there are dissidents who believe overlays are still better, since they avoid the system debacle - that’s just the flake misdesign discussion, though, which is a separate story) - for a short time, fenix was the only project that provided a flake with outputs that weren’t overlays, and during this period fenix was the only way to sensibly do these things with flakes.

oxalica’s rust overlay’s maintenance has however been resumed since, and has been offering flake-oriented utilities for a long time now, so there’s no real reason to use fenix anymore. Regardless of your opinion of overlays, oxalica/rust-overlay can be used both as an overlay and a flake-based package repo, so whatever your use case is, it tailors to it.

1 Like
12

Not to my knowledge, both projects really just offer the simplest possible API to get toolchains, no additional build support.

I’ve not ended up in that particular corner of the papercuts, though, so I’m just going by general knowledge of the codebases. It’s worth a double check.