gabyx
April 17, 2025, 9:48am
1
Problem
I have the following rust CLI tool default.nix which you can build with:
nix build "github:sdsc-ordes/catplus-converters?dir=tools/nix&ref=80025843a21e570ee86f6077360bffb07ae8f63d#catplus-converters" --print-out-paths
> /nix/store/pj638k9z33d5z125ainqhai16a79dax5-catplus-converters
I use the rust-overlay
from oxalica
:
# The Rust overlay to include the latest toolchain.
rust-overlay = {
url = "github:oxalica/rust-overlay";
inputs = {
nixpkgs.follows = "nixpkgs";
};
};
When I analyze this with nix-tree
:
nix-tree /nix/store/pj638k9z33d5z125ainqhai16a79dax5-catplus-converters
I see a huge dependency which is the /nix/store/fij1508sgwggn17ypmkj98ly5fcbrlmp-rust-default-1.88.0-nightly-2025-04-16
of about 1. 75 GiB :
The ldd
only reports the following deps:
ldd /nix/store/pj638k9z33d5z125ainqhai16a79dax5-catplus-converters/bin/converter
linux-vdso.so.1 (0x00007ffe5b57b000)
libc.so.6 => /nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/libc.so.6 (0x00007f8858800000)
/nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/ld-linux-x86-64.so.2 => /nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib64/ld-linux-x86-64.so.2 (0x00007f885919b000)
libgcc_s.so.1 => /nix/store/ik84lbv5jvjm1xxvdl8mhg52ry3xycvm-gcc-14-20241116-lib/lib/libgcc_s.so.1 (0x00007f885916b000)
Questions
Does anybody know why that happens cause these bloat dependencies seem to be build inputs only?? and what I can do to analyze that better, this is a fairly simple Rust app with some dependencies etc.
Does the rust-default somehow sneaks in by vendored dependencies?
Have a look here maybe:
rust
requires a lot to compile, analyze, debug, etc
There are some tweaks you can do to reduce the size but overall, projects tend to be large
Depencies listed (rust-docs
, rust-analyzer
, clippy
, etc) seem about right and what you would need to get a smooth development experience (formatting, debuggin, quick compile time, etc)
gabyx
April 18, 2025, 4:09am
3
Thanks for the references! I am just puzzled why rust-default even is part of the final derivation eventhough its not ? needed to run the application AFAIS… its just strange because these things seem to me like build dependencies…
rust-default
comes from the oxalica/rust-overlay
overlay you’re using or am I missing smth?
From repo examples, see the profile
arg is set to default
: rust-overlay/examples/hello-world/shell.nix at ec22cd63500f4832d1f3432d2425e0b31b0361b1 · oxalica/rust-overlay · GitHub
Which then uses: rust-overlay/manifests/profiles.nix at ec22cd63500f4832d1f3432d2425e0b31b0361b1 · oxalica/rust-overlay · GitHub (or from another line in the same file)
So there is complete
, default
and minimal
=> would assume using e.g complete
, you’ll get rust-complete
instead of your current rust-default
rust
is a compiled language so these are needed in order to run any rust
projects. One can not call a .rs
file, it needs to be compiled so then one can call the binary output.
Or maybe I’m misunderstanding you?
gabyx
April 18, 2025, 6:45am
5
There is a missunderstanding here: Its all about the single $out/converter
in the final derivation, which is a ELF executable and for which some how Nix thinks it needs /nix/store/...rust-default-...
because thats what Nix found as runtime dependencies when have done the anaylysis after the build etc. (I suspect it does not do a simple strings
analysis of the binary, right?).
The strange thing is using nightly-2024-06-17
instead of nightly
will not have this behavior and the link to rust-default-
store path is gone, so when using this
[toolchain]
channel = "nightly-2024-06-17"
components = [ "rustfmt", "rust-analyzer", "miri", "rust-docs", "clippy", "rust-src"]
profile = "default"
I am seeing patching beeing done for the toolchain
cargo> automatically fixing dependencies for ELF files
cargo> {'add_existing': True,
cargo> 'append_rpaths': [],
cargo> 'extra_args': [],
cargo> 'ignore_missing': [],
cargo> 'keep_libc': False,
cargo> 'libs': [PosixPath('/nix/store/hfkclq54gppdb9fshnkwfdph1safyvli-gnutar-1.35/lib'),
cargo> PosixPath('/nix/store/yip39c32hz27ix425p14q6pw7w8i598g-auto-patchelf-hook/lib'),
cargo> PosixPath('/nix/store/214vrsiv784x6marqh27ips2r4p6pq34-auto-patchelf-0-unstable-2024-08-14/lib'),
...
Is that the reason for it ? Maybe somebody from the Rust toolchain maintenance team in nixpkgs might answer this:
@emily , @shamrocklee : Do you might have a clue?
1 Like
gabyx
April 18, 2025, 7:05am
6
Ok, apparently the components =
section now influences runtime-dependencies on newer toolchains (?).
So the solution is to make a toolchain.toml
file for Nix building not use components
etc.
shimun
April 18, 2025, 12:33pm
7
I’ve had a similar issue months ago, which was caused by store paths being leaked into the ELF and thus added as dependencies. At the time I’ve solved the issue with RUSTFLAGS="--remap-path-prefix"
which then remapped the store path to something else.
1 Like
gabyx
April 22, 2025, 6:33am
8
Did you use RUSTFLAGS = "--remap-path-prefix=$NIX_BUILD_TOP=/";
I only see this used in
buildRustCrate
in nixpkgs
?
Thanks for the hint, so far adding this flag to the build did not really help.
gabyx
April 22, 2025, 2:23pm
9
The solution is the following:
opened 02:55PM - 04 Dec 24 UTC
closed 10:18PM - 04 Dec 24 UTC
bug
The commit that introduces Rust 1.83.0 (b259ef799b5ac014604da71ecd92d4a52603ed2d… ) appears to bloat the closure of packages built with the rust toolchain that have the `rust-src` component.
`rust-toolchain.toml`:
```toml
[toolchain]
channel = "stable"
components = [ "rust-src" ]
targets = [ "x86_64-unknown-linux-gnu" ]
```
`default.nix` (adapted the [example from the nixpkgs manual](https://nixos.org/manual/nixpkgs/stable/#using-rust-nightly-in-a-derivation-with-buildrustpackage)):
```nix
with import <nixpkgs>
{
overlays = [
(import (fetchTarball "https://github.com/oxalica/rust-overlay/archive/34971069ec33755b2adf2481851f66d8ec9a6bfa.tar.gz"))
];
};
let
rustToolchain = rust-bin.fromRustupToolchainFile ./rust-toolchain.toml;
rustPlatform = makeRustPlatform {
cargo = rustToolchain;
rustc = rustToolchain;
};
in
rustPlatform.buildRustPackage rec {
pname = "ripgrep";
version = "12.1.1";
src = fetchFromGitHub {
owner = "BurntSushi";
repo = "ripgrep";
rev = version;
hash = "sha256-+s5RBC3XSgb8omTbUNLywZnP6jSxZBKSS1BmXOjRF8M=";
};
cargoHash = "sha256-l1vL2ZdtDRxSGvP0X/l3nMw8+6WF67KPutJEzUROjg8=";
doCheck = false;
meta = {
description = "Fast line-oriented regex search tool, similar to ag and ack";
homepage = "https://github.com/BurntSushi/ripgrep";
license = with lib.licenses; [ mit unlicense ];
maintainers = with lib.maintainers; [];
};
}
```
```ShellSession
# nix-build
# nix path-info --recursive --size --closure-size --human-readable ./result
/nix/store/4hcdxyjf9yiq7qf3i4548drb6sjmwa1v-xgcc-13.3.0-libgcc 155.9 KiB 155.9 KiB
/nix/store/w19cxz37j5nrkg8w80y91bga89310jgi-libunistring-1.2 1.8 MiB 1.8 MiB
/nix/store/jwsdpq2yxw43ixalh93z726czz7bay2j-libidn2-2.3.7 352.6 KiB 2.1 MiB
/nix/store/sl141d1g77wvhr050ah87lcyz2czdxa3-glibc-2.40-36 28.8 MiB 31.1 MiB
/nix/store/5frxzv8fj2zaj04lbv3z848p8zvwpz3g-gmp-6.3.0 706.6 KiB 31.8 MiB
/nix/store/bklfkdh58y2hr24796hln00l2ly68fvg-mpfr-4.2.1 775.7 KiB 32.5 MiB
/nix/store/d72018202lbgj7ckmd020d56apny73wg-libmpc-1.3.1 277.4 KiB 32.8 MiB
/nix/store/haci0d018njygwdga95bsbx8ww2iswy7-zlib-1.3.1 125.2 KiB 31.2 MiB
/nix/store/kskaix9pdvh22pkgnaqd3k9yd7r1xh1z-isl-0.20 2.5 MiB 34.3 MiB
/nix/store/1qx77645wk0rd00h49rvjh9vkgs8vwvy-linux-headers-6.10 6.5 MiB 6.5 MiB
/nix/store/vpsla1ivhavzd4fmi95yzmgb4g9rd072-glibc-2.40-36-bin 2.7 MiB 33.7 MiB
/nix/store/n15bxkd7id2gvlwpihm4kp675aqkrwmj-glibc-2.40-36-dev 2.3 MiB 42.6 MiB
/nix/store/lw294qfdnf29lhxqwzy9byrmx975rvn9-gcc-13.3.0-libgcc 155.9 KiB 155.9 KiB
/nix/store/y3kdn61k93rq2jx1lj2x72lnsk0l92qh-gcc-13.3.0-lib 8.7 MiB 39.9 MiB
/nix/store/0vsyw5bhwmisszyfd1a0sdnwvnf4qa5a-gcc-13.3.0 220.7 MiB 276.5 MiB
/nix/store/56nav6x43mv8b5bipazq4d9ycr1ngvav-expand-response-params 16.4 KiB 31.1 MiB
/nix/store/717iy55ncqs0wmhdkwc5fg2vci5wbmq8-bash-5.2p32 1.6 MiB 32.6 MiB
/nix/store/j6z0q427f6jm3d9mmrb4pgz72i93yp2f-attr-2.5.2 90.6 KiB 31.2 MiB
/nix/store/4ac0simsl36mz7ksv64ag7nmpc86s1mh-acl-2.3.2 125.2 KiB 31.3 MiB
/nix/store/w3jiv6xd520k1d1rvhxn7mr0pqzindjs-gmp-with-cxx-6.3.0 749.0 KiB 40.6 MiB
/nix/store/ph44jcx3ddmlwh394mh1wb7f1qigxqb1-coreutils-9.5 1.4 MiB 42.3 MiB
/nix/store/88rlbc5ka2m5zir98ajkgpqkb1i82wcs-rustc-1.83.0-x86_64-unknown-linux-gnu 318.5 MiB 362.5 MiB
/nix/store/0w0sbmymx7xify583f78djpx0c47xhqv-rustfmt-preview-1.83.0-x86_64-unknown-linux-gnu 10.9 MiB 373.5 MiB
/nix/store/zxk207030dbrz6ikiq61sz2c08gpljxq-pcre2-10.44 1.8 MiB 32.9 MiB
/nix/store/20s3724qq3g0j14v6zc0s7z16k7i3j39-libselinux-3.7 1.2 MiB 34.1 MiB
/nix/store/hc7rrrmb1wilsldsii4fbmvcl68lijsl-util-linux-minimal-2.39.4-lib 1.8 MiB 32.9 MiB
/nix/store/pmf31rbzvv618qirwmzppjb63wsrl4zi-libffi-3.4.6 71.9 KiB 31.2 MiB
/nix/store/fxnchbq09sq279ssqdsgzigsyvp6hz06-glib-2.80.4 14.7 MiB 50.8 MiB
/nix/store/ncznv9xrbi34k4zlnd25dp7kyaqjscn7-libgpg-error-1.50 832.7 KiB 31.9 MiB
/nix/store/mxqw137xwdd4fgnna0z4a7j8cx969br8-libgcrypt-1.10.3-lib 1.4 MiB 33.3 MiB
/nix/store/spvci786gald6nsf7z5a095fsc1dc3xx-libsecret-0.21.4 672.8 KiB 53.6 MiB
/nix/store/9x0gj79aslszsgn8qcgwxv6ni27szl00-cargo-1.83.0-x86_64-unknown-linux-gnu 33.4 MiB 87.2 MiB
/nix/store/c1n8g5129aycgj4cxxgry8460ki7glfj-binutils-2.43.1-lib 2.8 MiB 34.0 MiB
/nix/store/jp66i2lg1p2s8lkwsqh33c3913rgdq8j-clippy-preview-1.83.0-x86_64-unknown-linux-gnu 13.8 MiB 376.3 MiB
/nix/store/nzmp6x4694fn4al0dxq4g1q4fr4yyg7l-rust-std-1.83.0-x86_64-unknown-linux-gnu 146.9 MiB 147.1 MiB
/nix/store/v9znmjrv9f18072n1y1g5caf9irzs6v2-rust-docs-1.83.0-x86_64-unknown-linux-gnu 659.1 MiB 659.1 MiB
/nix/store/vcvhwiilizhijk7ywyn58p9l005n9sbn-binutils-2.43.1 30.1 MiB 72.9 MiB
/nix/store/l7n97992gd5piaw8phkxzsz176gfk1yj-binutils-wrapper-2.43.1 50.0 KiB 88.3 MiB
/nix/store/lvnwdmnjm7nvaq0a3vhvvn46iy4ql7gr-gnugrep-3.11 926.6 KiB 33.8 MiB
/nix/store/vh9fsdhgxcnab2qk7vdp2palkkn6j3cp-gcc-wrapper-13.3.0 61.8 KiB 316.1 MiB
/nix/store/vpg1chc5fncwvd5fh6j9nywx0bcdpf04-rust-src-1.83.0-x86_64-unknown-linux-gnu 32.1 MiB 64.8 MiB
/nix/store/frx5kr6cmy43qij0kcif59z47pdhkik3-rust-default-1.83.0 180.5 MiB 1.7 GiB
/nix/store/hchmc1f3czwxxn1q8nk9ya9h50cnyn1p-ripgrep-12.1.1 4.9 MiB 1.7 GiB
```
When using b259ef799b5ac014604da71ecd92d4a52603ed2d:
```ShellSession
# nix-build
# nix path-info --recursive --size --closure-size --human-readable ./result
/nix/store/4hcdxyjf9yiq7qf3i4548drb6sjmwa1v-xgcc-13.3.0-libgcc 155.9 KiB 155.9 KiB
/nix/store/w19cxz37j5nrkg8w80y91bga89310jgi-libunistring-1.2 1.8 MiB 1.8 MiB
/nix/store/jwsdpq2yxw43ixalh93z726czz7bay2j-libidn2-2.3.7 352.6 KiB 2.1 MiB
/nix/store/sl141d1g77wvhr050ah87lcyz2czdxa3-glibc-2.40-36 28.8 MiB 31.1 MiB
/nix/store/lw294qfdnf29lhxqwzy9byrmx975rvn9-gcc-13.3.0-libgcc 155.9 KiB 155.9 KiB
/nix/store/y3kdn61k93rq2jx1lj2x72lnsk0l92qh-gcc-13.3.0-lib 8.7 MiB 39.9 MiB
/nix/store/41zm31n8rnd9wb207fpz430dwydhqg3s-ripgrep-12.1.1 4.9 MiB 44.8 MiB
```