On this computer I am finding I get random build failures from random rust programs (both my own and in nixpkgs). If I keep trying to rebuild, I get a different build error every time. If I keep retrying it eventually works, mostly on the 2nd attempt. Any ideas?
The errors come from code that is not mine, and code that looks correct to me.
Is driving me batty. So far I have not been able to reproduce on any other computer (maybe I haven’t tried enough times?), and all computers have the same nixos/home-manager config.
I think I might have asked about this somewhere else recently, but can’t find it right now.
Here is the latest example.
nix-deploys via ❄️ (nix-shell-env) took 2s
at 15:07:54 ❯ nix flake update robotica-rust
warning: updating lock file '"/home/brian/tree/personal/nix-deploys/flake.lock"':
• Updated input 'robotica-rust':
'github:brianmay/robotica-rust/f2d775818c605b7b13a149bea2a5fd578fd171fe?narHash=sha256-%2Btw1gvlpcLVYvwNjQYB02x8tzgFP9Sd2iiRGsd3m6WM%3D' (2025-10-06)
→ 'github:brianmay/robotica-rust/4d35aa89e6d1ae616116e456f1bd67dd8350a70a?narHash=sha256-KYfdxLcNgSNFisvpbTEBtoXBswzLNtZo%2BUgvxsZ2dXE%3D' (2025-10-07)
direnv: loading ~/tree/personal/nix-deploys/.envrc
direnv: using flake
evaluating derivation 'git+file:///home/brian/tree/personal/nix-deploys#devShells.x86_64-linux.defaudirenv: ([/nix/store/8cz47vsx8z9ksx63472i2p8jqk86r45j-direnv-2.36.0/bin/direnv export json]) is taking a while to execute. Use CTRL-C to give up.
error: builder for '/nix/store/acrk4rbdrkihlsjwfsm74cvn516l8kff-robotica-frontend-deps-0.0.0.drv' failed with exit code 101;
last 25 log lines:
> Compiling url v2.5.7
> Compiling robotica-common v0.1.0 (/build/source/robotica-common)
> error[E0080]: scalar size mismatch: expected 4294968006 bytes but got 4 bytes instead
> --> /nix/store/ji8h360nlg0da7kxxa865xajfnkb5g95-vendor-cargo-deps/c19b7c6f923b580ac259164a89f2577984ad5ab09ee9d583b888f934adbbe8d0/web-sys-0.3.77/src/features/gen_DragEvent.rs:5:1
> |
> 5 | #[wasm_bindgen]
> | ^^^^^^^^^^^^^^^ evaluation of `features::gen_DragEvent::_::_ENCODED_BYTES::_CHUNKS` failed inside this call
> |
> note: inside `flat_byte_slices::<3390, 1>`
> --> /nix/store/ji8h360nlg0da7kxxa865xajfnkb5g95-vendor-cargo-deps/c19b7c6f923b580ac259164a89f2577984ad5ab09ee9d583b888f934adbbe8d0/wasm-bindgen-0.2.100/src/rt/mod.rs:664:15
> |
> 664 | while i < slice.len() {
> | ^^^^^^^^^^^^^^^ the failure occurred here
> = note: this error originates in the attribute macro `wasm_bindgen` (in Nightly builds, run with -Z macro-backtrace for more info)
>
> note: erroneous constant encountered
> --> /nix/store/ji8h360nlg0da7kxxa865xajfnkb5g95-vendor-cargo-deps/c19b7c6f923b580ac259164a89f2577984ad5ab09ee9d583b888f934adbbe8d0/web-sys-0.3.77/src/features/gen_DragEvent.rs:5:1
> |
> 5 | #[wasm_bindgen]
> | ^^^^^^^^^^^^^^^
> |
> = note: this note originates in the attribute macro `wasm_bindgen` (in Nightly builds, run with -Z macro-backtrace for more info)
>
> For more information about this error, try `rustc --explain E0080`.
> error: could not compile `web-sys` (lib) due to 1 previous error
For full logs, run:
nix log /nix/store/acrk4rbdrkihlsjwfsm74cvn516l8kff-robotica-frontend-deps-0.0.0.drv
error: 1 dependencies of derivation '/nix/store/3xslb3390h46sdk37v39cbq248vrhn29-robotica-frontend-0.0.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/i191yy6iw93xm7c235rkkqa0h4cp82ww-robotica-frontend-bindgen.drv' failed to build
error: 1 dependencies of derivation '/nix/store/xg4drk6jzylrar2yhgvjx500bawxc2fv-robotica-backend-config.drv' failed to build
error (ignored): error: cannot unlink "/tmp/nix-build-robotica-backend-deps-0.0.0.drv-0/build/source/target/release/deps": Directory not empty
error: 1 dependencies of derivation '/nix/store/xb09p5mda4733p51ff5s71hffcavz77c-robotica-backend.drv' failed to build
error: 1 dependencies of derivation '/nix/store/6dys4mciih957ikihp9c3z155jj6r3kv-unit-robotica-backend.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/yaxdyk9a4848lf06x85j49l1pgpm7j09-system-units.drv' failed to build
error: 1 dependencies of derivation '/nix/store/g3isqqx0kv3pqhmkv4d42yh9b8pbmpbz-etc.drv' failed to build
error: 1 dependencies of derivation '/nix/store/kdv3ffkn99b63sb2hp3m8cq46w89x9i8-nixos-system-iot2-25.05.20250904.fe83bbd.drv' failed to build
error: 1 dependencies of derivation '/nix/store/g4rknp8nws807iva8wcsq4pc8v39x23l-activatable-nixos-system-iot2-25.05.20250904.fe83bbd.drv' failed to build
error (ignored): error: cannot unlink "/tmp/nix-build-robotica-freeswitch-deps-0.0.0.drv-0/build/source/target/release/deps": Directory not empty
error: 1 dependencies of derivation '/nix/store/38q922z6mnyh7pcsvgb7i3kpzrf4gkhv-deploy-rs-check-activate.drv' failed to build
error: 1 dependencies of derivation '/nix/store/dzh4czz0al6sgpr2ywqh048plrph1jzp-deploy.json.drv' failed to build
error: 1 dependencies of derivation '/nix/store/mszibnxcq1rrqfsyhmnqgril2d61kjg2-deploy.json.drv' failed to build
error: 1 dependencies of derivation '/nix/store/3w3y878h01ldc3q21a4qa0hzb9sfylk8-nix-shell-env.drv' failed to build
direnv: nix-direnv: Evaluating current devShell failed. Falling back to previous environment!
direnv: export +AR +AS +CC +CONFIG_SHELL +CXX +HOST_PATH +IN_NIX_SHELL +LD +NIX_BINTOOLS +NIX_BINTOOLS_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu +NIX_BUILD_CORES +NIX_CC +NIX_CC_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu +NIX_CFLAGS_COMPILE +NIX_DIRENV_DID_FALLBACK +NIX_ENFORCE_NO_NATIVE +NIX_HARDENING_ENABLE +NIX_LDFLAGS +NIX_STORE +NM +OBJCOPY +OBJDUMP +RANLIB +READELF +SIZE +SOURCE_DATE_EPOCH +STRINGS +STRIP +__structuredAttrs +buildInputs +buildPhase +builder +cmakeFlags +configureFlags +depsBuildBuild +depsBuildBuildPropagated +depsBuildTarget +depsBuildTargetPropagated +depsHostHost +depsHostHostPropagated +depsTargetTarget +depsTargetTargetPropagated +doCheck +doInstallCheck +dontAddDisableDepTrack +mesonFlags +name +nativeBuildInputs +out +outputs +patches +phases +preferLocalBuild +propagatedBuildInputs +propagatedNativeBuildInputs +shell +shellHook +stdenv +strictDeps +system ~PATH ~XDG_DATA_DIRS
nix-deploys via ❄️ (nix-shell-env)
at 08:36:14 ❯ direnv reload
direnv: loading ~/tree/personal/nix-deploys/.envrc
direnv: using flake
evaluating derivation 'git+file:///home/brian/tree/personal/nix-deploys#devShells.x86_64-linux.defaudirenv: ([/nix/store/8cz47vsx8z9ksx63472i2p8jqk86r45j-direnv-2.36.0/bin/direnv export json]) is taking a while to execute. Use CTRL-C to give up.
direnv: nix-direnv: Renewed cache
direnv: export +AR +AS +CC +CONFIG_SHELL +CXX +HOST_PATH +IN_NIX_SHELL +LD +NIX_BINTOOLS +NIX_BINTOOLS_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu +NIX_BUILD_CORES +NIX_CC +NIX_CC_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu +NIX_CFLAGS_COMPILE +NIX_ENFORCE_NO_NATIVE +NIX_HARDENING_ENABLE +NIX_LDFLAGS +NIX_STORE +NM +OBJCOPY +OBJDUMP +RANLIB +READELF +SIZE +SOURCE_DATE_EPOCH +STRINGS +STRIP +__structuredAttrs +buildInputs +buildPhase +builder +cmakeFlags +configureFlags +depsBuildBuild +depsBuildBuildPropagated +depsBuildTarget +depsBuildTargetPropagated +depsHostHost +depsHostHostPropagated +depsTargetTarget +depsTargetTargetPropagated +doCheck +doInstallCheck +dontAddDisableDepTrack +mesonFlags +name +nativeBuildInputs +out +outputs +patches +phases +preferLocalBuild +propagatedBuildInputs +propagatedNativeBuildInputs +shell +shellHook +stdenv +strictDeps +system ~PATH ~XDG_DATA_DIRS
Looks suspiciously like it could be hardware failure IMO. How is your RAM doing? Could be that the rust + nix builds tickle your memory enough to hit faulty areas.
Yeah, as long as this is just a bare metal system use that setting and it’ll deploy the binary as a boot menu entry. FWIW, using grub specifically is discouraged for most setups, but that option is generic between the boot managers.
I ran through all the memory tests for one pass, no errors detected.
Yes, I setup this computer using guides for using zfs as root before I realized grub is now discouraged, and am hesitant to change it in case I break my working computer :-). /boot is ext4 so it shouldn’t be a problem in theory…
Apparently it is good practice to use the EFI partition - FAT32 - as /boot, not /boot/efi. Sob. So maybe I am not keeping up here with what is best practice.
Back on topic: I considered the possibility that zfs is causing the random build failures, but this doesn’t seem likely to cause the errors I have been seeing IMHO.
This definitely seems like a hardware memory or CPU issue to me. If you want to rule out disk issues you could see if it still happens when building on tmpfs to largely remove the disk from the equation (though everything would probably be cached in memory given your amount of RAM anyways). Try stress and/or mprime to test CPU and RAM (arch wiki recommended tools: Stress testing - ArchWiki). Make sure to monitor system temperatures to ensure it isn’t overheating.
Do you have a 13th or 14th gen Intel CPU btw? That’s been a common cause of CPU instability recently. If it’s not the CPU, it could still be the memory (a lot of issues don’t show up in the first memtest pass as they’re probabilistic) but given the frequency with which you encounter this issue and the amount of RAM you have it does seem more like a CPU issue to me.
No, unfortunately if the problem appears it means the CPU itself is physically damaged (due to voltage issues from my understanding). Intel has released new firmware which fixes these issues but it can’t fix an already damaged CPU.
They’ve extended their RMA warranty so you can probably get a free CPU replacement. I did that and it was relatively painless, aside from figuring out the serial number. I’d still try to make sure it’s a CPU issue via stress testing first so that you don’t go through this process for nothing. Good luck!
warning: Git tree '/home/brian/tree/personal/robotica/robotica-rust' is dirty
error: builder for '/nix/store/6ccl58ixw27n5i2vngmlk9gfa8kvh31p-robotica-frontend-deps-0.0.0.drv' failed with exit code 101;
last 25 log lines:
> = note: this error originates in the attribute macro `wasm_bindgen` (in Nightly builds, run with -Z macro-backtrace for more info)
>
> note: erroneous constant encountered
> --> /nix/store/993fjkck66vsl08b20kyms27r41y5hkr-vendor-cargo-deps/c19b7c6f923b580ac259164a89f2577984ad5ab09ee9d583b888f934adbbe8d0/web-sys-0.3.77/src/features/gen_HtmlInputElement.rs:5:1
> |
> 5 | #[wasm_bindgen]
> | ^^^^^^^^^^^^^^^
> |
> = note: this note originates in the attribute macro `wasm_bindgen` (in Nightly builds, run with -Z macro-backtrace for more info)
>
> error[E0080]: scalar size mismatch: expected 4 bytes but got 0 bytes instead
> --> /nix/store/993fjkck66vsl08b20kyms27r41y5hkr-vendor-cargo-deps/c19b7c6f923b580ac259164a89f2577984ad5ab09ee9d583b888f934adbbe8d0/web-sys-0.3.77/src/features/gen_Window.rs:5:1
> |
> 5 | #[wasm_bindgen]
> | ^^^^^^^^^^^^^^^ evaluation of `features::gen_Window::_::_GENERATED` failed inside this call
> |
> note: inside `flat_byte_slices::<43049, 2>`
> --> /nix/store/993fjkck66vsl08b20kyms27r41y5hkr-vendor-cargo-deps/c19b7c6f923b580ac259164a89f2577984ad5ab09ee9d583b888f934adbbe8d0/wasm-bindgen-0.2.100/src/rt/mod.rs:664:15
> |
> 664 | while i < slice.len() {
> | ^^^^^^^^^^^^^^^ the failure occurred here
> = note: this error originates in the attribute macro `wasm_bindgen` (in Nightly builds, run with -Z macro-backtrace for more info)
>
> For more information about this error, try `rustc --explain E0080`.
> error: could not compile `web-sys` (lib) due to 2 previous errors
For full logs, run:
nix log /nix/store/6ccl58ixw27n5i2vngmlk9gfa8kvh31p-robotica-frontend-deps-0.0.0.drv
error: 1 dependencies of derivation '/nix/store/js7hpvvs9n272vg6i2gkd8mavbx34n7x-robotica-frontend-0.0.0.drv' failed to build
error (ignored): error: cannot unlink "/tmp/nix-build-robotica-slint-deps-0.0.0.drv-1/build/source/target/release/deps": Directory not empty
error (ignored): error: cannot unlink "/tmp/nix-build-robotica-freeswitch-deps-0.0.0.drv-1/build/source/target/release/deps": Directory not empty
error (ignored): error: cannot unlink "/tmp/nix-build-robotica-backend-deps-0.0.0.drv-1/build/source/target/release/deps": Directory not empty
error: build of '/nix/store/18z3fw6wd2np8yzrq3mk9rmwcavspwm3-robotica-backend-clippy-0.0.0.drv', '/nix/store/2ikdi9c8119dh97a2d1i41f070xnb196-robotica-freeswitch.drv', '/nix/store/2s4qhaxjm5nkfnmglfp2m1gckylk06k2-robotica-backend.drv', '/nix/store/37pdxkrl7xj4i5cdhlfvpj04xblb3pjd-robotica-freeswitch-clippy-0.0.0.drv', '/nix/store/8s8nhvcpdc8d1yy7ygwjkvcjfdy0dnp2-robotica-frontend-clippy-0.0.0.drv', '/nix/store/gi6priafdgi3wjhikaay9dd0cwcpz6a4-robotica-slint-0.0.0.drv', '/nix/store/js7hpvvs9n272vg6i2gkd8mavbx34n7x-robotica-frontend-0.0.0.drv', '/nix/store/wkwq7s7mirc53zh88brnmyjx10834dlc-robotica-slint-clippy-0.0.0.drv', '/nix/store/yn6mhp4mblazbl3jj15dxjdlzbc121y0-robotica-frontend-bindgen.drv' failed
First time I got an error without using nix build, just plain rust build. Yuck. Was trying to blame nix
Compiling icu_properties v1.5.1
error[E0080]: scalar size mismatch: expected 0 bytes but got 4 bytes instead
--> /home/brian/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/icu_properties-1.5.1/src/provider.rs:711:1
|
711 | / expand!(
712 | | (
713 | | // code point sets
714 | | (AsciiHexDigitV1Marker, "AHex"),
... |
929 | | );
| |_^ evaluation of `<provider::JoinControlV1Marker as icu_provider::KeyedDataMarker>::KEY::RESOURCE_KEY_MACRO_CONST` failed inside this call
|
note: inside `icu_provider::DataKey::construct_internal`
--> /home/brian/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/icu_provider-1.5.0/src/key.rs:394:15
|
394 | while i < leading_tag!().len() {
| ^^^^^^^^^^^^^^^^^^^^^^^^ the failure occurred here
= note: this error originates in the macro `data_key` which comes from the expansion of the macro `expand` (in Nightly builds, run with -Z macro-backtrace for more info)
note: erroneous constant encountered
--> /home/brian/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/icu_properties-1.5.1/src/provider.rs:711:1
|
711 | / expand!(
712 | | (
713 | | // code point sets
714 | | (AsciiHexDigitV1Marker, "AHex"),
... |
929 | | );
| |_^
|
= note: this note originates in the macro `data_key` which comes from the expansion of the macro `expand` (in Nightly builds, run with -Z macro-backtrace for more info)
note: erroneous constant encountered
--> /home/brian/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/icu_properties-1.5.1/src/provider.rs:693:15
|
693 | $($code_point_set_marker::KEY,)+
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
...
711 | / expand!(
712 | | (
713 | | // code point sets
714 | | (AsciiHexDigitV1Marker, "AHex"),
... |
929 | | );
| |_- in this macro invocation
|
= note: this note originates in the macro `expand` (in Nightly builds, run with -Z macro-backtrace for more info)
note: erroneous constant encountered
--> /home/brian/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/icu_properties-1.5.1/src/provider.rs:142:5
|
142 | icu_properties_data::impl_props_join_c_v1!(Baked);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: this note originates in the macro `icu_properties_data::impl_props_join_c_v1` (in Nightly builds, run with -Z macro-backtrace for more info)
For more information about this error, try `rustc --explain E0080`.
error: could not compile `icu_properties` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
Have you tried compiling large projects in other languages, e.g. the Linux kernel or Firefox? I’m still pretty sure this is some kind of hardware instability, probably CPU given that it seems to occur in certain codepaths. I’d also try other stress testing commands, e.g. letting stress --cpu $(nproc) run for a while.
