Hi!
I am tried to override the really-heavy package (chromium) to add missing functionality (krb5). And faced with memory limitation for this operation. Then I tried to google and look in manuals and could not find any information about where does nix-build and nixos-rebuild executes genericBuild scripts for packages and how to change it. I tries to use TMDIR variable but no unpacked sources appeared there - just empty folder.
How can I move building directory from limited temporary disk to huge ssd?
what file system is running out of space?
/tmp should be on disk.
opened 04:24PM - 27 Jan 19 UTC
## Issue description
I'm trying to rebuild with staging branch.
```
[root… @imac:~]$ df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 802M 0 802M 0% /dev
tmpfs 7.9G 2.1M 7.9G 1% /dev/shm
tmpfs 4.0G 7.0M 4.0G 1% /run
tmpfs 7.9G 360K 7.9G 1% /run/wrappers
/dev/sda4 458G 134G 300G 31% /
tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup
tmpfs 7.9G 17M 7.9G 1% /tmp
/dev/sda1 197M 174M 24M 89% /boot
tmpfs 1.6G 88K 1.6G 1% /run/user/1000
tmpfs 1.6G 4.0K 1.6G 1% /run/user/78
```
But the rebuild fails:
```
[ 48%] Building HexagonGenDAGISel.inc...
CXX menuattributeiter.lo
varobj.c:2539:1: fatal error: error writing to /build/ccE65Is0.s: No space left on device
}
^
/build/llvm/lib/ExecutionEngine/GDBRegistrationListener.cpp:243:1: fatal error: error writing to /build/ccUCp0u7.s: No space left on device
}
^
compilation terminated.
compilation terminated.
make[2]: *** [Makefile:1622: varobj.o] Error 1
make[2]: Leaving directory '/build/gdb-8.2.1/gdb'
make[1]: *** [Makefile:8825: all-gdb] Error 2
make[1]: Leaving directory '/build/gdb-8.2.1'
make: *** [Makefile:850: all] Error 2
make[2]: *** [lib/ExecutionEngine/CMakeFiles/LLVMExecutionEngine.dir/build.make:89: lib/ExecutionEngine/CMakeFiles/LLVMExecutionEngine.dir/GDBRegistrationListener.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:5574: lib/ExecutionEngine/CMakeFiles/LLVMExecutionEngine.dir/all] Error 2
make: *** [Makefile:152: all] Error 2
builder for '/nix/store/l6zb0p9x0qh61c2z01vjnpbhv0p0q93l-gdb-8.2.1.drv' failed with exit code 2
note: build failure may have been caused by lack of free disk space
building '/nix/store/kgfvxx6g7vhbihb9wpz9j2ha5zjlywh1-gstreamer-0.10.36.drv'...
cannot build derivation '/nix/store/5717l1jaw5lgavps9h2gd3g6ppbw631s-rustc-1.31.0.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/9wd6hx622jlg109crglkwcq6dzn7psh4-librsvg-2.44.11.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/j8h056pxlalrgxyyd6b2pmd8l74wv46h-gst-plugins-bad-1.14.4.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/ma311sr4vz5bb9kn7ljd49mfwqzhdsjg-wine-3.0.4.drv': 1 dependencies couldn't be built
building '/nix/store/zkpdcw0yyn94hd92qz222j052r45jpay-gts-0.7.6.drv'...
cannot build derivation '/nix/store/wcckb16wd1sc3i41d761fqwd8dsv3f1l-playonlinux-4.3.4.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/vnkwq8rkgwqpzgkfsr5kxfndw0hmkys6-system-path.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/gqd0jr4ljpszh8c63wcbk6bvwqx0r302-nixos-system-imac-19.03.git.319ebef.drv': 1 dependencies couldn't be built
error: build of '/nix/store/gqd0jr4ljpszh8c63wcbk6bvwqx0r302-nixos-system-imac-19.03.git.319ebef.drv' failed
```
### Steps to reproduce
`boot.tmpOnTmpfs = true;`
rebuild on staging, building all the things
### Workaround
`mount -o remount,size=15G /tmp`
## Technical details
- system: `"x86_64-linux"`
- host os: `Linux 4.19.16, NixOS, 19.03.git.8291376 (Koi)`
- multi-user?: `yes`
- sandbox: `yes`
- version: `nix-env (Nix) 2.2`
- channels(root): `"nixos-18.09.1829.0396345b794, nixos-hardware, nixos-unstable-19.03pre164715.201d739b0ff"`
- nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`
might gleam some issues.
NobbZ
March 2, 2022, 6:04pm
3
It is hard to help ypu here without the actual expression and error…
nix
does usually adhere to TMP
, TMPDIR
, and TEMP
, though I am not sure in which order of precedence or whatever…
Though I can say for sure I have seen builds happening in both /tmp
and /run/users/foo/tmp
or what the folder was when I was in a shell already…
So, more and detailed information here might be nice.
PS: by defailt /tmp
should be used which by default is on disk. So unless you changed things, there should be plenty of space…
PPS: Last month it took me an hour or two to realize that I am running out of space on /tmp
because of my choice to have a 10GiB /
…
1 Like
That is the root cause. I moved /tmp folder to tmpfs for the security reasons. Is there any way to change build directory without switching /tmp back to disk?
I try to build this package on 21.11 nixos:
chromium_browser_krb = pkgs.chromium.browser.overrideAttrs (attrs: {
gnFlags = lib.concatStringsSep " " [
attrs.gnFlags
"use_kerberos=true"
];
});
chromium_krb = pkgs.chromium.overrideAttrs (attrs: {
browser = chromium_browser_krb;
});
It of course may be incorrect, but I have no ability to check it because of subject.
P.S. Remount tmp with 64Gb, relaying on swap. Will see.
I’ve had good success with
zramSwap = {
enable = true;
algorithm = “zstd”;
};
in my configuration.nix… your mileage will vary, it will give you significantly more swap space at the cost of CPU.
You could try a symbolic link , but there must be a way to set it, which escapes me right now.
having tmp files in ram makes a lot of sense , especially with big builds. It can really keep wear and tear down of HDD’s and SSD’s… RAM has a pretty high mean time before failure… spinning rust and SSD’s are doomed to fail by their design.