There seems to be a problem with how nix handles large files. The two instances I’ve noticed are when doing a “nix copy” to a remote S3 store or when a build produces a large derivation and transfers it from the build environment to the store.
If it was just a matter of large memory usage that would be one thing, but Nix spins 100cpu on a thread for longer than it would take to copy/hash/upload the path. It’s noticeable with few hundred MB, but really bad with several GB. With enough time and memory, eventually the transfer succeeds. I’ve been on relatively recent nixUnstable or nixFlakes.
There is one confirmation here. 1 Can anyone else reproduce this or benchmark a few transfers? My suspicion is that the time needed is super linear. Confirmation and examples will help track down the issue and a fix.