Copy vs move - installPhase best practice

I’ve seen both cp and mv used in custom installPhase scripts for putting files under $out. Is there any reason to prefer one over the other?
As far as I can tell, the nixpkgs docs use both in examples.

2 Likes

It depends.

I want to say upfront that on most filesystems ( ext4 for example ) if you use cp -pr --reflink=auto -- . "$out"; you should just end up creating hardlinks; so really the performance hit you take is just syscalls rather than disk IO. But there are of course exceptions.

If you’re packaging something for your own usage that you don’t intend to PR into Nixpkgs you can absolutely use mv.

Just keep in mind that a handful of setup hooks related to tests, “source only” hooks, and some dist hooks ( scarcely used ) would likely fail since they often run after the install phase. For Nixpkgs we wouldn’t want to break these niche use cases so cp is preferred.

Personally a lot of the time I’ll just run builds in $out to avoid copying or moving anything; but if I were planning to PR Nixpkgs I would drop that optimization.

For Nixpkgs also keep in mind that in practice people consume built artifacts from caches, so the performance difference between a move vs copy can end up being a wash. What really matters is that a build is reproducible in this context.

1 Like

If /tmp and /nix/store are on different file systems, including different ZFS datasets or btrfs subvolumes, then there is no difference between moving and copying, is there? I don’t even think reflink works across file systems.

1 Like

If everything is on the same filesystem then mv is faster because it does not need to duplicate the data.

There is nothing forbidding using mv.

I can’t think of any hooks that would do that when manually specifying the install phase. Most places where such things occur use hooks for everything and the package managers commands.

This is dirty because it leaves temporary files. Don’t do that.

1 Like

I always assume that /nix/store is on a different file-system so I copy. And if it wasn’t I still prefer to copy because then the files in $out are allocated at the same time, rather than some allocated during the build and some not. Don’t take my word on that, that’s me being superstitious about fragmentation.

Also, for the sake of being functional its best to leave the artifacts of the buildPhase where they are and not mutate the build state after buildPhase. That allows someone to add a postInstall or fixupPhase hook that can work on the build artifacts seperately from what happened in installPhase.

2 Likes

Wouldn’t that leave changes in the build directory which really should be in $out?

It’s not something you would normally do, I just think it’s good practice in theory.

I think packagers from more “traditional” distros might suggest that both are wrong and tell you to use install.

But I have personally used all 3 just sort of at random so far :sweat_smile:

4 Likes

All of my builders have nix-daemon's TMPDIR pointing to a tmpfs filesystem. There is a very large performance benefit from this – with enough RAM incomplete build artifacts never hit the disk, and even if they do (go to swap) it’s still faster because swap doesn’t need to be journaled/sync’d/crashsafe.

Plus with install you don’t need mkdir -p:

installPhase = ''
  install -D frobnicate -t $out/bin/
'';
3 Likes

No you won’t. That command will create reflinks on certain filesystems which support them. Btrfs, XFS and bcachefs are the only Linux ones I know of.
There’s a catch here that /nix/store and $TMPDIR are usually behind different mounts in the VFS which prevents copies but with recent kernels, that has been alleviated.

On regular old filesystems like EXT4, it’ll create regular old copies. No hardlinks.

Unlike ZFS, you can move files between btrfs subvolumes.

Hosted by Flying Circus.