How to move nix store to external drive on macos?

Swordlash · June 7, 2022, 10:01pm

Hi, I have nix installed on my MacOS Monterey. Sadly I’m running out of space on the laptop and I’d like to make nix to use the external drive as its store (i.e. by symlinking). How to do it properly? I guess I’d need to copy the /nix catalogue contents into the drive, get rid of the Nix Store volume and then symlink the drive to /nix. Am I on a good track here?

abathur · June 8, 2022, 12:39am

While I haven’t tried it myself, IIUC a symlinked store can cause problems with some software. But the store volume doesn’t have to live on the same drive as /.

If starting over isn’t prohibitive, it’s probably easiest to follow the uninstall instructions (Installing a Binary Distribution) and then set the appropriate variables to specify a different disk before installing:

github.com

NixOS/nix/blob/b2dea231cf61d8d10cef89991571f57976b77f06/scripts/create-darwin-volume.sh#L64-L71

    
      
          # make it easy to play w/ 'Case-sensitive APFS'
          readonly NIX_VOLUME_FS="${NIX_VOLUME_FS:-APFS}"
          readonly NIX_VOLUME_LABEL="${NIX_VOLUME_LABEL:-Nix Store}"
          # Strongly assuming we'll make a volume on the device / is on
          # But you can override NIX_VOLUME_USE_DISK to create it on some other device
          readonly NIX_VOLUME_USE_DISK="${NIX_VOLUME_USE_DISK:-$(root_disk_identifier)}"
          NIX_VOLUME_USE_SPECIAL="${NIX_VOLUME_USE_SPECIAL:-}"
          NIX_VOLUME_USE_UUID="${NIX_VOLUME_USE_UUID:-}"

If starting over is prohibitive, it’ll be a little more DIY–but there’s inevitably some way to duplicate the existing volume to a new disk and then update /etc/fstab and /Library/LaunchDaemons/org.nixos.darwin-store.plist to specify the new volume’s UUID (fair warning: this will be more complicated if you’re using FileVault).

Swordlash · June 16, 2022, 11:04am

I’ll describe what I’ve done in case someone needs to do it on their own. The following steps seem to be working:

Install nix in a conventional way. This will create /nix entry in /etc/synthetic.conf and /nix entry in /etc/fstab.
Connect external drive. I formatted it to encrypted, case-sensitive APFS. Save the disk’s UUID (it’s shown in Disk Utility info and in diskutil apfs list). rsync the contents of /nix to this drive.
Delete the /nix disk created by the installer with sudo diskutil apfs deleteVolume /nix.
Using sudo vifs, edit /etc/fstab to point /nix to your disk’s UUID. For me,

mateusz@Mateuszs-MacBook-Pro ~ % cat /etc/fstab 
UUID=EBC0B7AA-1BD5-4F56-A677-6BDD47AC95F5 /nix apfs rw,noauto,nobrowse,suid,owners

This way, your disk will be automatically mounted to /nix when connected.

Reboot.
Connect the disk, if it’s not connected. With noauto it shouldn’t mount automatically; open Disk Utility and mount. This should automatically mount it to /nix.
Voila! Open the terminal and see whether the shortcuts to nix are sourced properly.

After first reboot, I had an issue with OS telling me that some nix software (the daemon I think) comes from an unknown source and won’t be launched. I opened security settings and allowed it from there.

abathur · June 16, 2022, 3:44pm

This is why I said to update /Library/LaunchDaemons/org.nixos.darwin-store.plist to specify the new volume’s UUID.

I’ll also reiterate that this will be more complicated for anyone trying to follow along with FileVault enabled. In that case, the installer creates an encrypted volume and stores the passphrase in your system keychain (using the volume UUID) and generates a slightly different mounting service to request this passphrase and use it to decrypt the volume.

The external disk/volume may or may not be encrypted (i.e., it is up to you if you are manually generating the volume)–and you’ll need to square these parts if you adapt an existing install. If you don’t encrypt the volume, you can use the below as a reference for how to replace the command in /Library/LaunchDaemons/org.nixos.darwin-store.plist:

github.com

NixOS/nix/blob/7e301fd74e929af916116a04d84e130f9e0112de/scripts/create-darwin-volume.sh#L287-L291

    
      
          case "$cmd_type" in
              encrypted)
                  cmd=(/bin/sh -c "/usr/bin/security find-generic-password -s '$volume_uuid' -w | /usr/sbin/diskutil apfs unlockVolume '$volume_uuid' -mountpoint '$mountpoint' -stdinpassphrase");;
              unencrypted)
                  cmd=(/usr/sbin/diskutil mount -mountPoint "$mountpoint" "$volume_uuid");;

If you do encrypt your disk/volume, you’ll need to update the UUIDs in the plist and put the passphrase in the system keychain. You can see how the installer encrypts the volume and stores the credential in:

github.com

NixOS/nix/blob/7e301fd74e929af916116a04d84e130f9e0112de/scripts/create-darwin-volume.sh#L703-L715

    
      
              _sudo "to mount your Nix volume for encrypting" \
                  /usr/sbin/diskutil mount "$volume_label"
          
          
    password="$(/usr/bin/xxd -l 32 -p -c 256 /dev/random)"
              _sudo "to add your Nix volume's password to Keychain" \
                  /usr/bin/security -i <<EOF
          add-generic-password -a "$volume_label" -s "$volume_uuid" -l "$volume_label encryption password" -D "Encrypted volume password" -j "Added automatically by the Nix installer for use by $NIX_VOLUME_MOUNTD_DEST" -w "$password" -T /System/Library/CoreServices/APFSUserAgent -T /System/Library/CoreServices/CSUserAgent -T /usr/bin/security "/Library/Keychains/System.keychain"
          EOF
              builtin printf "%s" "$password" | _sudo "to actually encrypt your Nix volume" \
                  /usr/sbin/diskutil apfs encryptVolume "$volume_label" -user disk -stdinpassphrase
          
          
    _sudo "to unmount the encrypted volume" \
                  /usr/sbin/diskutil unmount force "$volume_label"

Swordlash · June 16, 2022, 4:15pm

I have FileVault enabled, if that’s relevant.

abathur · June 16, 2022, 6:09pm

Indeed. I updated the previous response to leave more breadcrumbs.

Swordlash · June 16, 2022, 6:43pm

Thank you for the clarifications.