Per-user profiles not created when $HOME mounted on NFS

I’m using Nix (multi-user) on Ubuntu and i’m currently getting error: opening lock file '/nix/var/nix/profiles/per-user/.../profile.lock': No such file or directory, when trying to install something like nix-env -i nix-prefetch-git. It’s likely because the /nix/var/nix/profiles/per-user/ directory doesn’t contain a subdirectory for my user. What’s responsible for creating per-user subdirectories there? What could be causing it to not happen?

My setup is a bit unconventional in that the $HOME is mounted on an NFS drive (shared with other hosts), but i can’t figure out why that would matter. There are a few other hosts where Nix is working fine (also with $HOME on NFS), but on others i’m getting the above error (which were likely set up some time after the working hosts).

To investigate, i re-installed Nix on one of the failing hosts (removing Nix stuff first as instructed by the installer). But that also didn’t help - now i get error: creating directory '/nix/var/nix/profiles/per-user/...': Permission denied when doing anything with nix-env or nix-shell.

1 Like

home-manager documents the step to create the user profiles as well as per user gcroots as a manual process when not on nixOS.

Make sure you have a working Nix installation. If you are not using NixOS then you may here have to run

$ mkdir -m 0755 -p /nix/var/nix/{profiles,gcroots}/per-user/$USER

since Home Manager uses these directories to manage your profile generations. On NixOS these should already be available.

Still they were already available when I installed home-manager on my Arch Linux with Nix.

Perhaps it helps to create/adjust folders and permissions?

2 Likes

It does indeed help if i manually create the $USER subdirectory in those per-user directories. The issue is that it’s not happening automatically on some of the hosts. However, the per-user subdirectories are created on other hosts with, what i think is, the same setup (Ubuntu version, home mounted on NFS, etc., but maybe the initially installed Nix version was different). I would hate to have to manually create these subdirectories for every user on the system when afaik it should be happening automatically.

EDIT: ah, it turns out creating those directories DOESN’T help. Now i get

$ nix-shell -p nix-info --run "nix-info -m"
error: opening lock file '/nix/var/nix/db/big-lock': Permission denied

it seems that the permissions for the Nix daemon (or something?) are somehow messed up on this host. I have now tried multiple times to re-install different versions (2.3 and 2.3.2) of Nix on this host without success.

The reason, why i initially wrote that creating the subdirectories helped, was because it did help on another host with the same issue previously.

I can’t think of what’s different in all of these instances.

EDIT2: i noticed that the above error was the same as the one i got when trying out an older Nix version – 2.3. It led me to suspect that maybe Nix 2.3 did manage to create the per-user directories and this is a separate issue. So i uninstalled Nix 2.3.2 and installed Nix 2.3 again. I do get the same error as above

$ nix-shell -p nix-info --run "nix-info -m"
error: opening lock file '/nix/var/nix/db/big-lock': Permission denied

But checking ls /nix/var/nix/profiles/per-user/ does list my user’s directory.

It now seems that this might, at least partially, be due to a bug in Nix 2.3.2, so i guess i’ll submit an issue about this on GitHub.

I did some more experimenting that cleared up some things for me.

The big-lock permission denied issue

The

$ nix-shell -p nix-info --run "nix-info -m"
error: opening lock file '/nix/var/nix/db/big-lock': Permission denied

issue with Nix 2.3 and 2.3.2 was due to me not deleting/stopping nix-deamon.service and nix-deamon.socket prior to installing Nix. I.e., i didn’t follow the first step in the “Uninstalling nix” instructions presented by the installer.

Fixed it by following instructions properly.

Issue with per-user directories not being created

After the above was fixed, the issue with per-user directories not being automatically created when running nix-env -i ... is still present in Nix 2.3.2. I.e., i get

$ nix-env -i nix-prefetch-git
installing 'nix-prefetch-git'
error: opening lock file '/nix/var/nix/profiles/per-user/.../profile.lock': No such file or directory

If i roll back to Nix 2.3, the issue is gone – the per-user directory gets created (same as described in my previous post) and nix-prefetch-git gets installed. So it still seems to be a bug with Nix 2.3.2.

The discrepancy between Nix 2.3 and 2.3.2 is the cause of different behaviors of the various hosts i described above. Nix 2.3.2 was installed on the newer hosts and that’s why the issues started.

The effect of $HOME being on NFS

Turns out that Nix 2.3.2 acts differently depending whether i’m running nix-env -i ... from a user whose $HOME is on an NFS drive (that’s the case for all of the debugging done above) or from a regular user whose home resides on a local drive. When using a local user, i’m able to run nix-env -i nix-prefetch-git without an issue on Nix 2.3.2 – it successfully creates /nix/var/nix/profiles/per-user/$USER for the local user. nix-env -i ... also works for users with home on NFS if i manually create the per-user directories as suggested by @NobbZ above.

EDIT:
I’ve posted an issue on GitHub Per-user profiles not created when $HOME mounted on NFS · Issue #3356 · NixOS/nix · GitHub