How to use home-manager module inside a declarative container?

Help
1

I’m having some trouble using home-manager inside a container to manage a user in that container. While I seem to be able to have the module imported I’m only able see any effect by using the home-manager.useUserPackages option. In this case I will have programs listed in home.packages available on the user path, but there are no files (symlinks) created in my home directory. So at least I know it’s evaluating the home-manager config, and that would suggest it’s not the problem.

There is no .nix-profile directory created for the user which might be a clue, but I’m not quite able to figure out what that means. I wonder if there is some other part of the filesystem that needs to be bound to the container or something along those lines.

This is the config I’m working with (for context I’m using niv for the home-manager dependency):

{ config, lib, pkgs, ... }:

let
  sources = import ./nix/sources.nix;
  home-manager = builtins.fetchTarball {
    inherit (sources.home-manager) url sha256;
  };
in
{
  containers.mycontainer = {
    config = { config, lib, pkgs, ... }:
      {
        imports = [
          (import "${home-manager}/nixos")
        ];

        users = {
          mutableUsers = false;
          users.wkral = {
            isNormalUser = true;
          };
        };

        home-manager.useUserPackages = true;
        home-manager.users.wkral = import /home/wkral/home-nix/container.nix;
      };
  };
}

Any suggestions would be appreciated.

1 Like
2

Update, this doesn’t appear to just be a declaritive container issue, as I’m able to reproduce it with nixos-container using the imperitive style.

3

Hi there @wkral,

I have had a similar issue. Here is your example config modified to be self-contained:

let
  home-manager = builtins.fetchTarball "https://github.com/nix-community/home-manager/archive/90dd375eba16e11949d4a4a7538554dc8e9ceb8f.tar.gz";
  user = "wkral";
  container = "mycontainer";
in {
  containers.${container}.config = { config, lib, pkgs, ... }: {
    imports = [ (import "${home-manager}/nixos") ];

    # Alternatively, you can import home-manager at the nixos level
    # and share packages with the container:
    # nixpkgs.pkgs = pkgs;
    # home-manager.useGlobalPkgs = true;

    home-manager.useUserPackages = true;

    users = {
      mutableUsers = false;
      users.root.password = user;
      users.${user} = {
        isNormalUser = true;
        password = user;
        description = "User inside container";
      };
    };
    home-manager.users.${user} = {
      programs.home-manager.enable = true;
      home.file."home-manager-installed".text = "yes\n";
      home.packages = [ pkgs.hello ];
    };
  };
}

When I logged in to the container and checked systemctl status, I noticed that home-manager-wkral.service had failed. The error message was:

[root@mycontainer:~]# journalctl -u home-manager-wkral | grep error
May 06 10:07:31 mycontainer hm-activate-wkral[260]: error: opening lock file '/nix/var/nix/profiles/per-user/wkral/home-manager.lock': No such file or directory

The issue is that the home-manager activation script needs to put the user’s profile in /nix/var/nix/profiles/per-user. This is the always true regardless of the home-manager.useUserPackages setting.

But /nix is bind-mounted into the container read-only. But usually you would want read-only /nix for containers.

So we need to make some bind mounts for the container user’s profile. Like this:

let
  home-manager = builtins.fetchTarball "https://github.com/nix-community/home-manager/archive/90dd375eba16e11949d4a4a7538554dc8e9ceb8f.tar.gz";
  user = "wkral";
  container = "mycontainer";
in {
  systemd.services."${container}-setup" = {
    wantedBy = [ "container@${container}.service" ];
    script = ''
      mkdir -p /var/lib/${container}/per-user-{profile,gcroots}
    '';
    serviceConfig.Type = "oneshot";
  };

  containers.${container} = {
    bindMounts = {
      per-user-profile = {
        hostPath = "/var/lib/${container}/per-user-profile";
        mountPoint = "/nix/var/nix/profiles/per-user/${user}";
        isReadOnly = false;
      };
      per-user-gcroots = {
        hostPath = "/var/lib/${container}/per-user-gcroots";
        mountPoint = "/nix/var/nix/gcroots/per-user/${user}";
        isReadOnly = false;
      };
      "/nix/var/nix/profiles/per-user/root".isReadOnly = true;
    };

    config = { config, lib, pkgs, ... }: {
      imports = [ (import "${home-manager}/nixos") ];
      home-manager.useUserPackages = true;

      # Fix permissions on profile/gcroots directories before
      # home-manager activation.
      systemd.services."home-manager-${user}" = {
        preStart = ''
          chown -R ${user} /nix/var/nix/{profiles,gcroots}/per-user/${user}
        '';
        serviceConfig.PermissionsStartOnly = true;
      };

      users = {
        mutableUsers = false;
        users.root.password = user;
        users.${user} = {
          isNormalUser = true;
          password = user;
          description = "User inside container";
        };
      };
      home-manager.users.${user} = {
        programs.home-manager.enable = true;
        home.file."home-manager-installed".text = "yes\n";
        home.packages = [ pkgs.hello ];
      };
    };
  };
}

I hope this helps. Perhaps it’s a bug with home-manager or nixos-containers, but I’m not sure.

Rodney

2 Likes
4

Many thanks Rodney, this really helps.

I hadn’t thought to look at the service activation for the error, it just seemed to fail silently from my perspective. I didn’t even think it had created that service to be honest.

Is the root user profile required I wonder, if I’m not using that account and simply add myself to the wheel group? Looking at my host system it only seems to be managing channels in the root user profile. Perhaps in that situation I could just bindMount the per-user/{profiles,gcroots} folders and not create one for each individual user.

I’ll play around with it and see what I can do. I had resorted to using a VM to deal with my immediate needs, which comes with it’s own challenges.

Thanks again.

5

So after some playing around and figuring things out, I found that the critical piece of your solution is the creating of the directory for my user and changing ownership, but I was able to do that in the container config itself:

  containers.mycontainer = {
    config = { config, lib, pkgs, ... }: {
      systemd.tmpfiles.rules = [
        "d /nix/var/nix/{profiles,gcroots}/per-user/${user} - ${user} - - -"
      ];
      # ... rest of config
    };
  };

The directory tree was created, but not the user specific directories, I’m curious why not, and I think you’re right it might be a bug in one or the other of nixos-container or home-manager.

I wasn’t familiar with PermissionsStartOnly systemd directive and when it wasn’t listed in the man pages I went looking and found that it has been deprecated. That has a link to this PR which updated a bunch of NixOS modules. So I followed that example for using tmpfiles.d.

Thanks again, I had given up on the idea at least temporarily until you clued me in on the cause of the problem.

6

Thanks for the systemd.tmpfiles tip - that makes things simpler.

My own container also has user namespaces enabled, so the container root user can’t modify /nix at all, unless I bind mount writable directories into the container.

Is the root user profile required I wonder?

It’s not, but I was getting a warning about the directory not existing, so just added it in.

7

Sorry for reviving the old topic, but since the subject is still relevant, I might as well.

@rvl l how do you enable user namespaces?

@wkral it looks like you can use home-manager with just one user now, is that correct?

8

I’m afraid I haven’t used a container like this for a while, ultimately I ended up using a QEMU VM to solve my problem, it’s not ideal but it gets the job done.

I did make progress and got a user working with home manager and my home configuration thanks to @rvl. Ultimately, I was trying to be able to launch gui apps from the container and share my host’s wayland session but I was never able to get that to work. It might still be possible, but it certainly felt like nixos-container was not really designed for this kind of use-case, more for running services in isolated environmetns rather than having a full user environment. There were a lot of workarounds and it felt like I was fighting the tool. However, things may have expanded/improved in this area since I last tried.

9

Interesting, thanks. I don’t need to run gui apps, but i was trying to install home-manager with nix channels, and that didn’t work. I’m now trying a lxc guest. That might be better for this specific usecase.