Systemd permissions

mayl · November 22, 2020, 2:37pm

I’m trying to write my first NixOS service definition, and am hitting a wall. As a quick overview, I am trying to run the gns3server and have been referencing the upstream systemd config. I can run my server as my normal user, both from the shell and as a systemd unit. When I attempt to set the systemd user to a “dedicated” user for this service I get failures indicating the users’ home directory is not writable, but when I ls -la this same directory it appears the permissions are correct.

My service configuration is

{ config, lib, pkgs, ... }:

with lib; {
  options.gns3Server.services.gns3Server.enable = mkEnableOption "Activates GNS3 Server";

  config = mkIf config.gns3Server.services.gns3Server.enable {
    users.users.gns3 = {
      description = "gns3 server user";
      isSystemUser = true;
      group = "gns3";
      createHome = true;
      home = "/srv/gns3/gns3Server";
    };


    systemd.services.gns3Server = {
      wants = [ "network-online.target" ];
      after = [ "network-online.target" ];
      conflicts = [ "shutdown.target" ];
      wantedBy = [ "multi-user.target" ];

      serviceConfig = {
        User = "gns3";
        Group = "gns3";
        PermissionsStartOnly = "true";
        Restart = "on-failure";
        RestartSec = "30s";
        ExecReload = pkgs.writeShellScript "gns3_kill" ''
          kill -s HUP $MAINPID
        '';
        LimitNOFILE=16384;
      };
      script = let
        gns3Server = pkgs.gns3-server;
      in ''
        exec ${gns3Server}/bin/gns3server --config ${./gns3Config.ini}
      '';
    };
  };
}

journalctrl -u gns3Server.service | tail gives:

INFO run.py:221 Copyright (c) 2007-2020 GNS3 Technologies Inc.
INFO run.py:224 Config file /nix/store/83iwwyml1pl9pridm6c2x4r5424yix6k-gns3Config.ini loaded
INFO run.py:245 Running with Python 3.7.6 and has PID 5720
INFO run.py:79 Current locale is en_US.UTF-8
INFO web_server.py:294 Starting server on 0.0.0.0:3080
ERROR symbols.py:117 Could not create symbol directory '/srv/gns3/gns3Server/GNS3/symbols': [Errno 13] Permission denied: '/srv/gns3/gns
INFO __init__.py:62 Load controller configuration file /nix/store/gns3_controller.conf
INFO __init__.py:66 Controller is starting
CRITICAL web_server.py:87 Could not start the server: [Errno 13] Permission denied: '/srv/gns3/gns3Server'
Nov 22 14:31:06 larry-server systemd[1]: gns3Server.service: Succeeded.

sudo ls -la /srv/gns3 shows:

drwx------ 2 gns3 gns3 4096 Nov 22 14:34 gns3Server

Is there something basic I’m missing? if the gns3 user is the owner and has write permissions, why can’t it create directories in there?

Any help appreciated, I’ve been banging my head against this for hours and am pretty sure there’s something fundamental I’m missing…

aanderse · November 22, 2020, 2:51pm

Difficult to say exactly without seeing your configuration file. There are a number of paths you can set in the configuration file, so we would need to see what you’ve done there. If you haven’t set any paths then maybe default values are causing some troubles. Just an idea.

mayl · November 22, 2020, 3:04pm

Config file is pretty bare-bones and just a placeholder for now:

[server]
host = 0.0.0.0
port =  5555

I don’t think it’s related, because when I change the user in the systemd config to “my” user it works (so same config), and the logs tell me exactly which path has a permissions issue which I check with ls -la

freezeboy · November 22, 2020, 4:43pm

Not sure it helps but I see a strange line

INFO __init__.py:62 Load controller configuration file /nix/store/gns3_controller.conf

Why is this file not having a hash prefix?

Also, I don’t see the complete line above, but isn’t it expecting the folder to exits?

/srv/gns3/gns3Server/GNS3/

nixinator · November 22, 2020, 4:52pm

seems to be deprecated

systemd's PermissionsStartOnly is deprecated · Issue #53852 · NixOS/nixpkgs · GitHub

so, at first look these seems to be related to permissions in someway… might be a total red herring.

Gns3 has other problems when running a nixshell ,
https://github.com/NixOS/nixpkgs/issues/102536
but it is unrelated to this.

Nice that your trying to write a service for this… It certainly worth doing.

NobbZ · November 22, 2020, 5:20pm

What are permissions and ownership of /srv and /srv/gns3?

mayl · November 23, 2020, 12:43am

These are all good things to run down.

@nixinator:
Thanks for letting me know PermissionsStartOnly is deprecated. Since I’m no longer running an ExecPreStart phase, it should be redundant at this point and I should remove it. The discussion you linked turned me on to systemd.tmpfiles.rules which seems like a good thread for me to pull on as well.

@freezeboy:
The file in the nix/store that doesn’t have a hash is also an good observation, I don’t know what that config file is but it’s separate from the one I’m passing in. I wonder if I need to set a path in the config I provide gns3server to control that path. Thanks for pointing it out.

@nobbz
I’ll check the permissions of /srv/ and /srv/gns3 when I have access to the machine next. I let nixos users.users.createHome create those directories, so they should be whatever the defaults are in nixos.

I really appreciate everyone’s input, I’ll reply with where these ideas lead me as soon as I can!

mayl · November 23, 2020, 10:09am

@NobbZ

This is the result of ls -la /srv

total 12
drwxr-xr-x  3 root root 4096 Nov 22 14:34 .
drwxr-xr-x 17 root root 4096 Nov 22 14:34 ..
drwx------  3 root root 4096 Nov 22 14:34 gns3

@nixinator:
Removing the PermissionsStartOnly makes no difference (as expected).

@freezeboy:
I’m still looking for more details on that configuration file, so far haven’t been able to dig up much. That file does not exist in the nix store obvously, and upon closer inspection of the logs when I run as “my” user, this does create an error - but the service seems to start up anyway. When I try to start the server from my shell, it looks for this controller config in ~/.confg/gns3 which is not a path I set up explicitly.

I also came across this issue, which seems to imply that GNS3 infers where to find this gns3_controller.conf based on the path of the configuration file passed to it, which may explain why it looks in the nix store.

NobbZ · November 23, 2020, 10:10am

No one except for root is allowed to look into this folder.

Setting the folder to o+rx should do the trick.

mayl · November 23, 2020, 10:15am

Woo! That does seem to get me further thanks. Do you know how to get nixos to generate these folders with these permissions applied when the user is created?

NobbZ · November 23, 2020, 10:25am

Nope, I am not aware… Though you might be able to do something on your own using system.activationScripts.

aanderse · November 23, 2020, 11:42am

You are looking for systemd.tmpfiles.rules. Specifically you likely want systemd.tmpfiles.rules = [ "d /srv/gns3 - gns3 gns3" "d /srv/gns3/gns3Server - gns3 gns3" ];

From here you can remove the createHome directive for the user and the PermissionsStartOnly as neither are needed with the new tmpfiles declarations.

mayl · November 24, 2020, 12:36pm

Thanks for the pointer, I didn’t have time to try this today but will soon!

mayl · November 29, 2020, 7:52am

So I’ve got this “working” to some extent. @aanderse’s suggestion to use systemd.tmpfiles.rules was very helpful, once I did that I was able to overcome the permissions issue and have the required paths created for me by a nixos rebuild.

With that resolved, the second problem I hit was the strange config file with no hash in the nix store which @freezeboy noted. Based on this I think that gns3server infers where the controller config should be based on the path of the server config. Since I was writing my server config to the nix store and passing it in, gns3server tried to find the controller config there. The answer again was to use systemd.tmpfiles.rules and copy the config file into the desired /srv/gns3/gns3Server/ path. ~~Symlinking might work just as well, but I didn’t try that.~~ Tried it and it works fine.

With that all done, I can get the server to start and can successfully connect to it from a remote gns3 client. However, I haven’t been able to really test it. Anytime I try to import an appliance in the gui, the client crashes. @nixinator it seems you may have some experience with gns3 on Nixos, are you aware of this issue or do you know a workaround?

My current (hacky) config is below. Once I can get a client working to test it, I think it would be fairly straightforward for me to cleanup/modularize it based on other services in nixpkgs and put a pull request together for broader feedback.

{ config, lib, pkgs, ... }:

let
  gns3_group = "gns3";
  gns3_ip = "0.0.0.0";
  gns3_port = 5002;
  gns3_config_str = (import ./gns3Config.nix{inherit gns3_port gns3_ip;});
  gns3_config = builtins.toFile "gns3_config.ini" gns3_config_str;
in
with lib; {
  options.gns3Server.services.gns3Server.enable = mkEnableOption "Activates GNS3 Server";

  config = mkIf config.gns3Server.services.gns3Server.enable {
    users.users.gns3 = {
      description = "gns3 server user";
      isSystemUser = true;
      group = gns3_group;
      home = "/srv/gns3/gns3Server";
    };

    networking.firewall.allowedTCPPorts = [gns3_port];

    systemd.tmpfiles.rules = [ 
	"d /srv/gns3 - gns3 ${gns3_group}"
	"d /srv/gns3/gns3Server - gns3 ${gns3_group}"
	"L /srv/gns3/gns3Server/gns3_server.conf - gns3 ${gns3_group} - ${gns3_config}" ];

    systemd.services.gns3Server = {
      wants = [ "network-online.target" ];
      after = [ "network-online.target" ];
      conflicts = [ "shutdown.target" ];
      wantedBy = [ "multi-user.target" ];

      serviceConfig = {
        User = "gns3";
        Group = gns3_group;
        Restart = "on-failure";
        RestartSec = "30s";
        ExecReload = pkgs.writeShellScript "gns3_kill" ''
          kill -s HUP $MAINPID
        '';
        LimitNOFILE=16384;
      };
      
      script = let 
        gns3Server = pkgs.gns3-server;
        home_path = config.users.users.gns3.home;
      in ''
        ${gns3Server}/bin/gns3server --config "/srv/gns3/gns3Server/gns3_server.conf"
      '';
    };
  };
}

and gns3Config.nix is:

{ gns3_ip, gns3_port }:
''
[Server]
host = ${gns3_ip}
port = ${builtins.toString gns3_port}
auth = False
''

nixinator · November 30, 2020, 2:44pm

my. nix-shell version of this does run the client gui up , and doesn’t crash the client. My adventures are here GNS3 needs to setcap on ubridge · Issue #102536 · NixOS/nixpkgs · GitHub

mainly permission problems when running gns3 in a nix-shell. Gns3 seems like a ‘drive by commit’.

I appreciate you efforts trying to get it running as a module, send me your best working test module PR, and i’ll try it out here, and do some testing and debug for you.