Peertube in a Container: Configuration Issues

Hello everyone,

TL;DR: My Peertube container configuration doesn’t work and I don’t know why.

I am trying to run Peertube in a NixOS container with the following container configuration which is located in /etc/nixos/container.nix.

  containers.peertube = {
    autoStart = true;
    privateNetwork = true;
    hostAddress = "192.168.100.17";
    localAddress = "192.168.100.18";
    config = {config, pkgs, ...}: {
      system.stateVersion = "24.11";
      
      environment.etc = {
        "peertube/password-posgressql-db".text = "example_password";
        "peertube/password-redis-db".text = "example_password";
        "peertube/secrets".text = "example_password";
      };

      services = {
        peertube = {
          package = pkgs.peertube;
          listenWeb = 9000;
          enable = true;
          settings = {
              listen = {
              hostname = "0.0.0.0";
            };
            log = {
              level = "debug";
            };
            storage = {
              tmp = "/opt/data/peertube/storage/tmp";
              logs = "/opt/data/peertube/storage/logs/";
              cache = "/opt/data/peertube/storage/cache/";
            };
          };
          group = "peertube";
          enableWebHttps = false;
          user = "peertube";
          # redis.createLocally = false;
          redis.createLocally = true;
          localDomain = "127.0.0.1";
          configureNginx = false;
          smtp.createLocally = false;
          listenHttp = 9000;
          # database.createLocally = false;
          database.createLocally = true;
          secrets.secretsFile = "/etc/peertube/secrets";
          smtp.passwordFile = null;
          serviceEnvironmentFile = null;
          redis.port = null;
          redis.passwordFile = "/etc/peertube/password-redis-db";
          redis.host = null;
          redis.enableUnixSocket = true;
          database.user = "peertube";
          database.port = 5432;
          database.passwordFile = "/etc/peertube/password-postgresql";
          database.name = "peertube";
          database.host = "/run/postgresql";
          dataDirs = [ "/var/www/peertube" ];
        };
      };
    };
  };

A container is created, but the Peertube service fails to start. The following error is given when running the executable manually.

sudo nixos-container root-login peertube
[root@peertube:~]# /nix/store/6nyjcd5nrpp3g0csak0vw53nh4v66b1v-unit-script-peertube-start/bin/peertube-start 
cat: /run/secrets/peertube: No such file or directory
cat: /run/keys/peertube/password-redis-db: No such file or directory
/nix/store/6nyjcd5nrpp3g0csak0vw53nh4v66b1v-unit-script-peertube-start/bin/peertube-start: line 20: exec: node: not found

I am confused by this error because the files given are defaults or examples to options which were overwritten in the container configuration, specifically the redis.passwordFile and secrets.secretsFile from the NixOS options documentation. These could be remnants from a previous configuration. I have tried to recreate the container with nixos-rebuild switch and nixos-container destroy, but these commands were unsuccessful. The nixos-container destroy command gives the following output instructing me to remove it from the configuration instead.

nixos-container destroy peertube
/run/current-system/sw/bin/nixos-container: cannot destroy declarative container (remove it from your configuration.nix instead)

I am assuming that rebuilding the system configuration file does not destroy and recreate the entire container because databases would need to be preserved, but it seems like rebuilding the container in this case does not even transform trivial files to match a new state.

What is the correct way to rebuild a container after changing the configuration? Or if my assumption about the problem is incorrect, what could I do to further investigate the issue?

If I create a separate container, peertubeA, the errors about the password files being located in the incorrect locations disappear, but I still get a similar error:

/nix/store/ikaa3jhx93763aq9p7blmm28mlz0yc4x-unit-script-peertube-start/bin/peertube-start
/nix/store/ikaa3jhx93763aq9p7blmm28mlz0yc4x-unit-script-peertube-start/bin/peertube-start: line 20: exec: node: not found

And systemctl posts the same status for both scenarios:

...(code=exited, status=226/NAMESPACE)

The error message implies that node is missing, but installing node (fetched from the nix source, I don’t know why it wasn’t installed to begin with) still doesn’t fully resolve the issue. Adding the following gives a different error.

      environment.systemPackages = with pkgs; [
        nodejs_18
        yarn
        ffmpeg-headless
        openssl
      ];
/nix/store/2ymkvkvr2sk4xl705gyad13x33r1lfvb-unit-script-peertube-start/bin/peertube-start 
node:internal/modules/cjs/loader:1143
  throw err;
  ^

Error: Cannot find module '/root/dist/server'
    at Module._resolveFilename (node:internal/modules/cjs/loader:1140:15)
    at Module._load (node:internal/modules/cjs/loader:981:27)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:128:12)
    at node:internal/main/run_main_module:28:49 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

Node.js v18.20.6

At this point I am kind of stuck. I have looked on the wiki, forums, and other places for additional information but have had no luck, and I am very new to Nix so I am not sure where to start. Any pointers would be greatly appreciated.

Thank you for your help.

Why are you calling the unit script directly instead of using systemctl?

I didn’t get a good error message with systemctl so I called the unit script to see if I could get more details. There’s probably a journal with systemd that does this better, but I am not familiar with it.

If you call the script directly it will be missing env variables and other settings from the .service file.
You should be able to read the logs with journalctl -u peertube. I’m not really familiar with peertube so I can’t give you specific advice.

I’d also caution you about the NixOS containers a bit. They work, I doubt they are the reason for this problem, but I found them rather tedious to use.

2 Likes

Thank you for your reply.

I took your advice and found a more helpful error with journalctl. I was able to fix it and the Peertube service is now running. Now I am trying to access the service from outside the container for routing it through a reverse proxy. Using the address at http://192.168.100.17:9000 or 18:9000 in the web browser doesn’t return a webpage, even after I tried forwarding port 9000. Do you know of any resources which would help with this? I feel like one of hostAddress or localAddress would make the container accessible from the host machine, but I am having trouble appreciating the difference between them - it seems like they are both assigned to the interface for the container, just depending on an internal or external perspective.

Why do you caution about NixOS containers being tedious? Is there a better way to host this, in your opinion?

Details on how the previous problem was solved are below.

I have updated the configuration to the following.

containers.peertubeE = {
    autoStart = true;
    privateNetwork = true;
    hostAddress = "192.168.100.17";
    localAddress = "192.168.100.18";
    config = {config, pkgs, ...}: {
      system.stateVersion = "24.11";

      networking.extraHosts = ''
        127.0.0.1 peertube.local
      '';

      environment.etc = {
        "peertube/password-posgressql-db".text = "example_password";
        "peertube/password-redis-db".text = "example_password";
        "peertube/secrets".text = "example_password";
      };

      networking = {
        firewall = {
          enable=true;
          allowedTCPPorts=[9000];
        };
      };

      services = {
        peertube = {
          package = pkgs.peertube;
          listenWeb = 9000;
          enable = true;
          settings = {
            listen = {
              hostname = "0.0.0.0";
            };
            log = {
              level = "debug";
            };
            storage = {
              tmp = "/opt/data/peertube/storage/tmp";
              logs = "/opt/data/peertube/storage/logs/";
              cache = "/opt/data/peertube/storage/cache/";
            };
          };
          group = "peertube";
          enableWebHttps = false;
          user = "peertube";
          # redis.createLocally = false;
          redis.createLocally = true;
          localDomain = "peertube.local";
          configureNginx = false;
          smtp.createLocally = false;
          listenHttp = 9000;
          # database.createLocally = false;
          database.createLocally = true;
          secrets.secretsFile = "/etc/peertube/secrets";
          smtp.passwordFile = null;
          serviceEnvironmentFile = null;
          redis.port = null;
          redis.passwordFile = "/etc/peertube/password-redis-db";
          redis.host = null;
          redis.enableUnixSocket = true;
          database.user = "peertube";
          database.port = 5432;
          database.passwordFile = "/etc/peertube/password-postgresql";
          database.name = "peertube";
          database.host = "/run/postgresql";
          # dataDirs = [ "/var/www/peertube" ];
        };
      };
    };
  };

I receive the following error from journalctl -u peertube (repeated 10 times of course).

Mar 05 18:54:57 peertubeE systemd[1]: peertube.service: Main process exited, code=exited, status=1/FAILURE
Mar 05 18:54:57 peertubeE systemd[1]: peertube.service: Failed with result 'exit-code'.
Mar 05 18:54:57 peertubeE systemd[1]: peertube.service: Consumed 1.914s CPU time, 96.3M memory peak.
Mar 05 18:55:17 peertubeE systemd[1]: peertube.service: Scheduled restart job, restart counter is at 10.
Mar 05 18:55:17 peertubeE systemd[1]: Started PeerTube daemon.
Mar 05 18:55:18 peertubeE peertube[1152]: node:fs:1391
Mar 05 18:55:18 peertubeE peertube[1152]:   handleErrorFromBinding(ctx);
Mar 05 18:55:18 peertubeE peertube[1152]:   ^
Mar 05 18:55:18 peertubeE peertube[1152]: Error: ENOENT: no such file or directory, mkdir '/opt/data/peertube/storage/logs'
Mar 05 18:55:18 peertubeE peertube[1152]:     at Object.mkdirSync (node:fs:1391:3)
Mar 05 18:55:18 peertubeE peertube[1152]:     at File._createLogDirIfNotExist (/nix/store/wj06akqw413ykvkikx022m0bnn0x0h01-peertube-6.3.3/node_modules/wins>
Mar 05 18:55:18 peertubeE peertube[1152]:     at new File (/nix/store/wj06akqw413ykvkikx022m0bnn0x0h01-peertube-6.3.3/node_modules/winston/lib/winston/tran>
Mar 05 18:55:18 peertubeE peertube[1152]:     at buildLogger (file:///nix/store/wj06akqw413ykvkikx022m0bnn0x0h01-peertube-6.3.3/dist/core/helpers/logger.js>
Mar 05 18:55:18 peertubeE peertube[1152]:     at file:///nix/store/wj06akqw413ykvkikx022m0bnn0x0h01-peertube-6.3.3/dist/core/helpers/logger.js:69:16
Mar 05 18:55:18 peertubeE peertube[1152]:     at ModuleJob.run (node:internal/modules/esm/module_job:195:25)
Mar 05 18:55:18 peertubeE peertube[1152]:     at async ModuleLoader.import (node:internal/modules/esm/loader:337:24)
Mar 05 18:55:18 peertubeE peertube[1152]:     at async loadESM (node:internal/process/esm_loader:34:7)
Mar 05 18:55:18 peertubeE peertube[1152]:     at async handleMainPromise (node:internal/modules/run_main:106:12) {
Mar 05 18:55:18 peertubeE peertube[1152]:   errno: -2,
Mar 05 18:55:18 peertubeE peertube[1152]:   syscall: 'mkdir',
Mar 05 18:55:18 peertubeE peertube[1152]:   code: 'ENOENT',
Mar 05 18:55:18 peertubeE peertube[1152]:   path: '/opt/data/peertube/storage/logs'
Mar 05 18:55:18 peertubeE peertube[1152]: }
Mar 05 18:55:18 peertubeE peertube[1152]: Node.js v18.20.6
Mar 05 18:55:18 peertubeE systemd[1]: peertube.service: Main process exited, code=exited, status=1/FAILURE
Mar 05 18:55:18 peertubeE systemd[1]: peertube.service: Failed with result 'exit-code'.
Mar 05 18:55:18 peertubeE systemd[1]: peertube.service: Consumed 1.895s CPU time, 96.3M memory peak.

systemctl status peertube gives the following error status: (code=exited, status=1/FAILURE).

Removing the containers.peertubeE.config.services.peertube.settings block resolved the issue and the Peertube service is running successfully now. However, I am not able to access it from outside the container, even after forwarding port 9000. Does anybody know why that might be?

Does http://192.168.100.18:9000 work (from the host)?

Why do you caution about NixOS containers being tedious?

I mean you are basically already experiencing all of it. If it’s acceptable to you than you are good to go.

I’ve switched some of my services to run directly on the host rather than inside the container.

That is true. I still think that Nix containers look like the best solution here though. I feel like it is almost working but and I am just missing something simple, but if not I might change my mind about them.

I tried that address as well as peertube.local:9000 from the host and neither of them were able to bring up the peertube instance, but adding networking.extraHosts = 192.168.100.18 peertube.local outside the container configuration instead of inside the container configuration resolved the issue, which was somewhat obvious looking back, but certainly not going into it. Alternatively, leaving out the extra hosts option and using the localAddress as the localDomain also made the site accessible.

Thanks for helping me work through it. The working configuration is as follows:

{ config, pkgs, ...}
{
  # ...
  # ...Other configuration...
  # ...

  networking.extraHosts = "192.168.100.18 peertube.local";

  containers.peertube = {
    autoStart = true;
    privateNetwork = true;
    hostAddress = "192.168.100.17";
    localAddress = "192.168.100.18";
    
    config = {config, pkgs, ...}: {
      system.stateVersion = "24.11";

      environment.etc = {
        "peertube/password-posgressql-db".text = "example_password";
        "peertube/password-redis-db".text = "example_password";
        "peertube/secrets".text = "example_password";
      };

      networking = {
        firewall = {
          enable=true;
          allowedTCPPorts=[9000];
        };
      };

      services = {
        peertube = {
          package = pkgs.peertube;
          listenWeb = 9000;
          enable = true;
          settings = {
            listen = {
              hostname = "0.0.0.0";
            };
            # log = {
              # level = "debug";
            # };
            # storage = {
              # tmp = "/opt/data/peertube/storage/tmp";
              # logs = "/opt/data/peertube/storage/logs/";
              # cache = "/opt/data/peertube/storage/cache/";
            # };
          };
          group = "peertube";
          enableWebHttps = false;
          user = "peertube";
          # redis.createLocally = false;
          redis.createLocally = true;
          # localDomain = "192.168.100.18";
          localDomain = "peertube.local";
          configureNginx = false;
          smtp.createLocally = false;
          listenHttp = 9000;
          # database.createLocally = false;
          database.createLocally = true;
          secrets.secretsFile = "/etc/peertube/secrets";
          smtp.passwordFile = null;
          serviceEnvironmentFile = null;
          redis.port = null;
          redis.passwordFile = "/etc/peertube/password-redis-db";
          redis.host = null;
          redis.enableUnixSocket = true;
          database.user = "peertube";
          database.port = 5432;
          database.passwordFile = "/etc/peertube/password-postgresql";
          database.name = "peertube";
          database.host = "/run/postgresql";
          # dataDirs = [ "/var/www/peertube" ];
        };
      };
    };
  };
}

Both the localAddress and hostAddress options are required, but the container is accessed externally with the localAddress option. I have no idea what the hostAddress option is for. What does it mean to be assigned to the host interface? I

From the options search page:
localAddress:

The IPv4 address assigned to the interface in the container. If a hostBridge is used, this should be given with netmask to access the whole network. Otherwise the default netmask is /32 and routing is set up from localAddress to hostAddress and back.

hostAddress:

The IPv4 address assigned to the host interface. (Not used when hostBridge is set.)

Also, the peertube wiki page appears to be incorrect regarding this. I want to edit it. Should the existing configuration be removed and the configuration I wrote put in its place, or is there a reason the configuration present looks so different than the method used here? It doesn’t explain anything in the configuration, which tempts me to remove it entirely.

1 Like