Nginx won't start

Help
1

I’m trying to set up a simple personal web server using nginx. It won’t start, and I can’t even find a hint of what is wrong.

Here’s my NixOS config:

  services.nginx = {
    enable = true;
    virtualHosts."localhost" = {
      root = "/home/amy/nginx-test";
    };
  };

When I try to start it:

✦ ❯ systemctl restart nginx.service
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ====
Authentication is required to restart 'nginx.service'.
Authenticating as: Amy de Buitleir (amy)
Password: 
==== AUTHENTICATION COMPLETE ====
Job for nginx.service failed because the control process exited with error code.
See "systemctl status nginx.service" and "journalctl -xeu nginx.service" for details.

Trying the first suggestion to get more info:

✦ ❯ systemctl status nginx.service
● nginx.service - Nginx Web Server
     Loaded: loaded (/etc/systemd/system/nginx.service; enabled; preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Sat 2022-07-30 21:27:16 IST; 5s ago
    Process: 18229 ExecStartPre=/nix/store/ffnw8igsv4b12ffsfd8fqx51ivvyzbks-unit-script-nginx-pre-start/bin/nginx-pre-start (code=exited, st>
         IP: 0B in, 0B out
        CPU: 26ms

Jul 30 21:27:16 wombat11k systemd[1]: nginx.service: Control process exited, code=exited, status=135/n/a
Jul 30 21:27:16 wombat11k systemd[1]: nginx.service: Failed with result 'exit-code'.
Jul 30 21:27:16 wombat11k systemd[1]: Failed to start Nginx Web Server.

I don’t think that told me anything useful, so trying the second suggestion to get more info:

Jul 30 21:27:15 wombat11k systemd[1]: Starting Nginx Web Server...
░░ Subject: A start job for unit nginx.service has begun execution
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit nginx.service has begun execution.
░░ 
░░ The job identifier is 12992.
Jul 30 21:27:16 wombat11k nginx-pre-start[18229]: /nix/store/ffnw8igsv4b12ffsfd8fqx51ivvyzbks-unit-script-nginx-pre-start/bin/nginx-pre-star>
Jul 30 21:27:16 wombat11k systemd[1]: nginx.service: Control process exited, code=exited, status=135/n/a
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ An ExecStartPre= process belonging to unit nginx.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 135.
Jul 30 21:27:16 wombat11k systemd[1]: nginx.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ The unit nginx.service has entered the 'failed' state with result 'exit-code'.
Jul 30 21:27:16 wombat11k systemd[1]: Failed to start Nginx Web Server.
░░ Subject: A start job for unit nginx.service has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit nginx.service has finished with a failure.
░░ 
░░ The job identifier is 12992 and the job result is failed.
Jul 30 21:27:26 wombat11k systemd[1]: nginx.service: Scheduled restart job, restart counter is at 1.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ Automatic restarting of the unit nginx.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
Jul 30 21:27:26 wombat11k systemd[1]: Stopped Nginx Web Server.
░░ Subject: A stop job for unit nginx.service has finished
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A stop job for unit nginx.service has finished.
░░ 
░░ The job identifier is 13072 and the job result is done.
Jul 30 21:27:26 wombat11k systemd[1]: Starting Nginx Web Server...
░░ Subject: A start job for unit nginx.service has begun execution
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit nginx.service has begun execution.
░░ 
░░ The job identifier is 13072.
Jul 30 21:27:26 wombat11k nginx-pre-start[18324]: /nix/store/ffnw8igsv4b12ffsfd8fqx51ivvyzbks-unit-script-nginx-pre-start/bin/nginx-pre-star>
Jul 30 21:27:26 wombat11k systemd[1]: nginx.service: Control process exited, code=exited, status=135/n/a
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ An ExecStartPre= process belonging to unit nginx.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 135.
Jul 30 21:27:26 wombat11k systemd[1]: nginx.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ The unit nginx.service has entered the 'failed' state with result 'exit-code'.
Jul 30 21:27:26 wombat11k systemd[1]: Failed to start Nginx Web Server.
░░ Subject: A start job for unit nginx.service has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit nginx.service has finished with a failure.
░░ 
░░ The job identifier is 13072 and the job result is failed.
Jul 30 21:27:36 wombat11k systemd[1]: nginx.service: Scheduled restart job, restart counter is at 2.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ Automatic restarting of the unit nginx.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
Jul 30 21:27:36 wombat11k systemd[1]: Stopped Nginx Web Server.
░░ Subject: A stop job for unit nginx.service has finished
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A stop job for unit nginx.service has finished.
░░ 
░░ The job identifier is 13152 and the job result is done.
Jul 30 21:27:36 wombat11k systemd[1]: Starting Nginx Web Server...
░░ Subject: A start job for unit nginx.service has begun execution
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit nginx.service has begun execution.
░░ 
░░ The job identifier is 13152.
Jul 30 21:27:36 wombat11k nginx-pre-start[18339]: /nix/store/ffnw8igsv4b12ffsfd8fqx51ivvyzbks-unit-script-nginx-pre-start/bin/nginx-pre-star>
Jul 30 21:27:36 wombat11k systemd[1]: nginx.service: Control process exited, code=exited, status=135/n/a
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ An ExecStartPre= process belonging to unit nginx.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 135.
Jul 30 21:27:36 wombat11k systemd[1]: nginx.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ The unit nginx.service has entered the 'failed' state with result 'exit-code'.
Jul 30 21:27:36 wombat11k systemd[1]: Failed to start Nginx Web Server.
░░ Subject: A start job for unit nginx.service has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit nginx.service has finished with a failure.
░░ 
░░ The job identifier is 13152 and the job result is failed.
Jul 30 21:27:46 wombat11k systemd[1]: nginx.service: Scheduled restart job, restart counter is at 3.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ Automatic restarting of the unit nginx.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
Jul 30 21:27:46 wombat11k systemd[1]: Stopped Nginx Web Server.
░░ Subject: A stop job for unit nginx.service has finished
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A stop job for unit nginx.service has finished.
░░ 
░░ The job identifier is 13232 and the job result is done.
Jul 30 21:27:46 wombat11k systemd[1]: Starting Nginx Web Server...
░░ Subject: A start job for unit nginx.service has begun execution
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit nginx.service has begun execution.
░░ 
░░ The job identifier is 13232.
Jul 30 21:27:46 wombat11k nginx-pre-start[18353]: /nix/store/ffnw8igsv4b12ffsfd8fqx51ivvyzbks-unit-script-nginx-pre-start/bin/nginx-pre-star>
Jul 30 21:27:46 wombat11k systemd[1]: nginx.service: Control process exited, code=exited, status=135/n/a
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ An ExecStartPre= process belonging to unit nginx.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 135.
Jul 30 21:27:46 wombat11k systemd[1]: nginx.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ The unit nginx.service has entered the 'failed' state with result 'exit-code'.
Jul 30 21:27:46 wombat11k systemd[1]: Failed to start Nginx Web Server.
░░ Subject: A start job for unit nginx.service has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit nginx.service has finished with a failure.
░░ 
░░ The job identifier is 13232 and the job result is failed.
Jul 30 21:27:56 wombat11k systemd[1]: nginx.service: Scheduled restart job, restart counter is at 4.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ Automatic restarting of the unit nginx.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
Jul 30 21:27:56 wombat11k systemd[1]: Stopped Nginx Web Server.
░░ Subject: A stop job for unit nginx.service has finished
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A stop job for unit nginx.service has finished.
░░ 
░░ The job identifier is 13312 and the job result is done.
Jul 30 21:27:56 wombat11k systemd[1]: Starting Nginx Web Server...
░░ Subject: A start job for unit nginx.service has begun execution
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit nginx.service has begun execution.
░░ 
░░ The job identifier is 13312.
Jul 30 21:27:57 wombat11k nginx-pre-start[18364]: /nix/store/ffnw8igsv4b12ffsfd8fqx51ivvyzbks-unit-script-nginx-pre-start/bin/nginx-pre-star>
Jul 30 21:27:57 wombat11k systemd[1]: nginx.service: Control process exited, code=exited, status=135/n/a
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ An ExecStartPre= process belonging to unit nginx.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 135.
Jul 30 21:27:57 wombat11k systemd[1]: nginx.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ The unit nginx.service has entered the 'failed' state with result 'exit-code'.
Jul 30 21:27:57 wombat11k systemd[1]: Failed to start Nginx Web Server.
░░ Subject: A start job for unit nginx.service has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit nginx.service has finished with a failure.
░░ 
░░ The job identifier is 13312 and the job result is failed.
2

You could try

  systemd.services.nginx.serviceConfig.ReadWritePaths = [ "/var/spool/nginx/logs/" ];
3

I’m making some progress thanks to @Mapybara. After adding this to my config:

  systemd.services.nginx.serviceConfig.ReadWritePaths = [ "/var/spool/nginx/logs/" ];

I got this result

Jul 31 16:08:54 wombat11k systemd[89236]: nginx.service: Failed to set up mount namespacing: /run/systemd/unit-root/var/spool/nginx/logs: No such file or directory
Jul 31 16:08:54 wombat11k systemd[89236]: nginx.service: Failed at step NAMESPACE spawning /nix/store/ffnw8igsv4b12ffsfd8fqx51ivvyzbks-unit-script-nginx-pre-start/bin/nginx-pre-start: No such file or directory
4

That is not ideal!

I’m afraid we’ve reached the limit of my knowledge, but other readers are going to want to know whether you’re using NixOS stable or unstable (or a flake, or what)

5

I’m on NixOS unstable.

I was able to get lighttpd up and running, so I’m good with that for now. But if this might be a bug, I should report it.

6

nginx by default runs with ProtectHome = true, which means /home is empty. Pick a path outside /home for the root.

2 Likes
7

The systemd sandboxing also explains this error - you configured systemd to set up a mount to /var/spool/nginx/logs in the nginx container. I.e., it’s basically running mount --bind /var/spool/log/nginx/logs /run/systemd/unit-root/var/spool/nginx/logs.

/var/spool/log/nginx/logs doesn’t exist on your system though, so it fails.

Annoyingly hard to understand error message, but you really shouldn’t be playing around with this path anyway - nginx doesn’t log there by default (though some people configure it to, and those people probably see nginx failing because it won’t be allowed to write there due to ProtectSystem = "strict")

The unit you’re using is actually already configured to allow logging to /var/log through the LogsDirectory setting, which you should configure nginx to use instead (and I think it does this by default, there are just some bits in the nginx docs that suggest changing it to spool for some reason that I can only assume is related to commercial support for someone running a legacy system).

I wonder if we can do something about nginx crashing with no error in this scenario. It’s not easy to understand that nginx is sandboxed from a crash like this.

2 Likes
8

In the original post I don’t see any mention of a custom log directory. Was the entire config not pasted, or did I miss something?

9

@aanderse no, I think there was just some wire crossing that resulted in attempting

, which caused those errors to pop up, and I wanted to explain what they meant and (probably) what the origin is. @peterhoeg posted the real solution :slight_smile:

1 Like