Slowly going crazy trying to get nginx to not return a 403 error

Help
1

Ok I’ve read every single article or blog post on this but I can’t figure out how to get it to work. I’m setting up a new website with nginx as my web server. I just have a single index.html file in /home/nix/test/index.html, and my configuration.nix is:

{ modulesPath, pkgs, user, ... }:

{
  imports = [
    (modulesPath + "/virtualisation/digital-ocean-config.nix")
  ];

  users.users."nix" = {
    isNormalUser = true;
    extraGroups = [ "wheel" ];
    home = "/home/nix";
    shell = pkgs.fish;
  };

  networking = {
    hostName = "test";
    firewall = {
      allowedTCPPorts = [ 80 443 ];
    };
  };

  services.nginx = {
    enable = true;
    statusPage = true;
    recommendedOptimisation = true;

    virtualHosts."test.com" = {
      # addSSL = true;
      # enableACME = true;
      root = "/home/nix/test";
    };
  };
}

where I have my actual site instead of test.com. The nginx process is run by the nginx user, the /home/nix/test directory is owned by the nix user and has 744 permissions, and /home/nix/test/index.html file has 644 permissions and is also owned by nix.

If I curl 0.0.0.0:80 on my remote vps I get

<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx</center>
</body>
</html>

What am I doing wrong?

2 Likes
2

Seeing your Ngnix error logs would be helpful as well: journalctl -fu nginx.service.

Without further knowledge, I’d assume that Nginx sandboxing is at fault. The systemd service got sandboxed, which is a protection above the Unix user permissions. See this part in the NixOS 20.09 Release notes:

1 Like
3

@erictapen Thanks a lot for that link, the solution was setting

systemd.services.nginx.serviceConfig.ProtectHome = "read-only";
1 Like
4

I’m also having this issue, but the suggested solution doesn’t work.

From nginx logs:

nginx[6216]: [error] 6216#6216: *1 "/home/user/website/index.html" is forbidden (13: Permission denied), client: <IP>, server: HOSTNAME, request: "GET / HTTP/2.0", host: "HOSTNAME"
nginx[6216]: [error] 6216#6216: *1 open() "/home/user/website/favicon.ico" failed (13: Permission denied), client: <IP>, server: HOSTNAME, request: "GET /favicon.ico HTTP/2.0", host: "HOSTNAME", referrer: "https://HOSTNAME/"
nginx[6216]: [error] 6216#6216: *2 "/home/user/website/index.html" is forbidden (13: Permission denied), client: <IP>, server: HOSTNAME, request: "GET / HTTP/1.1", host: "<IP>"
nginx[6216]: [error] 6216#6216: *3 "/home/user/website/index.html" is forbidden (13: Permission denied), client: <IP>, server: HOSTNAME, request: "GET / HTTP/2.0", host: "HOSTNAME"
nginx[6216]: [error] 6216#6216: *3 open() "/home/user/website/favicon.ico" failed (13: Permission denied), client: <IP>, server: HOSTNAME, request: "GET /favicon.ico HTTP/2.0", host: "HOSTNAME", referrer: "https://HOSTNAME/"
nginx[6216]: [error] 6216#6216: *4 "/home/user/website/index.html" is forbidden (13: Permission denied), client: <IP>, server: HOSTNAME, request: "GET / HTTP/2.0", host: "HOSTNAME"
nginx[6216]: [error] 6216#6216: *4 open() "/home/user/website/favicon.ico" failed (13: Permission denied), client: <IP>, server: HOSTNAME, request: "GET /favicon.ico HTTP/2.0", host: "HOSTNAME", referrer: "https://HOSTNAME/"
nginx[6216]: [error] 6216#6216: *5 "/home/user/website/index.html" is forbidden (13: Permission denied), client: <IP>, server: HOSTNAME, request: "GET / HTTP/2.0", host: "HOSTNAME"
nginx[6216]: [error] 6216#6216: *5 open() "/home/user/website/favicon.ico" failed (13: Permission denied), client: <IP>, server: HOSTNAME, request: "GET /favicon.ico HTTP/2.0", host: "HOSTNAME", referrer: "https://HOSTNAME/"

The relevant nix config is:

    systemd.services.nginx.serviceConfig.ProtectHome = lib.mkForce false;
    systemd.services.nginx.serviceConfig.ProtectSystem = lib.mkForce false;

Which does seem to set the desired systemd [Service] options:

% cat /nix/var/nix/profiles/system/etc/systemd/system/nginx.service | grep Protect
ProtectClock=true
ProtectControlGroups=true
ProtectHome=false
ProtectHostname=true
ProtectKernelLogs=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectProc=invisible
ProtectSystem=false

Yet the issue persists.

5

Try using ReadOnlyPaths.

6

I didn’t mention before but I also tried

systemd.services.nginx.serviceConfig.ReadWritePaths = [ "/home/user/website" ];

Trying systemd.services.nginx.serviceConfig.ReadOnlyPaths = [ "/home/" ]; similarly updates the systemd service but the directory still isn’t readable.

7

Okay, nginx can read from /var so my workaround is to use /var/www/website and ln -s /var/www/website ~/website.

1 Like
8

I also faced the same.

Here’s what I am doing currently

services.nginx.user = "my_local_user_owning_the_directory_or_file";

idk if this introduces any security issues. If yes please share.

9

If it’s just for local dev and testing purposes, it’s fine to do, but if you are running this externally exposed on the internet, you should neither run nginx as a interactive local user nor grant access to other parts of the file system.
There is a reason that by default we lock things down and limit the access.

3 Likes
10

The reason being that now suddenly there is this always-on, fully internet connected nginx process that has access to all kinds of things on your host. Any kind of arbitrary code execution bug (and there have been some in the past, one of which last year) throws your system wide open on port 80/443, which is probably the worst kind of wide open.

Obviously we still need to run web servers, but turning off the sandboxing and/or giving nginx access to your home directory (especially read-write!) is a bad idea. Just put files in the paths that nginx is allowed to read. This is not a workaround but the correct way of doing it:

2 Likes
11

thanks for the feedbacks, reverted to nginx user and @ryang 's workaround.

12

I was also going crazy because your solution have not worked for me. It turned out that problem was that the home directory itself had insufficient permission. It can be set up by

users.users.<user>.homeMode=750

(My particular permission value is of course dependent on correct group membership - my user is in nginx group so only group permissions are necessary).

13

I am currently going crazy, Non of these solutions have worked for me, except for setting nginx.user = me because that seems like a security issue.

Is there any way to actually debug this thing?