Let's Encrypt certificate is not recognized by Traefik (works in nginx)

I am trying to run a demo Node.js app on a subdomain using Traefik reverse proxy. I am generating a Let’s Encrypt SSL certificate with the security.acme option. When I import the certificate in the Nginx config, it works. However, when I try to add it to Traefik, I get the following error:

Secure Connection Failed

An error occurred during a connection to hello.domain.com SSL peer has no certificate for the requested DNS name.

Error code: SSL_ERROR_UNRECOGNIZED_NAME_ALERT

The page you are trying to view cannot be shown because the authenticity of the received data could not be verified.
Please contact the website owners to inform them of this problem.

Here is my configuration.nix:

{ pkgs, ... }: {
  # ...

  config = {
    environment.systemPackages = with pkgs; [
      openssl
      git
      vim
      nodejs
      yarn
      nodePackages.npm
    ];

    security.acme = {
      acceptTerms = true;
      defaults.email = "name@domain.com";
      certs."domain.com" = {
        domain = "domain.com";
        extraDomainNames = [ "*.domain.com" ];
        dnsProvider = "ovh";
        dnsPropagationCheck = true;
        credentialsFile = "/etc/nixos/credentials.txt";
      };
    };

    services.traefik = {
      enable = true;
      staticConfigOptions = {
        global = {
          checkNewVersion = false;
          sendAnonymousUsage = false;
        };
        entryPoints = {
          web = {
            address = ":80";
            http.redirections.entrypoint = {
              to = "websecure";
              scheme = "https";
            };
          };
          websecure.address = ":443";
        };
        providers.docker.exposedByDefault = false;
      };
      dynamicConfigOptions = {

        tls = {
            stores.default = {
              defaultCertificate = {
                certFile = "/var/lib/acme/domain.com/cert.pem";
                keyFile = "/var/lib/acme/domain.com/key.pem";
              };
            };

            certificates = [
              {
                certFile = "/var/lib/acme/domain.com/cert.pem";
                keyFile = "/var/lib/acme/domain/key.pem";
                stores = "default";
              }
            ];
          };

        http.routers.hello = {
          rule = "Host(`hello.domain.com`)";
          entryPoints = [ "websecure" ];
          service = "hello";
          tls = true;
        };

        http.services.hello = {
          loadBalancer.servers = [{
            url = "http://localhost:8000";
          }];
        };

      };

    };


    systemd.services."hello-world" = {
      description = "Hello World";
      after = [ "network.target" ];
      wantedBy = [ "multi-user.target" ];
      serviceConfig = {
        Type = "simple";
        ExecStart = "${pkgs.nodejs}/bin/node /etc/nixos/app.js";
        Restart = "always";
        RestartSec = "3";
      };
    };

    networking.firewall.allowedTCPPorts = [ 80 443 8000 ];
    users.users.traefik.extraGroups = [ "docker" ];
  };
}

When I comment out this part:

       #  stores.default = {
       #      defaultCertificate = {
       #       certFile = "/var/lib/acme/domain.com/cert.pem";
       #       keyFile = "/var/lib/acme/domain.com/key.pem";
       #    };
       #  };

I get a “Warning: Potential Security Risk Ahead,” but I can reach the subdomain page. This is because Traefik uses the default certificate then:

| Common Name (CN)|TRAEFIK DEFAULT CERT|
|---|---|
|Organization (O)|<Not Part Of Certificate>|
|Organizational Unit (OU)|<Not Part Of Certificate>|
|Common Name (CN)|TRAEFIK DEFAULT CERT|
|Organization (O)|<Not Part Of Certificate>|
|Organizational Unit (OU)|<Not Part Of Certificate>|
|Issued On|Monday, May 29, 2023 at 10:48:12 AM|
|Expires On|Tuesday, May 28, 2024 at 10:48:12 AM|

How can i fix my config to use Let’s encrypt wildcard certificate with Treafik correctly?

I got it fixed. Here is the working config for reference:

{ pkgs, ... }: {
  # imports = [ ... ];

  config = {
    environment.systemPackages = with pkgs; [
      openssl
      git
      nodejs
      yarn
      nodePackages.npm
    ];

    security.acme = {
      acceptTerms = true;
      defaults.email = "name@domain.com";
      certs."domain.com" = {
        domain = "domain.com";
        extraDomainNames = [ "*.domain.com" ];
        dnsProvider = "ovh";
        dnsPropagationCheck = true;
        credentialsFile = "/etc/nixos/credentials.txt";
      };
    };

    services.traefik = {
      enable = true;
      staticConfigOptions = {
        global = {
          checkNewVersion = false;
          sendAnonymousUsage = false;
        };


        entryPoints = {
          web = {
            address = ":80";
            http.redirections.entrypoint = {
              to = "websecure";
              scheme = "https";
            };
          };
          websecure = {
            address = ":443";

          };

        };
        providers.docker.exposedByDefault = false;
      };
      dynamicConfigOptions = {

        tls = {
          stores.default = {
            defaultCertificate = {
              certFile = "/var/lib/acme/domain.com/cert.pem";
              keyFile = "/var/lib/acme/domain.com/key.pem";
            };
          };

          certificates = [
            {
              certFile = "/var/lib/acme/domain.com/cert.pem";
              keyFile = "/var/lib/acme/domain.com/key.pem";
              stores = "default";
            }
          ];
        };

        http.routers.hello = {
          rule = "Host(`hello.domain.com`)";
          entryPoints = [ "websecure" ];
          service = "hello";
          tls = {
            certResolver = "my-resolver";
            domains = {
              main = [ "domain.com" ];
              sans = [ "*.domain.com" ];
            };
          };
        };

        http.services.hello = {
          loadBalancer.servers = [{
            url = "http://localhost:8000";
          }];
        };

      };


    };


    # services.caddy = {
    #   enable = true;

    #   virtualHosts."hello.domain.com".extraConfig = ''
    #     reverse_proxy http://localhost:8000
    #   '';

    #   extraConfig = ''
    #     domain.com {
    #       root * /var/www
    #       file_server
    #     }

    #   '';
    # };

    virtualisation.docker.enable = true;
    systemd.services.copy-landing-page = {
      description = "Copy index";
      wantedBy = [ "multi-user.target" ];
      serviceConfig = {
        Type = "oneshot";
        ExecStart = "${pkgs.coreutils}/bin/cp ${./html/index.html} /var/www/index.html";
      };
    };

    systemd.services."hello-world" = {
      description = "Hello World Node.js app";
      after = [ "network.target" ];
      wantedBy = [ "multi-user.target" ];
      serviceConfig = {
        Type = "simple";
        ExecStart = "${pkgs.nodejs}/bin/node /etc/nixos/app.js";
        Restart = "always";
        RestartSec = "3";
      };
    };

    networking.firewall.allowedTCPPorts = [ 80 443 8000 ];
    users.users.traefik.extraGroups = [ "docker" "acme" ];
    #    users.users.traefik = {
    #   isSystemUser = false;
    #   isNormalUser = true;
    #   group = "traefik";
    # };

  };
}

The credentials content:

OVH_APPLICATION_KEY=<key>
OVH_APPLICATION_SECRET=<key>
OVH_CONSUMER_KEY=<key>
OVH_ENDPOINT=ovh-eu

Thanks, this was really helpful!

However there is one bit missing without which it doesn’t work:

security.acme.certs."domain.com".group = "traefik";

without that, Traefik can not read the certs provisioned by ACME and won’t serve anything, just giving a cryptic error failed to load X509 key pair: failed to find any PEM data in certificate input.

Additionally this line is not necessary and just produces another error in the logs:

certResolver = "my-resolver";