Forgejo git operations are very slow

Help
1

I have a very simple forgejo setup behind an nginx reverse proxy:

# Webserver
  services.nginx = {
    enable = true;

    # Use recommended settings
    recommendedGzipSettings = true;
    recommendedOptimisation = true;
    recommendedProxySettings = true;
    recommendedTlsSettings = true;

    # Only allow PFS-enabled ciphers with AES256
    sslCiphers = "AES256+EECDH:AES256+EDH:!aNULL";

    # Add virtual hosts:
    virtualHosts = {
      # Forgejo
      "forgejo.${config.networking.domain}" = {
        enableACME = true;
        forceSSL = true;
        root = "/var/www";
        extraConfig = "client_max_body_size 512M;";
        locations."/".proxyPass = "http://unix:/run/forgejo/forgejo.sock";
      };
    };
  };

# Forgejo
  services.forgejo = {
    enable = true;
    database.type = "postgres";
    lfs.enable = true;
    settings = {
      server = {
        DOMAIN = "forgejo.${config.networking.domain}";
        ROOT_URL = "https://forgejo.${config.networking.domain}/";
        PROTOCOL = "http+unix";
      };
      actions = {
        ENABLED = true;
        DEFAULT_ACTIONS_URL = "github";
      };
      service.DISABLE_REGISTRATION = true; 
    };
  };

When I do any git operations that require the server, such as push or pull, it always takes a lot longer than it would with github or gitlab or any other hosted git service I have used so far. Maybe 10-20 seconds for a simple push of a single small commit.

My server is mostly idle during that time. It has 8 cores and 64GiB of RAM, and is not virtualised, so its processing power should not be what makes it slow. Also the internet connection is fast. The web interface of forgejo is super swift.

Do I need to change some settings in forgejo or in my configuration.nix to make it work faster?