Issues building a new nixpkgs package on MacOS – Curl SSL Certificate Problem

Hi everyone,

I’m trying to add a new package to nixpkgs and build it before opening a PR. I’m on MacOS and have installed Nix using the Determinate Systems installer along with nix-darwin. My laptop uses an antivirus, which intercepts some HTTP requests with a custom certificate.

In the /Library/LaunchDaemons/org.nixos.nix-daemon.plist file, I noticed the following entry:

<key>NIX_SSL_CERT_FILE</key>
<string>/etc/ssl/certs/ca-certificates.crt</string>

I tried copying the appropriate certificate bundle to /etc/ssl/certs/ca-certificates.crt and restarting the nix-daemon, but it didn’t help.

Here is the problematic terminal output when I run nix build '.#cloudflare-dynamic-dns':

$ nix build '.#cloudflare-dynamic-dns'
warning: Git tree '/Users/glashevich/Workspace/code/github.com/NixOS/nixpkgs' is dirty
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
error: builder for '/nix/store/6yp7mjl23x4paxrzgxaby2nly762r0di-source.drv' failed with exit code 1;
       last 10 log lines:
       >   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
       >                                  Dload  Upload   Total   Spent    Left  Speed
       >   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
       > curl: (60) SSL certificate problem: unable to get local issuer certificate
       > More details here: https://curl.se/docs/sslcerts.html
       >
       > curl failed to verify the legitimacy of the server and therefore could not
       > establish a secure connection to it. To learn more about this situation and
       > how to fix it, please visit the webpage mentioned above.
       > error: cannot download source from any mirror
       For full logs, run 'nix log /nix/store/6yp7mjl23x4paxrzgxaby2nly762r0di-source.drv'.
error: 1 dependencies of derivation '/nix/store/fi5dy6h55fnabhh3z9p5y9d1ygkfrzfy-cloudflare-dynamic-dns-4.3.0.drv' failed to build

Running nix log yields the following:

$ nix log /nix/store/6yp7mjl23x4paxrzgxaby2nly762r0di-source.drv
warning: The interpretation of store paths arguments ending in `.drv` recently changed. If this command is now failing try again with '/nix/store/6yp7mjl23x4paxrzgxaby2nly762r0di-source.drv^*'
install: skipping file '/dev/fd/63', as it was replaced while being copied
error checking the existence of https://tarballs.nixos.org/sha256/:
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the webpage mentioned above.

trying https://github.com/zebradil/cloudflare-dynamic-dns/archive/refs/tags/4.3.0.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
^M  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0^M  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the webpage mentioned above.
error: cannot download source from any mirror

However, directly using curl from the shell works as expected when the SSL_CERT_FILE environment variable is set to the correct bundle.

$ curl https://github.com/zebradil/cloudflare-dynamic-dns/archive/refs/tags/4.3.0.tar.gz > /dev/null

Has anyone encountered a similar issue or has any ideas on how I can resolve this SSL certificate problem?

NOTE: the issue does not appear if hash is set here:

  src = fetchFromGitHub {
    owner = "zebradil";
    repo = "cloudflare-dynamic-dns";
    rev = "refs/tags/${version}";
    hash = "<HERE>";
  };

Also, building another package (e.g. nix build '.#myks') works as expected.

I worked around the issue by installing nix on another machine with Arch Linux and building there, but I still need to fix the root problem on the Mac.

I’m out of ideas on how to debug this further. Is there a way to pass extra arguments to the failing curl process somehow?

Thanks in advance for your help!

You think the anti virus is the cause of this issue?

The antivirus requires its certificate to be added to the system keychain. Otherwise, some websites didn’t open via browsers.

To use curl in a shell, I had to export all system certificates to a file and set SSL_CERT_FILE envvar.

I assume that curl that is used under the hood of nix build doesn’t use correct certificates. But I have no idea how I can reach out to this curl to pass the correct certificates path or to at least increase verbosity to see what it’s using.

Interesting. I never thought of certificates in the context of fetching. But of course some have. Does this help?

https://nix.dev/manual/nix/2.24/command-ref/conf-file.html?highlight=certificate#conf-ssl-cert-file

Unfortunately, this setting doesn’t change the behavior.
According to the doc you linked, the correct certificates file should be already used, as it is set via SSL_CERT_FILE.

I can reproduce the faulty behavior with curl by setting SSL_CERT_FILE to /etc/ssl/certs/ca-certificates.crt instead of the correct self-crafted certificates file:

SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt curl https://github.com/zebradil/cloudflare-dynamic-dns/archive/refs/tags/4.3.0.tar.gz > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the webpage mentioned above.

So it seems like, for some reason, the fetching module doesn’t respect the nix’s configuration.

The docs for convenience:


ssl-cert-file

The path of a file containing CA certificates used to authenticate https:// downloads. Nix by default will use the first of the following files that exists:

  1. /etc/ssl/certs/ca-certificates.crt
  2. /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt

The path can be overridden by the following environment variables, in order of precedence:

  1. NIX_SSL_CERT_FILE
  2. SSL_CERT_FILE

Do you think you’d be able to provide a public reproduction?

I prepared a host with self-signed certificate, and a repo with reproduction code: GitHub - Zebradil/nix-curl-repro

The README shows the reproduction steps and results.

Thank you. I suppose filing an issue would be the next step. Good luck.

Hi @Zebradil
I have a local cache on my LAN behind https and face the same issue.
What I do, is add my cert to the system trust store:
(as root)

cat /rootrmycert.crt >> /etc/ssl/certs/ca-certificates.crt

This is obviously a workaround and needs to be repeated when an update rebuilds the ca-certificates file.

I’m on Mac and certificates here aren’t working the same way as on Linux. Mac uses different certification stores and most of the apps understand that. Other apps look into the conventional locations like the one you mentioned (/etc/ssl/certs/ca-certificates.crt, /etc/ssl/certs/ca-bundle.crt) or obey the SSL_CERT_FILE environment variable.

I use the variable and it works for pretty much everything, except curl that is invoked by nix-build. For some reason, nix-build doesn’t pass the environment variable to the curl process and uses it’s own certificate (e.g. /nix/store/ix767x74k09h5ffj5nfarssvgbg1z7gw-nss-cacert-3.101.1/etc/ssl/certs/ca-bundle.crt).