I seem to be stuck because a binary cache is down

Help
1

I’ve dabbled with haskell.nix and set my nixos machine up to use IOG’s binary cache for Haskell packages. Now it seems like the cache is unreachable, because I’m getting errors like these with everything I try:

warning: error: unable to download 'https://hydra.iohk.io/6wpl0piasqslg5ll6lazzf1imasak2a7.narinfo': Timeout was reached (28); retrying in 326 ms
warning: error: unable to download 'https://hydra.iohk.io/6wpl0piasqslg5ll6lazzf1imasak2a7.narinfo': Timeout was reached (28); retrying in 581 ms
warning: error: unable to download 'https://hydra.iohk.io/6wpl0piasqslg5ll6lazzf1imasak2a7.narinfo': Timeout was reached (28); retrying in 1244 ms
warning: error: unable to download 'https://hydra.iohk.io/6wpl0piasqslg5ll6lazzf1imasak2a7.narinfo': Timeout was reached (28); retrying in 2810 ms

Even removing the cache from my configuration.nix and running nixos-rebuild switch doesn’t work because I get this error before the new configuration can be applied.

I’ve seen mentions of --option substitute false or --fallback, but those flags don’t seem to change anything.

How can I remove the binary cache?

1 Like
2

I observe this issue on my machine too currently.

Also I manage my nix.conf using nix, so manually removing the entry seems not an option.

@Noughtmare Can you try --option build-use-substitutes false when building then remove the entry and retry building regularly, see if the iohk entry is gone ?

4 Likes
3

That works, thanks a lot!

4

There are 2 issues here:

  1. an unavailable cache prevents you from moving forward, which is indeed a problem, and
  2. changing the nix configuration when you cannot run nixos-rebuild and friends

As for the latter, remember, there is nothing preventing you from manually editing the files under /etc.

So you can:

  sudo cp $(realpath /etc/nix/nix.conf) /etc/nix/nix.conf
  sudo chmod 644 /etc/nix/nix.conf
  # change whatever you want in your editor of choice
  sudo systemctl restart nix-daemon
4 Likes
5

just fyi since building without the cache for haskell.nix projects can be a real PITA, hydra.iohk.io is the old url. The current cache for IOG is cache.iog.io. Everything from the old hydra should be there and the same public key works for it. Might be helpful.

5 Likes
6

I’m still unable to resolve this issue on NixOS. I fixed the url from hydra.iohk.io to cache.iog.io in my configuration.nix and rebuilt but when I run nix build it’s still trying to download from the old URL and timing out.

7

What are you trying to build specifically? Does the flake you are trying to build have the old cache configured? Check .local/share/nix/trusted-settings.json for any saved settings? If so you can delete the file and Nix will prompt you if you want to use the cache, and you can just deny it.

Also, just run nix show-config as a sanity check to make sure the old cache is not getting picked up from somewhere else (user config, environment, etc).

8

Thanks - I’m just trying to build a basic Haskell project using the haskell.nix flake template. My trusted-settings.json had the old cache in it, so I deleted that, but when I ran nix show-config it was still showing there, along with some other substituters/trusted-public-keys that I previously commented out in my configuration.nix. It looks like at some point I created a ~/.config/nix/nix.conf file with the old binary cache settings in it - I probably followed the Nix instructions instead of NixOS instructions when I initially set up the cache. Thank you very much for your help!

1 Like
9

@curriedFunctions did you manually replace the substituter hydra.iohk.io with cache.iog.io in /etc/nix/nix.conf as well? as shown above: I seem to be stuck because a binary cache is down - #4 by peterhoeg

10

This has hit a lot of folks so we went ahead and setup a CNAME redirect from the old cache url to the new, so it should be less painful now.

11

Doesn’t seem to make things better, people report seeing HTTP 418 now…

12

I haven’t tried it yet as the projects I work on have used the new url for some time now, but at least curl works:

❯ curl hydra.iohk.io/nix-cache-info
StoreDir: /nix/store
WantMassQuery: 1
Priority: 50%

I tried this same curl yesterday and it wasn’t working yet, so perhaps the DNS just hadn’t propagated yet (for those reporting an error).

13

No I didn’t, I had already replaced the old cache url with the new one a while back in my configuration.nix file so my /etc/nix/nix.conf was correct - after deleting my superfluous ~/.config/nix/nix.conf file with the old url in it everything worked fine

14

Just wanted to let you know, that I am currently at 2/68/72 (9 failed), according to nix-top it is currently running the tests for OpenSfM in py3.9 and py3.10 respectively and does so for the last 10+ hours.

I indeed had a situation yesterday when my mem was used by 15/15 RAM/swap. System was barely usable, despite I had a browser and discord open. Still I was able to chat and browse, though clicks had a “latency” of 30 seconds.

After ~30 minutes that was over and next thing got built, memory consumption instantly fell to below 6 GiB on RAM, and over the next couple of minutes RAM/swap redistributed itself. Since then the RAM usage didn’t rise above ~10 GiB which is to be expected with my swappiness.