Unable to fix "too many open files" error

Help
1

Hello everyone,

I will be brief, after playing around with custom CPU flags I am unable to rebuild my system.
The process fails outputing a known error: “too many open files”.

After a quick internet search I have found this issue: Cannot raise maximum number of open file descriptors inside of nix build environment · Issue #4018 · NixOS/nix · GitHub
In which temporally augmenting the value of systemd.services.nix-daemon.serviceConfig.LimitNOFILE is suggest as a fix.

My problem is that even after raising the number up to a ludicrous amount the error does not disappear.

As a note:
I had to define it with lib.mkForce because the value was already defined somewhere else. (Not by me.)
My default value was already higher than the one suggested in the issue.

What am I missing?

Thanks in advice. :D

3 Likes
2

Possibly you accidentally altered an OS limit somehow. If you cannot rollback to before your “playing around”, you may have to investigate all the bits which can restrict the number of open files. Ideally you can find something to tell you what is limiting the build – maybe in build logs or in the system journal.

Poking around on my system, I found these interesting bits

  • cat /proc/sys/fs/nr_open returns 1073741816
  • ulimit -Hn returns 524288
  • sudo grep -R NOFILE /etc returns many files including
/etc/static/systemd/system/nix-daemon.service.d/overrides.conf:LimitNOFILE=1048576
/etc/static/systemd/system/nix-daemon.service:LimitNOFILE=4096

I have not set LimitNOFILE anywhere in my config; so these must be default values.

Here are some links which helped me understand more about NOFILE

2 Likes
3

I’ve managed to solve this issue, thanks for your reply. :slight_smile:

1 Like
4

Could you share what you did to fix the problem?

1 Like
5

After setting nixpkgs.hostPlatform.gcc.arch the build is failing with “Too many open files”. What did you do to resolve this?

6

Sorry for the late response but I’ve been away lately.

Right now I can exactly recall what steps I’ve took in order to fix the issue.

But I can tell you that if you want to build certain stuff you should disable boot.tmp.useTmpfs because if not building stuff will eat up your RAM.

You can also change where nix keeps its tmp files. I don’t know why do they use /tmp by default.

I’m not sure but you can try that and see if it helps.

7

At the risk of bumping an old thread, I encountered this issue while running a machine with quite a number of nixos containers.
I was able to fix it by increasing the systemd open files limit with:

  systemd.extraConfig = "DefaultLimitNOFILE=2048"; # defaults to 1024 if unset

This will require a reboot on the machine

3 Likes
8

Also sorry to necrobump, but this is one of the most prominent search results for the problem.

If you’re encountering this issue whilst doing nixos-rebuild switch, you might have tried to up the limit on nix-daemon.service (which is ludicrous by default anyway) and realised that it doesn’t work.

Turns out: it seems like nixos-rebuild switch does not use the Nix daemon to do the build (I tried disabling the daemon and noticing that builds don’t fail).
Apply a new limit in your shell that is doing the rebuild instead: ulimit -n 4096 for example. Then run the rebuild.

I am no expert but I think the Nix daemon is just there for multi-user support and probably only is there to make building possible by unprivileged users without letting them inject fake files into the store and making sure the builds are pure.
When you are the root user, I guess this does not apply.
This worked for me.

1 Like