Hello, I have a flip that I didn’t change in a long time, and suddenly it has become very, very slow. It is making it impossible to work with it.
At first, I thought it was just the end, but I removed the use flake directive, and then it loads instantly. I tried to execute nix shell, and it hangs. I have no idea of how to debug this, and it shows no output.
If this helps, the flake is at the root of several Node.js projects. I’m not sure if it’s just that it is trying to copy something to the store and hangs
Ok. The folder was 6 GB in size.
I removed some of the nested node_modules folders, and I slimmed it down to 2 GB
It took a bit long to load, but after a minute or two I saw a different log, this time it was copying something else to the store. I opened the folder it was copying and it was indeed my folder project.
And it finally loaded.
What I suspect it happened is that I just modified the .envrc file, invalidating the cache and forcing it to re-create the flake content. But since then, I added a bunch of stuff to the folder and it got bigger.
This is IMO a serious flaw. I’m a freelancer and not all my clients want their repositories polluted with my flake.nix or flake.lock or direnv files. Also, I don’t want to have a subfolder per client and then a subfolder per project just as a wrapper to have a flake.nix in it.
In node.js projects this is specially annoying because a) node modules is quite big and b) it contains thousands of files, which is even slower than just copying some big files
If anyone has a viable workaround, a way to ignore certain folders or who knows what, I will be very grateful
Yes! IMO it’s the main deal breaker for flakes. Also consider that any secrets or such in the directory end up being dumped into the nix store. It’s really, really silly.
If it’s a git repository, nix will by default only copy tracked directories. Make sure node_modules isn’t tracked.
If it’s not a git repository, make it a git repository.
Alternatively, use the traditional nix-shell with a shell.nix (or use nix in the direnv world) and forget flakes exist until the stalemate resolves and someone finds a proper solution to all these problems.
Could something as simple as this solve my problem? Will try it out.
I didn’t made it like that because all the nested folders are git reposiories already, but what you say makes sense.
The (in my opinion) better solution is one of these:
Have your flake with all the different projects shells in a different folder, and track it using .git, in the projct have the .envrc with use flake ../my-dev-flake#thisProject. You can alternatively have one flake per project if you have to be overly specific with nixpkgs revisions used.
Just use classic/legacy nix-shell + shell.nix. Most of the time you do not actually need a flake.
As a rule of thumb: You do not want to use a path shell if you expect the folder size to exceed a couple of ten MiBs.
That would work very nice except that I don’t want to have a .envrc in the project either. This are projects I don’t own, and I don’t want to put my personal configurations into them.
EDIT: Ok, thinking about it, I can still have all my flakes in a central “repository” and have a wrapper parent folder for each client with an .envrc pointing to the central repository of flakes. Ok, I can give that a try
The question is, if all node_modules were inside the subfolders, and the subfolders were all git projects… does that mean that nix only takes into account the root git project where it executes?
If nix starts to be a pain point in your development experience maybe try switching to docker based development. I had some success overcoming nix/nixos pitfalls by using Dockerfiles.
If you say nix, this is a path-flake, it will not bother to look out for git folders. And for the type inference only the top-level folder is considered, the so called flake root
Edit:
And to be exact, if nix gets told this is a “git” flake or infers this, then nix won’t bother to identify or filter (to be) ignored files and folders, it will just as git what files it knows about, and then copy those.
Well, so far I have been enjoying flakes.
I like the idea of having fixed inputs and predictable dev environments. With flakes I can see where each package is coming from, and I think shell.nix doesn’t offer that, and for me is important as each project may have different versions of the same packages.