Where I can find arguments against FU+ and FU? While I’m just starting I found that using FU greatly simplified my flake by not having to wrap every other bit of code into forEachSystem(system: ...). On the other hand I looked into FU+ and seemed overly complicated and not worth.
2 Likes
There’s certainly some discussion around how system should be handled. I think I’m on the side of using forAllSystems is usually an anti-pattern.
The idea is that, unless you’re really testing all architectures and deploying for them, you really should not be indicating that your flake works on them.
Note - and I think most people commenting on that issue miss this - that you can still force nix to use another arch’s output with e.g. nix build flake#packages.aarch64.default. So it’s not like you’re locking your flake to supported architectures only. You’re just advertising what you’ve tested.
If you do support multiple architectures, something like forEachSystem from FU is useful, and I wish that was a builtin.
But simpleFlake is less useful, unless you really consider the overlay aspect “boilerplate”. Personally I find it makes flakes harder to understand, since it focuses so much on overlays, and it produces a whole bunch of outputs that I might not even want my flake to have. And it fails to permit some others that I do want (e.g. checks), so that I now have to hack them into the overlay that it converts into a flake… simpleFlake is kind of weird and more designed around turning old nix repos into flakes.
This suggests to me that you might not realize you can do this globally:
{
outputs = {
self,
nixpkgs,
flake-utils,
}: let
systems = ["x86_64-linux" "aarch64"];
in
flake-utils.eachSystem systems (system: {
packages.default = {};
devShells.default = {};
})
// {
# Don't put system attributes on some outputs.
overlays.default = {};
};
}
I actually think a function that generates attrsets for each system and then just merges them would be a bit less magic and therefore nicer:
{
outputs = {
self,
nixpkgs,
}: let
systems = ["x86_64-linux" "aarch64"];
in
not-yet-builtins.merge-attr-map systems (system: {
packages.${system}.default = {};
devShells.${system}.default = {};
})
// {
# For performance and clarity, don't evaluate these attributes
# for all systems.
overlays.default = {};
};
}
At least I don’t think such a function exists yet, though you can do it with map, // and fold.
5 Likes
Personally, I feel like it’s a lot simpler to read a plain flake and will happily trade a little duplication for it.
Here’s an example of the flake for agenix that I’m happy with that doesn’t use flake utils.
{
description = "Secret management with age";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-21.11";
};
outputs = { self, nixpkgs }:
let
agenix = system: nixpkgs.legacyPackages.${system}.callPackage ./pkgs/agenix.nix {};
in {
nixosModules.age = import ./modules/age.nix;
nixosModule = self.nixosModules.age;
overlay = import ./overlay.nix;
packages."aarch64-linux".agenix = agenix "aarch64-linux";
defaultPackage."aarch64-linux" = self.packages."aarch64-linux".agenix;
packages."i686-linux".agenix = agenix "i686-linux";
defaultPackage."i686-linux" = self.packages."i686-linux".agenix;
packages."x86_64-darwin".agenix = agenix "x86_64-darwin";
defaultPackage."x86_64-darwin" = self.packages."x86_64-darwin".agenix;
packages."aarch64-darwin".agenix = agenix "aarch64-darwin";
defaultPackage."aarch64-darwin" = self.packages."aarch64-darwin".agenix;
packages."x86_64-linux".agenix = agenix "x86_64-linux";
defaultPackage."x86_64-linux" = self.packages."x86_64-linux".agenix;
checks."x86_64-linux".integration = import ./test/integration.nix {
inherit nixpkgs; pkgs = nixpkgs.legacyPackages."x86_64-linux"; system = "x86_64-linux";
};
};
}
2 Likes
I use FU+, but I’m planning to move away from it for my configs. I don’t like how functions like mkFlake and forEachSystem assume every definition is valid for every system, but but that’s never true for nixosConfigurations and darwinConfigurations. A NixOS config is never valid for Darwin (and vice versa with nix-darwin). It doesn’t hurt to have a Darwin config specified for Linux if you never evaluate it, but I would rather my flake outputs reflect actual things you can use. It feels messy otherwise.
It can also make things harder to understand when the magic breaks. A change to nix-darwin earlier this year broke evaluation with the v1.3 branch of FU+, which I was tracking. I ended up having to switch to FU+ master once I realized what the source of the error was. If I were specifying everything myself, I might not have run into that issue, or it would have happened in code I had written. I think it would have been easier to debug in that case.
3 Likes
Things can be tricky though, and not so straightforward, if you are trying to author a flake whose outputs are merely interpreted scripts, configuration, docs, text files, etc… Then what system(s) are you supposed to specify/declare? After all, flakes are supposed to be versatile when it comes to their outputs.
For interpreted scripts, its easy, the system of the host they shall be run on. This way you can make sure, the script actually is only available for those platforms that have a compatible interpreter (thats why I consider pythons “noarch” a lie!).
For generated content, eg. configuration files, PDFs, and others, I argued with myself back and forth quite a while.
At the end, they require that the tooling is available on the build hosts arch, while the output can be used anywhere, regardless the system.
Basically a crossbuild from “known to be able to build a LaTeX document”-systems to “noarch”, “multiarch”, or how ever you call it.
Flakes are not able to express this (yet). Not sure if this scenario will ever be able to be expressed. Though without content addressed derivations, and also given that LaTeX on Mac might produce some different output than on Linux (be it bug or a conscious choice), we might never take any benefit from such an architecture, while at the same time, if the provide the same output, we will have the benefit anyway, regardless of the build and target being seperate or just the same.
Thats why I eventually decided to simply not care, and only use the 4 major systems, of which I can actually test 2 and 2 half.
3 Likes
It’s also a platform tuple and not purely an arch. Build scripts often have platform-specific assumptions and settings, I’ve seen plenty of java/python build scripts check for MacOS and then produce a completely different output.
Of course you don’t need to worry about naughty build scripts when you’re writing really simple scripts from scratch, i.e. they don’t even shell out to anything native that might behave differently, but are you really writing flakes for such extremely simple scripts that you want to deploy to more than one platform without testing?
And separately, are you really writing latex projects that you’re going to need to build on anything but a few x86 machines? And are willing to advertise that it will work everywhere without testing?
Maybe you’ll want to eventually rebuild your flakes in 30 years on barch256-rustix or such, but I have my doubts that it will just work out of the box, and even if it does is it such a nightmare to just add an entry for the new platform after you’ve confirmed it does in fact work?
I feel like the band of projects that can actually usefully do this is so narrow, and that the edge cases are hard enough to spot, that forAllSystems should be considered an antipattern.
And hence I don’t like FU & co very much. They encourage blanket-enabling lots of platforms without testing, and despite their names don’t provide much besides helper functions that do this.
2 Likes
I prefer to use flake-utils. I build for both x86 and ARM so I already need both system arch outputs.
IME it’s a lot easier to define my flakes packages in overlays, then import that overlay to the flakes nixpkgs, and finally expose the package in flake outputs and devshell as appropriate.
This is my normal flake pattern:
{
inputs = {...};
outputs = { ... }:
rec {
# overlay that defines mypackage1, mypackage2,
# and packages under the mypackage_namespace attribute.
myOverlay = import .nix/overlays/default.nix;
# Also these packages can now be referenced in modules like normal packages.
nixosModules = rec {
mymodule = import .nix/modules/datava;
default = mymodule;
};
overlays.default = myOverlay;
} // (with flake-utils.lib; eachSystem [ system.x86_64-linux system.aarch64-linux ] (system:
let
pkgs = import nixpkgs {
inherit system self;
config = {...};
# import myOverlay into the nixpkgs for each system.
overlays = [
self.overlays.default
devshell.overlays.default
];
};
in {
devShells.default = pkgs.devshell.mkShell {
imports = [
(pkgs.devshell.importTOML ./devshell.toml)
{ packages = with pkgs; [...];}
];
};
# packages from this flake can now be used
# in packages.* output and devShells.* outputs, etc.
packages = {
inherit (pkgs) mypackage1 mypackage2;
default = pkgs.mypackage1;
} // (pkgs.mypackage_namespace);
}));
}
The verbosity of naming each system individually isn’t what bothers me (although it does since some flake.nix files are already >100 lines so more boilerplate is not ideal). But rather having to redefine packages in several locations just seems prone to bugs that break reproducibility and caching; if the wrong or a different nixpkgs is used then it can lead to build differences and errors that are hard to debug.
I also think I had issues using packages in custom modules if they weren’t defined by an overlay but I don’t recall what the issue was exactly… I think it’s that downstream flake wouldn’t have the package defined unless it’s merged with nixpkgs via an overlay.
Are overlays really a big anti-pattern concern? I’ve certainly found them the easiest way to define packages in a system-agnostic way that allows usage in both the current flake and in any downstream flake.
I guess if you don’t ever build for other architectures it might seem silly to use each-system but it’s becoming increasingly common in cloud environments. While you can force nixpkgs to build for other architectures w/o specifying them in the flake output, you can’t (easily) if you want to use the same upstream nixosConfiguration to share configuration modules.
I haven’t at least noticed any performance issues using overlays heavily (I mean building home-manager for every nixosConfiguration is already N*M evaluations and any package that uses a form of src = ./.; rebuilds on every commit so those are my worse performance issues right now with nix).
2 Likes
Yes, this is why I think the non-all systems generator is not an antipattern. I still disagree with the way it functions; hiding the ${system} behind magic makes things less clear and harder for newcomers.
But yeah, if you do have packages for multiple systems you do want to follow DRY principles. I think there’s a tradeoff in how many awkward non-multiarch outputs you need to define, and how you handle per-system changes to the build system, but there are definitely projects where having this makes sense.
It’s all the other things FU does that I take issue with.
Kinda, mostly when used the way you are using them. You have the problem that you are in fact creating duplicate instances of nixpkgs: 1000 instances of nixpkgs
The performance issues may not be hitting you yet, but you’re creating a cascade of nixpkgs evaluations.
Keep in mind not every nixosConfiguration is evaluated every time you run nix, however if you import nixpkgs {}; like that that is run almost every single time you do anything with the flake. Have enough flakes that do this and depend on each other and suddenly nix takes several GB of memory to evaluate anything.
That isn’t to say that overlays in general are always an antipattern - they’re just a potential problem for the ecosystem when used as an import nixpkgs {overlays = [];} in a flake.
I really don’t understand why. That is such a roundabout and complex way of defining packages. This is what I tend to do: dotfiles/pkgs/default.nix at 8dbcf2aa7e4e613e1b9cb235e64efa8dcf2ef480 · TLATER/dotfiles · GitHub
I then import that file in the flake.nix, giving it:
# flake.nix
{
outputs = {self, nixpkgs, ...} @inputs: {
# Add some mechanism to define $system as you see fit
packages.${system} = import ./pkgs {
inherit self;
pkgs = nixpkgs.legacyPackages.${system};
flake-inputs = inputs;
};
};
}
This is primarily to reduce boilerplate in the flake.nix, feel free to put that attrset directly in your flakes (or generate it by mapping over files in a directory). Technically you can even pull self from flake-inputs to cut down on yet another line of boilerplate.
This should not cause issues with downstream modules not “having access” to the packages? If you want to add/change a package in nixpkgs you’d use nixpkgs.overlays in the NixOS configuration, rather than doing this at the flake level. Any other scenario you just need an option that allows setting the package.
If you want an overlay for use in NixOS (say, because your modules rely on them and you don’t want your users to set a package option or force them to plug your flake input into their module args - which IMO we should, and provide support for at a nixosSystem level to avoid exactly this), you can then define it like so:
# Part of flake.nix
{
overlays.default = (final: prev: {
my-awesome-flake-pkgs = self.packages.${prev.system};
});
# These can still reference the packages as usual
nixosModules = {
mymodule = import .nix/modules/datava;
default = self.nixosModules.mymodule;
};
}
I know tastes differ, but I think this is objectively cleaner? Newcomers don’t have to figure out the likes of rec or understand overlays to mostly know what’s going on, and this definitely cuts down on the boilerplate of importing nixpkgs with all manner of arguments. The only pointless boilerplate left is what you do to map $system, and arguably defining callPackage.
Also, your use of rec really bothers me, no wonder you’re complaining about boilerplate, just use the self input instead.
3 Likes
I can live with that, and I definitely prefer this over flake-utils’ implementation! Encouraging explicitly specifying the systems is enough to fix most issues IMO, and I really like that this has a way around the awkward merge for system-independent outputs 
I still think it’ll be less clear to new users than raw outputs, but I suppose there’s a limit to how much you should cater to that.
Wonder if it’d be less magic to hide system-independent outputs behind a separate attribute rather than doing magic with attribute names, though.
1 Like
Following up on my previous post, I did eventually drop FU+ from my configs. It’s hard to say how much it improved clarity because I also took the opportunity to drop a lot of the magic directory structure. That did help quite a bit. I also moved my vendored packages to a separate repository because I don’t want anyone depending on my config repo. (I’d disable PRs if I could.)
I also don’t use any overlays now. I had a bad experience earlier this year where something broke with FU+ and how I had defined them, causing infinite recursions. If I need to access packages from other flakes, I use inputs, which I pass via _module.args.
2 Likes
fwiw, I’m starting to feel like genAttrs might be enough?
outputs =
let systems = [ "x86_64-linux" "aarch64-darwin"];
in {
packages = genAttrs systems (system: {
default = nixpkgs.legacyPackages.${system}.cowsay;
});
};
4 Likes