One thing the Nix ecosystem seems to be lacking is a common standard for how infrastructure code should be organized. This is understandably outside the scope of what Nix aims to achieve, since Nix has so many different uses and install bases, but I think it would be interesting to have a discussion about it.
There seems to be a lot of discussion already about Nix’s other use-cases, such as being a build tool or using NixOps/Morph for small/medium size deployments. Where I don’t see much discussion is for larger scale deployments (500+ nodes) where you’d traditionally use something like Puppet with the “roles and profiles” pattern and an ENC like Hiera.
Are there any best-practices for this sort of thing? How to organize your code? What should or shouldn’t go into a module? How to program in little differences per region/environment while keeping everything dry?
It’d be kind of nice to come up with a common structure. Combined with the new flakes feature it would make getting started with Nix/NixOS in a large environment much easier.
{ pkgs, mainUser, mainGroup, homeMaker }:
builtins.listToAttrs
(
map
(
incompleteUser:
{
name = incompleteUser.name;
value = incompleteUser.value //
{
home = homeMaker incompleteUser.name;
group = mainGroup;
isNormalUser = true;
extraGroups = [
# Sudo
"wheel"
# Network manager
"networkmanager"
# Run locate using locate db.
# Only for autogenerated db
"locate"
# [ CommentDb: 1 ]
"video"
"lxd"
];
shell = pkgs.versioned."19.09".zsh;
# mkpasswd -m sha-512 # Careful, mkpasswd is its own package
hashedPassword=
"$6$IV2x36/8$LYL29lIpAMeRg4xasdasasdiF1veU5uAKnasdfFmBIAZ1xHpZQF4tbpWl4D/";
};
}
)
(
[
{
name = mainUser;
value = {
uid = 1000;
};
}
{
name = "head";
value = {
uid = 20000;
};
}
] ++
(
map
(
id:
let uname = "tmpu${builtins.toString id}";
in
{
name = uname;
value =
{
uid = 20000 + id;
};
}
)
( pkgs.lib.lists.range 1 9 )
)
)
)
Then packages.nix further imports editors.nix, systemmanagement.nix, media.nix etc.
I had multiple disks and two computers and I wanted to make an arrangement that would help me plug and play without thinking much. Thing is there is no set answer I think, you just have to use your own categorization skills and categorize options into nix files.
(We use the “our” namespace, so none of our module names will conflict with existing NixOS modules and to tip us off that we are dealing with our configuration options.)
Some of those question and a good part of what’s motivating your post is being addressed by:
We would be very pleased to have your (real or hypothetical) use cases on board and I’d specifically ask you to open up github discussion on that repository.
This is especially a good moment, since we are at the brink of expanding into the enterprise use cases, by first incorporating kubenix as a kubernetes resource builder.
The per-region requirement as well as efficient nix evaluation at scale are probably points we can investigate further together.
We also have the suites concept to consolidate deployment scenarios and have an “inoculating iso” and are further researching the identity bootstrapping workflows (network & cryptographic), which are thought for ease and scaleability of bare metal deployments.
Unfortunate that I saw your post only until now.
EDIT: there is also a project which looks at those problems from a hashicorp ecosystem angle (and seems to have a little bit of mind share with devos):
In my company we are using the *-nixpkgs patttern, where we inherit upstream nixpkgs, and we basically overlay with our own, and layout the same way that nixpkgs lays out. We also then use logical and hardware profiles.
This then can be used with any combination of snowflake machines, auto-updating fleet, etc, all from one tree. When we first created this, we were also building it with hydra, and managing hydra with the same source tree as well. The hydra-built version of it now lives over at https://github.com/Holo-Host/holo-nixpkgs and has evolved/changed since we worked on it. But still built by hydra, still following the same core pattern. Deployed machines from holo-nixpkgs also auto-update themselves, so that one can be a good one to look at for large scale auto updating fleet of machines.