What if we had a repository of NixOS system, deployment and microservice configuration modules? Lets call it Nixcfgs.
While Nixpkgs contains fundamental service modules, Nixcfgs would contained stack modules that include several partially configured services integrated and combined. These modules would then automatically be hooked to a unified authentication system.
As a concrete example, I am interested in creating a NixData configuration, consisting of:
Python Data Science modules
Various basic command-line tools for data processing
Pandoc + TUI Editor toolchain for publication
JupyterHub + JupyterLab module
RocketChat + Ejabberd for communication
GitLab as a VCS
All of these services tied together with a single authentication system, allowing users to have a single login for various web apps hosted on the same server. At the moment the Nix community doesn’t have a place for this sort of project to live.
In Nixcfgs this stack would be one module among many serving different purposes, and users could then install them to their server and automatically. For the Nix community, this would mean a focal point for sharing system configuration and deployment knowledge. Eventually, we can have a docker-like interface for pulling and running modules imperatively while retaining the option for subsequent declarative configuration by hacking the modules or using overlays.
It sounds like a cool idea, altho I’m a relative newb with overextended obligations and don’t have the capacity to contribute directly. The nix community seems like it’s spread a bit thin right now; to the extent I can help, I’m focused on core flows.
At the moment the Nix community doesn’t have a place for this sort of project to live
The org at Nix community projects · GitHub describes itself as an incubator. But this also seems like it could be complementary to nixops.
I don’t see a technical reason why this cannot exist inside of nixpkgs already, it seems like nixcfgs is a collection of modules that enable and configure other modules, which already works.
Maybe this kind of thing live in a module namespace like nixcfgs.data.
Are you sure you want all those things running on one machine?
users of your NixData should trust you, because modules in NixOS module system can do whatever root can do.
But should they? This is same kind of problem, that Docker has - whether downloaded image has some 0-day-like builtin into main
executable. However, NixOS modules can be reviewed. This is why a complex set of modules in nixpkgs would be used much more than
same in standalone repo.
So yes, having a common nixcfgs which is reviewed by lots of eyes would be good.
I imagine that each maintainer/author has his own top-namespace, say
ixxie.NixData.enable = true;
and provide short README on what it is for and how to use it.
this wouldn’t suite nixpkgs. The stack you’ve described is a bit opinionated. And I’d like nixpkgs/NixOS to contain less opinionated stuff.
the PR review and merge policy should be defined.
each contribution should have some integration tests unless it is pinned to some nixpkgs revision
ideally, it should respect running standalone inside nixos-container (and disable features that can’t be run there).
@jcrben I understand and I see your point about the community being spread thin; on the otherhand I really believe that have configuration templates will help draw people and ultimately also save a lot of time spent on replicated system configuration effort.
The Nix Community org could be one place for this, but as @ryantm points out, there is a close affinity with Nixpkgs and so I think its natural that this either would be a part of Nixpkgs or at least under the NixOS foundation. I can definitely see the value on having it in Nixpkgs but was simply unsure if its appropriate.
@ryantm - I want a data science hobby server; I will not spend all my spare money on a cluster for this stack My goal is also to lower entry barrier to data science - allow anybody to spin up an instance and run with it (this is one of my hopes for Nixcfgs - make it easy to setup full stacks). That said I did mention NixOps/Disnix deployment templates should be included in Nixcfgs so they wouldn’t have to live on one machine even then.
@danbst good point about configs being more opnionated than Nixpkgs should be…
Regarding the namespaces, at first I thought user namespaces would be weird but I see your point; its perhaps possible to make Nixcfgs import various configurations as submodules, if they pass a certain standard (like the tests you describe), but then PRs and issues would be all over the place. This would however allow existing projects like musnix to live in their original repos while still being included.
Regarding the trust issue, it seems to be NixOS modules are a lot more transparent than docker images.
To address the the problem of ‘opinionation’ of Nixpkgs we would then have some sane defaults which correspond to the orthodox conception of such a stack with extra options to allow tweaking. Worst case a stack could be forked.
I think putting this in Nixpkgs could be very nice for new comers: they could follow a simple tutorial and very quickly have a full stack up and running. Eventually if and when some Nix language changes would take place, we could even consider distinguishing service modules from stack modules explicitly.
My goal is also to lower entry barrier to data science - allow anybody to spin up an instance and run with it (this is one of my hopes for Nixcfgs - make it easy to setup full stacks). That said I did mention NixOps/Disnix deployment templates should be included in Nixcfgs so they wouldn’t have to live on one machine even then.