Nice work!
I’m impatient to know how do you std’ize
bitte piece, my use case is
very close to it.
Could be great if a new nixosModule Cell Block Type (formerly Clade)
had an action to navigate interface options a la nixos-search.
EDIT: I guess NixOS module system will be a problem here because its global namespace.
I’m currently on paternity, but afaik this is in the backlog.
I think the semantic precedent that was first set by divnix/DevOS (now divnix/digga) with modules, profiles and profile aggregates (called “suites”) is a good abstraction.
Especially that particular interpretation of “suites” as profile combinators is yet largely unexploited in bitte and conditions the codebase to be sometimes unfortunately really messy.
If / once integration with colmena progresses, Standard can be more natively used for (NixOS) infrastructure-based management.
Currently, we know the “metal” (bitteProfile) and “cloud” (app deployment domain) Cells. Some examples can be seen in the public downstream adopters of Standard & Bitte.
Hey Blagg! I’ve had some ideas about std.
In my view, the goal is to build a portage-like metadata repository of configurations and deployments, which can be used to build, deploy and manage multiple Software-Defined and Network Functions,virtual machines and even bare-metal machines in (singularity-) containers.
Of course, not everything needs to be homogeneous and software-defined from the kernel to the user side or from bootloading to bare-metal networking. The goal is to come as close as possible with plenary power.
Having a canonical way to write conditions for each type of cell would be nice, but may fall short of our idealism.
This could be a good use for the Ultimate Dev Env spec that should be laying around somewhere.
The big goal is to have a single declarative interface to containers and VMs, especially those installed on bare-metal.
This is only part of the pie. We need to handle bootloaders, and, depending on Hardware Support personal interests, to how far all the way to bare metal we can get.
A special case are microservices deployed on bare metal, which we may want to run a hybrid CustomSoftware / CustomResources approach to, but the upper bound is still bare-metal containers.
@DavidRoland I may not grasp to the fullest extend your line of thought and the context of it.
But already, I have a sense that you may enjoy the following:
The Aura projects has set sails to blur the current runtime boundaries between pet machines and datacenter workloads. They seem to aim at a classical jail as their namespace primitive. One of the authors after some discussions and back and forth may be currently evaluating Nickel as the lingua franca configuration system for that new runtime. A k8s scheduler-shim is planned. The goal, however, is to replace systemd.
Some folks have started to get in touch with some OCI folks and we’ve been talking (just talking the walk as of now) about amending the OCI specification with a Nix image layer type. There are multiple upsides to this, such as quicker startup times, but fundamentally, it brings the Nix distribution model closer to the defacto industry standard for containers. And I also beleive it would disintermediate singluarity ce (at least parts of it), as each hashed resource locator (in the nix store) is typically already signed.
Since aura might use the OCI container spec as artifact interchange format (which I cheer), combining the two might just evolve into some very exciting new mainstream distribution mechanism over time. And one of the Aura authors is actually familiar with Nix and NixOS, which is a promising precursor.
Currently, my focus — as evidenced by the recent pattern pieces in the Standard docs — is on the SDLC and CICDO (CI, CD & Observability) side.
Next, I want to factorize documentation matter and expose close-to-source documentation nuggets onto the Nix repo-spanning connecting tissue via styx. Sort of taking shopify’s backstage heads-on (and partly disintermediate it) with the power of Nix.
I haven’t yet spent a whole lot on SDNs besides some excursions into northbound Yang models, which seemed to be a data description DSL, not unlike Nix/Nickel, and hence probably can be emulated and made be part of the same config lingua franca. A yangModels Cell Block Type could provide a stateless submit Action.
Behold — a paper from La Universidad del Cauca, just around the corner from where I live 
…
http://web.unicauca.edu.co/telematica/?q=node/513
conditions on cells
What kind of conditionals do you have in mind, here?
my bad, i thought you lived on a different planet.
For those interested, the std book has been created to provide a thorough introduction to the framework via a hands-on example project. It’s an evergreen work, so it will receive updates over time, but it’s got enough content to get most people going.
That’s really interesting! I wasn’t aware of the Aura project or the effort to add a Nix image layer type to OCI, but that could be really powerful. I’m also really interested in your work on the SDLC and CICD/Observability side. I think Nix is really well-suited for that sort of thing, and I’m excited to see what you come up with.
It sounds like you’re doing some really cutting-edge work here, and I’m really excited to see how it all develops. Keep up the good work Blaggacao, and thanks for sharing your thoughts with us, we’re all excited to see what you come up with next.
Thanks @blaggacao for point Aura.
I thought OAM (Open Application Model) with their only implementation, KubeVela (CUE based), could bring some fresh air to application delivery.
And I’m also wondering how do your CI/CD/O ideas in std will move from runtime to control plane/loop?
First, we’ve been exploring the “operable” abstraction as a layer in between “package” and “distribution” (OCI). If the package is already great stuff, it can be tiny. But it grows as a function of bad operational quality of the “package” itself and thereby accomodates important team dynamics over the SDLC.
The planned Standard CI/CD Preset is envisioned to only comprise the Dev environment, since after that we need a more complex state machine to decide on environment propagation. No current CI is really fit for that task. The best Candidate in my opinion for that stage would be zeebe.io (because it can interoperate well with less technical stakeholders), although at work we also have been working on GitHub - input-output-hk/cicero: event-driven automation for Nomad for that part of the SDLC focusing on nix-flavord technical stakeholders.
But as for the control plane, all that Std provides is the manifest of the 4th layer of packaging that is handed over to the control plane.
Perspectively, we can have a look at JuJu Charmed Operator Framework for another layer of packaging (“the [still] secret 5th layer”). The advantage is, it doesn’t strictly rely on the control plane’s implementation of distributed state, but brings it’s own and can therefore be ported to different control planes (potentially even systemd).
EDIT: @juaningan you got me hooked on OAM (let someone else bridge it to zeebe 
). If you have an idea or an elegant integration point for CI/CD/O, please comment on that PR: docs: add ci cd pattern piece by blaggacao · Pull Request #123 · divnix/std · GitHub — I guess the 4th layer has to be redefined LOL. And the 5th layer would become somewhat 3bis 
Support was recently added for generating devcontainers. Usage is straight-forward: supply an existing development shell and get an OCI image back that can optionally be made compatible with VSCode Devcontainers.
{
vscode = lib.ops.mkDevOCI {
name = "docker.io/std-vscode";
tag = "latest";
devshell = cell.devshells.default;
vscode = true;
labels = {
title = "std-dev";
version = "0.1.0";
url = "https://github.com/divnix";
source = "https://github.com/divnix";
description = ''
A prepackaged vscode devcontainer for hacking on std
'';
};
}
$ std //_automation/containers/vscode:copy-to-docker
$ devcontainer open .
In the above example, you’ll be dropped into a devcontainer session with the devshell already activated and ready to use. In the near future, the image will be available in a container registry, and it will be possible to use a development shell without having Nix installed locally.
Side note: The container is completely nix-driven and does not rely on using an FHS system like existing tooling.
I’ve started to prompt some phrases from the Standard docs to an AI text2image (pixray). Just wanted to share. I’m stunned.
“Nix is marvel to ones and cruelty to others.”
“Lazy configuration tree”
“Nix has superpowers”
“The Nix Operator persona …”
For the ones interested, I just managed to produce a decent integration betweem Standard and Colmena over here, thanks to a new colmena upstream interface that allows to relax on the schema (of which kost parts would have unused added complexity in this use case that would have to be muted in userspace). Now it’s muted by design in the evaluator amd yhe user interface is super simple (and there’s only one!):
i really want to understand std and hive… i can pretty much checkout any config and understand it… but with hive… i stare on it and don’t get it 
and i really want to understand it…
Well, it’s also just in the making. I’m not even using it for myself, so it might be a bit early.
But I can still say something about it.
First, std hasn’t been made with NixOS configurations in mind. This might be a potential use-case, though, and hence, I’m trying out stuff in divnix/hive.
You’ve got to understand how std does imports as a prerequisite. And probably familiarize with its few conventions.
Then, in the cellBlocks secion of divnix/hive we kind of see a pattern that should be familiar to divnix/digga / divnix/DevOS users: the distinction in modules, profiles & suites.
Then, we also have the “final” configurations. So in a way it’s 4 layers: module → profile → suite → configuration. (If “4 layers” triggers somebody in another context, please send me a 
)
Ignore (devshells "devshells"), it’s for the dev environment for the flake itself: rather standard Standard .
So what you’ve been looking at so far is the hive’s output type system that is intelligently leveraged by the importer. Just as any type system, this gives you a decent, but abstract overview over the semantic organization of this particular repo.
Next, the “# soil”. The soil are all additional variadic arguments (after the first one) to the growOn function, and not by coincidence is the soil located down (towards the earth). It’s the “soil”, after all. Easy to reference, easy to find (across all std repos). It is designed to host all the compatibility things, where some tooling X (including the nix CLI’s defaults) expects things. So if you know std and you see a “#soil”, it tells you something about tooling schemata (or just “legacy” stuff).
So in divnix/hive/flake.nix’ #soil, you see adapters for colmena, nixos-generators & home-manager (in the making, so maybe not yet). The functions have absolutely unrelated and undocumented fantasynames for a purpose: they are prototypes and by not choosing a final naming, they are less likely to slip into a mental corsette, already. Other than maybe making the reader dizzy lol! 
So what does make-{mead,honey,moonshine}.nix do? They harvest the cells in the schematic ways that each of the tools expect. This is where you read unlimited variations of boilerplate in those other repositories, that look “easier” at first glance. Yeah, each time, you’d be looking mostly at some variation of boilerplate in such a prime estate like a flake.nix, such as if “using Nix” was about “writing boilerplate”… 
So if you drop the mental presumption of the primacy of the tooling schema (read: “tool centric”), then what you’re left with is a perfectly “problem” or “domain”-centric (and typed, sic!) file organization with a compatibility layer of “#soil”. And the best: the problem domain is yours to explore. No opinion whatsoever does std have (as opposed to, for example, the Nix CLI).
At this point: nix repl / :lf . / outputs.<TAB> tells the rest of the story (and makes the small bits of magic that std actually really does intuitively experiencable). I’m afraid, that’s probably the “cost” of learning Standard.
std TUI/CLI? Won’t be helpful in divnix/hive, we have quite well-established vertical tooling (colmena/nixos-generator/home-manager). No need to wrap them in a horizontal one. The use case for the CLI/TUI is rather the one of a “nix” service provider to a “non-nix” configuration consumer.
@luxus just fiddled this together over the weekend, so maybe the NixOS use case becomes clearer with this example for you and others: feat(lina2358): add initial [working] configuration by blaggacao · Pull Request #2 · divnix/hive · GitHub
Some notes:
nixosProfiles/nixosSuites/nixosModules (not needed for a simple config)nixosConfigurations if we have colmenaConfigurations? → no real good answer, maybe just because colmena is an external evaluator while nixosConfigurations can be explored in the repl, natively. Maybe compat with nixos-generators’ CLI for what it’s worth.nix-community/disko’s module integration (very WIP)hardwareProfile so that there’s not going to be a layer violation and flipping the metal shall not change the system config (there are still a couple of layer violations in this config, but it’s good enough for now)ifwifi over the weekend.Still, at this point using std for NixOS is off-label use and probably will always be or maybe an off-shoot project might take that space, cause entering the fearless NixOS support pool is not necessarily something that I would want to invest in near term. It would also likely start to completely overshadow the fledgling Nix (not NixOS!) DevOps use-case & patterns.
But if it was to support NixOS more closely, it would take out a few “full repository cycles” as to when it comes to passing instances of nixpkgs (pkgs) down to the module system and settle on one single way (aligned across colmena, nixos & home-manager).
Quick Update: We’ve started to modularize some components of Standard and make them independently useful:
A little brush-up on the readme. Hope this is a better appetizer than what’s been there before…
And there is also an integration with https://flake.parts live now:
https://flake.parts/options/std.html
flake-parts with std as described there.std with flake-parts by simply invoking flake-part’s mkFlake function in one of your layers of “Soil”.I found std while searching if OAM had any nix-related stuff going on, and only found this open issue:
As nix newbie and fretting that nix would be too complex and overkill, std and the way that it is documented really made things click for me, and now I’m sold on Nix et al 
The context of my search is a project where I want to integrate Wasmcloud, that has a complex custom build environment, and uses wadm (wasmCloud Application Deployment Manager) based on OAM. I too feel that this open standard and nix can be complementary.
