It seems like some of the excitement is in allowing Nix expressions to refer to things that are very unlike to contents of the existing Nix store. But could this sort of layer also be useful for ‘unifying’ different package managers that actually do keep Nix-like promises, so-to-speak? One sort of use case I have in mind is things like a Nix user using Guix to pull in the latest Guile, or a Guix user relying on Nix to provide dependencies that may not be suitable for inclusion in Guix.
Similarly, since Nomia is interested in supporting other languages, it seems like it could be a really useful bridge for providing scaffolding for package sets
a Nix user on macOS could use Nomia to write impure Nix expressions that include a few packages or services from impure sources (MacPorts? Homebrew?) before gradually pulling them out one-by-one
developers of new languages (like Nickel, Guix’s DSL, or Distri’s package recipe format) could use Nomia to allow their new tools to depend on or refer to parts of Nixpkgs for things they haven’t packaged yet, without having to directly interoperate with the Nix CLI or language
users of other configuration languages could use Nomia to refer to packages and services provided by Nix idiomatically, if their language has a Nomia frontend
Are these kinds of things among the intended uses of Nomia?
Yes, precisely! Or, as I put it the other day in the Nomia discord channel:
One aspect of Nomia hinted at in the docs but not fully fleshed out that I’d like to cover better: Nomia as a general FFI whenever the integration point can be expressed via resource interactions. This is sort of like how Unix pipes act as a general FFI between arbitrary processes, except where named pipes let you even splat in the full source code of the other side of the pipe when you open it.
In particular this will allow any language to integrate with the Nix expression langauge. Not directly able to e.g. create a nixexpr function that is backed by some function in Haskell, but being able to reference and manipulate any packages that any nix expression evaluates to
Cool! I have some other questions that are just to sort of check my understanding of the vision of Nomia.
In a way, these ‘reductions’ remind me a little bit of nix-store --optimise, because of the way they deduplicate. In this case, it seems like what we’re really trying to save is time rather than disk space, but could Nomia likewise emit ‘reductions’ for things already built and dumped in the store if it notices that they’re equivalent in some sense after the fact?
It seems like this kind of elimination process could also handle lockfile generation for any project that is ultimately realized via Nix, no matter how it’s specified at the top level:
Specify a Nix build using nondeterministic signifiers (e.g., source control URIs, which go out to the network and eventually resolve to the latest committed version)
Use this kind of transformatino process until every source in the graph is a fixed-output derivation
Dump the names of the root of every tree in the forest
This is kinda cool because it leaves the Nix community free to modify and even mix and match schemas for specifying Nix resources without explicitly defining or ever changing a lockfile format. Nix lockfiles could effectively just be any collections of Nomia names that have the right properties of determinism
So in binary package managers, an implicit global namespace is how ‘packages find each other’. Packages are treated as equivalent to one another, regardless of where they come from, based on name. In the Nix world, packages in Nixpkgs share a global namespace and refer to each other by name at build time. Nixpkgs has a mechanism for inserting out-of-tree dependencies into that shared namespace so that packages written without that modified dependency in mind can refer to it: overlays.
Nomia could standardize and generalize that sort of functionality, pulling it down to the store layer instead of a library written in the Nix language, in the form of explicitly configured reductions. Instruct Nomia to emit a reduction for a high-level name, one which is not fully-determined, like ‘php’, and now everything else in that namespace which was declared to depend on ‘php’ will depend on our modified version of that resource.
But because Nomia wants to name things that are more general than just files on disk, we could also use user-configured reductions like this to override service definitions, or details of a deployment like which host a service is to run on.
Would this kind of stuff be an abuse of Nomia, or is it aligned with your vision? It seems to me like it could be really nice for rolling out simple collections of Nix-specified packages and services without implementing complicated libraries for managing overrides in the Nix DSL. (With in-language override mechanisms remaining a possibility for folks who want something custom and know the Nix language well but don’t know or care how Nomia is implemented/how to extend it.)
So Nomia is expected to know, and maybe reason, about the guarantees offered by certain names (or maybe whole namespaces). Could this be used to classify package isolation?
To describe what I mean, I want to borrow the notion of ‘exchange directory’ from Distri, a neat little research distro from a former Debian developer inspired in part by Nix. An exchange directory is a shared (impure) resource on the filesystem that an otherwise isolated package is allowed to consult. So packages can’t depend on things outside the Nix store, but applications are usually still expected to read configuration from $HOME if they find it, and often (but not always! ) /etc as well.
One of the historical goals with Nixpkgs has apparently been to ensure that packages always work when you run them from the Nix store, without depending on being installed to a profile. This means we have to do weird things to make software that inherently relies on impurity by design, like some runtime-loaded plugin systems, work right. In the case of Qt, we’ve opted to make applications unconditionally search for plugins based on the PATH that is set in their environment when they launch. This gives us a certain robustness (applications can be launched directly from the store) at the cost of a certain kind of impurity that can be difficult to manage, and for some use cases we might want to make a different tradeoff. Could judicious use of Nomia namespaces help track/manage these tradeoffs more explicitly without having to teach Nomia or Nix how to reason about different aspects of package isolation?
I’m picturing something like:
guarantees have names
those names could mean something to Nomia (e.g., this resource does not depend on any exchange directories; it promises not to read from any files outside the store)
but guarantee names don’t have to mean anything in that sense
guarantees can be presumptively satisfiable (satisfied unless a resource declares violating them) or explicitly satisfiable (satisfied only if a resource declares meeting them)
packages can declare that they meet or violate some guarantees
namespaces can declare that they meet or violate some guarantees
if you’re working in a namespace that declares a guarantee, and you depend on a package (perhaps from another namespace) which is incompatible with it, Nomia throws an error and tells you that resource can’t be instantiated
Is this kind of thing (i.e., managing not just impurity but robustness) part of the intended picture for Nomia?
I believe recursive Nix, for example, is mentioned in the original Flakes RFC as it is in your post. Let me know if I correctly understand how Nomia addresses the problems that Nix flakes has been an experiment in solving.
Does (or, more weakly, could) Nomia subsume the purpose of the input side of the flake.nix schema— and only that portion of the flake.nix schema? In other words: half of the work of the flakes schema does is to provide a deliberately impure syntax in the Nix language for a certain kind of resource (that kind of resource also being called ‘flakes’ of course ), but which can always be reduced to a fully determined (fixed-output) resource in the lockfile.
The other half (the outputs schema) is a promise that Nix tools will understand what can be done with resources of that kind once it is handed one:
nix run will know how to ‘run a flake’ by looking at its declaration of a default application
nixos-rebuild will know how to instantiate a flake as an operating system by looking for a host declared in the flake
nix develop will know how to set up a shell environment for the development of a package associated with the flake
… and so on
In a Nomia-based implementation of Nix flakes, I imagine the layering would be slightly different than with the current prototype, because it would be natural for Nomia to handle resolving a name equivalent to flake:nixpkgs#fish all the way down to the package, and not just the flake:nixpkgs part. But essentially, Nomia’s task would just be to hand over resources that nixos-rebuild or nix profile install understands, and it wouldn’t have to care about what those look like. The substantive features like Nix flakes aims to offer in the CLI could remain more or less independent of Nomia, but they would leverage Nomia for input URIs. Is that right?
Yes, for example I imagine we might want “multihashing” in a content-addressed namespace, where the same resource can be identified by multiple different hash algorithms.
Yes, one of the nice things about Nomia’s approach here is that the high level “unlocked” description and the lower-level “locked” description are all on the same level with respect to the system as a whole, so you can mix and match as well. You could for example have a notion of “airplane mode” locking where any indeterminism that requires network access is resolved but any other indeterminism (e.g. reading local files) is kept unresolved until later.
This is aligned with the vision. I expect some users will want to restrict their systems to the fully specified byte-for-byte compatible approach like Nix gives today, but in practice we often want to describe and share resources at different levels of abstraction.
If I understand what you’re saying here, yes. Different resource types have different guarantees, and different contextual names (i.e. those which depend on resources passed in from outside of Nomia) will each have their own semantics, which can be statically analyzed to determine properties of the whole.
Yes, exactly. In fact you could use those kinds of input specifications independent of the rest of the flakes mechanism, just with a normal nix expression.
Right— overrides within Nix can be much more minute than replacing one resource with another, both resources more or less given as wholes, and that’s what’s great about the various overriding mechanisms we have in Nixpkgs today. This can make them really convenient and useful despite the way that they rely on machinery that can be tricky if you have to dig into its guts for some reason. But user-configured reductions in Nomia would work in a very simple way, at the cost of being a bit coarser. So both would be good to have for different situations.
That’s a very cool example. And yeah, it suggests that Nomia would be useful for managing different kinds of guarantees like I was thinking about in the more complicated example I tried to outline in my earlier post.
I’m really excited about what Nomia might be able to do and be! Seems relevant to a whole range of real problems. It also seems clear that Nomia’s aims are conservative even though they are powerful, in that they seem unlikely to disrupt or hamper the development of features within Nix or products based on Nix that are unrelated to Nomia’s problem domain.
Looking forward to playing with it as soon as it’s ready for that!
This sounds really awesome (and way over my head at the moment…), thank you!
The “content-addressed Nix” annoucement was posted around the time this conversation wrapped up in May - did it affect any of the subsequent work on Nomia?
I assume the answer is no because Nomia seems to occupy a whole another level, and content-addressability in Nix is “just” an extra feature. I barely grasps the basics of either though so I could be totally wrong.
I’ve discussed the relationship of the projects with @edolstra and @thufschmitt , ultimately I hope for Nomia to cover what the CA store and other features currently provide but obviously there is significant work to be done before that’s feasible.
I’ve been wondering whether this project was still active, since signs of life on the GitHub repo have been scarce. I guess Scarf has a lot else on its plate and you’re waiting for a few more things to settle in Nix itself?
I still think Nomia is a very interesting idea. Care to make any adjustments to the timeline on the roadmap given on GH?
) 