Lots of great ideas here. Do we have a any plan of action or Github issue where all this is condensed?
1 Like
Anyone should feel free to open issues upstream and link them as I did for the inputs.patches idea.
Inlinging source here, since I should not trust the hedgedoc demo server:
Source
Unofficial Flakes Roadmap
=====
You wanted it. Even though, our apoligies, it's unofficial.
In fact this document just takes the gist out of all the feedback from https://discourse.nixos.org/t/improving-flakes/12831.
## What are flakes?
- Flakes are a vehicle to help nix philosophy to gain mass adoption (like snow-_flakes_ falling all over a place).
- They also are a way to promote ecosystem growth through _decentralization_.
- They _enforce_ clear input / output interfaces for an otherwise extremely flexible `nix` language.
- They are a `flake.nix` at the root of a folder tree, written in `nix`, that satisfies [certain contracts](https://github.com/NixOS/nix/blob/e6150de90d8db101209fc6363f5f7696ee8192c4/src/nix/flake.md)
- They are pure eval _by default_.
- They are a whole new class of CLI experience within the `nix 3.0` milestone. — a highly contested item ("solve everything at once")
## Ideas & Discussions
### Super Stupid Flakes (a.k.a. "legacy `nix` isn't going away")
###### Origin of Idea: @zimbatm
##### "✤" represent parts of the contract
Each repo top-level contains a `default.nix` (✤) . The file returns a function that takes `system` (✤) and `inputs` (✤) as attributes and which returns an attrset. That attrset has some reserved attributes like `lib` (✤) , `overlay` (✤) , `devShell` (✤) and `defaultPackage` (✤), and probably more (exact list to be determined).
Each repo top-level contains a `flake.toml` (✤) that contains project metadata. This means the inputs, the project description and the list of supported systems.
TODO: I’m assuming inputs would work the same as in the current implementation.
Turn on pure eval by default.
Restore recurse into attrs.
That’s it.
#### Can I make this backward-compatible?
Yes, use `{ system ? builtins.currentSystem, inputs ? import ./flake.lock.nix }` as the header.
Where `flake.lock.nix` (✤) is a poly-fill that plays a similar role as the current `flake-compat.nix` . There is a possible variant where Nix actually generates the `flake.lock` file so that it can be imported with Nix.
#### Is `lib` getting re-evaluated for each system?
Use the fact that `import` memorizes the result. So for example `{system, inputs }: { lib = import ./lib.nix; }` would return the same lib for each system.
#### Does it cache my eval?
No. It would be nice to cache it, but it can be done later. The current implementation is strictly tied to git and gets invalidated on each new commit.
#### What if I want to build a derivation that depends on multiple systems?
Avoid doing that. It makes it hard for contributors to use your repo as they now need access to all the target architectures and set them up as nix remote builder, to build the repo.
If you really need to, add a separate `release.nix` (✤) file that cobbles everything together. Classic Nix is not going away.
### System as an Input
###### Origin of Idea: @domenkozar
##### This probably was in part a thought-parent to Super Stupid Flakes (TM)
[In his own words](https://www.notion.so/Flakes-system-as-an-input-instead-of-an-output-2d2cdef8eac2434a833d6faae15b35c0).
```toml
[flake]
supportedSystems = [
"x86_64-linux",
"x86_64-darwin",
]
[inputs]
...
flake.toml
Origin of Idea: Original Flake Design
This is a psychological one: people not familiar with the nix language arguably prefer a familiar format as the entrypoint.
This idea combines with Super Stupid Flakes
Subflakes
Origin of Idea: @nrdxp
To preemt the “growth through decentralzation” aspect, we also need a way to work smoothly with in-tree sub-flakes to be compatible with out current mono-repo ecosystem model. We need to iron out the subflake experience to offer the best of both worlds and make transition smooth.
Dev-Only Dependencies
Origin of Idea: @blaggacao
flake.nix unlike their language-specific cousins, such as e.g. pyproject.toml do not have a distinction between “development” and “runtime” dependencies. However a growing ecosystem of nix powered devshells (with additional tooling) seems to indicate that such a distintion is needed to improve dependency clarity. Currently, those are often just carelessly “mixed” together.
Inputs Patching
Origin of Idea: @Worldofpeace / @blaggacao
Upstream Issue: Flake: `inputs.<name>.patches = []` · Issue #4433 · NixOS/nix · GitHub
Patches to source code that represents an input should be treated as inputs, too. Simple.
Overlays vs follows
Origin of Idea: @zimbatm / @Mic92 / @blaggacao
Overlays and inputs.<name>.follows are functional equivalents. We should settle on inputs.<name>.follows notation, since those:
- easier to understand
- don’t suffer infinite recursion when used without great care
- no namespace pollution
- can be unordered (as opposed to when applying several overlays that depend on each other).
No Flat Hierarchy (1 level deep)
Origin of Idea: unclear
Enforcing a one-level-deep hierarchy on the output contracts has been poorly understood and the original reasons can be traced back to “more efficient indexing” (aprox. Elco quote burried in some PR discussion).
It seems like people would prefer to abandon this contract.
Version Compatibility Boundaries
Origin of Idea: @blaggacao
Especially for libraries, it might be desirable to specify version boundaries for inputs on which compatibility is guaranteed.
Increasing nesting of the follows syntax might lead to situations where optimzing for closure size (= e.g. pining all inputs to a specific nixpkgs version) might lead to hard-to-understand compatibility problems.
By allowing (very conscientious) upstream libraries to declare explicitly their compatibility boundaries (e.g. git’s own <hash>..<hash> notation) would allow firendlier errors and tooling around this class of problems.
</details>
1 Like
Where does flake.toml come from? I don’t mind separation of metadata, but it’s metadata for a default.nix file that likely consists of more intense Nix code than what the metadata declarations would be. Also, at the risk of undermining the previous sentence, changing from Nix to TOML for this file also means that metaprogramming of that data, on the few occasions it does happen, will become a lot more opaque. Flakes will need friendly, official documentation to be approachable, and if that’s the case, the introduction stating
{
flake.supportedSystems = [
"x86_64-linux"
"x86_64-darwin"
];
inputs = {
...
instead of
[flake]
supportedSystems = [
"x86_64-linux",
"x86_64-darwin",
]
[inputs]
...
is not going to be a big deal. This amount of complexity, in Nix or TOML, all reads close enough to less-strict JSON that I don’t think people will be thrown off. Nixpkgs’s lack of comprehensive documentation discourages people from picking it up or sticking with it the most, in my opinion, and flakes are a worse version of that currently. All the documentation for flakes I regularly reference is unofficial, save for nix3-flake(1), and referencing the relevant parts of Nix’s source when I have questions is much less pleasant than referencing the relevant parts of Nixpkgs’s source.
Despite how flakes are unstable & the unofficial flakes roadmap is unofficial, I’d really love to see polished, comprehensive, as-official-as-possible documentation for flakes (+ the unofficial flakes roadmap) as they are right now. People who currently use flakes can have a single reference to use & share when others ask. People who are looking into flakes can have a reference they know about & feel confident trusting that won’t unexpectedly leave them to fend for themselves in a confusing section. People who are holding off on flakes can have an accurate way to gauge how flakes are shaping up and quote from an uncontentious source when discussing their thoughts on flakes.
If we have that sort of documentation, and people feel like those bits of metadata being written in Nix instead of something else is a hang-up, then go for trying out TOML. The current flake situation being disorienting for experienced Nix users though makes me hesitate for now.
3 Likes
As a newcomer to Nix (started in May this year), who comes from the ops/packaging/OS side with loads of experience in Debian but no prior interest in functional programming, I love Flakes and the new nix-command. The ergonomics and flags of the various subcommands I’ve encountered (build, develop, eval, hash, store, show-derivation, path-info, why-depends) all seem to basically make sense, locking is sane and easy to reason about, and when I’ve gone to hack on nixpkgs itself, I’ve found it straightforward to understand the difference between nix build .#foo (my local version) vs nix build nixpkgs#foo (the current master).
Honestly, my main complaint is still just that basically all of the documentation pages, tutorials, pills, etc still refer to nix-build or nix-shell, none of which seem to work without a default.nix that contains (what feels to me) like a lot of mystery-meat stuff centered around import <nixpkgs> {}.
So anyway, I understand that there are some legitimate technical/philosophical/governance issues at play here, but I want to register my appreciation for what has been done so far to get Flakes where it is.
25 Likes
I think flakes are a great, however the fact that you cannot compute the inputs seem to be overly restrictive. I have created an
issue, where I have included all the other issues on that point. If you know about any other issues that I have missed or if I’m missing something fundamental about flakes please let me know. Any feedback would be great 
.
3 Likes
This reminds me of the allowUnfree issue: it very likely must be a flake input too. The current way of importing nixpkgs with { config.allowUnfree = true; } means that a flake user has no chance to explicitly accept the license
12 Likes
There is more and more tutorials popping up, advertising the use of flakes. I am still hesitating because it is experimental, but would like to advertise it myself for projects of mine.
This is the best thread I could find in order to get a roadmap for the future of flakes.
Is there any progress for an RFC or similar?
1 Like
There’s this announcement: Nix release schedule and roadmap
nix 3.0 would mark the stabilization of flakes (as that roadmap states), so it’s the closest thing I’m aware of. But there may be a more up-to-date roadmap somewhere
4 Likes
There was a summer of nix talk given just a few hours ago where @edolstra discussed a little bit about the roadmap for flakes. Trying to formalize a roadmap process and a desire to stabalize flakes with an RFC. I got the impression that the stabalization discussion/RFC would begin fairly soon. But it sounds like he needs to work with the community to make anything concrete.
https://youtu.be/h8hWX_aGGDc?t=2900 links the portion of the talk where the roadmap is discussed.
7 Likes
For me, the main issue with flakes is copying the entire Git repository to the Nix store. As far as I understand, Nix does this for the evaluation cache, but in my opinion, the evaluation cache is not the main feature of flakes and could have been implemented as another experimental feature.
@misuzu That is being fixed soon™ with Source tree abstraction by edolstra · Pull Request #6530 · NixOS/nix · GitHub
5 Likes
To be clear, “soon” means “when it’s done”.
2 Likes
But also connotes the hope and eagerness of many users like me, who, in addition to recognizing that rushing such a complex PR through would cause more harm than good, have been looking forward to the completion of that work for a very long time (years now). 
Thanks in advance to @edolstra for all the time he’s put into that abstraction and @roberth for his work in reviewing that PR and trying to move it forward without breaking things. I’m going to be so happy when that feature lands!
3 Likes
It is not copying the full repository, but “only” the staging area/commit.
If it was copying full repository, working on nixpkgs would kill many peoples store…
Copying 50,000 files takes a while on my machine, and doing so for every single line change is very frustrating. That’s why I don’t use flakes when working on nixpkgs. At work, we have one large multi-package monorepo, and working on one package with flakes is an even worse experience than working on nixpkgs. Single checkout of this monorepo is around 300MB and building one package 50 times consumes ~15GB of space in /nix/store.
3 Likes
I wonder if nix.settings.auto-optimise-store = true; would mitigate most of the duplication you see from copying .nix expressions to the store. It’s still not instant. But at least it will save some space for you.
It is an “after the fact” optimisation, which means, the full store path is created, and then scanned for “duplicate” files, which will be replaced with hardlinks to identical files.
So instead of making things quicker, it will in fact make them slower, due to the after the fact scanning. Copying will happen anyway.
2 Likes
Deep down this whole thing come from the fact that we take the input as a git repo instead of a hash of a bundle of files.
This is my biggest problem with flakes. Treating a flake as a git repo should be an optimisation, not the default.
4 Likes
It is not the default.
The default is to infer the type if none is specified.
So nix will first check if the potential flake location is path-like. If it is, it will be checked whether or not this path is part of a source code repository. If it is, (git or mercurial or svnor whatever else is supported) then the repo type will be used.
If its not, the path will be interpreted as that… path:.
If that can not be resolved into a valid flake, flake: is infered and checked against the registry.
So git is not a default, but “prefered if available”.
If you do not want to use git (for whatever reason) for a local checkout/clone, use path: explicitely.
5 Likes