Public, reusable flake with private parts

Help
1

I am looking for a way to have a public flake that is reusable as inputs in other flakes, but some parts of the flake are private. I think this is a very common thing, people sharing their config, but don’t want to share their secrets of course or some network settings. If those private parts are just plain files, then agenix or sops can be used, encrypted versions of those secrets are published (still kinda meh, but okay). The problem arises if some nix code should be concealed as well.

Here are the partial solutions I see to this problem:

Git Annex

Git Annex allows you to mark certain files as ‘large’, causing only a pointer to be stored in git instead of the actual content. By default that pointer is a symlink into .git/annex/objects/..., but that doesn’t work with nix, so one needs to git annex unlock the file into a pointer file with content /annex/objects/<hash>.ext that a git smudge filter turns into the proper content on checkout.

This is what I do currently: https://gitlab.com/nobodyinperson/nixconfig

:white_check_mark: Good:: Any file can be selectively concealed. Only the hash of the file is public. Adding garbage comments to the file helps the brute-forcing paranoia. Already-encrypted public files such as age secrets get another layer of obscurity, which doesn’t harm.
:white_check_mark: Good: Git annex provides many other benefits, most notably a convenience command git annex assist [-m "these changes"] that will pull, merge, push to all configured remotes and ‘do the git stuff’ in one command.
:cross_mark: Bad: If the git tree is clean (everything nice and committed), nix only sees the pointer files as it just copies the git-tracked content into the nix store… That means one needs to (remember to) manually dirtify the tree before doing anything. Very annoying, but workable.
:cross_mark: Bad: People reusing the flake will hit eval errors because the private nix are not present on their machine (which is good!). Maybe this is handleable with something like this:

nix-repl> isAnnexedPointerFile = path: (builtins.substring 0 15 (builtins.readFile path)) == "/annex/objects/"                        
nix-repl> isAnnexedPointerFile ./modules/secrets/setup.nix
true

and then conditionally importing or doing graceful fallbacks where the secrets are used. I’d have to try though.

Private git repo as input, git submodule or fetched via builtins.fetchGit

{
  inputs.privateStuff.url = "git+ssh://myserver.com/...";
  # or as submodule
  self.submodules = true; # include submodules (build with nix>=2.27!)
}
# or manual builtins.fetchGit

:white_check_mark: Good: This would allow private parts to stay non-public completely (still ending up verbatim in the local nix store of course). No manual git-tree-dirtifying shenanigans necessary as for git annex.
:cross_mark: Bad: Flake is not reusable. People will hit fetch errors. Apparently builtins.tryEval also cannot catch errors like this to handle gracefully?

The public-private git subtree stunt

While writing this up, I just found this approach, with a funny flake.lock-symlinking trick and including a public flake as git subtree (i.e. duplicating it the git way) in a private repo. Maybe I don’t understand it fully, but I don’t really see how that differs from just having one public and one private flake.

:white_check_mark: Good: Public flake is nicely reusable, private stuff stays private
:cross_mark: Bad: Manual work to keep public and private flake pushed and synced, either with subtree or submodule approach. E.g. with the git annex approach, it’s really just git annex assist -m "these changes" and the state is synced to every configured remote. EDIT: :thinking: Although, git annex assist in the private all-encompassing flake should still work as usual, right? And one could selectively push the subtree to the public repo if one feels like it, this reconsideration might also help prevent accidental leakage of stuff that should have landed in the private part…
:cross_mark: Bad?: The public part might need massaging, e.g. a function that produces a certain nixosConfiguration with options for customization. Or maybe just recursive merging can work.

Thinking about it, this last approach might be good. Subtree shouldn’t be necessary, a submodule should be fine or even just using the public repo as url input (kills interactivity though…). And input following setup will be a mess, which is why they symlinked the flake.lock, makes sense…

Any other ideas or approaches I might be missing?

2

It’s not. You are structuring it incorrectly. Have your private flakes use your public flakes, not the other way around.

5 Likes
3

Indeed, wanting a public flake that has some parts concealed seems backwards if you think about it. The proper way is most likely to have a public flake, curated, everything designed to be nicely reusable, and using that flake (as everyone else) in your private flake.

Then, however, you can’t point people to the full configuration you’re using, just the modules, not how they’re applied. And it requires putting work into extracting the public bits in a reusable form. But okay.

1 Like
4

There is nix flake init which let’s you define example configurations that are provided as template.

I’m not sure what benefit a “full” configuration that doesn’t work because the private parts are missing would have beyond that.

1 Like
5

I just mean seeing how the module is used. Sure, seeing the lengthy module definition in the public flake is good, maybe even the inline documentation - if any - but nothing beats an in-use example of how to use a module. But good documentation and maybe the auto-generated nixosOptionsDoc is the key to replace that. nix flake init templates is also a good pointer, indeed, thanks.

6

You can .follows a repository that provides the same modules and fully reuse the flake.

I don’t think reusing someone else’s private flake often makes sense anyway, though, sharing your config is only really useful to help give folks examples.

But yes, as @truh says, doing it the “backwards” way is definitely better from a design perspective. It’s how the module system is intended to be used, and gives anyone who does want to use your config all the power of NixOS to tweak it to their use case.

If you have private stuff you want to be easily configurable, create a custom module for that, and make your private flake only set options in your custom module.

This isn’t true, by the way. You can keep the secrets outside your git repo and just keep them in a local file. Just replace the path with a string pointing at the absolute location of the secrets file.

That does mean you need to hand-copy the file instead of getting it copied around by nix, but honestly that’s probably a better way of managing secrets.

2 Likes
7

It is not that common, I haven’t seen many people who do it. I do it, and did private repo as SSH github input. It had the pitfalls. I’ve since converted my secrets to use absolute paths in sops-nix (which it and agenix can actually do), which “solved” the reusability problem but it introduces its own set of issues.

The flake now has increased impurity, since the secrets are not deterministically built with nix. It is no longer side effect free. However, another consideration is that secrets are not copied to the store anymore, encrypted or not. This may be beneficial to some.

My workaround was to manually deploy secrets to the hosts and put on a timer to periodically pull the needed updates. It’s been decidedly okay but I’ve been running this schema for a week so it’s a bit soon to make wide-scale determinations yet.

1 Like
8

IMHO the public subrepo stunt is the correct approach, and yes, it requires care to do nicely. Which is why I consider anyone taking the time to publish reusable (or at least educational) parts doing great community service — that stuff not often get praise on Discourse, but it doesn’t go unnoticed, people learn by example a lot.

Some handling overhead is unavoidable, but that’s because using Nix is software development. You’d want examples and tests if your private values can’t be shown to readers. And with that mindset it gets easier to imagine the public source code to be your deployment artefact: a CI job could splice it out and push to it a public repository.

Edit: I had misread the @TLATER’s argument; reworded mine.

1 Like
9

Yes, I have started externalising things into https://gitlab.com/nobodyinperson/yannix, which I included with git subtree in https://gitlab.com/nobodyinperson/nixconfig, which I will eventually turn into my private config repo. Thanks for your inputs.

There is no direct solution to my initial question yet though, so I’ll leave this as unsolved for now. Maybe someone shows up with an interesting solution we all haven’t thought of.

1 Like