# Secrets in Nix Suck (and how to fix them)

**URL:** https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822
**Category:** Nix
**Created:** [April 21, 2024, 2:46pm UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822 "2024-04-21T14:46:28Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![woobilicious](https://discourse.nixos.org/user_avatar/discourse.nixos.org/woobilicious/32/5279_2.png) [@woobilicious](https://discourse.nixos.org/u/woobilicious)
#### Post date: [April 21, 2024, 2:46pm UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/1 "2024-04-21T14:46:28Z")

</div>

# Premise

Currently it’s _really really_ easy to accidentally put your secret keys in to the nix-store, there’s plenty of [bad advise](https://discourse.nixos.org/t/using-an-external-secret-file-in-a-nix-sandboxed-build/3274/3) around, especially for resource acquisition, nix is great until you can’t use a fetcher, and then there’s a billion ways to mess up manually acquiring resources, It’s also just a pain in the ass (_coughheavilymoddedfactorioserver_), nix can fetch resources, and it has access to every application that one might use to download, the issue is that you cannot cleanly, nicely use them inside the sandbox.

If you want to see a package that does everything wrong, see the [Factorio Downloader](https://github.com/NixOS/nixpkgs/blob/41992852aa6862dcd63681b0c0433be3818a8ea0/pkgs/games/factorio/default.nix#L120-L124) that writes secrets in to the nix-store while trying avoid transient issues.

The entire point of nix to me is that I can type `nix build` and it builds an entire workable package, or full system, I should not need to wrangle all sorts of weird things, that otherwise could be just a prompt.

A solution should not leak secrets to arbitrary users via nix-build, it should not need raised privileged, and it should not need me to do non-nix things, or manage a reverse-proxy, or accidentally write your secrets to bash history.

Current solutions include:

- NIX\_PATH

- netrc

- impureEvars

- ad-hoc scripting to load the files in to store manually

- Setup a http proxy injecting headers or full custom backend

- providing secrets over http/udp/tcp

- extra-sandbox-paths

## Actual solution

Nix really needs a way so that with minimal setup `nix build` can work, providing a prompt to the user if possible, and then passing these secrets from the front end, to the builders safely and securely.

It seems like a MVP could be constructed using `pre-build-hook` and `post-build-hook`, and some per-derivation sandbox paths, Still not exactly sure how to authenticate which derivation gets which secrets. but it seems trivial to implement.

But this would need to be added to `nix` itself, so that nixpkgs could start accepting fetchers of this sort, I assume any packages that use these fetchers would be no different to the requireFile, except you could wrap your package in a “secret provider” override for nixos-config, it could be easily documented in nixpkgs, the fetchers that use this functionality can be _audited_ by nixpkg maintainers _in one location_ instead of the many different ad-hoc solutions inside nixpkgs or outside it.

---

<div class="post-metadata">

### Author: ![ElvishJerricco](https://discourse.nixos.org/user_avatar/discourse.nixos.org/elvishjerricco/32/984_2.png) [@ElvishJerricco](https://discourse.nixos.org/u/ElvishJerricco)
#### Post date: [April 21, 2024, 4:23pm UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/2 "2024-04-21T16:23:30Z")

</div>

There is no solution. The entire premise of binary caching is that all aspects of a build are public knowledge. There’s no way around this. Secrets is a stateful problem, fundamentally unsuitable for Nix to solve.

---

<div class="post-metadata">

### Author: ![rhendric](https://discourse.nixos.org/user_avatar/discourse.nixos.org/rhendric/32/5699_2.png) [@rhendric](https://discourse.nixos.org/u/rhendric)
#### Post date: [April 21, 2024, 5:01pm UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/3 "2024-04-21T17:01:51Z")

</div>

> [@ElvishJerricco](#):
>
> Secrets is a stateful problem

This. If your secrets don’t need to be read at build time, put them where you put the rest of your application state (`/var`), not where you put your immutable, world-readable software. If your software really expects those secrets to be in a particular store location, symlink the `/var` location into the store. If your software requires the secrets to live in a configuration file along with a bunch of other things that aren’t secret, petition your maintainer to fix that or patch it yourself, because that’s just not great design in general.

The catch is if you’re using secrets to sign software; that’s when `extra-sandbox-paths` or secret servers can be appropriate.

---

<div class="post-metadata">

### Author: ![volth](https://discourse.nixos.org/user_avatar/discourse.nixos.org/volth/32/272_2.png) [@volth](https://discourse.nixos.org/u/volth)
#### Post date: [April 21, 2024, 7:52pm UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/4 "2024-04-21T19:52:39Z")

</div>

As the author of the mentioned bad advise, I primarily meant setting root’s and user’s passwords, which finally end up in Nix Store in hashed form and symlinked from `/etc` – the existing NixOS design.

`builtins.readFile` let to put the secrets separately from the `*.nix` codebase, into `git-crypt`’ed directories, etc; so, it is about how to manage the server’s secrets on the workstation we develop server’s software on, not about how to deploy the secrets onto the server and not about how to avoid having plain-text secrets in world-readable `.drv` files: there are at least 3 separate issues with the secrets.

---

<div class="post-metadata">

### Author: ![Infinisil](https://discourse.nixos.org/user_avatar/discourse.nixos.org/infinisil/32/26523_2.png) [@Infinisil](https://discourse.nixos.org/u/Infinisil)
#### Post date: [April 21, 2024, 7:59pm UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/5 "2024-04-21T19:59:50Z")

</div>

Related RFC:

> <https://github.com/NixOS/rfcs/pull/143>
>
> \# Summary
> 
> This RFC proposes a system of Access Control Lists in the Nix Store…, allowing multiple users to share software in a single store without exposing it to each other.
> 
> \[Rendered\](https://github.com/tweag/rfcs/blob/acls/rfcs/0143-nix-store-acls.md)

And implementation PR:

> <https://github.com/NixOS/nix/pull/9287>
>
> \# Motivation
> 
> 
> Add functionality to manage POSIX ACLs (access control lists) on …Nix store paths (including \`.drv\` files) and derivation build logs.
> 
> In particular:
> 
> + Adds a \`nix store access\` command with the following subcommands:
> - \`info\`
> - \`protect\` / \`unprotect\`
> - \`grant\` / \`revoke\`
> + Adds a \`--protect\` flag to \`nix build\` and \`nix store add-\*\`
> + Adds \`\_\_permissions\` argument to \`builtins.derivation\`, and \`permissions\` argument to \`builtins.path\`, which allow to control the permissions on corresponding store objects.
> 
> For now, all of these are hidden behind \`--experimental-flags acls\`
> 
> \# Context
> 
> 
> https://github.com/NixOS/rfcs/pull/143
> 
> 
> 
> \## Implementation strategy
> 
> \- Add a C++ interface for POSIX ACLs
> \- Add a Nix data structure to describe ACLs of a store path (\`AccessStatus\`), which can be 
> + \`protected\` (if NOT protected, readable and executable by everyone; else readable and executable only by entities)
> + \`entities\` (list of entities (users and groups) that have access to a path if it is protected)
> \- Implement setting and getting \`AccessStatus\` on a local store and remote daemon store
> + If the path/derivation does not exist yet, apply the AccessStatus as soon as the path appears
> \- Add CLI subcommands and language primitives to manage \`AccessStatus\`-es
> 
> 
> 
> 
> 
> \# Priorities
> 
> Add :+1: to \[pull requests you find important\](https://github.com/NixOS/nix/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc).

---

<div class="post-metadata">

### Author: ![bobvanderlinden](https://discourse.nixos.org/user_avatar/discourse.nixos.org/bobvanderlinden/32/59_2.png) [@bobvanderlinden](https://discourse.nixos.org/u/bobvanderlinden)
#### Post date: [April 21, 2024, 8:25pm UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/6 "2024-04-21T20:25:52Z")

</div>

Just to mention another thing to take into consideration: with most of the options and the solution, the secret is available inside a build environment, which can thus leak to the build output on accident.

That isn’t the case for netrc with builtins.fetchurl. The secret never touches the build environment. Nixpkgs fetchers are based on being a derivation, which does depend on having the secret in a build environment.

That said, I do like your suggestion. It would indeed be really nice if the Nix frontend would be able to pass through secrets. Better multi-user experience and isolation. It helps the UX quite a bit as well.

---

<div class="post-metadata">

### Author: ![woobilicious](https://discourse.nixos.org/user_avatar/discourse.nixos.org/woobilicious/32/5279_2.png) [@woobilicious](https://discourse.nixos.org/u/woobilicious)
#### Post date: [April 22, 2024, 4:05am UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/7 "2024-04-22T04:05:10Z")

</div>

Security is more important than 100% hermetic build systems, and none of this matters for FODs.

And you’re missing the forrest for the trees, nix is meant to be public yes, so we should embbed as much _public facts_ about our build process as possible, _requireFile_ is a UX nightmare _because_ it doesn’t document enough about the build process, it explicitly requires _extra knowledge_ to be passed via a side channel. This is stupid.

Factorio mod downloads needs secret passed in to a http custom header, nix should _describe this fact_ and handle it, it should _just work_ with tools like `nixos-rebuild build --target-host`, and nixops and other “nix deployment tools”.

An impure side channel for secrets, on the file system, is _just as bad_ for reproducibility as http-proxy injection, and netrc for inputs that require secrets, that are currently allowed in FOD’s, the issue is that they’re inflexible and end up with hacky broken solutions that put secrets in to the build system by accident (Which also breaks reproducibility).

---

<div class="post-metadata">

### Author: ![woobilicious](https://discourse.nixos.org/user_avatar/discourse.nixos.org/woobilicious/32/5279_2.png) [@woobilicious](https://discourse.nixos.org/u/woobilicious)
#### Post date: [April 22, 2024, 4:10am UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/8 "2024-04-22T04:10:36Z")

</div>

This does sound good, especially for stuff like nixos-config files (like icecast password inside xml), I’m very much in favour of anything that is transient remaining as transient as possible, ideally we should avoid ever storing passwords / secrets on disk in the nix-store.

---

<div class="post-metadata">

### Author: ![woobilicious](https://discourse.nixos.org/user_avatar/discourse.nixos.org/woobilicious/32/5279_2.png) [@woobilicious](https://discourse.nixos.org/u/woobilicious)
#### Post date: [April 22, 2024, 4:33am UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/9 "2024-04-22T04:33:17Z")

</div>

`readFile` is read in to nix var, and all nix vars are unsafe, and end up being written to your nix-store. And NixOS does support directly specifying hash/KDF’s, which should be better here.

I usually don’t consider low security level users as “trusted” these days.

---

<div class="post-metadata">

### Author: ![woobilicious](https://discourse.nixos.org/user_avatar/discourse.nixos.org/woobilicious/32/5279_2.png) [@woobilicious](https://discourse.nixos.org/u/woobilicious)
#### Post date: [April 22, 2024, 4:41am UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/10 "2024-04-22T04:41:46Z")

</div>

My entire OP is about secrets in builders, especially fetchers…if we throw our hands up in the air and say “can’t be improved” then people will not use nix/NixOS because if you’re build process requires nix and non-nix tooling, then you have the worst of both worlds, the answer will be “don’t use nix, use something that is easy to reason about from a security perspective”.

---

<div class="post-metadata">

### Author: ![RaitoBezarius](https://discourse.nixos.org/user_avatar/discourse.nixos.org/raitobezarius/32/1835_2.png) [@RaitoBezarius](https://discourse.nixos.org/u/RaitoBezarius)
#### Post date: [April 22, 2024, 9:04am UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/11 "2024-04-22T09:04:19Z")

</div>

> [@ElvishJerricco](#):
>
> There is no solution. The entire premise of binary caching is that all aspects of a build are public knowledge. There’s no way around this. Secrets is a stateful problem, fundamentally unsuitable for Nix to solve.

> [@rhendric](#):
>
> The catch is if you’re using secrets to sign software; that’s when `extra-sandbox-paths` or secret servers can be appropriate.

(I agree with you rhendric on that.)  
I would say, there’s only one exception to all of this.

Take for example: [https://packages.debian.org/sid/grub-efi-amd64-signed](https://packages.debian.org/sid/grub-efi-amd64-signed) – this is signed in the build environment of Debian, a random user cannot “reproduce” this package.

We don’t have similar packages yet in nixpkgs, but in the future, it’s possible to envision that [hydra.nixos.org](http://hydra.nixos.org) or similar can have `extra-sandbox-paths` to some control socket of some HSM or signing server to sign artifacts that are de-facto impossible to reproduce.

But @mschwaig provided some interestings leads regarding this problem: [https://www.youtube.com/watch?v=-CUa3yVTK5U&list=PLgknCdxP89ReD6gxl755B6G\_CI65z4J2e&index=20](https://www.youtube.com/watch?v=-CUa3yVTK5U&list=PLgknCdxP89ReD6gxl755B6G_CI65z4J2e&index=20).

> [@woobilicious](#):
>
> But this would need to be added to `nix` itself, so that nixpkgs could start accepting fetchers of this sort, I assume any packages that use these fetchers would be no different to the requireFile, except you could wrap your package in a “secret provider” override for nixos-config, it could be easily documented in nixpkgs, the fetchers that use this functionality can be _audited_ by nixpkg maintainers _in one location_ instead of the many different ad-hoc solutions inside nixpkgs or outside it.

FYI, nixpkgs has strict compatibility constraints w.r.t. Nix. Even if it was added to Nix, it would take a long time to use it.

---

<div class="post-metadata">

### Author: ![Yajo](https://discourse.nixos.org/user_avatar/discourse.nixos.org/yajo/32/5602_2.png) [@Yajo](https://discourse.nixos.org/u/Yajo)
#### Post date: [April 29, 2024, 3:41pm UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/12 "2024-04-29T15:41:51Z")

</div>

FWIW you can use `builtins.fetchGit` to fetch your private code using your local Git auths, such as SSH keys.

Of course, the private code itself lands in the nix store. I don’t know what’s worse. But what else could it do? 🤷‍♂️

---

<div class="post-metadata">

### Author: ![MatthieuB](https://discourse.nixos.org/user_avatar/discourse.nixos.org/matthieub/32/7083_2.png) [@MatthieuB](https://discourse.nixos.org/u/MatthieuB)
#### Post date: [April 29, 2024, 7:17pm UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/13 "2024-04-29T19:17:35Z")

</div>

On servers sops-nix does the job for me.

On desktops, I found a trick by having only secrets encrypted with gpg in the nix-store, and using some shell wrappers to decrypt them with a yubikey. It doesn’t fit for all usages, but it works well for me for wireguard and other stuff, the secrets are decrypted/read on demand and only used in memory. That’s the most secure solution I found, but it’s pretty hacky 🤯

---

<div class="post-metadata">

### Author: ![andrewhamon](https://discourse.nixos.org/user_avatar/discourse.nixos.org/andrewhamon/32/8387_2.png) [@andrewhamon](https://discourse.nixos.org/u/andrewhamon)
#### Post date: [February 1, 2025, 4:05am UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/14 "2025-02-01T04:05:16Z")

</div>

builtin fetchers block the nix evaluator, which is very bad for eval performance (and would mean that you can not evaluate nix expressions without being able to authenticate to your build resources).

---

<div class="post-metadata">

### Author: ![andrewhamon](https://discourse.nixos.org/user_avatar/discourse.nixos.org/andrewhamon/32/8387_2.png) [@andrewhamon](https://discourse.nixos.org/u/andrewhamon)
#### Post date: [February 1, 2025, 4:12am UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/15 "2025-02-01T04:12:09Z")

</div>

I think there is a narrower case here which would still prove extremely useful without as many pitfalls: allow secret passing only in fixed-output-derivations.

FoDs already suffer from non-reproducibility. There is no guarantee that the net will give back the same bytes. We accept this because the fixed-output guarantees that we will detect when we fail to produce the expected output.

So allowing secret passing to a FoD wouldn’t make it any less pure than it already is(n’t). It would provide tremendous utility without significantly devaluing Nix in use cases that don’t require secrets (i.e. it shouldn’t cause harm to nixpkgs).

---

<div class="post-metadata">

### Author: ![hacker1024](https://discourse.nixos.org/user_avatar/discourse.nixos.org/hacker1024/32/7091_2.png) [@hacker1024](https://discourse.nixos.org/u/hacker1024)
#### Post date: [February 6, 2025, 1:08am UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/16 "2025-02-06T01:08:56Z")

</div>

We have another use case for secrets in the sandbox: Custom NixOS VM tests.

Our software relies heavily on third party APIs, and we’d like to run full end-to-end tests, from the OS up. Internet access can be enabled in the sandbox fairly easily, but there’s no straightforward way to use API keys.

Output “reproducability” is not really a concern here, because no-one’s using the output of the test derivations (aside to check the occasional screenshot). All we care about is whether the test passes (so the derivation builds successfully) or fails. Like a FOD, we don’t really care how we got there (what secrets we used) if it passes.

---

<div class="post-metadata">

### Author: ![TLATER](https://discourse.nixos.org/user_avatar/discourse.nixos.org/tlater/32/4257_2.png) [@TLATER](https://discourse.nixos.org/u/TLATER)
#### Post date: [February 6, 2025, 3:00am UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/17 "2025-02-06T03:00:28Z")

</div>

To be fair, that’d be a very flaky set of tests. Usually the solution to this kind of thing is mocking.

---

<div class="post-metadata">

### Author: ![hacker1024](https://discourse.nixos.org/user_avatar/discourse.nixos.org/hacker1024/32/7091_2.png) [@hacker1024](https://discourse.nixos.org/u/hacker1024)
#### Post date: [February 6, 2025, 3:01am UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/18 "2025-02-06T03:01:33Z")

</div>

Understood, and we have many tests with mocks as well. But at some point we _do_ need to test the system as a whole before release.

---

<div class="post-metadata">

### Author: ![rhendric](https://discourse.nixos.org/user_avatar/discourse.nixos.org/rhendric/32/5699_2.png) [@rhendric](https://discourse.nixos.org/u/rhendric)
#### Post date: [February 6, 2025, 3:02am UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/19 "2025-02-06T03:02:55Z")

</div>

> [@hacker1024](#):
>
> But at some point we _do_ need to test the system as a whole before release.

That doesn’t have to be part of the build process though, does it? I would make that a separate derivation that produces a test executable or script that is run entirely outside of the sandbox.

---

<div class="post-metadata">

### Author: ![hacker1024](https://discourse.nixos.org/user_avatar/discourse.nixos.org/hacker1024/32/7091_2.png) [@hacker1024](https://discourse.nixos.org/u/hacker1024)
#### Post date: [February 6, 2025, 3:03am UTC](https://discourse.nixos.org/t/secrets-in-nix-suck-and-how-to-fix-them/43822/20 "2025-02-06T03:03:33Z")

</div>

We could, but that becomes harder to integrate with our existing CI systems (Hydra + GitLab integration) that are built on derivations.

It’d also make it harder to take advantage of the existing NixOS VM testing infrastructure. We are building an embedded device, so testing the whole OS is appropriate.
