Best practice for managing individual secrets on shared nix flake project

Help
1

Hi,

I am part of a team migrating to using Nix flakes for our development environments and deployments. One of the things we need to solve is secret management. Having done some research, it seems like agenix and sops-nix are the generally recommended ways of sharing secrets across machines.

From what I am able to gather, these tools are mainly intended for shared secrets. However, our infrastructure also relies on developer-specific API-tokens for accessing our internal repositories. Before moving to Nix flakes we used direnv to load the developers’ own tokens into the environment variables of the development environment, which could be read from the Makefile. However, Nix flakes consider environment variables to be impure, and therefore they are not visible within the flake.

So the problem we’re trying to solve is how to reference a secret by a single name in the Flake (eg. “GIT_TOKEN”), even while it refers to different concrete tokens on each developer machine/when deployed. Admittedly this feels like an impure requirement, however I feel like it is not so unusual a requirement that nobody has encountered it before.

Is there a standard way to handle secrets like these in Nix flakes?

2

Use .env files, which you not commit but do load through the .envrc.

Direnv has a failing and a silenced tool to load the .env in it’s stdlib.

3

It might just be that I’m not understanding your suggestion, but I don’t see how this helps this problem.

The issue is not that we cannot place our tokens into the environment variables, the problem is that in Nix flakes, attempting to access environment variables (for example through builtins.getEnv) simply returns an empty string, because it is considered impure to depend on machine-specific environment variables. So what we’re looking for is a way to do this without relying on environment variables. I apologise if that was unclear from my original question.

4

Avoid anything that needs the secrets to be done in nix.

1 Like
5

netrc probably? nix.conf - Nix 2.34.7 Reference Manual

6

I.e., write scripts that access those secrets at runtime, don’t bake secrets into derivations. Baking secrets into derivations isn’t just bad practice, it leaks those secrets.

Scripts that use the secrets at runtime can be executed with e.g. direnv. But to give more specific advice we really need to see what kind of use cases you have for reading these secrets.

2 Likes
7

Not sure if this would fit your environment.

I use the command line tool pass to provide secrets either as argument (<command> $(pass show <secret>) or as input (pass show <secret> | <command>).

A git token could be provided with: git clone https://token:$(pass show <token>)@<gitserver>/<owner>/<repo>.git

The pass store (= encrypted) can be shared in your team (I use git to share the store with my laptops and workstation).

8

Our concrete use case is that we use uv2nix to build our python project workspace in our Nix flake. However, some of our dependencies are for other projects on our internal git infrastructure. These require authentication using a per-developer access token. When running uv as a standalone package, it seems like this token is being read from the environment variables (or .netrc, I’m not entirely sure actually) correctly. However, since uv2nix abstracts this away within the flake, it cannot see my machine specific .netrc or environment variables.

According to the documentation for uv2nix, we should be able to make it use our local netrc-file by creating a custom overlay like so:

manualOverrides = final: prev: {
  internally-hosted-py-lib = prev.internally-hosted-py-lib.overrideAttrs(old: {
    src = old.src.overrideAttrs(_: {
      curlOpts = "--netrc-file /home/user/.netrc";
      SSL_CERT_FILE = "${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt";
    });
  });
};

However, this fails with an the following error: curl: (26) .netrc error: no such file. I assume this is for a similar reason of impurity as is the case when loading environment variables.

That failing, we tried simply passing the credentials directly to curl. In the following example, glUser and glToken are the environment variables loaded with builtins.getEnv. This is the one that fails because it results in empty strings.

manualOverrides = final: prev: {
  internally-hosted-py-lib = prev.internally-hosted-py-lib.overrideAttrs(old: {
    src = old.src.overrideAttrs(_: {
      curlOpts = ''-u ${glUser}:${glToken}'';
    });
  });
};

It is a good point that we should not bake secrets into derivations. The difficulty is that we aren’t the ones in control of when the secrets are loaded, since that is abstracted away by uv2nix, hence our issues.

EDIT: Updated to absolute .netrc path, since ~ is not resolved - for clarity, the absolute path is resulting in the same error.

9

Right, so you’re working around an issue you’ve identified the root cause of. I figured this’d be XY.

Mind opening a new post about the actual root cause? I’ve not used uv2nix, and the kind of people who will read this post are less likely to have experience with it, too.

Most likely it’s a small mistake somewhere that can be fixed with a one line change to docs or a script somewhere, or maybe you’re holding it wrong. If we fix whatever causes uv2nix to fail here you can actually securely manage your secrets.

1 Like
10

Sure, that’s probably a good idea. Thank you for the suggestion and the help. I’ll get that done first thing tomorrow :blush:

2 Likes
11

From the URL you linked:

Your best bet is to give each user a standard (non user-dependent) netrc path (preferably /etc/nix/netrc) and put their netrc there, then add it to extra-sandbox-paths in nix.conf.

14

We’ve released https://secretspec.dev a while ago to address this (as it support multiple providers) and secretspec - devenv to integrate it with Nix.