Tip: keeping AI API keys and endpoint config out of my Nix configs

Guides
1

Small tip that took me embarrassingly long to figure out, sharing in case it helps others who mix LLM tooling into their NixOS setup.

I use a couple of little helper scripts that call LLM APIs (commit message generation, log triage). My first approach was hardcoding the API base URL and key into my scripts, then keeping the scripts out of git. That was ugly: my Nix expression for the machine and my actual scripts drifted apart, and swapping providers meant editing several files.

What I settled on:

1. The scripts read two environment variables (API_BASE, API_KEY) and nothing else.

2. Those variables are set through my Nix configuration, so the whole machine stays declarative while the secrets live in one place outside the repo.

3. For the endpoint itself I use an aggregated gateway (y-api.bestvirtualgoods.com) with an OpenAI-compatible interface. The nice property for a Nix setup: the “model” becomes just another config option. I can switch to a cheaper or faster model by changing one line in configuration.nix instead of editing every script.

Since everything is OpenAI-compatible, none of the tooling cares what the upstream actually is. Rolling back a bad model choice is now literally `nixos-rebuild switch` to the previous generation.

Not affiliated with the gateway, just describing my setup. Curious how others handle API keys in their Nix configs - sops-nix feels like overkill for one key, but maybe I’m wrong.

1 Like
2

Going further, it’s not the best security if you’re setting secrets via env vars system-wide. Every process on your system will have access to it and can easily leak it. For example a program that crashes and dumps telemetry which includes all env vars and then sends it to a remote, all automatically.

Instead, (if you still don’t want to use agenix/sops) a better idea would be to create a secret file where only your user can read, and read it from your scripts. That’s what agenix/sops in part does, but here you would do it manually.

3 Likes