Regardless of how it was written. I found the content to be useful to invoke thinking about how I want to approach secret deployment in my future workflows.
SecretSpec looks interesting, I plan to look at it in the future.
LLM Assisted? Yes. Slop? I do not think so.
Requested publicly, so I’m responding publicly. I don’t think that’s merited, for a few reasons:
Thanks! We’ve spent a lot of energy making the design sound and well tested.
I was surprised to see there’s even Windows users (also all SDKs are built for Windows) besides Linux and macOS.
What a time to be alive!
I personally wish there was more moderation on people who are against LLMs and rant in any form, because they are derailing the conversations from the topic (while there’s plenty of existing threads to vent).
Ironically, it seems like they argue for higher quality content while not adding much besides repeating the word slop.
Also as @aanderse pointed out, there are unnecessary personal attacks all over the place needlessly tolerated.
I get some people don’t like it, but noone of this is productive. We’re going to keep using LLMs.
my 2c to moderation team
We’re going to keep using LLMs.
Well some of us haven’t started and dont plan to. I do agree with Domen here that the discussion regarding LLMs should have been left at “please disclose”, which Domen has done and I thank him for it.
Thats all from me. So i am in favor of everything LLM related being moved or just straight up deleted as its not relevant anymore.
Agreed that there should be stricter moderation of the off-topic anti-LLM comments. The link you posted is interesting and worth discussing. SecretSpec is interesting and worth discussing. Users of this forum deserve to be able to have those discussions without having to scroll past pages of endlessly regurgitated arguments about AI.
Speaking personally, I see the duty of moderation (in the sense of a debate moderator) as not to put the thumb on the scale on any particular side of the issue, but to provide the structure and protection needed for the best arguments on both sides to surface.
I agree it would be nice, in a librarian-mindset way, for the debate over LLM use to be isolated to dedicated topics and not to flow over into others. I have a hard time seeing how to do that in a way that doesn’t effectively put a thumb on the scale, when those other topics are examples of people using LLMs in the community, and are therefore ipso facto pro-LLM arguments. So implicit pro-LLM arguments can happen anywhere, but their rebuttals are confined to a flame thread? That doesn’t seem like an effective way to get to the best arguments on both sides.
I will moderate personal attacks regardless of who makes them. The closest thing to a personal attack I’ve seen here is the thing about personal anxieties, which to my reading is less than maximally sensitive but it was in response to a person saying ‘my concern: […] I feel like I need […]’; characterizing that as a personal anxiety is not obviously an attack, to me. (Not to be dismissive of the person saying that it was interpreted as a personal attack — your reaction is also valid. Ideally your interlocutor would listen to your reaction, clarify their non-hostile intent, you each would extend good faith to each other, and hands would be shaken all around.)
And to keep this from being more off-topic: further personal feedback to me or @moderation-team in general is welcome via PMs, and if you have anything to say in public about these comments, please use the Meta > Moderation category.
My 2c: I’m happy to not interact at all with LLM posts if they’re stated as such up-front.
+1, and I’d go one step further and expand this to also protect secrets at runtime: secrets don’t belong in applications.
Keys belong in HSMs/TPMs, or exposed via authenticating proxies.
To me you are overreacting to this. He just said that in his opinion people don’t talk like that. That’s all. You can’t remove opinions from people’s mind, and I think this is a free speech forum. He saw enough evidences and pointed them out to be allowed to also drop some gut feelings on his post.
Can you stop please with this non sense? You’re not adding anything to the conversation that is about secrets.
This is actually really nice and the solution makes a lot of sense, I feel like this is the kind of thing that seems so obvious upon seeing it it’s surprising it doesn’t already exist. Great work! I have a couple thoughts on secretspec generally:
First, is secretspec.toml intended to be a stable, versioned schema that you could theoretically interface with without specifically using the secretspec tool? Or is it intended to always essentially be intertwined with whatever version of secretspec you’re using? I see there’s a revision tag, and I saw it should be 1.0, but there’s an example in the docs where revision is 2.1.0 so i’m not really sure what that’s for. If it is actually a schema version, what happens when v1.1 releases? (I ask because tbh the most value I see for me personally is this clearly defined secrets interface, but not necessarily using the tool itself, but maybe that’s the wrong way to look at the project)
Also i’m a bit dubious of require_reason = "agents" being default. This makes secret access function differently based on a third-party heuristic, which is bad for both false positives (could make a CI job fail if detection is too aggressive) and false negatives (agents not providing a reason on access because the heuristic was outdated or something). I’m not familiar with the detect-coding-agent crate but from a skim of the crate listing it’s defo not pitched as a security tool, but here you’re leaning on it for a detection that is almost implied to be a control by the docs.
That’s what I thought when we were solving secrets management in devenv and we made it a separate project so that everyone can benefit from sane secrets management.
That’s the reason it’s called spec, once secretspec 1.0 hits it will be set in stone and new revisions will be made for any changes. At the moment we keep backwards compatibility but we’re adding new fields in each secretspec release.
Addressed in Document revision compatibility · cachix/secretspec@51d2452 · GitHub
It’s a convenient default, I’m happy to make it more robust if you have concrete ideas. That’s why you can set require_reasons = true if you want stronger guarantees.
I agree, I’ve started exploring a modern local keyring provider that is written in Rust and uses TPM by default with 2FA options in factorseal.
I’ve worked on caching and fallback configuration for upcoming secretspec 0.17 as providers like 1Password need like 1s to fetch a secret, so we have to find a way how to bootstrap these from local providers and cache the secrets for some time locally to avoid expensive lookups.
Feedback welcome!
perfect thank you!
I suppose it depends on the threat model. If most people using this are using LLMs in their development workflows, which I’d guess is the case, I’d turn it on by default.
At the very least, the Audit Logging docs imply that logging with reason is on by default:
secretspec records every secret access to a local audit log so you can review, after the fact, what secret was accessed, when, by whom, with what reason, and what the outcome was. Auditing is on by default.
Which is not true, and in the secretspec.toml Reference there’s this:
If your agent isn’t auto-detected, set
SECRETSPEC_AGENT=1(or userequire_reason = trueto require a reason from everyone).
Which I think doesn’t really address the problem. How can I know something autonomous can access secrets without a reason before it’s already done it? I didn’t actually notice before that an auditing failure doesn’t block secrets access so it’s less of a control than I thought (it would be nice if that could be optionally enabled) but hopefully you still get my point that without looking closely it seems like auditing is more of a security feature than it actually is.
Regardless this is just a small thing and I’m still very positive overall on the project, and I’m providing this feedback to hopefully be helpful not just for the sake of being annoying/pedantic. Also these are just my thoughts without much context, and you have far more context on what users will actually want from this, so ultimately you can make a better choice about it than I can :))
I’ve opened Clarify audit reason defaults by domenkozar · Pull Request #203 · cachix/secretspec · GitHub, let me know if that addresses your concerns, I’d like to be precise here what it guarantees and provides.
Concrete feedback is very welcome, thank you for taking the time to write constructive feedback!
I think that is much improved, thanks for taking on the feedback!
I like the idea. As author of the contracts RFC, I’m all for adding interfaces to decouple the code, allowing for multiple backends to fulfill the same role and let the end user choose their preferred backend.
At the start of the blog post, you showed a table summarizing the various mechanisms used to pass secrets for NixOS modules. I assumed somehow SecretsSpec would help those modules that implement a less ideal or just plainly wrong mechanism. Am I understanding SecretsSpec correctly in that it must be implemented in the upstream service directly and cannot be bolted on in the NixOS module?
The idea of secretspec is to fix it upstream.
We’ve since created IPC for resolver and provider that Nix will use.
We provide integrations for docker and git using existing credentials helpers but those protocols are subpar.
There’s also devenv machines which allows you to use secretspec when provisioning NixOS machines.
Fixing upstream software will take time, but with integrations guides and coding agents these days it’s not that much work.
How do you expect upstream to just accept “LLM-generated patch to use our library for secrets management”? /genq
I use LLMs frequently and do like the approach taken by SecretSpec for application-side secret loading. But I do wonder what your approach for convincing people outside the Nix/Cachix/Devenv/SecretSpec sphere to use this will be.