How should we handle software created with LLMs?

Well, given that Ghostscript’s -dSAFER is not likely to ever have been safe against malicious inputs but we only mark it insecure when someone bothers to report a specific exploit, sure insecure is not a super reliable marker.

Not sure we have a chance to maintain a useful exposure-weighted contamination list in the case of absence of any public legible data, given the precedents of other metadata.

(And yes, given the absence of a policy, I have by now merged at least one LLM-generated PR to Nixpkgs where the submitter did understand the feedback and did make sure it is applied — and obviously have seen PRs with which submitter was not able to do requested changes properly so they went nowhere)

2 Likes

One place where I’ve used Claude Code when contributing to nixpkgs is to create the VM test in this PR: nixos/postgresql: don't perform postgresql-setup when in standby mode by bouk · Pull Request #469863 · NixOS/nixpkgs · GitHub

I changed the module and had it create the NixOS regression test (it iterated on a couple different versions until it worked). Without an AI coding tool this PR would’ve probably landed without the test, but this is obviously better and was still low-effort. I (and the PR reviewer) still reviewed the test of course.

It’s now much easier to expand the testing we’re doing, which will reduce bugs in NixOS.

1 Like