I would argue that it is even more important to take a firm stance against use of generative LLMs and agents by people with a “proven track record”, because a one-off “agentic” contribution by someone outside the project is likely to be something simple and context-free, like a package update, that is feasible to review despite having no mind behind it. People with proven track records take on ambitious changes. Changes where the context and change history matters. Changes where, in practice, nobody actually reads every line of the code looking for every last bug, they rely on the reputation of the human author and the shared understanding of what the code’s supposed to do that you develop by talking through it with them. That works (better than anything else we’ve tried) as long as there’s actually another human being on the other end of the line.
But LLMs have no long-term memory. The “context” doesn’t feed back and modify the weights, it gets discarded at the end of the dialogue. So all those decisions you (the reviewer) thought you were making, in the review conversation, about how the code ought to be maintained going forward: if the author is laundering LLM output, you might as well have been pissing into the wind.
If you have a trusted maintainer laundering slop LLM output, you have a much bigger problem than AI. At this point said maintainers is not not adhering to the current AI policy, never mind a harsher one, despite being educated about it. This is no longer an act of carelessness. They could also just write bad code by hand or even inject malicious obfuscated code into your code base (hello xz).
This is an issue of trust in people which has nothing to do with the technology. Forbidding maintainers access to a tool because they could do bad things with it, is ultimately saying that you don’t trust the maintainers. At that point this becomes a different discussion entirely.
There are very legitimate reasons to be opposed to AI usage, and the general impact it has on open source software (and other parts of our life). This is not one of them.
Yes. It is my considered professional opinion as both a software engineer and a cognitive scientist that no one can be trusted to use LLMs to do nontrivial software development work, because “find the bugs in a large body of code that looks mostly okay but might have subtle problems somewhere” is something that human beings are bad at. And because it’s easy to fool yourself into thinking you are checking the large body of LLM-generated code more carefully than you actually are.
If that is your opinion please communicate that directly next time. The way this was structured as an answer to chemonke led me to believe that you implied that this applied specifically to repeated contributors as an issue of trustworthiness, not generally to everyone as an issue of capability.
Edit: I apologize that came out harsher than intended.
I wonder how this thread will look in 5 years. We are far past the point whether LLMs are useful. Outright banning LLMs will leave NixOS project boxing with one arm behind the back. And I think this poll all but helps bringing discussion about the real issues (I am guessing that sloppy overly large PRs are one), instead it polarizes people further. Of course that was not the author’s intent.
LLMs won’t replace judgement. An idiot with a tool is still an idiot. Yet I sense here deeply religious sentiments. It sounds to me like how some people would like to keep Nix/NixOS an elitist project for those with Masters Thesis in pure functional programming.
I think I would have given up NixOS without LLMs that have been hugely useful in debugging my config and helping me to grasp the Nix-language.
I love the idea of Nix and NixOS. My biggest concern have always been the community “culture wars”.
I disagree with LLMs being anywhere near this project, but ive madw that abundantly clear.
I will respond to one thing:
It sounds to me like how some people would like to keep Nix/NixOS an elitist project for those with Masters Thesis in pure functional programming.
No, in fact I dont have a masters degree myself. I was planning to, but I couldnt stand the bullshit.
A lot of folks that are well respected dont have a masters degree, many dont have a bachelor either. Education isnt a requirement to participate and never will be.
I am sorry that you had to resort to LLMs to debug your config. I view that as a failure of the Nix community in two different ways:
Our documentation wasn’t sufficient to help you
We didn’t build a friendly enough space for you to reach out
Please reach out to a human instead of an LLM, we would all be happy to help you and perhaps build a friendship on a mutual interest (Nix or other things).
Just why would I reach out to the community where even a quick answers mean 30min of waiting if my option is to ask LLM and get an immediate answer?
NixOS documentation leaves a lot to hope for, yet the repo itself is very understandable by the LLMs. I’m a person who always wants to understand how things work instead of outsourcing all thinking to LLMs. I use LLMs to learn and ask them references to documentation or source code.
And why does it bother you if my LLM-assisted configs are applied to the computers I own? It’s kind of my private matter, isn’t it? I get it it’s a different thing if I start to contribute sloppy AI-PRs to NixPkgs. I definitely won’t do that.
And apologies for sounding bit hostile. From my point of view people seeking to “ban LLMs” indirectly are about “to take NixOS away from me”. NixOS is a big deal for me. I have used Linux since 1996 and tried many distros. Yet NixOS blew my mind and I fell in love with it.
Just why would I reach out to the community where even a quick answers mean 30min of waiting if my option is to ask LLM and get an immediate answer?
Because we would be happy to help you and the reason why i know so many great folks online and many now offline, is because I asked for help. I am just worried about what we’re loosing as a community and society by talking to statistical models instead of each other.
NixOS documentation leaves a lot to hope for, yet the repo itself is very understandable by the LLMs. I’m a person who always wants to understand how things work instead of outsourcing all thinking to LLMs. I use LLMs to learn and ask them references to documentation or source code.
Every person asking an LLM burns energy and their money instead of us having docs and everyone being able to cheaply seatch those. Generating the docs with an LLM would still be better for the environment than everyone asking their own LLM.
NixOS documentation leaves a lot to hope for, yet the repo itself is very understandable by the LLMs. I’m a person who always wants to understand how things work instead of outsourcing all thinking to LLMs. I use LLMs to learn and ask them references to documentation or source code.
I mean as you said, I don’t. I would for my own code, because code I write I license under a strong copyleft license, which should prohibit training an LLM on said code, who am I to know that by copy pasting my code into a LLM they don’t treat it as free real estate for training .
But in the nixpkgs I truly shouldn’t care and don’t have the right to prohibit you anyway. But as I have said, i fear what we’re loosing here as a society and community.
And apologies for sounding bit hostile. From my point of view people seeking to “ban LLMs” indirectly are about “to take NixOS away from me”. NixOS is a big deal for me. I have used Linux since 1996 and tried many distros. Yet NixOS blew my mind and I fell in love with it.
No worries, you did sound a bit hostile, but so do I a lot of the time
This anti-LLM mood feels a lot like the old monolithic vs microkernel, Linux vs BSD, Vi vs Emacs arguments. I am also concerned of LLM use without human judgement and getting flooded buy sloppily written documents, emails and messages. But yet LLMs are technical progress and very useful when used correctly. The fight against LLMs is a futile fight against the windmills.
This sentiment resembles the emergence of other technologies like how e.g. UK introduced a “Red Flag Law” (Locomotives Act 1865) when the first self-propelled (steam) vehicles emerged at the roads requiring a red flag being carried in front of the vehicle. I would love to see the focus being in how LLMs could be useful and how to limit the adverse effects. My 50c.
This sentiment resembles the emergence of other technologies like how e.g. UK introduced a “Red Flag Law” (Locomotives Act 1865) when the first self-propelled (steam) vehicles emerged at the roads requiring a red flag being carried in front of the vehicle. I would love to see the focus being in how LLMs could be useful and how to limit the adverse effects. My 50c.
And wouldn’t you know it, the red flag law was correct. Personal cars should be banned from all cities. The earth is here for creatuers (including humans) to walk on, not cars to drive on.
Unclear, published Nix configs seem to be easier to average into something pointing to the correct function of the source code than the talking-past-each-other flamewars or firehose ten-lines-of-discussion-hashed-together (and I don’t think there are any large Nix-related discussion channels that are on the open web but don’t have either property)
I would imagine that help questions and answers would be more closely associated with each other, and data from these threads, while presumably part of the same dataset, would likely not be invoked. But I haven’t looked deeply into the implementations of these models in the last few years, so my knowledge might be outdated on that All I’m familiar with is their output and very high-level theory, which from experience, requires massive cleaning up and double-checking to the point that I haven’t found them acceptable for use in a large complex codebase with limited training data (especially given a fair % of nix code on github doesn’t eval).
EDIT: to be clear, my knowledge of LLM output is not outdated, as I observe it daily. I just have not read the maths papers backing their design.
.
