@TLATER I think what @hashelq meant to say is something like “new programmers may not know how to code as well as senior programmers with 10+ years programming experience, but they shouldn’t be discouraged”. It’s not about 10 year old children I believe.
6 Likes
i think the elephant in the room here is the social contract behind collaboration on open-source, consisting of authors, consumers and reviewers.
nixpkgs famously has thousands of open pull requests, so if authors were to widely vibe-code their contributions, it would be understandable if reviewers were to feel such authors were shirking their side of the social contract to shift more burden to the reviewer’s side.
this pre-supposes the capitalist class’s perspective of textile and software as widgets used to further the accumulation of capital.
to be fair, the prevalence of Silicon Valley figureheads in the oligarchy definitely underlines that dynamic – but any of this has little to say as to the intent of our hypothetical open-source contributor.
1 Like
I second the general direction of @sergek’s and @TLATER’s arguments, but try to add a piece of clarity that I’ve been missing in the discussion so far:
The problem with auto-generated code and text is not that it came out of the machine. As many noted, asymptotically it would be indistinguishable from human-produced content anyway. The problem is that our assets are — for me, more than anything else — means of communication between people, and if even a few use it carelessly by flooding it with their unfiltered noise, it degrades the value for everyone.
Beginner developers and new community members may naturally have a lot of misunderstandings, including what can arguably be considered rude — such as spammy LLM use. We can help by making the unwritten or hard-to-find rules explicit and put learning them on the onboarding path. Yes, this means there’d be a soft barrier to active participation, but that’s not necessarily a bad thing if active participants produce high quality content that passive readers would also enjoy.
12 Likes
There are better solutions to this problem than trying to determine whether and to what extent the author used AI in their development process. E.g. GitHub - mitchellh/vouch: A community trust management system based on explicit vouches to participate. · GitHub
2 Likes
Specifically I started this post after seeing an announcement for a new tool that presented itself as an alpha for a marketable production ready piece of software licensed BSL. It was very openly developed using AI with little human oversight (they called it spec driven development, I still take issue with the term). I dug through the codebase a bit and it was a mess, but more than that, it had major security red flags.
I’ve seen an increasing number of AI generated projects, many of which are small scope hobby projects that I don’t mind seeing. But there’s a certain point at which I think the proliferation of AI shovelware in our announcements channels will smother the work of people who actually care and are invested and want to learn from the community.
So, to answer your question, I intended the scope of this topic to be about sofrware announcements made by the authors of said software, specifically in discourse. But I’m just happy to collect people’s thoughts about this and provide a place to talk about AI in the forum.
7 Likes
Thank you Valentin! I think we’re indeed thinking in the same direction, and you indeed add clarity. “Our assets are […] means of communication between people” and, as a corollary, if they should interact with LLMs at all,
…our assets are an LLM Inputs, not LLM Outputs.
IMO vibe coding should be disclosed at least in the body of an announcement, possibly in the tittle, and i things get really bad then maybe even give it a separate section. No need to ban it completely.
I imagined it, and I am fine with it.
3 Likes
Imagine being a newbie programmers who wants to make a Nix tool, and hear something like: “AI slop is not allowed, request rejected“ from maintainers, just because they have some “AI policy“ and you may not know how to code as an 10years+ senior.
I mean… yes? A junior cannot hope to write the same code as someone with 10 year of experience. That’s expected, normal and okay. As a programmer trying to get better, a major part of your betterment is knowing how to pick your battles and picking projects you can complete and won’t just be overwhelmed by. Reminds me of me back in the say trying to setup an email server following guides online, i didn’t learn anything nor did i have a properly functional email server. I skipped ahead way too far. First you gotta learn to walk, then you can start learning to run.
6 Likes
Your motivation makes sense, but in practice we don’t really have a way to classify posts that are “software announcements made by the authors of said software”. Do you have any proposal for how to identify/ classify such posts?
1 Like
To start, I don’t think I’ve ever seen a proper announcement that was not by an author of a tool. Sometimes people will share cool projects they find in Links. Regardless of AI, it’s generally good to disclose if you are or are not the author of what you’re sharing, as thus can and has caused confusion. Even if it’s not an explicit rule. Whether or not someone is the author of a tool they’re sharing is either made obvious explocitly or through context, or corrected within a couple posts when someone assumes wrongly.
Identifying AI work is much easier in practice than in theory. As IT adjacent people here, I think it’s very common for us to look at and bikeshed about the edge cases. But, to barrow a legal maxim, hard cases make bad law. If someone has code that is high enough quality that it could fool someone, and at the same time they’re deliberately obfuscating AI usage to avoid dealing with a hypothetical rule, that’s a hard case we should handle individually. But 9 times out of 10, identifying AI work is simple:
- Is there a claude (or equivalent) folder?
- Does the author openly brag about using an LLM?
- Do commits disclose they were made by an LLM?
- Does the code contain any of the tells that it was made by AI?
- inconsistent style
- makes simple mistakes or wrong assumptions about its domain
- many redundant test cases
- disables things instead of fixing them
- tendency to cross interface bounderies that shouldn’t be crossed
Step four is difficult and prone to false positives, fortunately you almost never need to go as far as step four. And if you do get to step four, you’re better off just giving them the benefit of the doubt.
I’m not really concerned about flagging every individual instance of AI, because that would be a sisyphusean task. Rather, I just want to keep the Announcements category (and Nix forums more broadly) as a place for community to share ideas and projects, instead of a ticker for new AI generated tools that are of dubious quality (on average) and don’t lead to greater contributions to NixOS or the community. That last point is important to me. If a real human is writing software for nix, they ask questions, learn new things which they can use to answer questions, complain about and update the wiki, attend NixCon, make connections, etc. It’s so much more valuable than just the tool itself. That is in addition to the issues I’ve outlined earlier.
3 Likes
@ttamttam1 I honestly agree with your overall sentiment, but I’m just not sure how we could implement your suggestions, because your concepts are ill-defined. I think these are the main issues you would need to clarify:
(1) How do we track the relationship between discourse posts / comments and projects that are linked from those posts?
(2) What exactly is the definition of “AI work” and what is the threshold for this?
– – –
(1) How do we track the relationship between discourse posts / comments and projects that are linked from those posts? And what kind of moderation policy do we want to enforce for those links? You say:
But what exactly is a “proper announcement” and how would we classify this? And why does this requirement only apply to “proper announcements”?
(2) What exactly is the definition of “AI work” and what is the threshold for this? You say:
But what exactly does this mean? What happens if a post / project meets some of these criteria but not all? Also none of the individual criteria are clear to begin with. For example you say:
But how many commits are we looking for? Is it an absolute number or a percentage of commits?
Again, I just want to emphasize that even if I agree with your high level motivation, the devil is in the details. If you could translate your proposal into some very clear criteria like “these posts are allowed vs. these posts are not allowed” it would be easier to debate the merits of such a proposal.
Any post advertising a new project in Announcements. Or new topics that should be moved to Announcements.
I’m not suggesting we do anything about links from individual posts, just new Announcements topics.
That’s not really for me to decide, but the community. My personal opinion is that any llm generated code should come with a disclaimer, others may disagree. But I suggest reading over my “hard cases make bad law” point again. Just because it’s hard to draw distinct lines between things doesn’t mean you shouldn’t moderate it. Let’s look at a curated list from the nixos discourse terms of service:
the Content is not pornographic, does not contain threats or incite violence, and does not violate the privacy or publicity rights of any third party;
We could rules lawyer this all day; how pornagraphic? If something’s considered pornagraphic in one country, but not a another, does it count? What if it contains nuditity but is not pornagraphic? What if it contaons no nuditity but is pornagraphic?
the Content is not spam, is not machine- or randomly-generated, and does not contain unethical or unwanted commercial content designed to drive traffic to third party sites or boost the search engine rankings of third party sites, or to further unlawful acts (such as phishing) or mislead recipients as to the source of the material (such as spoofing);
What if the content is AI generated? What about helpful bots? Does self promotion count as SEO boosting?
you have, in the case of Content that includes computer code, accurately categorized and/or described the type, nature, uses and effects of the materials, whether requested to do so by NixOS Foundation or otherwise.
What does it mean to adequetly categorize code? What is type? What is nature?
We could go on but the fact is that rules are full of vagueries and that’s not usually a mistake, but a feature. It lets things be addressed with nuance instead of unbreakable rules. And even with all of these ill-defined rules, the forums are still (mostly) moderated.
A good idea! Personally I wanted to see what other people were thinking with regards to AI. Making a proposal would be the next step. But that requires a bit more thought and effort. It’s also worth noting that, from a practical perspective, changes to the discourse haven’t been easy to bring about. Take for example the petition to add new announcement subcategories.
1 Like
As a practical note on effecting change, I know the SC is currently internally discussing broad AI questions with respect to various parts of this community. You might want to reach out to your favorite SC member and see if they have your point of view in mind.
5 Likes
In the metadata of a vibecoded package, it can be include that it was vibecoded. Just a simple tag. I feel like this is good enough for transparency for those who care to avoid while letting potential contributors/maintainers know to tread more carefully around the package. This is just for nixpkgs and modules in there. External flake modules in other repos can disclose another way, like talked above.
Enforcement tho, would be a whole another issue as pointed out by tamttam1 above.
The difference is there is a broad consensus on the definition of “pornographic”, but that’s not the case for “vibe-coded”. We could even prove or disprove this via a simple experiment:
–
(1) Collect a set of ~20 images which contain a mix of pornographic, non-pornographic, and borderline images. Ask a random sample of ~100 users from this forum to classify each image into porn vs. not-porn.
(2) Same experiment as above, but classify github repos into vibe-coded vs. not-vibe-coded.
My hypothesis is that you will see broad consensus on (1) but not on (2).
–
Any idea how we could conduct experiment (2)? I’d be happy to help! 
You mean “coded” right? we can drop the “vibe” now…
Umm, there is? I would say this is a definition / threshold that’s shifted substantially even within my lifetime, and I’m not that old 
1 Like
Mhm, we can drop the vibe and replace with slop
10 Likes
Can you please stop discrediting everyone/thing that used the help of an LLM in one way or the other (seen you do that on the other thread as well) and rather make productive comments on the topic?
5 Likes