HuggingFace has announced The Stack - an enormous dataset, composed almost entirely out of stolen data people have not consented to. For GitHub repositories which ended up in it, there’s a way of opting out (after it has already been collected, heh), by creating a PR. I’d like someone with high enough privileges in the NixOS organization to submit an opt out request, as all the repositories under NixOS have been included. I don’t consent to my contributions being in the dataset, but I also don’t have the power to opt out in the name of the organization.
4 Likes
Genuinely curious: why would you (or anyone with the patience to answer) contribute to an MIT-licensed project — a license which explicitly grants anyone who gets a copy of the software the permission to use the software for any purpose — if you do not consent to having your contributions used for this purpose?
13 Likes
Because I don’t consider it to contradictory to wish others being able to benefit from my contributions while not consenting having my contributions be in a training dataset. Especially not without opting into it. Without even mentioning the fact that it’s a blatant MIT violation, despite how permissive said license is.
5 Likes
Sure, those things aren’t mutually exclusive, but the text of the MIT license references neither. It says, ‘Permission is hereby granted […] to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software’. Is that permission, without restriction, granted or not? If not, I would really reconsider participating in any MIT-licensed projects at all, because that’s what the license explicitly says you’re granting! (The copyleft people were right all along…)
Oh? I figured that GPL violations could be present in this sort of thing, but it surprises me to hear that there are MIT license violations as well. Which of the two MIT license conditions are being violated — the inclusion of the copyright/permission notice in the dataset, or the absence of a warranty?
4 Likes
I don’t understand why you’re trying to gotcha me so hard. Yes, the copyright clause is very obviously being violated. Quoting from the page I linked, they brag it’s a “source code across 713 programming languages”. If you look into the dataset itself, it is quite literally a dump of ~all of GitHub. As soon as you begin training, there’s no copyright information to be had. The only copyright information in the dataset is if you, yourself associate LICENSE file with source files in the same repository, or map the GitHub metadata to the files you’re training on.
We all know what this training set will be used for, and what the LLM output looks like so we don’t have to pretend that everyone will make sure to, not only check all the licenses, but also for those licenses which give permission, include copyright information.
Even if you do want to argue that AI companies could in theory uphold license terms, the fact that said burden is put on the consumer of the dataset is not enough, and is enough of a reason for me to want to opt out of if, if you so insist on me simply not wanting big tech to profit.
2 Likes
I’m not trying to gotcha you. I have two things in mind.
- I really do want this to make sense. I value understanding people with different points of view. My interpretation of the MIT license is that this is clearly permitted, and not just in a ‘the law allows it so it’s okay for me to do it’ obnoxious way, but in a ‘the license is explicitly written to permit this and I have a hard time understanding how someone would see things differently’ way.
- When I was a young free software advocate, one thing was hammered pretty hard into me: you have to imagine, when writing free software, that your worst enemy is going to use what you write, in the way that would make you the least happy. The pro-social benefits of gifting your software to the commons have to outweigh that for you (and for me, it usually does); otherwise, you shouldn’t do it! I think this value may not be held as highly among the current generation of open source (I use the term, as distinct from free software, very intentionally) advocates. I think that’s a mistake, and I hope to prevent others from making it.
I get that this looks like I’m coming out swinging for big tech; maybe I’m being a ‘useful idiot’, but what I really mean to do is come out swinging for licenses matter — the difference between MIT and GPL and more bespoke licenses that exclude certain uses and industries matters — and people like you (and sometimes me!) who have strong views on how your contributions are used should be speaking up at licensing time, not long after the fact!
14 Likes
The other thing to consider is that all of this data has been fed into LLMs already. How much of a benefit do we gain by removing it now? The stack has been around for years, I’m sure every commercial coding agent has been trained on it already. Not to be a doomer, but I don’t see a point in removing it from the stack at this point.
To be clear on my point, I’m not an LLM user. I have no interest in using them in nixpkgs, in my configuration, or anywhere for that matter. Just thought I would mention what I’d consider a pretty important clarification.
1 Like
Well, you are, but I’ll bite – if not for the fact that there’s no benefit in being in it, it doesn’t cost anything, and while worst case nothing changes, best case future models won’t be trained on those repositories.
This is why, even though for personal projects nowadays I’ll stay away from permissive licenses, I consented to MIT when contributing to e.g. nixpkgs, which requires copyright notice, as you yourself have noted, and as you haven’t refuted my claim of the license requirement not being sufficiently upheld.
There’s also greater discussion in how this (and other like this) dataset came to be, which is problematic in of itself. That should also be enough of a reason to want to opt out, if you believe in open source, and are such a strong proponent of precise licensing.
2 Likes
The specific component of the MIT license that LLMs could violate is the requirement for attribution (“The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.”)
The problem is that it’s scoped to entire copies and “substantial portions” of the work. I doubt that an LLM is likely to reproduce an exact copy of a work that is long enough for it to count as that. Substantial is also never defined and I don’t know enough about copyright law to know if that has a definition elsewhere.
I also know that the U.S. S.C. ruled that training an LLM is a transformative work, qualifying as Fair Use, and thus not counting as a derivative which means that the model isn’t bound by the terms of the MIT license. If it manages to reproduce a “substantial portion” then that code is bound by the MIT license and the code then follow the MIT license requirement for attribution.
I do want to say that this is just what I know about this topic from occasional research and that I don’t really know much about U.S. copyright law and also that this only applies in the U.S. (please correct me if I misunderstood something)
I assume that it would qualify as a license violation in countries that don’t have fair use but I’m not sure?
2 Likes
My current understanding of your argument is that, in the two-step process
- from NixOS repository to giant dataset
- from giant dataset to trained LLM weights
that the MIT license violation happens in step 2. In other words, that The Stack is not in and of itself a violation of the MIT license, but that it’s clearly and explicitly intended to be an input to a step that allegedly is. And therefore opting out of The Stack is justified as a protest against the violation to which The Stack is an accessory. Is that basically right?
I think I already understand the disagreement over whether training is fair use or IP theft; I’m not trying to rehash that argument here. 
3 Likes
A little bit of both. I agree that step 2 is, in my opinion, in violation. But also step 1 I think is, because the files aren’t sufficiently associated with the necessary information. They’re just placed as-is, and it’s on the consumer of the dataset to associate those files with GitHub metadata to find out the license, its terms, and the copyright notice.
2 Likes
What happens with forks? Is each fork its own repository in this context?
1 Like
Hmm. I don’t know if it’s flawed in some way, but at least nominally there’s a license detection step in HuggingFaceCode/stack-v3-train · Datasets at Hugging Face that suggests that this isn’t being left to the consumer.
I tried looking for Nixpkgs in the online dataset viewer, to see if the license metadata was correct, and I couldn’t find it. Maybe I just don’t know how to use it effectively, or maybe we’ve already opted out? (Edit: presumably we have not opted out if Am I in The Stack? is correct. Guess I’m just too inexperienced at using the viewer.)
It’s limited to the first 5GB so it does not show up there. Tried querying the parquet but too slow.
1 Like
I believe that this is worth repeating–especially in the NixOS community.
A lot of us do open-source work because we believe in ratcheting the commons forward and helping humanity, and sometimes that is gonna mean that people we don’t like (as a subset of the general set of humans) will benefit.
A lot of people who probably dislike (or would dislike) me have written code that I use, and vice versa. The neat thing about software is that we can still benefit from each others’ work.
5 Likes
This is likely up to the Steering Committee to have the final say on. I’ll add it to the agenda for next week for discussion.
Personally, I’m of the opinion that my Nixpkgs contributions can be used for such training and I encourage people doing such training to do so. Apart from the license question; purely from a pragmatic standpoint it leads towards models that are better at Nix. And that is better than hoping the models stay ignorant enough that people don’t use them for Nix-related things.
7 Likes
Apart from the license question; purely from a pragmatic standpoint it leads towards models that are better at Nix.
This is a strategic benefit to the community–even if we dislike (and have rules about!) LLMs contributing back to the ecosystem itself, it is strictly better for end-users to have access to better tooling when they’d like to consider NixOS adoption in their own orgs and projects.
4 Likes
This distinction is why I’ve always hated the term ‘open source’. It obscures the specific criteria and values driving these licenses. Making the source available for anyone to use, however they want (given they comply with the license), is not optional. Why someone may want to arbitrarily restrict access is understandable, but at that point, it’s simply not open source/free software.
I remember this discussion happening long ago when someone took their ruby library down after discovering that I.C.E was using it. This is an understandable reaction, but I was a bit confused, because I thought it was clear that Open Source licenses permit anyone to use/learn from/redistribute the code, so long as they comply with the license. That means me, you, I.C.E, North Korea, whatever. I mean, it’s also code posted publicly on the internet. I’m not sure where these values got lost in favor of “only people I like and/or whose morals align with mine can use this code”.
That said, it is regrettable that Big Tech and AI companies are breaking the implicit social contract around freely sharing code. Where it should be a give-and-take, but they just take and take and take. And massively profit off of it, too. You could take the route e.g. MongoDB did with its Server Side Public License, but it’s no longer open source.
So I’m fine with open source licensed code being using for training, as long as they follow the license terms.
This is why I think permissive licenses like the MIT are a mistake in general, and why I find it concerning that many “rewrite” projects also silently drop the GPL in favor MIT/BSD/etc. That said, it may not matter in the end; it definitely seems like LLMs are a “cheatcode” to get around GPL licensing currently.
This is U.S. specific, but the recent Bartz v. Anthropic case held that
In short, the purpose and character of using copyrighted works to train LLMs to generate
new text was quintessentially transformative. - pg 13
I’m not a legal expert by any means, but if copyrighted works can be used, what does that mean for open source-licensed code freely available on github, etc?
Don’t get me wrong, I absolutely do think licenses are important and that attributions are necessary, particularly for GPL/copyleft licensed code.
2 Likes