Opt out of HuggingFace's "The Stack"

Hi,

My retraction stands generally for my contributions. I am opting out “my code”, and since they have no mechanisms to extract “my code” from a project’s code, I don’t think it’s correct to now opt-in the project.

While the license will be used as an argument about how this is allowed, it is factually irrelevant to the topic at hand. This is about a process unrelated to copyright law and licensing. They are allowing people to opt-out, I am taking them up on the offer.

As far as I know, I never signed a CLA that handed over my intellectual property to the NixOS project. So even considering the license, I will argue that it is still “my code”. I will also argue that anyone else who hasn’t handed over their intellectual property is given the option to opt-out.

Especially for the following projects, of which most or virtually all the meaningful contributions are mine. I don’t think you can make the call.

Especially considering that I never added licensing information on nix-community/nix-review-tools, I object even more so.

12 Likes

I forgot about NixOS/nixos-common-styles. It’s in large part my contributions. All authorship information was lost when the project was created with existing code. The information can be pieced together by going through the nixos-homepage history.

Of which, I would also request opting out. Since I assumed they have ingested the histories, and not just an arbitrary latest commit. There is a significant amount of meaningful work that stood for years in it. Even though it is not the case anymore.

4 Likes

I don’t think it should be considered an accident, or a forceful move. If someone thinks their commits are used in a way that goes against their beliefs/values, what would be their recourse?

Yes it was unilateral but so was the training on their contributions. The whole planet is learning on how to deal with this new situation, I don’t understand why there’s so little empathy around this topic.
It goes both ways, not everyone should be forced to take a moral stand.

Regarding the dataset I find it dishonest, “opt-out” is a dark pattern and shifts the burden to the wrong place. We would probably have a similar discussion if they did an “opt-in” though.

14 Likes

The dataset’s opt-out process is a request that you submit. It does not seem binding in any way. I think it’s perfectly reasonable for the dataset’s maintainers to deny your request (or part of your request) if the alternative would mean excluding all of NixOS/nixpkgs.

(My personal take: the dataset’s maintainers should not offer an opt-out at all. They do not need, nor should they be concerned with, getting a person’s “consent” before copying their publicly available code to a dataset. Ironically, giving people the ability to opt out of the dataset (and a dedicated repository to file those requests in) has probably led to more outrage than just not allowing opt-outs in the first place.)

And that’s for the dataset “maintainers” to make that call.

Mind you that I said that I was making the opt-out request of my code from the dataset. Using their process.

And mind you, they have not only taken code they were allowed to. They have scraped all public repositories. PII included. Copyright infringement [not theirs] included.

And for those tempted to again beat the dead horse of the license… Their dataset, AFAIUI, (1) does not adhere to the license of the MIT license (lacking the “attribution” needed), and (2) goes against the GPL’s terms (by adding further restrictions). (Pardon my sloppy wording, I am obviously not a lawyer.) So for all intent and purpose — and I could very well be wrong, but that would require lawyers and likely court time — they made themselves ineligible to a lot of the open source code they have included in the dataset.

Which anyway is orthogonal to the situation. I have made my request clear: I am opting-out my contributions.

15 Likes

This could not be more opposite from the legal truth. But at this point AI companies (and big tech in general) have successfully gaslit the public into believing “all public information is public domain”. The entire opt-out process is theater to give people the illusion of control, and a smoke grenade to hide the fact that they are operating way beyond the law. Even when taking that process at face value, who can check that they are honoring an opt-out? Who can hold them accountable if they don’t?

Either way, none of this ultimately matters, unless someone musters the resources to actually sue them (lol). So this entire discussion is merely a hypothetical and principled one, and not one with anything actionable.

17 Likes

I am happy that now the latest film from marvel is considered public domain because it is on a random website.

Luckily the laws are the same for everyone…

… as the theft was on a larger scale we will see a lot of people going to jail. Right?

7 Likes

Oh that means people using the dataset can completely ignore the opt-out by just doing setV1 union setV2 union setV3, amazing …
(does not apply for repos which are opted-out before they make it into a release)

How so? The dataset only aggregates repositories with permissive licenses, and it preserves the license files and other notices in its data. If I clone a bunch of repositories off GitHub, zip them up, and send them to a friend, there’s no point in that process where I need to get any repository author’s permission first.

This is going to serve as training data for LLMs that will absolutely be used to generate code that lacks any licensing indicators that it should as derivative works.

13 Likes

Hi @tomberek,

Could you actually provide an answer that you have read and heard what I said, and the concerns I highlighted previously?


First of all, I never retracted or intended to retract the request for opting-out. While it might have been made mistakenly, it ultimately is not an error. I would, and do, intend to request that all of my intellectual property is opted-out of the data set.

Will you make it clear in your communications with them that I have explicitly said that I am not retracting my request? Because as it stands, your last message implies you think you have the authority to make the call on my behalf, and have decided to overrule my action.

I will add that I believe that your course of action was improper.

You should have stopped at “adding this to the SC agenda”, it was unnecessary to be proactive. If you had only added to the SC agenda that you needed to discuss the issue, it would have been fine. But now you have gone and made an explicit decision. This is not restorative, the status quo was not restored. Now an action was taken in the name of the project and contributors. Despite the wishes of at least one contributor, and I suspect many more.

While having the weight of the SC still wouldn’t make me happy about whether or not the project should leave itself be in such datasets, it would at least come from a source of authority, which I would respect, and not an arbitrary unilateral decision.

But Nixpkgs isn’t my main concern. I’ll take it or leave it, I did stepped-down, after having been clearly told that my contributions were not valued compared to substantially small monetary contributions of literally anyone.


Now, do I need to remind you that you have seriously overstepped your authority here, @tomberek?

While there is a tenuous “organizational” claim you could be making about the ofborg-viewer repository, you are not a copyright holder, I am the only copyright holder of the ofborg-viewer. (With exception to the packaging done by Graham while we were setting that up, and that is minor and separate from the core implementation. The other two fixes wouldn’t cross some significance criteria.)

As such, and as the copyright holder, I am asking that you confirm that you have understood that I have not retracted the request to opt-out, that I will, for the foreseeable future, opt-out of the project being included in training data set, and don’t intend to opt-it-in.

And where you definitely have crossed a line is the nix-review-tools repository. While the organization has, AFAIUI, now been made a part of the NixOS project outright, I would expect the repositories are still owned and operated by their owners, as was the case beforehand.

Especially since in the case of nix-review-tools, not only am I the copyright holder, but as I’ve learned by looking at it to see what was up with it, there is no license for the project. (And it is the case mainly because this was initially published from a pile of hacks for the benefit of friendly collaborators.) So while the previous users had an implicit tacit agreement about being able to use it, neither you personally, nor the organization, nor the community, have any claims to make about it and acting on its behalf.

Then, there’s the whole question of the repositories in which my code is a meaningful part, or was the major or a large part of at some point. Those were still in the status quo, as they didn’t end-up in the accidental inclusion. I still am at a loss about how to get my request heard.


At the risk of escalating this, I now also have formulated this additional question:

How will all of you @tomberek, the SC, and the NixOS organization, ensure that such decisions and actions that are made on behalf of contributors respect the wishes of each of the people involved?

19 Likes

You are right that nix-review-tools does not have a license. For that reason, I think that repository should be dropped from the nix-community organization unless it adopts an appropriate open source license. The organization should not host software that is merely source-available. Thankfully, that project is very simple, and would, thanks to modern tools, be trivial to rebuild from scratch with a proper license if necessary.

Even if tomberek did send a message to the dataset maintainers that “overruled your decision”, and it is absolutely not clear to me at all why anyone would think that he did, it would still seem reasonable to me if he was not aware of the perculiar licensing situation regarding nix-review-tools.


To be clear: you are the one who was “proactive” when you put in an opt-out request for entire repositories that you were not the sole contributor or owner of. An action I would describe as “improper” or “overstepping your authority”.

Regarding ofborg-viewer: your status as copyright holder should not be considered. Nothing about the license of that repository prevents it from being included in the dataset, and the repository is obviously part of an organization that consists of many individuals other than yourself. It should be up to the leadership of that organization, not any individual contributor, no matter how sole their contributions to any given repository, whether the repositories of that organization can be included in the dataset or not.

2 Likes

First of all, rude. But no worries, I dropped the nix-review-tools repo since it’s apparently unwanted, “very simple”, and “trivial”.


Bullet points for the rest.

  • The fact that the code has an open source license is entirely orthogonal, as proven by their disregard of it.
  • The opt-out process allows a developer to opt-out their code.
  • I am still the author of the code I authored.
  • I am making use of the process.
  • I still am requesting that my code is removed, as per their process.
    • Even though their process obviously cannot cope with what it claims to allow.
  • Two seemingly conflicting facts can be true at once:
    • A mistake was made, and the process is broken, causing an accidental removal.
    • Explicitly opting-in a repository is not equivalent to it having been left on a list.
  • (Not about Nixpkgs)
    • The fact that a repository is on the NixOS org is irrelevant.
    • Having the license to use code does not make you the owner or author of it.

That’s it. You don’t need to continue arguing against imagined points. Never have I argued that they were not allowed to. Feel free to build a new straw man though.


Addendum: Please do not interact with me any further. Until you are in any form representing the NixOS community, the NixOS project, the NixOS organization, or the NixOS SC.

You have clearly proven that you do not care for other people. That you will gleefully create a straw man, hoping to ensnare the other person into unneeded arguments, and this disrespectful behaviour is unwanted.

15 Likes

All right. This isn’t going in a great direction.

I’d like to try a little experiment here. Haven’t done this before, might go terribly; if it does, blame me individually. I’m only supposed to have this job for the remainder of the month anyway.

This topic is now closed for anything other than:

  1. non-judgmental and objective observations or reports on the state of the world, or
  2. questions and answers offered in a spirit of curiosity and collaboration.

Some examples of 1:

  • Descriptions of what files are present in The Stack.
  • Updates of changes to the contents of The Stack.
  • Reports from the SC about what they’re discussing and/or doing on this topic.

Some non-examples of 1:

  • Claims about how intellectual property works, either legally or in your ethical framework.
  • Claims along the lines of how LLMs are here and one should get used to it.
  • Anything about how people here or elsewhere have behaved poorly in your opinion.

With respect to 2, note that I will enforce do-not-interact requests made by anyone, addressed to anyone, for any reason.

I will delete any posts below this one that don’t meet these conditions to the letter. I intend to be more strict than I generally am elsewhere in this forum. This is an attempt to salvage this topic, rather than fully lock it, so given that the alternative is nobody posts for any reason I’m not going to let myself feel bad about stifling speech.

PM me — do not reply below — if you have questions or concerns about this experiment.

30 Likes

We’ve mostly had people reach out with questions but not discussed this yet in actual committee quorum. I’d say there isn’t a clear direction from the SC yet. It may become a meeting topic next week, as this week is Steering Committee + Board sync.

6 Likes

For most of these projects (e.g. uutils) the license change to MIT is the primary objective of the rewrite.

2 Likes