# Petition to build and cache unfree packages on cache.nixos.org
**URL:** https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440
**Category:** Uncategorized
**Created:** [February 1, 2022, 6:50am UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440 "2022-02-01T06:50:33Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@samuela](https://discourse.nixos.org/u/samuela)
#### Post date: [February 1, 2022, 6:50am UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/1 "2022-02-01T06:50:33Z")
I’m all about FOSS but for some things there are simply no sensible FOSS alternatives. In particular, anything and everything that touches CUDA is unfree and therefore not built by Hydra or cached on [cache.nixos.org](http://cache.nixos.org). Can we relax this constraint, at least for some packages?
I’m really running into trouble trying to use `jaxlibWithCuda` and `tensorflowWithCuda` right now. Building `tensorflowWithCuda` takes 80 CPU-hours. I maintain my own cachix cache, and build a subset of packages in GitHub Actions, automatically pushing them to my cache. But `tensorflowWithCuda` is so nasty that it times out the GitHub Actions limits (see eg [this run](https://github.com/samuela/nixpkgs-upkeep/runs/5012733644?check_suite_focus=true)).
How are users supposed to use CUDA-related software on NixOS?
Relevant PRs:
- [TensorFlow ecosystem: `tensorflow` should never be in `propagatedBuildInputs` by samuela · Pull Request #157611 · NixOS/nixpkgs · GitHub](https://github.com/NixOS/nixpkgs/pull/157611)
- [JAX ecosystem: `jaxlib` should never be in `propagatedBuildInputs` by samuela · Pull Request #156808 · NixOS/nixpkgs · GitHub](https://github.com/NixOS/nixpkgs/pull/156808)
---
### Author:  [@knedlsepp](https://discourse.nixos.org/u/knedlsepp)
#### Post date: [February 1, 2022, 7:17am UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/2 "2022-02-01T07:17:42Z")
There’s some recent activity here: [https://github.com/NixOS/nixpkgs/issues/83884](https://github.com/NixOS/nixpkgs/issues/83884)
---
### Author:  [@asymmetric](https://discourse.nixos.org/u/asymmetric)
#### Post date: [February 1, 2022, 4:15pm UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/3 "2022-02-01T16:15:11Z")
> [@samuela](#):
>
> I maintain my own cachix cache, and build a subset of packages in GitHub Actions, automatically pushing them to my cache. But `tensorflowWithCuda` is so nasty that it times out the GitHub Actions limits (see eg [this run](https://github.com/samuela/nixpkgs-upkeep/runs/5012733644?check_suite_focus=true)).
You could try using [nixbuild.net](http://nixbuild.net) - it even has a [GH action](https://github.com/marketplace/actions/nixbuild-net). Making the resulting cache publicly available could be an alternative to your proposal, in a way.
---
### Author:  [@samuela](https://discourse.nixos.org/u/samuela)
#### Post date: [February 1, 2022, 8:46pm UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/4 "2022-02-01T20:46:21Z")
[My cachix](https://app.cachix.org/cache/ploop#pull) is actually public already. I’m aware of [nixbuild.net](http://nixbuild.net) but unfortunately I don’t have the means to pay for all these builds myself.
---
### Author:  [@Ma27](https://discourse.nixos.org/u/Ma27)
#### Post date: [February 1, 2022, 9:16pm UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/5 "2022-02-01T21:16:16Z")
Have you checked btw if you’re legally allowed to redistribute this? IIRC that’s one of the reasons why we don’t do that on [cache.nixos.org](http://cache.nixos.org) .
---
### Author:  [@samuela](https://discourse.nixos.org/u/samuela)
#### Post date: [February 1, 2022, 10:34pm UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/6 "2022-02-01T22:34:34Z")
I would imagine that even if cudatoolkit/cuDNN are not redistributable, packages that compile against them ought to be redistributable?
---
### Author:  [@vcunat](https://discourse.nixos.org/u/vcunat)
#### Post date: [February 2, 2022, 9:21am UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/7 "2022-02-02T09:21:31Z")
Then beware of keeping non-redistributable stuff in closures. Otherwise the current substitution algorithm wouldn’t work well I think, even if you somehow convinced the cache to keep some holes in its closures.
---
### Author:  [@samuela](https://discourse.nixos.org/u/samuela)
#### Post date: [February 4, 2022, 3:17am UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/8 "2022-02-04T03:17:09Z")
Ok, now I’m even more confused… It appears that `python3Packages.pytorch` _is_ cached on [cache.nixos.org](http://cache.nixos.org) even though it links against CUDA/cuDNN as well. Why is PyTorch cached but `tensorflowWithCuda` and `jaxlibWithCuda` are not?
For example,
```auto
$ nix-shell -p python3Packages.pytorch
...
copying path '/nix/store/klmb0q1kyd8k00n0mfzyhjmvkz54vzvf-python3.9-pytorch-1.9.0' from 'https://cache.nixos.org'...
copying path '/nix/store/f5ghfax668nswhh1r60j4c5dyxbyhjps-python3.9-pytorch-1.9.0-dev' from 'https://cache.nixos.org'...
...
```
---
### Author:  [@jonringer](https://discourse.nixos.org/u/jonringer)
#### Post date: [February 4, 2022, 4:14am UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/9 "2022-02-04T04:14:28Z")
> [@samuela](#):
>
> It appears that `python3Packages.pytorch` _is_ cached on [cache.nixos.org](http://cache.nixos.org) even though it links against CUDA/cuDNN as well.
From the pytorch-bin package:
```auto
# Includes CUDA and Intel MKL, but redistributions of the binary are not limited.
# https://docs.nvidia.com/cuda/eula/index.html
# https://www.intel.com/content/www/us/en/developer/articles/license/onemkl-license-faq.html
license = licenses.bsd3;
```
I agree, it’s odd though.
---
### Author:  [@samuela](https://discourse.nixos.org/u/samuela)
#### Post date: [February 4, 2022, 4:22am UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/10 "2022-02-04T04:22:37Z")
So presumably `tensorflowWithCuda` and `jaxlibWithCuda` could be made redistributable as well?
---
### Author:  [@jonringer](https://discourse.nixos.org/u/jonringer)
#### Post date: [February 4, 2022, 5:49am UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/11 "2022-02-04T05:49:23Z")
Well, I included the license bit, to demonstrate that it’s odd. It has a bsd3 license, when it should probably have a unfreeRedistributable license.
One option would be to have a `config.acceptCudaLicense`, and just have the meta check it before someone attempts to install.
---
### Author:  [@FRidh](https://discourse.nixos.org/u/FRidh)
#### Post date: [February 4, 2022, 8:59am UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/12 "2022-02-04T08:59:25Z")
The way we handle licenses in Nixpkgs is just not correct. The license that is added in meta is typically the license of the source. The license of the built artifact is not necessarily the same.
>
>
> \*\*Describe the bug\*\*
> In the \`meta.license\` of a package we typically state the …licenses that apply to the source, not the licenses that apply to the built project.
>
> \*\*Additional context\*\*
> The actual licenses that apply to the built package can be different. Exactly which apply is difficult to say as it depends entirely on the licenses of the source, as well as other (native) build inputs. This makes it impossible to reason about at evaluation time and near impossible by inspecting build artifacts.
>
> Even so, it is possible to take a cautious approach by propagating licenses. This allows for example organizations to block software that \*may\* be GPL-licensed.
>
> In my opinion \`meta.licenses\` as we have now should be part of the \`src\` derivation, and \`meta.license\` of a package should be taken from there by default. Additionally, I think we could have a \`config.meta.propagateLicenses\` to propagate licenses of all build inputs. I think there could be an additional switch for native build inputs.
>
> I imagine this could need an RFC.
>
> \*\*Notify maintainers\*\*
> Note I have no need for this myself and merely wrote this because it again \[came up on IRC\](https://logs.nix.samueldr.com/nixos/2020-12-09#4344676;).
>
> \*\*Related issues\*\*
> \- default value for packages that have no license https://github.com/NixOS/nixpkgs/issues/43231
> \- building of \`unfreeRedistributable\` packages https://github.com/NixOS/nixpkgs/issues/83884
>
> \*\*Considerations\*\*
> \- propagating licenses may impact evaluation severely
> \- not propagating licenses of build tools will encourage the users of this feature to fix separation of native versus non-native inputs which is also good for cross-compilation
> [@Obtaining source locations and licenses recursively for a derivation](https://discourse.nixos.org/t/obtaining-source-locations-and-licenses-recursively-for-a-derivation/3934):
>
> Checking what licenses apply to a binary is not easy. With Nix we declare the license that applies to a source, although we declare it in the derivation of the built artifact. I’m looking for a solution that can yield for me for a derivation what components potentially leave traces, what source those are built from, and what licenses apply. Clearly, recursion plays a role here. We do not declare in our expressions what parts leave traces. We do know that buildInputs are run-time dependencies a…
---
### Author:  [@vcunat](https://discourse.nixos.org/u/vcunat)
#### Post date: [February 4, 2022, 9:55am UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/13 "2022-02-04T09:55:12Z")
The `meta.license` system was never designed to handle such details. It was just a dumb string not too long ago.
---
### Author:  [@samuela](https://discourse.nixos.org/u/samuela)
#### Post date: [February 5, 2022, 1:49am UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/14 "2022-02-05T01:49:14Z")
Ok, my solution for now is just to give up on the source builds entirely ([related commit](https://github.com/samuela/nixpkgs-upkeep/commit/4543e5997f60ff61fbd62dd5678220e46fe5173d)). This may not be the nix-iest way but these massive builds make it impossible for me to be productive otherwise. I’ve already spent way too much time on this…
It’s a shame, because I really believe that Nix/NixOS has a lot to offer the reproducible research community. But without a “just works” solution for CUDA packages I can’t recommend Nix to other ML people in good conscience.
---
### Author:  [@jonringer](https://discourse.nixos.org/u/jonringer)
#### Post date: [February 5, 2022, 9:07am UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/15 "2022-02-05T09:07:15Z")
> [@samuela](#):
>
> It’s a shame, because I really believe that Nix/NixOS has a lot to offer the reproducible research community. But without a “just works” solution for CUDA packages I can’t recommend Nix to other ML people in good conscience.
I think that’s fair. But I would also like to point out that cuda and similar frameworks have always been awkward when used in tandem with FOSS. If nvidia had cuda under mpl where they reserved patent rights, but allowed for a permissive license, this would be a non-issue.
---
### Author:  [@samuela](https://discourse.nixos.org/u/samuela)
#### Post date: [February 5, 2022, 9:56pm UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/16 "2022-02-05T21:56:40Z")
Yeah, NVIDIA is not without fault here. They haven’t been the best community members in Linux/OSS world to say the least…
Unfortunately dealing with CUDA is just a necessary reality for the kind of work that I do. And as much as possible I’d like to do that work with NixOS 😛
---
### Author:  [@alevy](https://discourse.nixos.org/u/alevy)
#### Post date: [February 6, 2022, 1:08pm UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/17 "2022-02-06T13:08:34Z")
@samuela if this is solvable for your particular use case with resources, and a binary cache of a few libraries is the last barrier to making reproducible research in (some area of) ml much better (using Nix), there are certainly those of us in academia who would be able to help out
What is expensive for an individual to front can often be really negligeable in an academic setting (especially given free bandwidth and spare servers).
Please DM or email me if you want to see if we can make this happen!
---
### Author:  [@anon20203487](https://discourse.nixos.org/u/anon20203487)
#### Post date: [February 11, 2022, 2:19pm UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/18 "2022-02-11T14:19:25Z")
I will be a bit egotistical here, but why should us cache unfree software? Our lives are already harder because they are not open source.
Why should us make the lives of _unfree_ software projects easier by giving them bandwidth, disk space and processor cycles _for free_?
---
### Author:  [@samuela](https://discourse.nixos.org/u/samuela)
#### Post date: [February 11, 2022, 6:59pm UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/19 "2022-02-11T18:59:25Z")
@anon20203487 How would you suggest that users do machine learning on NixOS?
(Also not to nitpick but in many cases unfree != open source != non-redistributable.)
---
### Author:  [@anon20203487](https://discourse.nixos.org/u/anon20203487)
#### Post date: [February 11, 2022, 9:13pm UTC](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440/20 "2022-02-11T21:13:52Z")
Do not ask me. However, you have asked, so I will answer.
I am on the same side of Linus and [his advice to Nvidia](https://www.youtube.com/watch?v=_36yNWw_07g) here. We should not bow to them merely because they monopolize the market.
If you are taking a more pragmatical way, just try to convince people with bandwidth and disk space and money to pay the eventual lawyers and money to pay the eventual sysadmins about your idea. I am only a humble user of an AMD Ryzen while I am trying to save some money for a RISC-V desktop…
[Next page](https://discourse.nixos.org/t/petition-to-build-and-cache-unfree-packages-on-cache-nixos-org/17440.md?page=2)