Hosting binary 'source' tarballs

danieldk · March 8, 2020, 7:34am

A while ago I have packaged Softmaker Office and FreeOffice, both are closed source, but are pretty useful (as least to me) because they have fairly good Office compatibility. The derivations work fine, except that they break regularly, because the source tarballs are replaced. E.g., at some point the release version is 972 with the corresponding download URL

https://www.softmaker.net/down/softmaker-office-2018-972-amd64.tgz

Then then Softmaker Office is updated, and not only do they release a new tarball such as:

https://www.softmaker.net/down/softmaker-office-2018-974-amd64.tgz

But the previous version’s tarball is 301-redirected to the new version. Of course, this breaks the derivation until the hash is updated. I asked about this, and they seem to do redirections because people click on links in older release emails:

https://forum.softmaker.com/viewtopic.php?f=320&t=19327&p=63557&hilit=nixos#p63557

They suggest hosting the source tarballs ourselves to avoid that hashes change. Is such a facility available?

tokudan · February 22, 2020, 10:24pm

Well, there is http://tarballs.nixos.org/ but not sure that’s actually intended for non-free software, as that could easily violate licences.

d-goldin · February 28, 2020, 1:30pm

There are some derivations where archive.org is leveraged, such as https://github.com/NixOS/nixpkgs/blob/0d7ca4ff088a55fd53825b9518e868b3e6d9ba33/pkgs/applications/networking/instant-messengers/skypeforlinux/default.nix#L64 - maybe that’s an option.

danieldk · March 8, 2020, 7:33am

That’s a nice idea, though I guess for new releases one would have to wait some time before they become available?

Another option that I have considered is asking them if we could use GitHub releases. I could make a new repository, commit/tag some metadata, and upload the releases. This is e.g. how spaCy distribute models.

d-goldin · March 8, 2020, 4:20pm

Yeah, that could work. The question is mostly of trust - if upstream already provides binary blobs for sth, it would be nicer to not have the need to trust some additional third party (in this case you, also depends on whether the tarballs are signed or not and the license). The archive.org workaround was mostly due to license and trust.

cdepillabout · March 9, 2020, 6:16am

memtest86 falls into this category as well:

github.com

NixOS/nixpkgs/blob/42895406af2ef474d287f657744fa85e459bdc61/pkgs/tools/misc/memtest86-efi/default.nix#L7-L23


src = fetchzip {
  # TODO: The latest version of memtest86 is actually 8.2, but the
  # company developing memtest86 has stopped providing a versioned download
  # link for the latest version:
  #
  # https://www.passmark.com/forum/memtest86/44494-version-8-1-distribution-file-is-not-versioned?p=44505#post44505
  #
  # However, versioned links for the previous version are available, so that
  # is what is being used.
  #
  # It does look like redistribution is okay, so if we had somewhere to host
  # binaries that we make sure to version, then we could probably keep up
  # with the latest versions released by the company.
  url = "https://www.memtest86.com/downloads/memtest86-${version}-usb.zip";
  sha256 = "1x1wjssr4nnbnfan0pi7ni2dfwnm3288kq584hkfqcyza8xdx03i";
  stripRoot = false;
};

makefu · March 9, 2020, 9:46am

For one of my derivations i used the ability to upload packages onto archive.org ( https://git.shackspace.de/rz/stockholm/blob/master/krebs/5pkgs/simple/fortclientsslvpn/default.nix#L15 ). This could also be used if the webspace is rejecting the archive.org bot or the file is too large for scraping.

khumba · March 22, 2020, 9:45pm

Similarly, I’ve got a derivation for the Obsidian X cursor theme, since it’s a solid theme and I was missing it from Gentoo. The OpenDesktop sites used to allow direct downloading, but they seem to have stopped, and an email I sent to them went unanswered. The package itself is derived from another GPLv2+ theme so hosting it isn’t a problem legally. Is there a service for cases like this? Is this tarballs.nixos.org?

(Of course, I’m scratching a personal itch here, I’d understand if we don’t want tons of random themes in Nixpkgs.)