Help making a derivation fixed

Help
1

Hello good people!

I’m wrapping a non-free software in a fhs wrapper.

  1. source is an archive that is extracted by nix
  2. the contents is actually another proprietary archive/installer. I run it as part of the build process
  3. now I get final derivation
  4. wrap that derivation into a FHS env

All good and works fine.
But
the derivation at p.3 is supposed to be fixed. There should be no difference in what dependencies I used in running that installer - in the end it unarchives the same exact content.
But currently, if I for example update nixpkgs used - it will force all steps above to happen again, even though derivation at p.3. does not change.

I hope I’ve described it in an understandable way. please correct my incorrect terminology.

The question is - how to make that derivation at p.3 to not depend on it’s build inputs?

p.s. for specifics - the specific derivation in question is this: nixpkgs/pkgs/applications/misc/houdini/runtime-build.nix at 7150b43a80469432abe828322ac49f0025c44ed8 · NixOS/nixpkgs · GitHub , but I hope for a general case answer

2 Likes
2

The derivation is “fixed output” when it contains outputHash, outputHashMode and outputHashAlgo. When you add them to a derivation, it is fixed, thus can access network and will always check if hash of output path is the same.

See Advanced Attributes - Nix Reference Manual.

Now we have a convenient function to get a similar experience to other fetchers: lib.fetchers.withNormalizedHash.

Suppose your derivation is:

stdenvNoCC.mkDerivation { ... }

Now you just have to wrap it like this:

stdenvNoCC.mkDerivation (lib.fetchers.withNormalizedHash { } { ...; hash = ""; })

Build it, and you’ll get the correct hash.

1 Like
3

That is something that is only present in unstable right?

4

Why not simply do 1. and 2. in one step?

Why the FHS-wrapper? Have you tried using patchelf instead? That’s usually preferable.

Related work:

(My refactor of that currently under review: teamspeak3: modernise by Atemu · Pull Request #354161 · NixOS/nixpkgs · GitHub)

5

source archive is not available for automatic download, it’s behind registration, so stage 2 relies on requireFile.

I should try to do that actually. But i’m expecting a bunch of hardcoded fhs paths to be present in different scripts in that software, so the process probably won’t be trivial.

Also, technically, does patchelf count as tampering with software which would be a EULA violation? since all that has a custom proprietary license

6

thank you, I will try that.

I wonder, what happens if I add those hashes, but it will turn out that the software’s custom installer actually do behave differently with different versions of build dependencies?

Will it result in noone else being able to build that derivation unless using the same exact nixpkgs version I did when calculating hashes?

7

Yeah, if the fixed-output derivation is actually not deterministic, it will fail for people, possibly only sometimes. This is unfortunately a pretty real problem :slight_smile:

On the other hand, since fixed-output derivations only care about the output, you can possibly munge the output to “fix” non-deterministic results.

8

Right so where’s the issue? requireFile is a fixed output derivation by definition.

If all of those scripts are available after unpacking and it doesn’t download anything global-FHS-dependent at runtime, you should be able to just patch the shebangs. The autoPatchelfHook might even do that automatically but I’m not sure.

IANAL.

Technically, it could. Though I’d argue that dynamically linking the software against all the deps of the FHSEnv would be much the same and that is also allowed (or at least tolerated) somehow.

9

so, first - we need the archive. it’s contents - is the “installer” (this archive is a requireFile derivation)
then - we need to run “installer” that will once again unpack a bunch of stuff from itself.

And this second step contents, that “installer” produces, that I want to be a fixed derivation too, since it basically just extracts files from it’s internal archives.

10

Why not simply let the installer run in the same derivation you do all the other stuff in?

11

late reply, sorry,

so basically that is what I do now: one derivation requires the installer archive (downloaded from the site manually), as install phase it runs that installer - and the result of that installation is always the same, only depends on that installer archive. therefore this derivation can be a fixed derivation.
Another derivation uses this fixed one and wraps it into a buildFHSEnv.

Now when you want to update nixpkgs - only that wrapper gets rebuilt, and the whole installation from the initial source archive does not happen again (which saves ~3 GB of space)