Is it possible to get hashString of current derivation?

#1

Inside my derivation I want to get the hashString of this derivation.
Is it somehow possible to get to the source file of the derivation to use it as input to hashString?

#2

Can you explain a little bit more what you are trying to do?

The out variable will have the input hash as part of the path: `(basename $out) | cut -d - -f 1`.

#3

I have a fixed output derivation for testing my package. https://github.com/NixOS/nixpkgs/blob/3ff636fb2e756ac57d7f0007dc2c6c2425401997/pkgs/development/compilers/ldc/default.nix#L160

Problem is that it isn’t rebuild if only something inside this fixed output derivation is changed.
That could be solved if I get somehow the contents of that derivation into the outputHash.
I am currently just trying with outputHashMode = “recursive” but I don’t think it will work as it computes just the hash of the contents without the derivation itself as far as I understood it.

#4

Doing what you do is heresy according to https://github.com/NixOS/nix/issues/2270

Let the heresy begin! But first TLDR: you have to replace inputString = ldcBuild.outPath; with outPath to test derivation. Fixing infinite recursion shouldn’t be hard.

let
  pkgs = import <nixpkgs> {};
  lib = pkgs.lib;
in rec {

Let’s also make some abstractions out of this. First, we’ll reuse @Profpatsch test runner, but heavily modified one:

  drvSeqL = drvDeps: drvOut: let
    drvOutOutputs = drvOut.outputs or ["out"];
  in
    lib.overrideDerivation drvOut (_: {
      inherit drvDeps;
      buildCommand =
        (lib.concatMapStrings (output: ''
        target=${lib.escapeShellArg drvOut.${output}}
        # if the target is already a symlink, follow it until it’s not;
        # this is done to prevent too many dereferences
        target=$(readlink -e "$target")
        # link to the output
        ln -s "$target" "${"$"}${output}"
        '') drvOutOutputs);
    });
  drvSeq = drvDep: drvOut: drvSeqL [drvDep] drvOut;

Then let’s define our compiler:

    compiler = pkgs.runCommand "" {
        pname = "compiler";
        version = "0.1";
        meta.description = "A compiler!";
        outputs = [ "out" "doc" ];
    } ''
      echo BUILDING COMPILER
      mkdir -p $out/bin
      echo -n '#!/bin/sh
        echo hello $1, really compiled with compiler v'"$version"'
      ' > $out/bin/compiler
      chmod +x $out/bin/compiler

      # multiple outputs!
      echo run like "compiler world" to produce "hello world" > $doc
    '';

Pretty nice compiler, read help cat $(nix-build -A compiler.doc) on how to use this compiler.

We want some test also, right?

    compilerTest = pkgs.runCommand "${compiler.name}-tests" {
        buildInputs = with pkgs; [ curl compiler ];
    } ''
      echo RUNNING TESTS 1
      curl --insecure -LO http://nixos.org/nix/install
      set -e
      compiler world | grep -q "hello world"
      echo -n $inputString > $out
    '';

Very nice test, but it doesn’t work:

$ nix-build -A compilerTest
these derivations will be built:
  /nix/store/37s0s3amkkm4697mz62mf7pi1zqg9m0g-compiler-0.1-tests.drv
building '/nix/store/37s0s3amkkm4697mz62mf7pi1zqg9m0g-compiler-0.1-tests.drv'...
RUNNING TESTS 1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: nixos.org
builder for '/nix/store/37s0s3amkkm4697mz62mf7pi1zqg9m0g-compiler-0.1-tests.drv' failed with exit code 6
error: build of '/nix/store/37s0s3amkkm4697mz62mf7pi1zqg9m0g-compiler-0.1-tests.drv' failed

Uh oh. This is because you can’t run test without getting package out. We will fake our packages with

    test = testedPackage compilerTest (
        pkgs.runCommand "compiler-tests" { } "touch $out");
$ nix-build -A test
these derivations will be built:
  /nix/store/5fpcvplpfd4frcx7dhhp4f1y6a7mf37p-compiler-0.1-tests.drv
  /nix/store/avby8nf3224mvd8lspicnmcnv8gr8amn-compiler-tests.drv
building '/nix/store/5fpcvplpfd4frcx7dhhp4f1y6a7mf37p-compiler-0.1-tests.drv'...
RUNNING TESTS 1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   237  100   237    0     0   1234      0 --:--:-- --:--:-- --:--:--  1234
100  2476  100  2476    0     0   4559      0 --:--:-- --:--:-- --:--:--  4559
building '/nix/store/avby8nf3224mvd8lspicnmcnv8gr8amn-compiler-tests.drv'...
/nix/store/qyskkrclp5ix4wx6f00dqkzfg9lsp55s-compiler-tests

Look ma! It curls internet in sandbox mode! But getting dummy package isn’t interesting. Let’s get real package:

  compilerOut = testedPackage compilerTest compiler;
$ nix-build -A compilerOut
/nix/store/2flxjqal2fjj5y9nb5s14w3srk63xpjx-compiler-0.1

Let’s check, if test is rerun when compiler version is changed (I’m changing it to 0.6):

$ nix-build -A compilerOut
these derivations will be built:
  /nix/store/82ya3qa1120by2aqm8mz3zwm5q8fazkn-compiler-0.6.drv
  /nix/store/frjxiczb32q57wblb6m09zwqhpmnaag3-compiler-0.6-tests.drv
  /nix/store/45nndb36xip8jb51j3phljkjksgj0axj-compiler-0.6.drv
building '/nix/store/82ya3qa1120by2aqm8mz3zwm5q8fazkn-compiler-0.6.drv'...
BUILDING COMPILER
building '/nix/store/frjxiczb32q57wblb6m09zwqhpmnaag3-compiler-0.6-tests.drv'...
RUNNING TESTS 1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   237  100   237    0     0   1346      0 --:--:-- --:--:-- --:--:--  1338
100  2476  100  2476    0     0   4602      0 --:--:-- --:--:-- --:--:--  4602
building '/nix/store/45nndb36xip8jb51j3phljkjksgj0axj-compiler-0.6.drv'...
/nix/store/qmj5y5lw79q9l2m4ysycwr1sx0b9gj7s-compiler-0.6

$ nix-build -A compilerOut
/nix/store/qmj5y5lw79q9l2m4ysycwr1sx0b9gj7s-compiler-0.6

Good, so it reruns tests when compiler was changed. But now let’s introduce wrong test (compiler world | grep -q "bla bla"):

$ nix-build -A compilerOut
these derivations will be built:
  /nix/store/3gx6k899zl8w5fgq2f0vaqifyzawrl46-compiler-0.6-tests.drv
  /nix/store/5zdbvjp2ws5pdvrqgbs3l5kksmda2ppg-compiler-0.6.drv
building '/nix/store/3gx6k899zl8w5fgq2f0vaqifyzawrl46-compiler-0.6-tests.drv'...
RUNNING TESTS 1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   237  100   237    0     0   1227      0 --:--:-- --:--:-- --:--:--  1227
100  2476  100  2476    0     0   4298      0 --:--:-- --:--:-- --:--:--  805k
builder for '/nix/store/3gx6k899zl8w5fgq2f0vaqifyzawrl46-compiler-0.6-tests.drv' failed with exit code 1
cannot build derivation '/nix/store/5zdbvjp2ws5pdvrqgbs3l5kksmda2ppg-compiler-0.6.drv': 1 dependencies couldn't be built
error: build of '/nix/store/5zdbvjp2ws5pdvrqgbs3l5kksmda2ppg-compiler-0.6.drv' failed

Very good. I mean, it’s bad, tests shouldn’t fail, but it’s good it reruns tests, but doesn’t rebuild compiler if compiler isn’t changed. Let’s fix test and rerun:

$ nix-build -A compilerOut
these derivations will be built:
  /nix/store/f3pairqlnrlyqhd4j55n3xbyap5ki9b8-compiler-0.6-tests.drv
  /nix/store/8n4zz64fp2lqsycv20ni2lj9vh4s6k4j-compiler-0.6.drv
building '/nix/store/f3pairqlnrlyqhd4j55n3xbyap5ki9b8-compiler-0.6-tests.drv'...
RUNNING TESTS 1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   237  100   237    0     0   1240      0 --:--:-- --:--:-- --:--:--  1240
100  2476  100  2476    0     0   3844      0 --:--:-- --:--:-- --:--:--  3844
building '/nix/store/8n4zz64fp2lqsycv20ni2lj9vh4s6k4j-compiler-0.6.drv'...
/nix/store/bx0z3yll0ajhincwgrkpyg96yk6hg8d3-compiler-0.6

$ cat $(nix-build -A compilerOut.doc)
run like compiler world to produce hello world

@ThomasMader is that what you wanted? If yes, then here is the missing part of the puzzle:

  testedPackage = test: package:
    let
      _tested = derivation (test.drvAttrs // rec {
        inputString = builtins.unsafeDiscardStringContext test.outPath;
        outputHashAlgo = "sha256";
        outputHash = builtins.hashString "sha256" inputString;
      });
    in drvSeq _tested package;
2 Likes
#5

Thank you very much for your detailed answer.

I can see that it opens the door for heresy stuff but in my case I need to test in a fixed output derivation just because the socket implementation is tested via the loopback address (127.0.0.1).
See https://github.com/dlang/phobos/blob/56175eb0e9b5fc41f72ed1ad50233fc316b8893b/std/socket.d#L779 for example.

I fail to see how the loopback address can introduce any inpurities.
But I see that in the long run I need to go for another solution.