Inside my derivation I want to get the hashString of this derivation.
Is it somehow possible to get to the source file of the derivation to use it as input to hashString?
2 Likes
Can you explain a little bit more what you are trying to do?
The $out variable will have the input hash as part of the path: $(basename $out) | cut -d - -f 1.
I have a fixed output derivation for testing my package. https://github.com/NixOS/nixpkgs/blob/3ff636fb2e756ac57d7f0007dc2c6c2425401997/pkgs/development/compilers/ldc/default.nix#L160
Problem is that it isn’t rebuild if only something inside this fixed output derivation is changed.
That could be solved if I get somehow the contents of that derivation into the outputHash.
I am currently just trying with outputHashMode = “recursive” but I don’t think it will work as it computes just the hash of the contents without the derivation itself as far as I understood it.
Doing what you do is heresy according to Restrict fixed-output derivations · Issue #2270 · NixOS/nix · GitHub
Let the heresy begin! But first TLDR: you have to replace inputString = ldcBuild.outPath; with outPath to test derivation. Fixing infinite recursion shouldn’t be hard.
let
pkgs = import <nixpkgs> {};
lib = pkgs.lib;
in rec {
Let’s also make some abstractions out of this. First, we’ll reuse @Profpatsch test runner, but heavily modified one:
drvSeqL = drvDeps: drvOut: let
drvOutOutputs = drvOut.outputs or ["out"];
in
lib.overrideDerivation drvOut (_: {
inherit drvDeps;
buildCommand =
(lib.concatMapStrings (output: ''
target=${lib.escapeShellArg drvOut.${output}}
# if the target is already a symlink, follow it until it’s not;
# this is done to prevent too many dereferences
target=$(readlink -e "$target")
# link to the output
ln -s "$target" "${"$"}${output}"
'') drvOutOutputs);
});
drvSeq = drvDep: drvOut: drvSeqL [drvDep] drvOut;
Then let’s define our compiler:
compiler = pkgs.runCommand "" {
pname = "compiler";
version = "0.1";
meta.description = "A compiler!";
outputs = [ "out" "doc" ];
} ''
echo BUILDING COMPILER
mkdir -p $out/bin
echo -n '#!/bin/sh
echo hello $1, really compiled with compiler v'"$version"'
' > $out/bin/compiler
chmod +x $out/bin/compiler
# multiple outputs!
echo run like "compiler world" to produce "hello world" > $doc
'';
Pretty nice compiler, read help cat $(nix-build -A compiler.doc) on how to use this compiler.
We want some test also, right?
compilerTest = pkgs.runCommand "${compiler.name}-tests" {
buildInputs = with pkgs; [ curl compiler ];
} ''
echo RUNNING TESTS 1
curl --insecure -LO http://nixos.org/nix/install
set -e
compiler world | grep -q "hello world"
echo -n $inputString > $out
'';
Very nice test, but it doesn’t work:
$ nix-build -A compilerTest
these derivations will be built:
/nix/store/37s0s3amkkm4697mz62mf7pi1zqg9m0g-compiler-0.1-tests.drv
building '/nix/store/37s0s3amkkm4697mz62mf7pi1zqg9m0g-compiler-0.1-tests.drv'...
RUNNING TESTS 1
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: nixos.org
builder for '/nix/store/37s0s3amkkm4697mz62mf7pi1zqg9m0g-compiler-0.1-tests.drv' failed with exit code 6
error: build of '/nix/store/37s0s3amkkm4697mz62mf7pi1zqg9m0g-compiler-0.1-tests.drv' failed
Uh oh. This is because you can’t run test without getting package out. We will fake our packages with
test = testedPackage compilerTest (
pkgs.runCommand "compiler-tests" { } "touch $out");
$ nix-build -A test
these derivations will be built:
/nix/store/5fpcvplpfd4frcx7dhhp4f1y6a7mf37p-compiler-0.1-tests.drv
/nix/store/avby8nf3224mvd8lspicnmcnv8gr8amn-compiler-tests.drv
building '/nix/store/5fpcvplpfd4frcx7dhhp4f1y6a7mf37p-compiler-0.1-tests.drv'...
RUNNING TESTS 1
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 237 100 237 0 0 1234 0 --:--:-- --:--:-- --:--:-- 1234
100 2476 100 2476 0 0 4559 0 --:--:-- --:--:-- --:--:-- 4559
building '/nix/store/avby8nf3224mvd8lspicnmcnv8gr8amn-compiler-tests.drv'...
/nix/store/qyskkrclp5ix4wx6f00dqkzfg9lsp55s-compiler-tests
Look ma! It curls internet in sandbox mode! But getting dummy package isn’t interesting. Let’s get real package:
compilerOut = testedPackage compilerTest compiler;
$ nix-build -A compilerOut
/nix/store/2flxjqal2fjj5y9nb5s14w3srk63xpjx-compiler-0.1
Let’s check, if test is rerun when compiler version is changed (I’m changing it to 0.6):
$ nix-build -A compilerOut
these derivations will be built:
/nix/store/82ya3qa1120by2aqm8mz3zwm5q8fazkn-compiler-0.6.drv
/nix/store/frjxiczb32q57wblb6m09zwqhpmnaag3-compiler-0.6-tests.drv
/nix/store/45nndb36xip8jb51j3phljkjksgj0axj-compiler-0.6.drv
building '/nix/store/82ya3qa1120by2aqm8mz3zwm5q8fazkn-compiler-0.6.drv'...
BUILDING COMPILER
building '/nix/store/frjxiczb32q57wblb6m09zwqhpmnaag3-compiler-0.6-tests.drv'...
RUNNING TESTS 1
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 237 100 237 0 0 1346 0 --:--:-- --:--:-- --:--:-- 1338
100 2476 100 2476 0 0 4602 0 --:--:-- --:--:-- --:--:-- 4602
building '/nix/store/45nndb36xip8jb51j3phljkjksgj0axj-compiler-0.6.drv'...
/nix/store/qmj5y5lw79q9l2m4ysycwr1sx0b9gj7s-compiler-0.6
$ nix-build -A compilerOut
/nix/store/qmj5y5lw79q9l2m4ysycwr1sx0b9gj7s-compiler-0.6
Good, so it reruns tests when compiler was changed. But now let’s introduce wrong test (compiler world | grep -q "bla bla"):
$ nix-build -A compilerOut
these derivations will be built:
/nix/store/3gx6k899zl8w5fgq2f0vaqifyzawrl46-compiler-0.6-tests.drv
/nix/store/5zdbvjp2ws5pdvrqgbs3l5kksmda2ppg-compiler-0.6.drv
building '/nix/store/3gx6k899zl8w5fgq2f0vaqifyzawrl46-compiler-0.6-tests.drv'...
RUNNING TESTS 1
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 237 100 237 0 0 1227 0 --:--:-- --:--:-- --:--:-- 1227
100 2476 100 2476 0 0 4298 0 --:--:-- --:--:-- --:--:-- 805k
builder for '/nix/store/3gx6k899zl8w5fgq2f0vaqifyzawrl46-compiler-0.6-tests.drv' failed with exit code 1
cannot build derivation '/nix/store/5zdbvjp2ws5pdvrqgbs3l5kksmda2ppg-compiler-0.6.drv': 1 dependencies couldn't be built
error: build of '/nix/store/5zdbvjp2ws5pdvrqgbs3l5kksmda2ppg-compiler-0.6.drv' failed
Very good. I mean, it’s bad, tests shouldn’t fail, but it’s good it reruns tests, but doesn’t rebuild compiler if compiler isn’t changed. Let’s fix test and rerun:
$ nix-build -A compilerOut
these derivations will be built:
/nix/store/f3pairqlnrlyqhd4j55n3xbyap5ki9b8-compiler-0.6-tests.drv
/nix/store/8n4zz64fp2lqsycv20ni2lj9vh4s6k4j-compiler-0.6.drv
building '/nix/store/f3pairqlnrlyqhd4j55n3xbyap5ki9b8-compiler-0.6-tests.drv'...
RUNNING TESTS 1
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 237 100 237 0 0 1240 0 --:--:-- --:--:-- --:--:-- 1240
100 2476 100 2476 0 0 3844 0 --:--:-- --:--:-- --:--:-- 3844
building '/nix/store/8n4zz64fp2lqsycv20ni2lj9vh4s6k4j-compiler-0.6.drv'...
/nix/store/bx0z3yll0ajhincwgrkpyg96yk6hg8d3-compiler-0.6
$ cat $(nix-build -A compilerOut.doc)
run like compiler world to produce hello world
@ThomasMader is that what you wanted? If yes, then here is the missing part of the puzzle:
testedPackage = test: package:
let
_tested = derivation (test.drvAttrs // rec {
inputString = builtins.unsafeDiscardStringContext test.outPath;
outputHashAlgo = "sha256";
outputHash = builtins.hashString "sha256" inputString;
});
in drvSeq _tested package;
4 Likes
Thank you very much for your detailed answer.
I can see that it opens the door for heresy stuff but in my case I need to test in a fixed output derivation just because the socket implementation is tested via the loopback address (127.0.0.1).
See https://github.com/dlang/phobos/blob/56175eb0e9b5fc41f72ed1ad50233fc316b8893b/std/socket.d#L779 for example.
I fail to see how the loopback address can introduce any inpurities.
But I see that in the long run I need to go for another solution.