nix-fod: just like nix-build but replaces the outputHash with a fakeHash and produces the hash you needed at the end of the build with no error. Why? Because I recently was poking at nix-update and ended up adding the following to a private package of mine
genHash =
let
outputHash = lib.fakeHash;
in
package.yarnOfflineCache.overrideAttrs (prev: prev // { inherit outputHash; });
update =
let
root = builtins.toString ./.;
in
pkgs.writeShellScriptBin "update" ''
export PATH=${
lib.makeBinPath [
pkgs.coreutils
pkgs.jq
pkgs.gawk
pkgs.nix
]
}:$PATH
set -euo pipefail
log=$(mktemp)
trap 'rm -f "$log"' EXIT
exec 2> >(tee "$log" >&2)
set +e
nix-build -A genHash --no-out-link ${root}/default.nix
set -e
hash$(tail -n1 "$log" | awk '/got: / { print $2; }')
jq -n --arg hash $hash '$hash' > ${root}/hash.nix
'';
to me this seems absolutely bonkers, but I also am no c++ dev and have never peeked at the internals of nix so maybe there are reasons why what I am suggesting canât work, but it seems to me:
If I can do it outside of nix-build, then surely nix-build itself could be taught
FODs are never going away, and you could think of this as a nix-prefetch-url but for derivations, and no-one disputes the utility of prefetching urls
Its crazy to me that the alternative seems to be trying to inject json into the logs rather than just saying what you mean which is âplease sir, calculate me the hash I wantâ.
There is one complication that would make this⌠messy. FODs donât have ssl certs available. Even when they download an https url, they actually just accept certificates blindly. This is fine normally, since the specified hash protects you against malicious action anyway. This is also fine these days when you explicitly change the hash argument in the code, because thereâs a condition in the code which brings in the local nixpkgsâ version of the certificate files when the supplied hash is fakeHash. (and hopes that youâre working with a recent enough nixpkgs that this worksâŚ) However nix itself only knows about the FOD itself, not the function in nixpkgs that generates it. If it just inserted a fakeHash at the bottom level, youâd be downloading with no authentication.
Speaking of which, Iâm not 100% sure your override isnât causing the same issue in the code you wrote above. If youâre overriding the underlying derivation and not the fetcher invocation, then it would. Not sure if fetchers have been changed to be more properly overrideable than they used to beâŚ
Ha! I was just about to ask, am I potentially opening a wormhole? I guess I need to look harder at what nix-update is doing, because I was basically transposing that.
The problem isnât how you wrote the script, itâs how you abstracted out the changed FOD. I believe nix-update actually modifies the source code in place, so itâs changing the arguments to the fetcher itself, not invoking any overrides.
EDIT: Yeah, a quick look at the code suggests fetchers havenât gotten any more overrideable since I last checked. Youâre very likely running without certs here.
Well now I am very glad I posted this. I guess in a world with perfect memory safety I could just run the script âtwiceâ i.e. the second time with the calculated hash, and if it worked then no one fooled me. Not really a great game to play tho. I guess Iâll make it work with in place updates.
All the more reason to maybe try to push something into nix itself with certs.
You might want to consider an external pinning system like nvfetcher. Generally a lot less painful than rolling a custom update script. Though it does depend what kind of sources youâre following.
I am an avid npins user, the problem is that the yarnOfflineCache which is what I am trying to override isnât something that you can just âfetchâ as far as I am aware without running something like yarn2nix to generate nix from the lock file? It doesnât seem from a skim that nvfetcher supports this.
Also, maybe I am being dumb here, but looking at the nix_prefetch function combined with how itâs called when updating yarn (called here from here) makes me suspicious that if I have a problem then so does nix-update?
EDIT: actually isnât this all a bit upside down: yarn is fetching all the things, not nix, and then dumping that fetch into a folder and we are checking the hash of that? So nixâs treatment of ssl doesnât really come into it?
nix isnât doing the fetching in the case of any FOD. In pkgs.fetchurl, curl does the fetching, without regard to nix. The issue is how that derivation is written, not anything to do with ânixâs treatment of sslâ. Itâs really more ânixpkgsâ treatment of sslâ.
You are not dumb. Weâre oblivious, apparently. What nix-update is doing is indeed not safe. I didnât quite believe it, since I know people worry about this exact issue and I knew it was thought of as handled correctly in nix-update⌠it seems it isnât.
Thatâs fair I was being sloppy with terminology. I meant nixpkgs fetchers / nix builtin fetchers vs yarn, which surely is built with tls support, irrespective of how where it is called from.
Iâm just now realizing nix-update isnât the same as nixpkgs-update, which is whatâs used to automatically update nixpkgs packages, and does handle this safely. So the issue isnât quite as serious as I thought it was.
I think it does get a fair amount of play (rg 'nix-update ') seems to show a decent number of packages using it directly for their update scripts: for instance.
Because reproducibility means youâre using old certs when you run old FODs, so theyâd start failing after a while with complaints about certs not verifying, which doesnât actually matter, since you already have an even stronger guarantee that youâre getting the bits you want in the form of the specified hash.