UpdateScript written in nix considered bad practice (vs. bash)?

Help
21

I know that you are against it. That is why we are having this discussion :grinning:. Please let the others think on the problem themselves.

If the internal call to nix-shell was impure or something like that, this would be update-script-run-time, and way less concerning, indeed.

Then we are back to your bash only version, I know.

1 Like
22

If the internal call to nix-shell was impure or something like that, this would be update-script-run-time, and way less concerning, indeed.

Then we are back to your bash only version, I know.

Wait what? The updater would still be written in Nix, just, uhm, processed at a more fitting moment of time.

23

Can you show me what you mean in (pseudo) code?

24

I would probably get lazy.

That code with writeScript would probably be just yet another attribute in update-utils.nix. The file would need to be parameter to the script, so "$1" instead of the current immediate expansion via Nix. I would probably use ${common-updater-scripts}/bin/update-source-version notation, and probably use writeScriptBin. Then updateScript would be something like nix-shell -p '(import ${builtins.toString ./update-utils.nix}).updater' --run 'update-libreoffice.sh ${builtins.toString ./default.nix}'

(Pretty sure this is a suboptimal solution because this is not what I usually try to do… but the idea is to make impure evaluation happen only once updateScript is evaluated)

25

Hm, I am not sure if I follow you completely. You want to achieve that the evaluation of passthru.updateScript is pure?

In other words: nix eval .#libreoffice-bin.passthru should succeed without error: in pure evaluation mode, ‘fetchurl’ requires a ‘sha256’ argument?

I am not sure if that is possible. I need to think about it.

26

@7c6f434c

I tried to do what you mean with the latest commit here: https://github.com/NixOS/nixpkgs/pull/175160

But I don’t think it is possible to achieve what you want with the current code. Let me explain:

I use impure nix functions at evaluation time. This leads to the following sequence:

  1. Get/Precompute the latest libreoffice version in nix and write it to update.sh file with writeScript function (impure/side-effect at eval time)
  2. Run the update.sh file to update

If you want the side effect to happen during (== runtime) the update.sh file, then you must write the impure code in bash. So step 1 is not needed and therefore we can just write the complete update script in bash.

So why do I want this impurity in step 1? It isolates the side effect to eval-time (hard cut). If everything works, then I gain nothing. But if the side-effect (e.g. can’t download version/sha256) fails I immediately stop with a custom error message and don’t run the update.sh file.

We also have other benefits of isolating the side-effect:

  • We can use nix.
  • Even if you design your updateScript poorly, you will always isolate the side-effect to eval time, no matter how you write your code. Therefore you automatically get these benefits listed here if you use that approach.
  • We can test it easily. If we want to be extra safe we can even build the updateScript first with nix-build -A libreoffice-bin.updateScript and look at the update.sh file if it really contains the latest version instead of garbage.

We could of course do the same in bash and isolate the side-effect/download of version to a single bash-function and stop execution if it fails. This has the following disadvantages in my opinion:

  • Often if the bash script fails, the real error is swallowed by the update-source-version or other scripts and remains hidden.
  • We don’t get automatic isolation of the side-effect, you need to encapsulate into a function yourself.
  • The update script usually needs the following info: url, sha256, platform strings (possibly mapped) etc. All this info needs to be passed from the derivation (nix) or either duplicated.

In general, bash is a good approach and despite my comparison here, my main point is not nix is better than bash but the following: It is ok if you use impure nix at runtime too.

The only disadvantage of the nix approach is its impurity. As a result, you can only run it with the --impure flag: nix build .libreoffice-bin.passthru.updateScript --impure. Does this currently have any impact on nixpkgs?

No because the passthru attributes are removed before the stdenv.mkDerivation call and therefore the nixpkgs derivation remains pure. That is the nice purpose of the passthru attributes.
Only if you want to introduce a pure eval check of the passthru.updateScript attribute then it will fail.

But it will fail by design, because in my opinion the updateScript is not like any other derivation: It is an impure derivation by design.

To sum it up: It is ok to write your updateScript in impure nix at runtime.

Edited because it is possible to run the side effect at runtime. See post below.

27

You already have some of the impure work in bash. You already launch a Nix evaluation in the runtime (nix-shell). You can as well move the impure part there.

No, by design it is purely evaluated code for something impure and, worse, side-effect-ful to happen at runtime.

No.

28

It is indeed possible to move the side-effects/impurity to the nix-shell runtime. In my case, I did the following:

In the updateScript I only declare the impure dependencies using nix-shell as a script interpreter:

writeScript "update-libreoffice.sh" 
''
    #!/usr/bin/env nix-shell
    #!nix-shell -i bash impure-update.nix # computes and sets $newVersion, $newAarch64Sha256 bash variables using impure nix functions and pulls in all dependencies, e.g. update-source-version
    set -eou pipefail
    update-source-version libreoffice-bin $newVersion $newAarch64Sha256 --system=aarch64-darwin
    # ...
'';

and impure-update.nix:

# Impure functions, for passthru.updateScript runtime only
{ arg1
, arg2
, pkgs ? import ../../../../../default.nix { } 
}:
let
    getLatestStableVersion = ...
    getSha256 = ...
in
pkgs.mkShell rec {
  buildInputs = [ pkgs.common-updater-scripts ];
  newVersion = getLatestStableVersion; # impurity happening here
  newAarch64Sha256 = getSha256 arg1 ...
  newX86_64Sha256 = getSha256 arg2 ...
}

If you need to pass arguments from your derivation to your update.nix functions you can do that with:
nix-shell -i bash -argstr arg1 value1 --argstr arg2 value2.

Thank you @7c6f434c for the interesting discussion. It is always nice to learn something new :grinning:.

1 Like