Is it a reasonable idea to use Nix as a replacement for shell/python scripting?

Context: I have a transpiling toolchain built in Haskell, I’d like to write a helper script that invokes the toolchain for transpiling input file. Having nix available is not an issue, since the toolchain mandates the availability of nix and using it to have a reproduciable environment.

Choices: Since maximizing the portability is not a goal, I don’t have to stick to shell script. And since Nix is available, I could actually use any other scripting system. I also do want to minimize dependency. Hence Nix come in to my mind.

Is it a reasonable idea? Any similar open source project I could borrow some ideas from?

It’s reasonable, but there are at least a few different patterns here and I’m not (personally) sure there’s enough context here to know what’s ideal for your project.

Maybe a good place to start is whether you’re building the toolchain itself as one or more Nix packages, or whether you’re just using Nix to provide dependencies? (If that isn’t clear, a similar question is: can people use your toolchain via Nix without cloning the repo, or are all users working with a local clone?)

2 Likes

Okay, I am giving it some mental space, but first challenge is, is there a ready to be used getopts equivalent to bash scripting?

I don’t think so, but you can use --argstr myvariable myvalue as named parameters

I may need to revise my statement a bit.

I read the quote above (well, the previous edit of it) as suggesting that you are looking to use Nix to automatically provide an interpreter and dependencies for an executable script because it avoids system preconditions other than installing Nix (which people will need anyways). I imagined you might be curious about writing it in Haskell to avoid adding additional dependencies to your project.

But this sounds like you’re asking about how to parse arguments in the Nix language itself, so I imagine my first impression was wrong. I guess you’re asking about whether it makes sense implement the logic of your scripts in the Nix language?

If that’s right, I don’t recommend trying to do this directly in Nix.

2 Likes

yea, I could use Haskell. But it feels it’s the sort of job better suited using scripting language.

Yes, using Nix for “scripting”.

It does sound like not a good idea, probably because Nix is more optimized for the build sytem domain not general purpose scripting.

1 Like

It’s easy enough to find people inlining scripts written in one language or another within a nix expression, but I don’t recall seeing, cli argument parsing in nix.

It’s definitely possible to write an ~API wrapper in nix that’ll convert nix datastructures into, say, a command-line invocation of some tool–but that’s more like the opposite of what you mean.

Maybe folks in the guix ecosystem have patterns for this?

I wonder, do you just not want to use shell scripts (because it’s ugly, etc.), do you want to stay in the functional ecosystem or is there another reason?

Shell is usually used as “glue” because it’s so easy to run external programs with it. Therefore many build systems fall back to it for the actual work. E.g. the nix build “phases” are all usually shell scripts aswell.

I don’t exactly know what your “transpiling toolchain” exactly does, but if it’s “dynamic” (e.g. various options to build something), then you might create a nix derivation which takes inputs like for example the emacs derivation (though the emacs derivation is quite complex, so I’m not sure that’s the best example. It’s just the first that came to my mind).
But Nix isn’t really targeted at being able to build a thing in different ways. Rather the opposite: if you have a tool with a complex build process, Nix is extremely good at defining one process to build that thing in a reproducible and easily shareable manner (with usually no configuration options at all).

I’d argue that a bash script (even with it’s ugly argparsing and stuff) might be the best way to get a simple build tool unless it’s already clear that the script will have to do some heavy lifting in the future… Typically the main alternative would then be either make/ninja or your suggestion of haskell(/python/etc.)

Depending on what you want, maybe a script like the following might already do the job. Note the last part:

if test $# -eq 0; then
  # clean all if no args
  all
else
  "$@"
fi

That means if you run the script like clean.sh it will run the “default” all function inside this script, but you can also run clean.sh cpp to run the cpp function. That’s an easy way around the ugly shell argparsing if all you want is provide some simple options.

1 Like

In all seriousness, I am just brainstorming before getting hands dirty for this part of the project. And Bash is my comfort zone, going to it would otherwise be a default option.

Thanks for your thoughtful musing together!

Well, you can try google/zx or if you want something fancy this is example from shh:

writers.writeHaskellBin "example" {libraries = [haskellPackages.shh];} ''
  {-# LANGUAGE TemplateHaskell #-}
  import Shh

  -- Load binaries from Nix packages. The dependencies will be captured
  -- in the closure.
  loadFromBins ["${git}", "${coreutils}", "${curl}"]

  main :: IO ()
  main = do
    cat "/a/file"
    cp "/a/file" "/b/file"
''
7 Likes

We should consider bringing NIx into scripted and automated work gracefully rather than as a replacement for shell/python scripting.

Brainstorm from the real use case:

  • Hybrid mode (bash + getFlake)

    I don’t think so, but you can use --argstr myvariable myvalue as named parameters

First, use nixModues to generate a CLI interface https://github.com/nixosbrasil/climod (bash based)

Second, use the getFlake/import to get a Nix Project, ensuring we can pass the values from the shell to the Nix expr.(The flake doesn’t support passing an argStr so far)

case: tenzir/nix/static-binary.sh at 9999396a5d8cf74687a776778725a41b7778e289 · tenzir/tenzir · GitHub, to build the toolchain with different options quickly (this solution might be what you want).

  • NuScript

I like @tobim approach, which used NuScript to deal with the pipeline of the building status. Case: tenzir/.github/workflows/nix.nu at 9999396a5d8cf74687a776778725a41b7778e289 · tenzir/tenzir · GitHub

  • Python Command Interface

Use the Python library to create command-line scripting tools and through envrs to pass the values of the nix expression.

case: https://github.com/GTrunSec/std-ext/blob/44e045e3e682b7c8919129497a90d0a6c8038e5a/cells/cliche/entrypoints/default.nix#L11

  • Julia Command Interface

Use the Julia library to create command-line scripting tools and through envrs to pass the values of the nix expression.

  • Dag pipeline Jobs/tasks, powered by Tullia (CI/CD solution)

Building our toolchain by a Dag runner without writing a complex bash script.

Case: scienix/nix/julia/pipelines.nix at e3d4f5fb02a504c603920d0f2480f4b0e946c30f · GTrunSec/scienix · GitHub

Case: workflow-template - 3  prefect tasks

5 Likes

Wow! Bookmarked, such a great answer and extensive list to look into! Thanks!