I’ve got several shell scripts I want to turn into packages. But, I want to keep them stored in my Git repo as ‘.sh’, read via my Flake (builtins.readFile), then process into a package via pkgs.writeShellScriptBin, and then put them in my overlay.
I know - it’s overly complex.
I’ve taken a crack at it here, which doesn’t work yet.
I wanted to see if anyone has done this before. Am I over-complicating things? A major requirement is for the shell scripts to be in the repo as .sh files, not in a Nix file.
pkgs.runCommand "shymega-scripts" { scripts = path/to/scripts; } ''
for script in "$scripts"/*; do
install -DT "$script" "$out/bin/$(basename "$script" .sh)"
# Call any substitution functions you may need here...
done
''
If you specifically want a separate package for each script, I’d focus on how you intend to specify the dependencies for the scripts when they differ since that may serve as a clarifying question.
If the answer is that you’re just going to specify the union of the dependencies for every script anyways, I’d just keep it simple as Ryan’s suggesting.
If you’re going to tailor the dependencies to each, most of the fancy logic for iterating an entire directory of scripts will be moot since you’ll need a spec per script.
This is unfortunately one of those places where you’ll find a lot of approaches out in the wild. There are a few ways to approach the basics, and then some people also layer on some functions to customize/automate bits of it.
The simplest probably goes something like:
That’ll work best if the scripts are short and you know all of the dependencies already.
You can also do something very similar (but more flexible) by using stdenv.mkDerivation, makeWrapper, and lib.makeBinPath. The same file even has an example:
If some of the scripts are long or complicated enough that forgetting about dependencies and getting runtime errors is a concern, you can also do something similar with resholve:
(resholve is basically blocking the build if it can’t find all of the dependencies. The grammar for specifying everything is probably overkill for simple scripts when you know the dependencies, but it helps reliably package more complex scripts.)
Some people will write one or more utility functions that augment these main approaches. I stumbled on a few of these while trying to find the examples above, so I’ll leave them here for reference. It’s easy to over-engineer this stuff, so I’ll caution I would personally just keep it simple until you’ve repeated something enough to annoy you:
I personally don’t use resholve as it uses an end-of-life version of python and is lacking documentation and widespread understanding. For shell scripts I stick to writeShellApplication which at least runs shellcheck to avoid common shell mistakes. It’s also fairly straightforward to use.
If the scripts were already written elsewhere, and you just want to drop them in your config wholesale, makeWrapper is a suitable solution as well.
For anything fairly complex (I set an arbitrary threshold for myself here), I find shell languages unsuitable due to how “quirky” bash and friends can be, and instead prefer more “normal” programming languages. Unfortunately, that does mean a bit more effort in terms of packaging, though I’m accustomed to packaging stuff for nixpkgs anyway, so doing the same for my own config is fine for my usecase.
PS none of these solutions ever need readFile; you would only use this function if you need the nix evaluator to see the contents of the file (which you don’t need at all for this usecase).
Despite having read the link you provided in the past, I don’t have a clear picture of how to use resholve, or even what problem it solves that the “trivial builders” don’t solve already. I could nitpick about the docs if you’d really like (under a new topic), but docs aside, I feel biased against the solution already - especially for someone new to nix, I’d rather steer them towards well-known solutions.
Happy to field nits anywhere that is easy for you to raise them. (Good timing. Some updates to the README are next on my TODO list after I get past some family stuff and the Nix installer updates / migration script for trouble caused by the upcoming macOS Sequoia update.)
Since the README includes basic examples for each Nix function, it may help if one nit picks at what parts are unclear.
As stated in my comment earlier and in resholve’s own README, it blocks the build if it sees external dependencies in the script that you haven’t specified.
(But it’s fair to object that the in-tree README in nixpkgs doesn’t explicitly connect these dots. It’s trying to strike a balance between the maintenance drag of a full Nth copy of documentation in resholve’s repo + man page and a terse reference for the Nix API. Maybe it’s a lukewarm compromise.)
Not sure why this is framed as a disagreement (i.e., I’d rather steer). My comment also steers Dom towards writeShellApplication and makeWrapper by mentioning them first. It also directly cautions that resholve is likely overkill unless they’ve got long/complex scripts.
I appreciate everyone’s input. In the end, I decided to make my derivations for my tools, which I prefer. I have public and private Flakes for my packages and modules.
Well to be clear, all of the solutions provided are derivations…end of the day there are a dozen ways to do it, whichever works best for your use case.
