Hi, I’ve been a happy user of Nix Flakes (on Manjaro and Fedora Linux) for development for a while. Currently, I am attempting to package the Velociraptor script runner/manager so that I can use it in some applications I’m working on, but there appears to be very little documentation on how to build Deno packages with Nix.
Velociraptor’s documentation suggests that running deno install is their preference, which downloads the main script, aggregates all dependencies, caches them in the local Deno cache, and then adds a shell script to launch. This seems less-than-ideal for Nix, so I’m looking at using the deno compile command, in conjunction with fetchFromGitHub to build a self-contained binary that can be installed in the Nix store. My current attempt looks like this:
{ stdenv, fetchFromGitHub, deno }:
stdenv.mkDerivation rec {
name = "velociraptor-${version}";
version = "1.2.0";
src = fetchFromGitHub {
owner = "jurassiscripts";
repo = "velociraptor";
rev = "acb9e6eb5c1bbf84e9f7b0c4c5d46222f61df279";
sha256 = "sh4LnzLa0TL0KaL8lAsiWl90pKJvmphGf4u4PncGvww=";
};
buildInputs = [ deno ];
postUnpack = "deno cache $sourceRoot/cli.ts";
buildPhase = "deno compile --output vr ./cli.ts";
installPhase = "mkdir -p $out/bin && install -t $out/bin vr";
}
The postUnpack step fails, however, with a DNS failure. This makes sense to me, as Nix builds are supposed to be hermetic. This leads to a few concrete questions:
- Is there movement within Nixpkgs to create a standard framework (similar to those for Python, Rust, and Haskell) for fetching Deno dependencies?
- Is there a recommendation for this kind of build? The
deno info command is capable of emitting a complete list of transitive dependencies with URLs and checksums – is there a way to use this to create a Deno cache (that lives in the Nix store) that I can then feed to deno compile in the buildPhase of the velociraptor package?
3 Likes
(I believe the thread Status of lang2nix approaches is an attempt to come up with a standardized solution to this issue.)
You need to essentially generate a list of sources that deno cache would install and then organize them such that deno can load them as if they were cached. It’s enough work that it is encourage to write a dedicated converter e.g deno2nix. There’s some standardization to make this easier but it’s in the early stages.
As far as getting the data you need from deno, it appears that getting the URL & sha256 sums should be easy.
XDG_CACHE_HOME=/tmp/deno deno cache --lock-write --lock deno.lock https://deno.land/std@0.111.0/http/file_server.ts
This will produce a deno.lock file with each of the URL + sha256 checksum. I have confirmed that this is identical to the hash nix would procuce.
For example:
# Using nix store prefetch-file
nix store prefetch-file https://deno.land/std@0.111.0/_util/assert.ts
Downloaded 'https://deno.land/std@0.111.0/_util/assert.ts' to '/nix/store/8cy592azy5vbg5gb4ggdafvrdznks415-assert.ts' (hash 'sha256-L4aBRaBCoR1a0KPHSNz1gK3YoNvA6HbqoAJjA6VIj1g=').
# Using deno cache
"https://deno.land/std@0.111.0/_util/assert.ts": "2f868145a042a11d5ad0a3c748dcf580add8a0dbc0e876eaa0026303a5488f58"
# Convert to base64 sha256:
nix hash to-base64 --type sha256 2f868145a042a11d5ad0a3c748dcf580add8a0dbc0e876eaa0026303a5488f58
L4aBRaBCoR1a0KPHSNz1gK3YoNvA6HbqoAJjA6VIj1g=
This is sufficient to use fetchurl to grab the source:
src = fetchurl {
url = "https://deno.land/std@0.111.0/_util/assert.ts";
sha256 = "2f868145a042a11d5ad0a3c748dcf580add8a0dbc0e876eaa0026303a5488f58";
};
# I think it's OK the leave the sha256 in the current base but not 100%. Otherwise just convert using nix hash
This src can then be an input to a function that recreates the deno cache directory structure… along the lines of this:
transformedSrc = deno2nix.convert2nix {
name = "std_util_assert";
another_needed_variable = "maybe the version?";
src = fetchurl {
url = "https://deno.land/std@0.111.0/_util/assert.ts";
sha256 = "2f868145a042a11d5ad0a3c748dcf580add8a0dbc0e876eaa0026303a5488f58";
};
};
I looked a bit at the structure but I’m not sure how it generates it from the downloaded files. There is also probably a better way to set the cache location than using XDG_CACHE_HOME. I’d recommend looking at some other ‘2nix’ converts like composer2nix and yarn2nix (in nixpkgs actually now).
1 Like
Just noticed the replies, thanks!
I decided to dig into both the Deno internals and the Nixpkgs Rust infrastructure, and there I discovered the fetchCargoTarball function. I decided to attempt to emulate that with Deno, and I had some success with that approach1. That said, I’m curious if people here think that’s an appropriate way to go about packaging a Deno application, as I’d like to be able to merge some of this work to Nixpkgs proper.
I did successfully build a Velociraptor package2 and look forward to the discussions that will lead to a deno2nix in Nixpkgs itself!
1 Like
Looks good to me. If you can add it to the nixpkgs repo in a similar format & structure as the rust tool, I think it would be fine to open a PR. If there’s specific concerns that come up it’s nice to have them associated with a PR anyway.
1 Like
@moosingin3space I have hit the exact same issue as you did, how did you solve this at the end?
FYI, I created a flake, deno2nix, for packaging application written by deno. If you interested it, try this.
1 Like
does i make any sense to “compile” a deno package in nixos, i mean the size are the same
deno compile --unstable https://deno.land/std@0.79.0/examples/cat.ts
packaging instead a bash script like deno run (with source and deps) isnt it more memory efficient?
