Semver eval and NPM resolution in pure Nix

Development
1

This is a working implementation I wrapped up today that parses semver to SAT conditions and resolves closures against NPM package sets

This is implemented 100% with Nix using no IFD and if you lock the upstream metadata this operation can be purified.

This took months, and I’m happy to finally see it working. This implementation has some modest limitations in the closure, but the underlying evaluator can simplify expressions and merge ranges - ultimately this will solve SAT in the next week or two when I have time to wrap it up.

The semver closure is exposed in a top level flake ( just finished it still need to do real testing ), but it’s working on non-trivial sets with thousands of packages in them.

I’m also happy to report that it’s fast, especially with a warm TTL cache.

$ nix eval --impure --json  \
      github:aameen-tulip/at-node-nix#lib.libsat.packumentSemverClosureSerial  \
      --apply 'p: p { ident = "bunyan"; version = "1.8.15"; }'  \
    |jq;
{
  "isFlat": true,
  "roots": ["bunyan/1.8.15"];
  "packages": {
    ....
    "bunyan/1.8.15": {
      "ident": "bunyan",
      "sats": {
        "dtrace-provider": {
          "descriptor": "~0.8",
          "satisfies": ["0.8.0", ..., "0.8.7, "0.8.8"]
        },
        "moment": {
          "descriptor": "^2.19.3",
          "satisfies": [...]
        },
        "mv": {
          "descriptor": "~2",
          "satisfies": [...]
        },
      ...
      },
      "dtrace-provider/0.8.8": { ... },
     ...
  }
}

This is the abbreviated format meant for writing to JSON. If you drop Serial at the end of the function name you get the full thing that basically lets you walk the whole resolution process.

That routine doesn’t “deduplicate” or merge overlapping ranges yet, but the underlying evaluator can. I will get that hooked in soon. The isFlat field basically indicates that “there’s no point in deduplicating/merging constraints, so this should match NPM”.

they “keyed” version represents the “resolved version”, being the latest from the pool which satisfies the descriptor.

the underlying closure routine is written such that you can swap out alternative lookup routines. So you could for example do:

myInit = packumentClosureInit // {
  __lookupMeta = { packumenter, ident, version } @ args:
    if ident == "@foo/bar"
    then { inherit packumenter; versionInfo = lib.importJSON ./package.json; }
    else packumentClosureInit.__lookupMeta args;
};
# if any package depends on `@foo/bar' they'll get the info in `package.json' instead of the registry.
myClosure = packumentSemverClosure {
  startSet = [( myInit {
    ident = "@foo/quux"; version = "4.2.0";
  } )];
};

This is one piece of a new Nix+Node.js framework I will be releasing near the end of this year, floco, but I’m happy to share this milestone with y’all :grin:

2 Likes