My view is that we should have wasm derivations, and wasm derivation IFD should be good. It should basically be the same thing as builtins.wasm, and in a world with parallel eval, IFD in general should be no worse than builtins.wasm or builtins.exec.
7 Likes
I mean the binary you need to turn your input data (e.g. yaml) into JSON. And sure you can āpreprocessā it and e.g. commit it to the repo, but how are you going to review that it matches the input?
Parallel eval certainly would alleviate most of the bottleneck around IFD. But Iām just not convinced that setting up the nix sandbox and forking out a new process will ever be as performant as running a JIT WASM module from within Nix. Iām imagining scenarios where builtins.wasm is in a hot path that might run the same module 100ās or 1000ās of times, potentially nested. WASM should have no issue with that. Parallel IFD might handle it fine, but Iām skeptical.
Give Nix an IFD and it can do arbitrary data transformations efficiently once. Teach nix to use WASM and it can do arbitrary data transformations efficiently forever.
Binary blobs are certainly an issue but IMO you can IFD wasm once, publish it to your cache, and then use it forever. If IFD is truly just as performant as WASM when used in a hotpath, than thereās no reason to use WASM over IFD. Maybe one could make an argument for aesthetics, I certainly find WASM sandboxing to be cleaner than containers, which in turn are cleaner than Nixās diy sandboxing.
But again, Iām not convinced that IFD truly is just as performant as WASM. At the end of the day, this performance talk is all just speculation. We need data. And that data still wouldnāt help much because weāre comparing WASM to an imagined more performant version of IFD/eval that doesnāt exist.
2 Likes
IIRC determinant nix has parallel eval so this could be done in theory
whether we like it or notā¦
I donāt approve of this feature. Purity is key. This goes completely against Eelcoās original thesis.
2 Likes
Greenspunās Tenth Rule
(Although usually thatās a comment on what needs to be added to an inferior language; I donāt think thatās whatās going on here and Iād prefer nix to remain pure.)
1 Like
I donāt quite see how you both think this is impure. The WASI stuff @lucperkins was talking about sound like it is, but just builtins.wasm, as it now stands, seems at least as pure as IFD. Still would prefer parallel eval+IFD like mentioned here:
1 Like
builtins.wasm has one nice feature over IFD: it doesnāt make eval system-dependent.
4 Likes
Could be solved by installing a binfmt handler for wasm or something? Then can have your derivation generate wasm and execute it?
Required watching for anyone interested in these ideas
5 Likes