I need to create a project involving electron and some libraries (easymidi), but I can’t find how to do:
If I install electron from Nix directly, for instance using nix-shell -p electron_17, I can start a basic hello world project, but then I can’t import other node libraries (if I install them with yarn, I get an error about unknown symbols, I guess because yarn and the nix electron runs different nodejs version
If I do everything using yarn or npm, electron can’t even start a basic hello world folder:
$ nix-shell -p yarn
$ git clone https://github.com/electron/electron-quick-start
$ cd electron-quick-start/
$ yarn install
$ yarn start
yarn run v1.22.15
$ electron .
events.js:377
throw er; // Unhandled 'error' event
^
Error: spawn /tmp/c/electron-quick-start/node_modules/electron/dist/electron ENOENT
at Process.ChildProcess._handle.onexit (internal/child_process.js:274:19)
at onErrorNT (internal/child_process.js:469:16)
at processTicksAndRejections (internal/process/task_queues.js:82:21)
Emitted 'error' event on ChildProcess instance at:
at Process.ChildProcess._handle.onexit (internal/child_process.js:280:12)
at onErrorNT (internal/child_process.js:469:16)
at processTicksAndRejections (internal/process/task_queues.js:82:21) {
errno: -2,
code: 'ENOENT',
syscall: 'spawn /tmp/c/electron-quick-start/node_modules/electron/dist/electron',
path: '/tmp/c/electron-quick-start/node_modules/electron/dist/electron',
spawnargs: [ '.' ]
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Any idea what I did wrong? How can I setup electron with some random libraries like easymidi?
EDIT
Here is the log file (tried the same with npm, same error):
0 info it worked if it ends with ok
1 verbose cli [
1 verbose cli '/nix/store/06680csx89n416lvsy2yljmw6rvibj8y-nodejs-14.18.0/bin/node',
1 verbose cli '/nix/store/06680csx89n416lvsy2yljmw6rvibj8y-nodejs-14.18.0/bin/npm',
1 verbose cli 'start'
1 verbose cli ]
2 info using npm@6.14.15
3 info using node@v14.18.0
4 verbose run-script [ 'prestart', 'start', 'poststart' ]
5 info lifecycle electron-quick-start@1.0.0~prestart: electron-quick-start@1.0.0
6 info lifecycle electron-quick-start@1.0.0~start: electron-quick-start@1.0.0
7 verbose lifecycle electron-quick-start@1.0.0~start: unsafe-perm in lifecycle true
8 verbose lifecycle electron-quick-start@1.0.0~start: PATH: /nix/store/06680csx89n416lvsy2yljmw6rvibj8y-nodejs-14.18.0/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/tmp/d/electron-quick-start/no
de_modules/.bin:/nix/store/yx24h6ywbkg7rm5x7lrm8mx8ypcf4ywi-bash-interactive-5.1-p8/bin:/nix/store/fj3ywsx22xvjd4mly4323ikjcavyv91v-patchelf-0.13/bin:/nix/store/s5hkav7whndbfz0szshpb46h4idqdq9a-gcc-wrapper-10.3
.0/bin:/nix/store/a2w3l1m75908yd05a0h40vnrzvxfd0gd-gcc-10.3.0/bin:/nix/store/mfz26azl9561jgd5n73nkszzp6qhsaal-glibc-2.33-50-bin/bin:/nix/store/xyn0240zrpprnspg3n0fi8c8aw5bq0mr-coreutils-8.32/bin:/nix/store/5d5j
1z9bg01wqxahihy2x5n22ykc32w8-binutils-wrapper-2.35.1/bin:/nix/store/a4mmjm3bblxwp8h53bcfx3dly80ib0ba-binutils-2.35.1/bin:/nix/store/06680csx89n416lvsy2yljmw6rvibj8y-nodejs-14.18.0/bin:/nix/store/xyn0240zrpprnsp
g3n0fi8c8aw5bq0mr-coreutils-8.32/bin:/nix/store/1nq62klcc9n2jv2ixaf77makkzdcghrh-findutils-4.8.0/bin:/nix/store/xwkxkx4bk005q35hsdhqbkbdv7g28cz5-diffutils-3.8/bin:/nix/store/dy4ylp9439la4lq35ah2mj80fi87pk4w-gnu
sed-4.8/bin:/nix/store/xxgddhdi57bbgd1yxza44plq6krjmiz1-gnugrep-3.6/bin:/nix/store/31pkw5yi08fj4l0glzvpf1cp4ywkxh86-gawk-5.1.0/bin:/nix/store/nwg8in201f7y6vdm787v3j84jjrn0ayw-gnutar-1.34/bin:/nix/store/ygzg6wzh
gxf51ianb4zjvrzq4ilx9jd7-gzip-1.10/bin:/nix/store/gakfgapj20lv13vkcz6c38j8i9vz4ypi-bzip2-1.0.6.0.2-bin/bin:/nix/store/xgp0bgw4rpnbc3vr2qdsdbixp3zy4v1l-gnumake-4.3/bin:/nix/store/wadmyilr414n7bimxysbny876i2vlm5r
-bash-5.1-p8/bin:/nix/store/347zp4r9a7gm5gk0gwijqw294nnyypcs-patch-2.7.6/bin:/nix/store/f2x98vk07px8916b9xid7jq6ky86sfmi-xz-5.2.5-bin/bin
9 verbose lifecycle electron-quick-start@1.0.0~start: CWD: /tmp/d/electron-quick-start
10 silly lifecycle electron-quick-start@1.0.0~start: Args: [ '-c', 'electron .' ]
11 silly lifecycle electron-quick-start@1.0.0~start: Returned: code: 1 signal: null
12 info lifecycle electron-quick-start@1.0.0~start: Failed to exec start script
13 verbose stack Error: electron-quick-start@1.0.0 start: `electron .`
13 verbose stack Exit status 1
13 verbose stack at EventEmitter.<anonymous> (/nix/store/06680csx89n416lvsy2yljmw6rvibj8y-nodejs-14.18.0/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:332:16)
13 verbose stack at EventEmitter.emit (events.js:400:28)
13 verbose stack at ChildProcess.<anonymous> (/nix/store/06680csx89n416lvsy2yljmw6rvibj8y-nodejs-14.18.0/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack at ChildProcess.emit (events.js:400:28)
13 verbose stack at maybeClose (internal/child_process.js:1058:16)
13 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:293:5)
14 verbose pkgid electron-quick-start@1.0.0
15 verbose cwd /tmp/d/electron-quick-start
16 verbose Linux 5.10.75
17 verbose argv "/nix/store/06680csx89n416lvsy2yljmw6rvibj8y-nodejs-14.18.0/bin/node" "/nix/store/06680csx89n416lvsy2yljmw6rvibj8y-nodejs-14.18.0/bin/npm" "start"
18 verbose node v14.18.0
19 verbose npm v6.14.15
20 error code ELIFECYCLE
21 error errno 1
22 error electron-quick-start@1.0.0 start: `electron .`
22 error Exit status 1
23 error Failed at the electron-quick-start@1.0.0 start script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]
The reason is that pretty much every native binary you download from the internet will fail to run on NixOS, because its interpreter will point at a file that’s not in the place it is on other Linux distros. You can try to patch every binary your package manager downloads, but this will keep breaking every time it updates.
npm projects, unfortunately, really like bundling pre-built binaries with dubious provenance, and won’t let you use pre-existing system tools. I think electron is usually one of them, sadly.
The solution are shell.nixes with various 2nix projects, but not every package manager is supported. I’m not aware of a project that permits this for yarn, at least not without some hacking.
Thanks for your answer. But I did try some of these tools without much success. For instance, with npmlock2nix:
$ cd /tmp
$ git clone https://github.com/tobiasBora/quickstart-electron
$ cd quickstart-electron
$ nix-shell -p niv
$ niv init
$ niv add nix-community/npmlock2nix
$ cat nix/default.nix # Edit this file to have this content:
# nix/default.nix
let
sources = import ./sources.nix;
pkgs = import <nixpkgs> {};
in
import sources.nixpkgs {
overlays = [
(self: super: {
npmlock2nix = pkgs.callPackage sources.npmlock2nix { };
})
];
}
$ cat shell.nix # Edit this file to have this content
let
pkgs = import ./nix/default.nix;
in
pkgs.npmlock2nix.shell {
src = ./.;
}
$ nix-shell
[...]
> process-nextick-args@2.0.1 preinstall /build/node_modules/process-nextick-args
> /build/node_modules/.hooks/preinstall
7m ......] - install:yallist: info lifecycle yallist@4.0.0~install: y
> electron@19.0.9 postinstall /build/node_modules/electron
> node install.js
RequestError: getaddrinfo ENOTFOUND github.com
at ClientRequest.<anonymous> (/build/node_modules/got/source/request-as-event-emitter.js:178:14)
at Object.onceWrapper (events.js:520:26)
at ClientRequest.emit (events.js:412:35)
at ClientRequest.origin.emit (/build/node_modules/@szmarczak/http-timer/source/index.js:37:11)
at TLSSocket.socketErrorListener (_http_client.js:475:9)
at TLSSocket.emit (events.js:400:28)
at emitErrorNT (internal/streams/destroy.js:106:8)
at emitErrorCloseNT (internal/streams/destroy.js:74:3)
at processTicksAndRejections (internal/process/task_queues.js:82:21)
npm WARN my-electron-app@1.0.0 No repository field.9.0.9~postinstall:[0m
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! electron@19.0.9 postinstall: `node install.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the electron@19.0.9 postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /build/.npm/_logs/2022-07-25T11_35_17_790Z-debug.log
error: builder for '/nix/store/6x7ynmm42ijjy4bmd15a77kwbvc2cfy0-my-electron-app-1.0.0.drv' failed with exit code 1;
last 10 log lines:
> npm ERR! errno 1
> npm ERR! electron@19.0.9 postinstall: `node install.js`
> npm ERR! Exit status 1
> npm ERR!
> npm ERR! Failed at the electron@19.0.9 postinstall script.
> npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
>
> npm ERR! A complete log of this run can be found in:
> npm ERR! /build/.npm/_logs/2022-07-25T11_35_17_790Z-debug.log
>
For full logs, run 'nix log /nix/store/6x7ynmm42ijjy4bmd15a77kwbvc2cfy0-my-electron-app-1.0.0.drv'.
$ cd tmp && git clone https://github.com/tobiasBora/quickstart-electron
$ nix flake init -t github:nix-community/dream2nix#simple
$ nix eval --impure --raw --expr 'builtins.currentSystem' > ./nix_systems && git add ./nix_systems
$ nix flake show
$ nix build .#default
warning: Git tree '/tmp/quickstart-electron' is dirty
error: Electron binary hashes are missing for required version 19.0.9
Please add the hashes in the override below the origin of this error.
To get the hashes, execute:
/nix/store/x6qa4ickfvbagkp9zj86ac49s6ffghvb-nodejs/electron/print-hashes.sh 19.0.9
(use '--show-trace' to show detailed location information)
$ /nix/store/x6qa4ickfvbagkp9zj86ac49s6ffghvb-nodejs/electron/print-hashes.sh 19.0.9
error: unable to download 'https://atom.io/download/electron/v19.0.9/node-v19.0.9-headers.tar.gz': HTTP error 403 ('Forbidden')
response body:
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>W44G2TW1WQEQYV9Y</RequestId><HostId>ijUyhkPzI3jD7+MnmrB4A7amot/QQTZlsLMB/z6AQM7qoInwWb7pCRT+KT6Ym9JYzvv/sb7MDog=</HostId></Error>
Hm, hard to help much further here. This will be quite a debugging story. That said:
Seems interesting, sounds like some binary is trying to download something at build time, which npmlock2nix simply can’t control for.
If you have the cycles to dig into it, I’ve resolved similar problems by figuring out which package is trying to do that, and then going to their source and seeing how on earth they are expecting their software to be built.
Often you’ll find that there is some kind of condition that checks whether you have something downloaded or installed already, and you can circumvent the problem by pre-downloading or installing that thing. Sometimes you can’t, and then you have to patch the package in question.
This is a serious smell, they’re downloading random pre-built stuff raw from github, and you have no control over it and can’t review what it’s doing even if you had an engineering crew of 10000 people to do so. These kinds of broken builds always make me wonder how much malware there is in the average npm project.
If you do figure out a workflow that works, that’d be awesome - would be nice to add a description of how you do electron builds with nix to the wiki or such.
I have less experience with dream2nix, but did you actually try to do what the error message says? Even if not, looks like the package.json might be outdated, that atom release can no longer be downloaded (you can verify this by trying to open the URL manually with a browser). Who knows why you need an atom release for electron builds. You should really delete it and run npm install --lock-file-only once before trying this. Ignore me, misread the message. Still, that file is simply not permitted to be downloaded by anyone, this is probably an electron problem?
Also, I think it’s important to mention the reason this is a pain is because of the specific npm library/libraries.
The majority of npm libraries are just javascript and CSS; they’ll work no problem. Even when there’s a node version mismatch, it’s usually not a big deal. No node-2-nix needed.
However, stuff like:
easymidi
puppeteer (needs headless chrome)
video-editing tools (need ffmpeg)
bluetooth libraries
git tools
notification-system
serial communication
Those are all going to hook into system level stuff.
Why do they work everywhere but nix? Well actually they don’t, they only “probably” work outside of nix. And there’s a giant gap, a large canyon of tech-debt, between “working 99%” and “working 100%”. Basically devs that use nix have to pay the tech-debt that the original author created because nix libraries require 100%.
So (as the other people in the thread are talking about) either someone already did the work to make easymidi reliable, or you’ll have to do it yourself. Sometimes it’s really really easy to do it yourself, there’s tools, like node2nix to help but they’re not bullet proof.
A quick guide is:
If the module is just bundling/downloading/looking-for binaries, it probably won’t be too bad. Either pre-install the needed binaries (ex: ffmpeg), or use nix patching tools to patch the downloaded binaries.
However… if npm module uses “node-gyp” then you’re in for a generally bad/painful time. Node-gyp is where the real electron/node version mismatches are a problem. Last I fought with it, they were not easy to solve.
Thanks for these precisions. For the record, I found that I wanted right now to spend more time prototyping than writing Nix derivations for stuff I might never use (I might if the project is published to more than me), so for now I just enable nix-ld, this way I can simply type
$ nix-shell -p nodejs_latest
And then type any npm command I wish like in a normal distribution (which is faster to use, but of course this is not pure anymore). And if later I need more reproducibility, I’ll investigate ^^’