mkYarnPackage: ENOENT error from dependency during nix build, but not manual build in dev shell

Help
2

I solved this eventually, so I will summarize for other new nix users who may need to mess around with mkYarnPackage. The main thing I knew was that the error is coming from something called yazl, which is related to zip files, so possibly trying to package the .vsix file (which is just a zip).

       > Error: ENOENT: no such file or directory, stat '/build/5sg0ck2mfpyccf2pzv1jr133lk15ravx-source/deps/command-server/command-server'
       > Emitted 'error' event on ZipFile instance at:

After enabling the pkgs.breakpointHook as a buildInput (which I only just learned about), I entered the build sandbox with cntr and searched for references to command-server using:

find . -iname "*command-server*" 2>/dev/null

This led me to see the following the dep folder, where we see there is some invalid symlink. This link corresponds to the file that was being complained about above.

[nixbld@localhost:.]$ ls -al build/9jpv38f3v36hqnm82w99aginlrsynl6p-source/deps/command-server/
total 188
drwxr-xr-x 9 nixbld nixbld  4096  1月 18 04:00 .
drwxr-xr-x 3 nixbld nixbld  4096  1月 18 04:00 ..
drwxr-xr-x 3 nixbld nixbld  4096  1月 18 04:00 .cache
-rw-r--r-- 1 nixbld nixbld   584  1月  1  1970 CHANGELOG.md
lrwxrwxrwx 1 nixbld nixbld    19  1月 18 04:00 command-server -> deps/command-server
-rw-r--r-- 1 nixbld nixbld     0  1月 18 04:00 command-server.vsix
-rw-r--r-- 1 nixbld nixbld    10  1月  1  1970 .envrc
-rw-r--r-- 1 nixbld nixbld   540  1月  1  1970 .eslintrc.json
-rw-r--r-- 1 nixbld nixbld   569  1月  1  1970 flake.lock
-rw-r--r-- 1 nixbld nixbld  2300  1月  1  1970 flake.nix
drwxr-xr-x 2 nixbld nixbld  4096  1月  1  1970 .github
-rw-r--r-- 1 nixbld nixbld    43  1月  1  1970 .gitignore
-rw-r--r-- 1 nixbld nixbld  1076  1月  1  1970 LICENSE
lrwxrwxrwx 1 nixbld nixbld   105  1月 18 04:00 node_modules -> /nix/store/ys3bczil4nlvaa7n5hyshrivafs46k84-command-server-modules-0.9.0/deps/command-server/node_modules
drwxr-xr-x 3 nixbld nixbld  4096  1月 18 04:00 .npm
drwxr-xr-x 3 nixbld nixbld  4096  1月 18 04:00 out
-rwxr-xr-x 1 nixbld nixbld  3652  1月  1  1970 package.json
-rw-r--r-- 1 nixbld nixbld  3760  1月  1  1970 README.md
drwxr-xr-x 3 nixbld nixbld  4096  1月  1  1970 src
-rw-r--r-- 1 nixbld nixbld   584  1月  1  1970 tsconfig.json
-rw-r--r-- 1 nixbld nixbld  2616  1月  1  1970 vsc-extension-quickstart.md
drwxr-xr-x 2 nixbld nixbld  4096  1月  1  1970 .vscode
-rw-r--r-- 1 nixbld nixbld   145  1月  1  1970 .vscodeignore
drwxr-xr-x 3 nixbld nixbld  4096  1月 18 04:00 .yarn
-rw-r--r-- 1 nixbld nixbld 95815  1月  1  1970 yarn.lock
-rw-r--r-- 1 nixbld nixbld    21  1月  1  1970 .yarnrc

There is no deps subfolder above, so the link is invalid. Simply by removing the file in my buildPhase hook, I could prevent the error. I suspect there might be something in
mkYarnPackage’s configurePhase that is doing this automatically as I can see a bunch of special function setting up dependency links, but I need to dig more into what exactly it’s doing to know for sure.

Although that fixed the build problem, that’s not actually enough to get the extension to work. It builds, but it doesn’t actually do anything when loaded. To diagnose this I took a real .vsix file that I had built manually, and compared the contents to one that I built with the flake. It was clear that the folder that the packaging was occurring from was incorrect, as there was no out or node_modules folders, which were in the working extension. Further poking around the build sandbox made me realize that the deps/command-server folder has most of what we want, including out. The main problem is that the node_modules file in that folder is a symlink, and moreover the linked path doesn’t seem to be the one we actually want, as it only contains a @types folder. More digging around made me realize where to find the node_modules folder that actually contains the dependencies that we want.

So the final solution was to something like this:

buildPhase = ''
  # yarn tries to create a .yarn file in $HOME. There's probably a
  # better way to fix this but setting HOME to cwd works for now.
  export HOME="."
  yarn --offline run compile
  # Remove strange non-existent symlink created during packaging
  rm ./deps/command-server/command-server
  rm ./deps/command-server/node_modules
  cp -R ./node_modules ./deps/command-server
  pushd ./deps/command-server
  echo y | yarn --offline vsce package --yarn -o ./$pname.vsix
  popd
'';

I think this is a interesting example of how powerful knowing about using cntr and pkgs.breakpointHook is, as otherwise I think I would have been totally lost.

I did notice some other people do the delete and copy dance for node_modules, such as
nixpkgs/pkgs/applications/office/micropad/default.nix, so I’m guessing maybe what I had to figure out above is normal process for packaging yarn-related things?

I hope that helps someone!

3 Likes