Fetchzip results in do not know how to unpack source archive due to wrong nix store path

daniel-siepmann · September 26, 2022, 7:21am

Hi all,

I’m trying to create a derivation based on a .tar.gz file from the internet via the following .nix file:

{
  vimUtils,
  fetchzip,
  makeWrapper,
  lib,
  php80,
  php80Packages,
  git
}:

vimUtils.buildVimPluginFrom2Nix rec {
  pname = "phpactor";
  version = "3e7cb48b0ca429f92bcd57c236c39f170e9d23b1";
  src = fetchzip {
    url = "https://daniel-siepmann.de/fileadmin/${pname}-${version}.tar.gz";
    sha256 = "sha256-9LvkuQrOBw+BzKgDROY0xP77qc3m5vm25iSsmDQzuMs=";
  };

  nativeBuildInputs = [
    makeWrapper
  ];

  postInstall = ''
      wrapProgram $out/bin/phpactor \
        --prefix PATH : ${lib.strings.makeBinPath [ php80 ]} \
        --prefix PATH : ${lib.strings.makeBinPath [ php80Packages.composer ]} \
        --prefix PATH : ${lib.strings.makeBinPath [ git ]}
  '';
}

This worked in the path, but no longer with the latest version of the file. I used an empty string for sha256 and inserted the generated hash.
But the build fails with:

building '/nix/store/nbgg5x6kj8p1lrssvflf8bgh37bc5mm7-source.drv'...

trying https://daniel-siepmann.de/fileadmin/phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5723k  100 5723k    0     0  5746k      0 --:--:-- --:--:-- --:--:-- 5746k
unpacking source archive /build/phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.tar.gz
building '/nix/store/2f5f0x7axxib2wd1vaxxd1cf3bf7vkpv-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv'...
unpacking sources
unpacking source archive /nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source
do not know how to unpack source archive /nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source
error: builder for '/nix/store/2f5f0x7axxib2wd1vaxxd1cf3bf7vkpv-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv' failed with exit code 1
error: 1 dependencies of derivation '/nix/store/1kl15q3lrwslfz32gy7i7zw4yrdvd8kl-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv' failed to build
error: 1 dependencies of derivation '/nix/store/xlxfj0xh7fnn64sb5krxvdc9z9s3csc3-hm_nviminit.lua.drv' failed to build
error: 1 dependencies of derivation '/nix/store/xsfpz1fkbpakap9gxk7z4xhsybisc5bc-vim-pack-dir.drv' failed to build
error: 1 dependencies of derivation '/nix/store/mdkbf5ks28n311nsjlxyj8mmgnv3lzhz-home-manager-files.drv' failed to build
error: 1 dependencies of derivation '/nix/store/agsp4wawll33gwvnggdc24q1gr0dv5pl-neovim-0.7.2.drv' failed to build
error: 1 dependencies of derivation '/nix/store/makgn4padvzzsp5ychspyqd4f1gji6kn-home-manager-generation.drv' failed to build

And /nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source does not exist. I’ve just started with nix and any help would be welcome I guess the hash is somehow wrong? Looks like the extracted source is available under a different path.

BTW: I’m using nix 2.11.0 and home-manager 22.05 with nixpkgs master on an ubuntu system.

tobiasBora · September 26, 2022, 8:13am

How are you building your derivation? I tried to run it in a simple default.nix and it is working for me (and the hash is correct):

$ cat default.nix
{ pkgs ? import <nixpkgs> {} }:
pkgs.callPackage ./derivation.nix {}

$ cat derivation.nix
{
  vimUtils,
  fetchzip,
  makeWrapper,
  lib,
  php80,
  php80Packages,
  git
}:

vimUtils.buildVimPluginFrom2Nix rec {
  pname = "phpactor";
  version = "3e7cb48b0ca429f92bcd57c236c39f170e9d23b1";
  src = fetchzip {
    url = "https://daniel-siepmann.de/fileadmin/${pname}-${version}.tar.gz";
    sha256 = "sha256-9LvkuQrOBw+BzKgDROY0xP77qc3m5vm25iSsmDQzuMs=";
  };

  nativeBuildInputs = [
    makeWrapper
  ];

  postInstall = ''
      wrapProgram $out/bin/phpactor \
        --prefix PATH : ${lib.strings.makeBinPath [ php80 ]} \
        --prefix PATH : ${lib.strings.makeBinPath [ php80Packages.composer ]} \
        --prefix PATH : ${lib.strings.makeBinPath [ git ]}
  '';
}

$ nix-build
/nix/store/xmqrk30wr5k7dq3jg30gqccq6j6arady-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1

Note that I have not tested it further as I am not a vim user.

Can you also build it if you compile it as I do? If not maybe try to upgrade your channels, check that you are not behind a proxy (~~but a proxy would have cause an issue before as the hash would not be the good one~~ edit: it may compute the hash after the extraction)…

daniel-siepmann · September 26, 2022, 8:31am

Thanks for you help.

I use home-manager switch. But I get the following output following your approach:

nix-build
these 3 derivations will be built:
  /nix/store/nbgg5x6kj8p1lrssvflf8bgh37bc5mm7-source.drv
  /nix/store/2f5f0x7axxib2wd1vaxxd1cf3bf7vkpv-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv
  /nix/store/1kl15q3lrwslfz32gy7i7zw4yrdvd8kl-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv
these 30 paths will be fetched (12.59 MiB download, 64.13 MiB unpacked):
  /nix/store/0yw0w4ds9q34c1j7ghrg21gaj415dlym-gnumake-4.3
  /nix/store/35yhfk61ml4myj45fx1xsblh1iwys9b2-libssh2-1.10.0-dev
  /nix/store/3diddd01msw7v4rvd9f2sr69b07kbjlz-mirrors-list
  /nix/store/3jyz02sr95r9i1wnnlmh7pry2mlzmj4f-vim-gen-doc-hook
  /nix/store/541ykirikh68a1djyk5cz7w21f896hj3-patch-2.7.6
  /nix/store/56lap6lbsmhx0a6jshf4phc1b5ik6m01-stdenv-linux
  /nix/store/87nh2syrnydp10pcay26bvq23nv1a44j-nghttp2-1.47.0-dev
  /nix/store/8k0x0w978m2aakchrvzhzjpdr83yjnd8-vim-command-check-hook
  /nix/store/93711z38v7yqhyjh2z64z621wfrq82bj-nghttp2-1.47.0-lib
  /nix/store/9h54g26xv6hq0zb2kr127gc6iwdb835v-libssh2-1.10.0
  /nix/store/9kyw8a27xp8hspxv2lpvccdlbvnd181v-hook
  /nix/store/a0qsiqnwfbmw70k6q9wwldrjzk6nzak2-curl-7.84.0-man
  /nix/store/b88gp699lnznvsqcbkfqcm11sgnsdr6n-neovim-require-check-hook
  /nix/store/bgr48d8v22dqvha5vpabi2mbazc4jj47-openssl-3.0.5
  /nix/store/cbywlz71x7khq6s3qbnmn61g8ri6202z-nghttp2-1.47.0-bin
  /nix/store/chx7x4b12dpwzv74dsavcxqadhbxr71z-vim-9.0.0244
  /nix/store/cjl2qfa8h01l0qk1748ynbhwi1zv8vf6-stdenv-linux
  /nix/store/f1yk9kbnv0zikz64hn9hx218d04jq73l-patchelf-0.15.0
  /nix/store/gqjdfxidlf0430xiiz1nyada2vym5vbs-libkrb5-1.20
  /nix/store/i38j3pqxbiil11ngd1h5rndj4xr32krv-curl-7.84.0-dev
  /nix/store/j53sa7pgay8j6ds49vk3b900qqnk98jk-openssl-3.0.5-bin
  /nix/store/kacsc14cgv2jhcxh1sf6gdpz4m1w0896-libkrb5-1.20-dev
  /nix/store/ksly0020hw7i3wrdz5d170np7bbmpl79-nghttp2-1.47.0
  /nix/store/lm47r1xqyn4xyyq4fg4zmf11m7dxyblv-curl-7.84.0-bin
  /nix/store/mi645v4kmn1jixqx50wkdri6454m40x3-openssl-3.0.5-dev
  /nix/store/qbp8qpmxyz39mgn3qvz42nrbhh3ny752-curl-7.84.0
  /nix/store/rpa8lnnwccyak9nhfk5f49zhsbx68rvb-glibc-locales-2.35-163
  /nix/store/sd2s5syapsykjy5xdkx6407cx1bv9wpn-file-5.42
  /nix/store/wjh9x6hk6brpgx4ycgild0pdi4jak2s7-ed-1.18
  /nix/store/zwh8ddk6fjnpb28y5hv0vy8iv368sz5p-hook
copying path '/nix/store/9kyw8a27xp8hspxv2lpvccdlbvnd181v-hook' from 'https://cache.nixos.org'...
copying path '/nix/store/a0qsiqnwfbmw70k6q9wwldrjzk6nzak2-curl-7.84.0-man' from 'https://cache.nixos.org'...
copying path '/nix/store/wjh9x6hk6brpgx4ycgild0pdi4jak2s7-ed-1.18' from 'https://cache.nixos.org'...
copying path '/nix/store/sd2s5syapsykjy5xdkx6407cx1bv9wpn-file-5.42' from 'https://cache.nixos.org'...
copying path '/nix/store/rpa8lnnwccyak9nhfk5f49zhsbx68rvb-glibc-locales-2.35-163' from 'https://cache.nixos.org'...
copying path '/nix/store/0yw0w4ds9q34c1j7ghrg21gaj415dlym-gnumake-4.3' from 'https://cache.nixos.org'...
copying path '/nix/store/zwh8ddk6fjnpb28y5hv0vy8iv368sz5p-hook' from 'https://cache.nixos.org'...
copying path '/nix/store/gqjdfxidlf0430xiiz1nyada2vym5vbs-libkrb5-1.20' from 'https://cache.nixos.org'...
copying path '/nix/store/3diddd01msw7v4rvd9f2sr69b07kbjlz-mirrors-list' from 'https://cache.nixos.org'...
copying path '/nix/store/kacsc14cgv2jhcxh1sf6gdpz4m1w0896-libkrb5-1.20-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/b88gp699lnznvsqcbkfqcm11sgnsdr6n-neovim-require-check-hook' from 'https://cache.nixos.org'...
copying path '/nix/store/ksly0020hw7i3wrdz5d170np7bbmpl79-nghttp2-1.47.0' from 'https://cache.nixos.org'...
copying path '/nix/store/cbywlz71x7khq6s3qbnmn61g8ri6202z-nghttp2-1.47.0-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/93711z38v7yqhyjh2z64z621wfrq82bj-nghttp2-1.47.0-lib' from 'https://cache.nixos.org'...
copying path '/nix/store/bgr48d8v22dqvha5vpabi2mbazc4jj47-openssl-3.0.5' from 'https://cache.nixos.org'...
copying path '/nix/store/87nh2syrnydp10pcay26bvq23nv1a44j-nghttp2-1.47.0-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/9h54g26xv6hq0zb2kr127gc6iwdb835v-libssh2-1.10.0' from 'https://cache.nixos.org'...
copying path '/nix/store/j53sa7pgay8j6ds49vk3b900qqnk98jk-openssl-3.0.5-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/qbp8qpmxyz39mgn3qvz42nrbhh3ny752-curl-7.84.0' from 'https://cache.nixos.org'...
copying path '/nix/store/35yhfk61ml4myj45fx1xsblh1iwys9b2-libssh2-1.10.0-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/lm47r1xqyn4xyyq4fg4zmf11m7dxyblv-curl-7.84.0-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/mi645v4kmn1jixqx50wkdri6454m40x3-openssl-3.0.5-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/541ykirikh68a1djyk5cz7w21f896hj3-patch-2.7.6' from 'https://cache.nixos.org'...
copying path '/nix/store/i38j3pqxbiil11ngd1h5rndj4xr32krv-curl-7.84.0-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/f1yk9kbnv0zikz64hn9hx218d04jq73l-patchelf-0.15.0' from 'https://cache.nixos.org'...
copying path '/nix/store/chx7x4b12dpwzv74dsavcxqadhbxr71z-vim-9.0.0244' from 'https://cache.nixos.org'...
copying path '/nix/store/56lap6lbsmhx0a6jshf4phc1b5ik6m01-stdenv-linux' from 'https://cache.nixos.org'...
copying path '/nix/store/cjl2qfa8h01l0qk1748ynbhwi1zv8vf6-stdenv-linux' from 'https://cache.nixos.org'...
copying path '/nix/store/8k0x0w978m2aakchrvzhzjpdr83yjnd8-vim-command-check-hook' from 'https://cache.nixos.org'...
copying path '/nix/store/3jyz02sr95r9i1wnnlmh7pry2mlzmj4f-vim-gen-doc-hook' from 'https://cache.nixos.org'...
building '/nix/store/nbgg5x6kj8p1lrssvflf8bgh37bc5mm7-source.drv'...

trying https://daniel-siepmann.de/fileadmin/phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5723k  100 5723k    0     0  6282k      0 --:--:-- --:--:-- --:--:-- 6289k
unpacking source archive /build/phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.tar.gz
building '/nix/store/2f5f0x7axxib2wd1vaxxd1cf3bf7vkpv-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv'...
unpacking sources
unpacking source archive /nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source
do not know how to unpack source archive /nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source
error: builder for '/nix/store/2f5f0x7axxib2wd1vaxxd1cf3bf7vkpv-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv' failed with exit code 1
error: 1 dependencies of derivation '/nix/store/1kl15q3lrwslfz32gy7i7zw4yrdvd8kl-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv' failed to build

No need to verify if everything is working, I would already be happy if it builds.
I’m not behind a proxy.
I’ve updated channels this morning already. Did it once more, same result.

I probably lack the necessary understanding of nix/nixpkgs/nix store in order to find the issue. Looks like there is something wrong with my system? But I thought that’s one benefit of nix, this should not be the case.

I also run nix-store --gc in case there was some left over from previous runs with same hash but broken result.

tobiasBora · September 26, 2022, 9:18am

After reporting your issue on the Matrix help channel Sandro proposed to check the content of /nix/store/…-source by running it with

nix-build -K

(this will not remove the file after a build error)

daniel-siepmann · September 26, 2022, 10:16am

nix-build -K
these 3 derivations will be built:
  /nix/store/nbgg5x6kj8p1lrssvflf8bgh37bc5mm7-source.drv
  /nix/store/2f5f0x7axxib2wd1vaxxd1cf3bf7vkpv-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv
  /nix/store/1kl15q3lrwslfz32gy7i7zw4yrdvd8kl-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv
building '/nix/store/nbgg5x6kj8p1lrssvflf8bgh37bc5mm7-source.drv'...

trying https://daniel-siepmann.de/fileadmin/phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5723k  100 5723k    0     0  6002k      0 --:--:-- --:--:-- --:--:-- 5999k
unpacking source archive /build/phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.tar.gz
building '/nix/store/2f5f0x7axxib2wd1vaxxd1cf3bf7vkpv-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv'...
unpacking sources
unpacking source archive /nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source
do not know how to unpack source archive /nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source
note: keeping build directory '/tmp/nix-build-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv-1'
error: builder for '/nix/store/2f5f0x7axxib2wd1vaxxd1cf3bf7vkpv-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv' failed with exit code 1
error: 1 dependencies of derivation '/nix/store/1kl15q3lrwslfz32gy7i7zw4yrdvd8kl-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv' failed to build

ls -lah /nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source
ls: cannot access '/nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source': No such file or directory

I would have expected that folder/file to exist also after running home-manager switch. That’s what I checked already and why I wrote:

I guess the hash is somehow wrong? Looks like the extracted source is available under a different path.

But ls -lah /nix/store/*-source doesn’t look promising as well.

tobiasBora · September 26, 2022, 12:41pm

Hum that is strange… In my system I have the same hash but I can look at it without any issue:

$ ls -lah /nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source
total 17M
dr-xr-xr-x    14 root root     28 janv.  1  1970 .
drwxrwxr-t 13414 root nixbld  56K sept. 26 14:29 ..
dr-xr-xr-x     4 root root      5 janv.  1  1970 autoload
dr-xr-xr-x     2 root root      3 janv.  1  1970 bin
-r--r--r--     1 root root    37K janv.  1  1970 CHANGELOG.md
-r--r--r--     1 root root   4,4K janv.  1  1970 composer.json
-r--r--r--     1 root root   277K janv.  1  1970 composer.lock
dr-xr-xr-x     7 root root      9 janv.  1  1970 config
dr-xr-xr-x    15 root root     28 janv.  1  1970 doc
dr-xr-xr-x     3 root root      4 janv.  1  1970 ftplugin
dr-xr-xr-x     8 root root     14 janv.  1  1970 .git
dr-xr-xr-x     3 root root      5 janv.  1  1970 .github
-r--r--r--     1 root root    451 janv.  1  1970 .gitignore
…

My inital guess would be that nix is unable to download that file for some reasons (banned ip…) but then I don’t understand why you don’t have a more explicit error before. Can you download the https://daniel-siepmann.de/fileadmin/phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.tar.gz and successfully extract it manually on your system?

$ curl https://daniel-siepmann.de/fileadmin/phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.tar.gz --output phpactor.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5723k  100 5723k    0     0  12.9M      0 --:--:-- --:--:-- --:--:-- 12.9M

$ md5sum phpactor.tar.gz
878ece561b50b64862516cd467e6c188  phpactor.tar.gz

Also, which version of nix are you using?

$ nix --version
nix (Nix) 2.8.0pre20220411_f7276bc

In order to maximize reproducibility I tried to create a flake gist, can you try it?

$ git clone https://gist.github.com/4bcb9c7d43639fa7a5d5ecbad8933bd9.git 
$ cd 4bcb9c7d43639fa7a5d5ecbad8933bd9  
$ nix --experimental-features 'nix-command flakes' build .

daniel-siepmann · September 26, 2022, 12:48pm

I can download the file (It is my own server ;))

/tmp > curl https://daniel-siepmann.de/fileadmin/phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.tar.gz --output phpactor.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5723k  100 5723k    0     0  5696k      0  0:00:01  0:00:01 --:--:-- 5700k
/tmp > md5sum phpactor.tar.gz
878ece561b50b64862516cd467e6c188  phpactor.tar.gz
/tmp > nix --version
nix (Nix) 2.11.0

/tmp > tar -xf phpactor.tar.gz
/tmp > ls -lah phpactor-build
total 668K
drwxrwxr-x 14 daniels daniels  560 Sep 26 09:16 .
drwxrwxrwt 41 root    root    1.3K Sep 26 14:44 ..
drwxrwxr-x  4 daniels daniels  100 Sep 26 09:16 autoload
drwxrwxr-x  2 daniels daniels   60 Sep 26 09:16 bin
-rw-rw-r--  1 daniels daniels  37K Sep 26 09:16 CHANGELOG.md
-rw-rw-r--  1 daniels daniels 4.4K Sep 26 09:16 composer.json
-rw-rw-r--  1 daniels daniels 277K Sep 26 09:16 composer.lock
drwxrwxr-x  7 daniels daniels  180 Sep 26 09:16 config
drwxrwxr-x 15 daniels daniels  560 Sep 26 09:16 doc
drwxrwxr-x  3 daniels daniels   80 Sep 26 09:16 ftplugin
drwxrwxr-x  8 daniels daniels  280 Sep 26 09:16 .git
drwxrwxr-x  3 daniels daniels  100 Sep 26 09:16 .github
-rw-rw-r--  1 daniels daniels  451 Sep 26 09:16 .gitignore
drwxrwxr-x 19 daniels daniels  420 Sep 26 09:16 lib
-rw-rw-r--  1 daniels daniels 1.1K Sep 26 09:16 LICENSE
-rw-rw-r--  1 daniels daniels 1.2K Sep 26 09:16 Makefile
-rw-rw-r--  1 daniels daniels  21K Sep 26 09:16 phpactor.schema.json
-rw-rw-r--  1 daniels daniels  409 Sep 26 09:16 phpbench.json
-rw-rw-r--  1 daniels daniels 1.3K Sep 26 09:16 .php-cs-fixer.dist.php
-rw-rw-r--  1 daniels daniels 277K Sep 26 09:16 phpstan-baseline.neon
-rw-rw-r--  1 daniels daniels  591 Sep 26 09:16 phpstan.neon
-rw-rw-r--  1 daniels daniels  953 Sep 26 09:16 phpunit.xml.dist
drwxrwxr-x  2 daniels daniels   60 Sep 26 09:16 plugin
-rw-rw-r--  1 daniels daniels 2.8K Sep 26 09:16 README.md
-rw-rw-r--  1 daniels daniels   50 Sep 26 09:16 requirements.txt
drwxrwxr-x  4 daniels daniels   80 Sep 26 09:16 templates
drwxrwxr-x  9 daniels daniels  200 Sep 26 09:16 tests
drwxrwxr-x 22 daniels daniels  460 Sep 26 09:16 vendor

You’re flake is not working for me:

4bcb9c7d43639fa7a5d5ecbad8933bd9 > nix --experimental-features 'nix-command flakes' build .
error: builder for '/nix/store/2f5f0x7axxib2wd1vaxxd1cf3bf7vkpv-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv' failed with exit code 1;
       last 3 log lines:
       > unpacking sources
       > unpacking source archive /nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source
       > do not know how to unpack source archive /nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source
       For full logs, run 'nix log /nix/store/2f5f0x7axxib2wd1vaxxd1cf3bf7vkpv-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv'.
error: 1 dependencies of derivation '/nix/store/1kl15q3lrwslfz32gy7i7zw4yrdvd8kl-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv' failed to build

4bcb9c7d43639fa7a5d5ecbad8933bd9 > nix log /nix/store/2f5f0x7axxib2wd1vaxxd1cf3bf7vkpv-vimplugin-phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.drv
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source
do not know how to unpack source archive /nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source

I still don’t use flakes and need to switch some day …

I guess one issue is that there is a subfolder after extracting. Will check that. But this would not explain why it is working for you.

Thanks for all your time so far

tobiasBora · September 26, 2022, 1:05pm

Super weird… I have no idea what’s going on on your system (but I am not a nix guru…). Are you on NixOs or on another system? The only idea I have so far would be to try first to download the file manually and manually set src = ./phpactor.tar.gz; to see if the issue is still there (in my case it does work flawlessly). Also even if I don’t think the issue is linked to this, double check that you still have free space on the partition containing /nix/store and maybe try to garbage-collect/repair the store… but I’m sorry I don’t have other ideas for now, maybe wait for a more experimented user to come.

jtojnar · September 26, 2022, 1:10pm

It is weird.

I can also reproduce, running the following commands in Nixpkgs checkout df67b7738617ed1ddcfbae879b77d222ee531d89:

$ cat test.nix 
let
  pkgs = import ./. {};
in
pkgs.fetchzip {
  url = "https://daniel-siepmann.de/fileadmin/phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.tar.gz";
  sha256 = "sha256-9LvkuQrOBw+BzKgDROY0xP77qc3m5vm25iSsmDQzuMs=";
}

$ nix-build test.nix
this derivation will be built:
  /nix/store/nbgg5x6kj8p1lrssvflf8bgh37bc5mm7-source.drv
building '/nix/store/nbgg5x6kj8p1lrssvflf8bgh37bc5mm7-source.drv'...

trying https://daniel-siepmann.de/fileadmin/phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5723k  100 5723k    0     0  3916k      0  0:00:01  0:00:01 --:--:-- 3917k
unpacking source archive /build/phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.tar.gz
/nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source

$ file /nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source
/nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source: cannot open `/nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source' (No such file or directory)

Unless the archive is dynamically generated (i.e. the hash of the file can change), I would recommend just using fetchurl and letting mkDerivation unpack it.

daniel-siepmann · September 26, 2022, 1:25pm

I’m on Ubuntu using Nix and home-manager.

Your suggestion of pre downloading is working.
There is enough free space 159G. And I already run the nix-store --gc. nix-store --verify end with success. What exactly do you mean with repair the store?

daniel-siepmann · September 26, 2022, 1:25pm

Unless the archive is dynamically generated (i.e. the hash of the file can change), I would recommend just using fetchurl and letting mkDerivation unpack it.

The hash should normally not change. I didn’t know I don’t need to use the fetchzip.

Using fetchurl solves the issue. I fear you don’t have an explanation?

tobiasBora · September 26, 2022, 1:44pm

Good at least that you found a solution (good point jtojnar).

Funnily I cannot reproduce it even with your hash, pinning the nix version and after garbage collection:

$ cat aaa.nix                                                                                                                                                                                                                                                                       
let
  pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/df67b7738617ed1ddcfbae879b77d222ee531d89.tar.gz") {};
in
pkgs.fetchzip {
  url = "https://daniel-siepmann.de/fileadmin/phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.tar.gz";
  sha256 = "sha256-9LvkuQrOBw+BzKgDROY0xP77qc3m5vm25iSsmDQzuMs=";
}
$ nix shell 'github:NixOs/nixpkgs?rev=df67b7738617ed1ddcfbae879b77d222ee531d89#nix'
$ nix --version
$ nix (Nix) 2.11.0
$ nix-build aaa.nix
this derivation will be built:
  /nix/store/nbgg5x6kj8p1lrssvflf8bgh37bc5mm7-source.drv
these 24 paths will be fetched (4.87 MiB download, 26.68 MiB unpacked):
  /nix/store/0yw0w4ds9q34c1j7ghrg21gaj415dlym-gnumake-4.3
  /nix/store/35yhfk61ml4myj45fx1xsblh1iwys9b2-libssh2-1.10.0-dev
  /nix/store/3diddd01msw7v4rvd9f2sr69b07kbjlz-mirrors-list
  /nix/store/541ykirikh68a1djyk5cz7w21f896hj3-patch-2.7.6
  /nix/store/56lap6lbsmhx0a6jshf4phc1b5ik6m01-stdenv-linux
  /nix/store/87nh2syrnydp10pcay26bvq23nv1a44j-nghttp2-1.47.0-dev
  /nix/store/93711z38v7yqhyjh2z64z621wfrq82bj-nghttp2-1.47.0-lib
  /nix/store/9h54g26xv6hq0zb2kr127gc6iwdb835v-libssh2-1.10.0
  /nix/store/a0qsiqnwfbmw70k6q9wwldrjzk6nzak2-curl-7.84.0-man
  /nix/store/bgr48d8v22dqvha5vpabi2mbazc4jj47-openssl-3.0.5
  /nix/store/cbywlz71x7khq6s3qbnmn61g8ri6202z-nghttp2-1.47.0-bin
  /nix/store/f1yk9kbnv0zikz64hn9hx218d04jq73l-patchelf-0.15.0
  /nix/store/gqjdfxidlf0430xiiz1nyada2vym5vbs-libkrb5-1.20
  /nix/store/i38j3pqxbiil11ngd1h5rndj4xr32krv-curl-7.84.0-dev
  /nix/store/irg1wjsdl9iwwm1wdmpx1nfw82v04dmj-zlib-1.2.12-dev
  /nix/store/j53sa7pgay8j6ds49vk3b900qqnk98jk-openssl-3.0.5-bin
  /nix/store/kacsc14cgv2jhcxh1sf6gdpz4m1w0896-libkrb5-1.20-dev
  /nix/store/ksly0020hw7i3wrdz5d170np7bbmpl79-nghttp2-1.47.0
  /nix/store/lm47r1xqyn4xyyq4fg4zmf11m7dxyblv-curl-7.84.0-bin
  /nix/store/mi645v4kmn1jixqx50wkdri6454m40x3-openssl-3.0.5-dev
  /nix/store/qbp8qpmxyz39mgn3qvz42nrbhh3ny752-curl-7.84.0
  /nix/store/rpa8lnnwccyak9nhfk5f49zhsbx68rvb-glibc-locales-2.35-163
  /nix/store/sd2s5syapsykjy5xdkx6407cx1bv9wpn-file-5.42
  /nix/store/wjh9x6hk6brpgx4ycgild0pdi4jak2s7-ed-1.18
copying path '/nix/store/rpa8lnnwccyak9nhfk5f49zhsbx68rvb-glibc-locales-2.35-163' from 'https://cache.nixos.org'...
copying path '/nix/store/3diddd01msw7v4rvd9f2sr69b07kbjlz-mirrors-list' from 'https://cache.nixos.org'...
copying path '/nix/store/a0qsiqnwfbmw70k6q9wwldrjzk6nzak2-curl-7.84.0-man' from 'https://cache.nixos.org'...
copying path '/nix/store/ksly0020hw7i3wrdz5d170np7bbmpl79-nghttp2-1.47.0' from 'https://cache.nixos.org'...
copying path '/nix/store/cbywlz71x7khq6s3qbnmn61g8ri6202z-nghttp2-1.47.0-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/wjh9x6hk6brpgx4ycgild0pdi4jak2s7-ed-1.18' from 'https://cache.nixos.org'...
copying path '/nix/store/sd2s5syapsykjy5xdkx6407cx1bv9wpn-file-5.42' from 'https://cache.nixos.org'...
copying path '/nix/store/0yw0w4ds9q34c1j7ghrg21gaj415dlym-gnumake-4.3' from 'https://cache.nixos.org'...
copying path '/nix/store/gqjdfxidlf0430xiiz1nyada2vym5vbs-libkrb5-1.20' from 'https://cache.nixos.org'...
copying path '/nix/store/93711z38v7yqhyjh2z64z621wfrq82bj-nghttp2-1.47.0-lib' from 'https://cache.nixos.org'...
copying path '/nix/store/bgr48d8v22dqvha5vpabi2mbazc4jj47-openssl-3.0.5' from 'https://cache.nixos.org'...
copying path '/nix/store/541ykirikh68a1djyk5cz7w21f896hj3-patch-2.7.6' from 'https://cache.nixos.org'...
copying path '/nix/store/f1yk9kbnv0zikz64hn9hx218d04jq73l-patchelf-0.15.0' from 'https://cache.nixos.org'...
copying path '/nix/store/irg1wjsdl9iwwm1wdmpx1nfw82v04dmj-zlib-1.2.12-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/87nh2syrnydp10pcay26bvq23nv1a44j-nghttp2-1.47.0-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/56lap6lbsmhx0a6jshf4phc1b5ik6m01-stdenv-linux' from 'https://cache.nixos.org'...
copying path '/nix/store/kacsc14cgv2jhcxh1sf6gdpz4m1w0896-libkrb5-1.20-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/9h54g26xv6hq0zb2kr127gc6iwdb835v-libssh2-1.10.0' from 'https://cache.nixos.org'...
copying path '/nix/store/j53sa7pgay8j6ds49vk3b900qqnk98jk-openssl-3.0.5-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/35yhfk61ml4myj45fx1xsblh1iwys9b2-libssh2-1.10.0-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/qbp8qpmxyz39mgn3qvz42nrbhh3ny752-curl-7.84.0' from 'https://cache.nixos.org'...
copying path '/nix/store/mi645v4kmn1jixqx50wkdri6454m40x3-openssl-3.0.5-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/lm47r1xqyn4xyyq4fg4zmf11m7dxyblv-curl-7.84.0-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/i38j3pqxbiil11ngd1h5rndj4xr32krv-curl-7.84.0-dev' from 'https://cache.nixos.org'...
building '/nix/store/nbgg5x6kj8p1lrssvflf8bgh37bc5mm7-source.drv'...

trying https://daniel-siepmann.de/fileadmin/phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5723k  100 5723k    0     0  12.8M      0 --:--:-- --:--:-- --:--:-- 12.9M
unpacking source archive /build/phpactor-3e7cb48b0ca429f92bcd57c236c39f170e9d23b1.tar.gz
/nix/store/5j311rg5i7cd4zk8naksx0fc713dvsgf-source

Really strange… No idea why I am the only one with a working version here ^^

jtojnar · September 26, 2022, 2:13pm

fetchurl produces a fixed-output derivation whose output is the file downloaded from the URL as is. This does not work for files that can change, such as source archives generated by Git forges, since git archive output is not guaranteed to be stable, because when that happens, the specified hash would no longer match.

fetchzip produces a FOD from the extracted contents of an archive so the hash will still match if the archive’s metadata change, as long as the extracted files are the same.

mkDerivation accepts both archives and directories as src (in fact, fetchzip mostly just calls mkDerivation and passes it the fixed-output derivation attributes).

fetchurl is generally preferred for static files. But fetchzip should still work fine, looks like a bug in Nix.

jtojnar · September 26, 2022, 5:09pm

@lilyinstarlight pointed out on Matrix that this is FOD build with input references crashes nix/returns invalid store path · Issue #6660 · NixOS/nix · GitHub (triggered by Nix store paths in .git/hooks/*.sample files in the archive).

daniel-siepmann · September 27, 2022, 5:53am

Thanks for explaining. I’ll stick to fetchurl then. The file is created once and never changed. And is a workaround due to my bad knowledge of Nix.

I thought fetchzip would be the better approach but lacked the necessary knowledge.