fetchFromGitHub vs fetchgit differences

Hello,

I have this simple derivation that uses fetchFromGitHub:

stdenv.mkDerivation rec {
  pname = "pgmq";
  version = "1.3.3.1-no-alter-extension";

  #   # https://github.com/tembo-io/pgmq/
  src = fetchFromGitHub {
    owner = "garganscript";
    repo = "${pname}";
    rev = "${version}";
    sha256 = "sha256-ZPnuAjieJM+C/ebNEr04ZN3Iw+J/mquHekJXXMNHelI=";
  };

  prePatch = ''
    ls -lah pgmq-extension
  '';
  configurePhase = ''
  '';
  buildPhase = ''
  '';
  installPhase = ''
  '';
}

(~cgenie/postgres-pgmq-nix-flake (fetchgit-problems): nix-tests/fetch-from-github/default.nix - sourcehut git).

I can run it with:

nix -L develop 'git+https://git.sr.ht/~cgenie/postgres-pgmq-nix-flake?ref=fetchgit-problems&dir=nix-tests/fetch-from-github'

and it produces this output:

pgmq> -rw-r--r-- 1 nixbld nixbld 1.6K Jan  1  1970 Makefile
pgmq> -rw-r--r-- 1 nixbld nixbld  445 Jan  1  1970 Trunk.toml.in
pgmq> -rw-r--r-- 1 nixbld nixbld  195 Jan  1  1970 pgmq.control
pgmq> drwxr-xr-x 2 nixbld nixbld 1.1K Jan  1  1970 sql
pgmq> drwxr-xr-x 5 nixbld nixbld  100 Jan  1  1970 test

Now I have a similar derivation but this time I use fetchgit:

stdenv.mkDerivation rec {
  pname = "pgmq";
  version = "1.3.3.1-no-alter-extension";

  #   # https://github.com/tembo-io/pgmq/
  src = fetchgit {
    url = "https://github.com/garganscript/${pname}";
    branchName = "${version}";
    sha256 = "sha256-OugBw+zW98ECg2PhSMLyOLaBzzWUv2hCMzLpXye/yCQ=";
  };

  prePatch = ''
    ls -lah pgmq-extension
  '';
  configurePhase = ''
  '';
  buildPhase = ''
  '';
  installPhase = ''
  '';
}

(~cgenie/postgres-pgmq-nix-flake (fetchgit-problems): nix-tests/fetchgit/default.nix - sourcehut git)

I run it with:

nix -L develop 'git+https://git.sr.ht/~cgenie/postgres-pgmq-nix-flake?ref=fetchgit-problems&dir=nix-tests/fetchgit'

and it produces this output:

pgmq> -rw-r--r-- 1 nixbld nixbld 1.3K Jan  1  1970 META.json.in
pgmq> -rw-r--r-- 1 nixbld nixbld 1.6K Jan  1  1970 Makefile
pgmq> -rw-r--r-- 1 nixbld nixbld  445 Jan  1  1970 Trunk.toml.in
pgmq> -rw-r--r-- 1 nixbld nixbld  193 Jan  1  1970 pgmq.control
pgmq> drwxr-xr-x 2 nixbld nixbld 1.1K Jan  1  1970 sql
pgmq> drwxr-xr-x 5 nixbld nixbld  100 Jan  1  1970 test

Please note that this time, the META.json.in file is included, unlike in fetchFromGitHub.

But obviously the META.json.in file is there: pgmq/pgmq-extension at 1.3.3.1-no-alter-extension · garganscript/pgmq · GitHub

In the prePatch step I added cat pgmq-extension/pgmq.control and I do see new version of the file (1.3.3.1) so I conclude that fetchFromGitHub fetched the correct branch. It just, for some unknown reason, skips the META.json.in file.

What gives? Why different outputs? Obviously the fetchFromGitHub fetcher is wrong here. I don’t understand this.

OK apparently the GitHub tarball is missing this file as well:

curl -L https://github.com/garganscript/pgmq/tarball/1.3.3.1-no-alter-extension | tar -zx --strip-components=1

So it seems fetchFromGitHub uses that tarball?

fetchFromGitHub fetches the tarball from github, which is generated by git archive, which is modified by the .gitattributes file in the repo. That .gitattributes file ignores some of the files in the git repo, including the META.json.in.

.gitattributes is a fun mechanic

2 Likes