Cargo2nix (network failure OR erroneously fetching master) with git repo dependency

This repo contains a flake which uses cargo2nix.

If I add a git repo dependency (any git repo dependency) to Cargo.toml, it fails with

++ crate2nix generate -f ./Cargo.toml -o Cargo-generated.nix -h /nix/store/rhri0m5m71kbjjchcd4wv34pdfh89qi9-nixified-rust-project-crate2nix/crate-hashes.json
Error: while retrieving metadata about ./Cargo.toml: `cargo metadata` exited with an error:     Updating git repository `https://github.com/BurntSushi/ripgrep`
warning: spurious network error (2 tries remaining): failed to resolve address for github.com: Name or service not known; class=Net (12)
full error message
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/dya46arma9crfayrq0df5rzdrban9skb-yivygx3r9w5zwfq4hmx7zcp77yhqwy9w-source
source root is yivygx3r9w5zwfq4hmx7zcp77yhqwy9w-source
@nix { "action": "setPhase", "phase": "buildPhase" }
building
++ crate2nix generate -f ./Cargo.toml -o Cargo-generated.nix -h /nix/store/rhri0m5m71kbjjchcd4wv34pdfh89qi9-nixified-rust-project-crate2nix/crate-hashes.json
Error: while retrieving metadata about ./Cargo.toml: `cargo metadata` exited with an error:     Updating git repository `https://github.com/BurntSushi/ripgrep`
warning: spurious network error (2 tries remaining): failed to resolve address for github.com: Name or service not known; class=Net (12)
warning: spurious network error (1 tries remaining): failed to resolve address for github.com: Name or service not known; class=Net (12)
error: failed to get `ripgrep` as a dependency of package `rust-darwin-flake v0.1.0 (/build/yivygx3r9w5zwfq4hmx7zcp77yhqwy9w-source)`

Caused by:
  failed to load source for dependency `ripgrep`

Caused by:
  Unable to update https://github.com/BurntSushi/ripgrep

Caused by:
  failed to clone into: /nix/store/rhri0m5m71kbjjchcd4wv34pdfh89qi9-nixified-rust-project-crate2nix/cargo/git/db/ripgrep-c5ec4b80e634fc7f

Caused by:
  network failure seems to have happened
  if a proxy or similar is necessary `net.git-fetch-with-cli` may help here
  https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli

Caused by:
  failed to resolve address for github.com: Name or service not known; class=Net (12)
crate2nix failed.
== cargo/config (BEGIN)
    [source.crates-io]
    replace-with = "vendored-sources"
    
    
    [source.vendored-sources]
    directory = "/nix/store/9lp24wm0cdfplxqnrwmx2d4p8aslnr4s-deps"

== cargo/config (END)

== crate-hashes.json (BEGIN)
    {}
== crate-hashes.json (END)

== ls -la (BEGIN)
total 32
drwxr-xr-x 3 nixbld nixbld 4096 Jan  1  1970 .
drwx------ 3 nixbld nixbld 4096 Feb 25 17:01 ..
-rw-r--r-- 1 nixbld nixbld  161 Jan  1  1970 Cargo.lock
-rw-r--r-- 1 nixbld nixbld  334 Jan  1  1970 Cargo.toml
-rw-r--r-- 1 nixbld nixbld 2559 Jan  1  1970 flake.lock
-rw-r--r-- 1 nixbld nixbld 4467 Jan  1  1970 flake.nix
drwxr-xr-x 3 nixbld nixbld 4096 Jan  1  1970 src
== ls -la (END)

Will this keep on failing?

If this were genuinely caused by a temporary network failure, I guess it would just keep failing, even after the network problem is resolved, because Nix would notice that we have already evaluated inputs which produce this hash, and it would retrieve the result (containing the error) from the Nix store.

Is that true?

If so, how would you get around this problem?

But the network is fine

Needless to say, I have verified that my network is up, that github, gitlab, codeberg, bitbucket, nobug, etc. are actually reachable, and get the same problem with a variety of repos across a variety of hosts over a prolonged period of time.

How should I go about getting to the bottom of this?

Except that …

This isn’t even my original problem!

This repo is a simplified, toy one, in which I experiment with my flake for Rust projects. I came back to this toy repo because a real, much more complex project I have, which has a flake based on this one, is failing with a completely different problem. I wanted to reproduce the problem in the simpler context of the toy repo, but it seems to suffer from a more fundamental problem.

In the real project, when (just like in the problem described above) I add a git repo dependency—crucially, one in a repo that does not have a master branch— to Cargo.toml, it has no problem using the network, but instead it fails because it tries to download stuff from the non-existent master branch while evaluating an attribute which explicitly specifies a different branch. Here are the key points from the error message:

  • couldn't find remote ref refs/heads/master
  • while fetching the input 'git+https://github.com/jacg/ndhistogram?ref=master&rev=3a37241b8292ee2eaa3410916573ad4c178b46da'
  • while evaluating the attribute '"ndhistogram 0.7.0 (git+https://github.com/jacg/ndhistogram?branch=cyclic-axis#3a37241b8292ee2eaa3410916573ad4c178b46da)"'

Compare and contrast:

  • fetching          ref    = master&rev  = 3a37241b8292ee2eaa3410916573ad4c178b46da
    
  • while evaluating  branch = cyclic-axis # 3a37241b8292ee2eaa3410916573ad4c178b46da
    

Why is it trying to get 3a3724 from master when evaluating 3a3724 on cyclic-axis? Is this a bug?

Here is the error message in full
fetching Git repository 'https://github.com/jacg/ndhistogram'fatal: couldn't find remote ref refs/heads/master
warning: could not update local clone of Git repository 'https://github.com/jacg/ndhistogram'; continuing with the most recent version
error: Cannot find Git revision '3a37241b8292ee2eaa3410916573ad4c178b46da' in ref 'master' of repository 'https://github.com/jacg/ndhistogram'! Please make sure that the rev exists on the ref you've specified or add allRefs = true; to fetchGit.

       … while fetching the input 'git+https://github.com/jacg/ndhistogram?ref=master&rev=3a37241b8292ee2eaa3410916573ad4c178b46da'

       … while evaluating the attribute 'buildCommand' of the derivation 'hash-of-ndhistogram'

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/pkgs/stdenv/generic/make-derivation.nix:205:7:

          204|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          205|       name =
             |       ^
          206|         let

       … while evaluating the attribute '"ndhistogram 0.7.0 (git+https://github.com/jacg/ndhistogram?branch=cyclic-axis#3a37241b8292ee2eaa3410916573ad4c178b46da)"'

       at /nix/store/j622kkg43r2nfqh9l8q6p22r5p470mlb-source/tools.nix:245:13:

          244|             name = toPackageId attrs;
          245|             value = builtins.readFile hash;
             |             ^
          246|           };

       … while evaluating the attribute 'outputHash' of the derivation 'ndhistogram-0.7.0'

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/pkgs/stdenv/generic/make-derivation.nix:205:7:

          204|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          205|       name =
             |       ^
          206|         let

       … while evaluating the attribute 'buildCommand' of the derivation 'ndhistogram-0.7.0'

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/pkgs/stdenv/generic/make-derivation.nix:205:7:

          204|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          205|       name =
             |       ^
          206|         let

       … while evaluating the attribute 'name'

       at /nix/store/j622kkg43r2nfqh9l8q6p22r5p470mlb-source/tools.nix:277:21:

          276|                   {
          277|                     name = builtins.baseNameOf source;
             |                     ^
          278|                     path = source;

       … while evaluating 'escapeShellArg'

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/lib/strings.nix:318:20:

          317|   */
          318|   escapeShellArg = arg: "'${replaceStrings ["'"] ["'\\''"] (toString arg)}'";
             |                    ^
          319|

       … from call site

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/pkgs/build-support/trivial-builders.nix:480:33:

          479|       ${lib.concatMapStrings (x: ''
          480|           mkdir -p "$(dirname ${lib.escapeShellArg x.name})"
             |                                 ^
          481|           ln -s ${lib.escapeShellArg x.path} ${lib.escapeShellArg x.name}

       … while evaluating anonymous lambda

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/pkgs/build-support/trivial-builders.nix:479:31:

          478|       cd $out
          479|       ${lib.concatMapStrings (x: ''
             |                               ^
          480|           mkdir -p "$(dirname ${lib.escapeShellArg x.name})"

       … from call site

       … while evaluating 'concatMapStrings'

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/lib/strings.nix:53:25:

           52|   */
           53|   concatMapStrings = f: list: concatStrings (map f list);
             |                         ^
           54|

       … from call site

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/pkgs/build-support/trivial-builders.nix:479:9:

          478|       cd $out
          479|       ${lib.concatMapStrings (x: ''
             |         ^
          480|           mkdir -p "$(dirname ${lib.escapeShellArg x.name})"

       … while evaluating the attribute 'buildCommand' of the derivation 'deps'

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/pkgs/stdenv/generic/make-derivation.nix:205:7:

          204|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          205|       name =
             |       ^
          206|         let

       … while evaluating the attribute 'text' of the derivation 'vendor-config'

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/pkgs/stdenv/generic/make-derivation.nix:205:7:

          204|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          205|       name =
             |       ^
          206|         let

       … while evaluating the attribute 'buildPhase' of the derivation 'petalo-crate2nix'

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/pkgs/stdenv/generic/make-derivation.nix:205:7:

          204|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          205|       name =
             |       ^
          206|         let

       … while evaluating 'flatten'

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/lib/lists.nix:137:13:

          136|   */
          137|   flatten = x:
             |             ^
          138|     if isList x

       … from call site

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/pkgs/build-support/mkshell/default.nix:19:36:

           18|     (attrs.${name} or [ ]) ++
           19|     (lib.subtractLists inputsFrom (lib.flatten (lib.catAttrs name inputsFrom)));
             |                                    ^
           20|

       … while evaluating 'mergeInputs'

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/pkgs/build-support/mkshell/default.nix:17:17:

           16| let
           17|   mergeInputs = name:
             |                 ^
           18|     (attrs.${name} or [ ]) ++

       … from call site

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/pkgs/build-support/mkshell/default.nix:36:17:

           35|
           36|   buildInputs = mergeInputs "buildInputs";
             |                 ^
           37|   nativeBuildInputs = packages ++ (mergeInputs "nativeBuildInputs");

       … while evaluating 'chooseDevOutputs'

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/lib/attrsets.nix:498:22:

          497|   /* Pick the outputs of packages to place in buildInputs */
          498|   chooseDevOutputs = drvs: builtins.map getDev drvs;
             |                      ^
          499|

       … from call site

       … while evaluating the attribute 'buildInputs' of the derivation 'nix-shell'

       at /nix/store/ldx6mi3dk80pshjhlmw5ziynh3jln8nd-source/pkgs/stdenv/generic/make-derivation.nix:205:7:

          204|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          205|       name =
             |       ^
          206|         let

Grrrr !

But even this isn’t my original problem, though that’s not relevant here.

I’m so far down a rabbit hole of n-th order problems which are taking me far away from what I was originally trying to achieve! It’s driving me nuts.

1 Like