Horribly lost by "cat: not found" errors when configuring a nixpkg

Help
1

Hello all!

Super new to nix, so I imagine this is a bit of a simple fix but I can’t figure it out for the life of me…

I’ve got a default.nix:

{ lib, stdenv, fetchzip }:

stdenv.mkDerivation rec {
  name = "cobalt";
  version = "2.1.0";
  src = fetchzip {
    url = "ftp.ncbi.nlm.nih.gov/pub/cobalt/executables/${version}/ncbi-cobalt-${version}-src.tar.gz";
    sha256 = "NTlZFFCOCm3WjyKqpihmA3GnU8UV0mRZJDFoRn55+Ns=";
  };

  configurePhase = "cd c++ && ./configure";
}

And I’ve been running nix-build -E 'with import <nixpkgs> {}; callPackage ./default.nix {}' to build things.

But I keep getting some confusing errors that seem to imply things like rm and cat are missing:

> nix-build -E 'with import <nixpkgs> {}; callPackage ./default.nix {}'
this derivation will be built:
  /nix/store/gahinm75sjmf6z2g3z9vz8kq1q2dq1pl-cobalt.drv
building '/nix/store/gahinm75sjmf6z2g3z9vz8kq1q2dq1pl-cobalt.drv'...
unpacking sources
unpacking source archive /nix/store/5z0qmkxz2nhqqidvvqcrhz0b0zl80mls-source
source root is source
patching sources
configuring
./scripts/common/impl/get_lock.sh: line 14: whoami: not found
Waiting for /build/source/c++/configure.lock./scripts/common/impl/get_lock.sh: line 84: sleep: not found
../scripts/common/impl/get_lock.sh: line 22: expr: not found
sh: out of range
./scripts/common/impl/get_lock.sh: line 19: rm: not found
./scripts/common/impl/get_lock.sh: line 89: cat: not found
./scripts/common/impl/get_lock.sh: line 89: cat: not found
./scripts/common/impl/get_lock.sh: line 89: cat: not found
./scripts/common/impl/get_lock.sh: line 89: cat: not found
./scripts/common/impl/get_lock.sh: line 96: fmt: not found
error: builder for '/nix/store/gahinm75sjmf6z2g3z9vz8kq1q2dq1pl-cobalt.drv' failed with exit code 1;
       last 10 log lines:
       > ./scripts/common/impl/get_lock.sh: line 14: whoami: not found
       > Waiting for /build/source/c++/configure.lock./scripts/common/impl/get_lock.sh: line 84: sleep: not found
       > ../scripts/common/impl/get_lock.sh: line 22: expr: not found
       > sh: out of range
       > ./scripts/common/impl/get_lock.sh: line 19: rm: not found
       > ./scripts/common/impl/get_lock.sh: line 89: cat: not found
       > ./scripts/common/impl/get_lock.sh: line 89: cat: not found
       > ./scripts/common/impl/get_lock.sh: line 89: cat: not found
       > ./scripts/common/impl/get_lock.sh: line 89: cat: not found
       > ./scripts/common/impl/get_lock.sh: line 96: fmt: not found

Absolutely no idea what’s going on here since I can use rm and cat just fine in fields like configurePhase…

Any help would be greatly appreciated!

2

I’d look at the configure script and along its path to ./scripts/common/impl/get_lock.sh.

If I had to guess, something along that chain is trying to do some kind of PATH sanity-checking like setting a bog-standard unix PATH (maybe imperatively, or maybe just if it doesn’t detect “normal” locations in the PATH).

3

doing a very quick grep, it seems these shell scripts are modifying the PATH quite a lot, and setting absolute paths to /bin/ /usr/bin .

These are really big no no, and one of the reason nix exists.

You need going to need to patch them. here a quick grep results in the impl dir.

Good luck!

collect_outside_libs.sh:PATH=/bin:/usr/bin
collect_outside_libs.sh:export PATH
collect_outside_libs.sh:unset CDPATH
collect_outside_libs.sh:search=`echo ${2-$LD_LIBRARY_PATH} | tr : ' '`
create_flat_tuneups.sh:PATH=/bin:/usr/bin
create_flat_tuneups.sh:export PATH
favor-static:        sys0=$LD_LIBRARY_PATH
get_lock.sh:PATH=/bin:/usr/bin
get_lock.sh:export PATH
if_diff.sh:orig_PATH=$PATH
if_diff.sh:PATH=/bin:/usr/bin
if_diff.sh:  PATH=$orig_PATH
if_diff.sh:  PATH=/bin:/usr/bin
make_lock_map.sh:PATH=/bin:/usr/bin
make_lock_map.sh:export PATH
maybe_reconfigure.sh:environment variables (particularly PATH) are set correctly.
python-config.py:    elif want == 'LIBPATH':
reconfigure.sh:old_PATH=`sed -ne 's/^PATH: //p' ${status_dir}/config.log 2>/dev/null | tr '
reconfigure.sh:if test -n "$old_PATH"; then
reconfigure.sh:    echo "Restoring PATH to $old_PATH"
reconfigure.sh:    PATH=$old_PATH
run_with_lock.sh:orig_PATH=$PATH
run_with_lock.sh:PATH=/bin:/usr/bin
run_with_lock.sh:export PATH
run_with_lock.sh:        (PATH=$orig_PATH; export PATH; "$@"; echo $? > "$status_file") 2>&1 \
run_with_lock.sh:        PATH=$orig_PATH
run_with_lock.sh:        export PATH
strip_for_install.sh:PATH=/bin:/usr/bin:/usr/ccs/bin
strip_for_install.sh:export PATH
update_configurable.sh:PATH=/bin:/usr/bin
update_configurable.sh:export PATH
4

Yeah, it’s looking like a bit of a disaster so far… Any tips for patching / links to good places to learn? Are there any Nix specifics or should prePatch and some find / sed do the trick?

Thanks so much for helping me find the PATH issues!

5

No worries, welcome to Nix.

Yeah, those shell scripts seems pretty nightmarish…

You can scan through nixpkgs, and look at other nix derivations (build scripts) and see that patching in action.

In NixOS , normal binary and library areas don’t exist…

I’m not sure why this software messes around with PATH so much… it might be fun to find out.

I’ve seen worse than this…

1 Like
6

debugging this might be fun to use the new tmux debugger… so you can in enter into a derivation and have a poke around…

@matthewcroughan , is that thing ready for prime time?

7

Additionally, as a backup plan if this proves more difficult than some patching (as I’m now getting totally new errors that pop up after all of the path stuff I think is patched out), is it possible to use buildFHSUserEnv to build something like this?

I was trying something like:

buildFHSUserEnv {
     name = "COBALT";
     targetPkgs = pkgs: [ gcc9Stdenv wget ];
     extraInstallCommands = ''
        wget ftp.ncbi.nlm.nih.gov/pub/cobalt/executables/${version}/ncbi-cobalt-${version}-src.tar.gz
        cd c++
        ./configure
        make
      '';
     runScript = "bash";

But it can’t seem to find wget?

Here is the whole disaster I have right now which I might leave here while I head to bed if anyone fancies a challenge…

{ lib, gcc9Stdenv, fetchzip }:

gcc9Stdenv.mkDerivation rec {
  name = "cobalt";
  version = "2.1.0";
  src = fetchzip {
    url = "ftp.ncbi.nlm.nih.gov/pub/cobalt/executables/${version}/ncbi-cobalt-${version}-src.tar.gz";
    sha256 = "NTlZFFCOCm3WjyKqpihmA3GnU8UV0mRZJDFoRn55+Ns=";
  };

  sourceRoot = "source/c++";

  NCBICXX_TESTING_REQS = "0";

  prePatch = ''
    substituteInPlace scripts/common/impl/get_lock.sh scripts/common/impl/update_configurable.sh --replace 'PATH=' 'PATH=$PATH:'
    substituteInPlace src/build-system/configure --replace "PATH='/" '#'
    substituteInPlace src/build-system/Makefile.meta.in src/build-system/Makefile.mk.in --replace "/bin/mkdir" 'mkdir'
    substituteInPlace src/build-system/Makefile.meta.in src/build-system/Makefile.mk.in src/build-system/Makefile.meta_p src/build-system/Makefile.configurables.real --replace "/usr/bin/dirname" "dirname"
    substituteInPlace src/build-system/Makefile.meta.in src/build-system/Makefile.mk.in src/build-system/Makefile.meta_p src/build-system/Makefile.configurables.real --replace "/bin/cp" "cp"
    substituteInPlace src/build-system/Makefile.meta.in src/build-system/Makefile.mk.in src/build-system/Makefile.meta_p src/build-system/Makefile.configurables.real --replace "/bin/rm" "rm"
    substituteInPlace src/build-system/Makefile.meta_l --replace "/bin/date" "date"
  '';
}

# let 
#   pname = "cobalt";
#   version = "2.1.0";
#
#   # cobalt = gcc9Stdenv.mkDerivation {
#   #   inherit pname version;
#   #
#   #   src = fetchzip {
#   #     url = "ftp.ncbi.nlm.nih.gov/pub/cobalt/executables/${version}/ncbi-cobalt-${version}-src.tar.gz";
#   #     sha256 = "NTlZFFCOCm3WjyKqpihmA3GnU8UV0mRZJDFoRn55+Ns=";
#   #   };
#   #
#   #   sourceRoot = "source/c++";
#   #
#   #   # NCBICXX_TESTING_REQS = "0";
#   #   # __noChroot = true;
#   #   #
#   #   # prePatch = ''
#   #   #   substituteInPlace scripts/common/impl/get_lock.sh scripts/common/impl/update_configurable.sh --replace 'PATH=' 'PATH=$PATH:'
#   #   #   substituteInPlace src/build-system/configure --replace "PATH='/" '#'
#   #   # '';
#   # };
#   in buildFHSUserEnv {
#     name = "COBALT";
#     targetPkgs = pkgs: [ gcc9Stdenv wget ];
#     # extraInstallCommands = ''
#     #   wget ftp.ncbi.nlm.nih.gov/pub/cobalt/executables/${version}/ncbi-cobalt-${version}-src.tar.gz
#     #   cd c++
#     #   ./configure
#     #   make
#     # '';
#     runScript = "bash";

Please do forgive those ungodly substituteInPlace lines — I just kept copy-pasting lines until the errors went away and a new one popped up…

8

in nix , you have to explicitly bring things like wget in, if you don’t it wont be available.

This is by design! and it’s good design.

only use FHS as a last resort.

you can crack this…

and then get it committed to nixpkgs, and bingo! your a contributor.