Bazel build error on dependencies

Help
1

I am trying to package torch-tensorrt, which uses bazel as its build system. I have managed to make some early stages of the build use its dependencies from nixpkgs, but now I am stuck on a later error, and I cannot figure out where the BUILD.bazel file mentioned in the error is coming from.

ERROR: /build/output/external/libtorch/BUILD.bazel:88:11: SolibSymlink _solib_k8/_U@libtorch_S_S_Ccaffe2___Ulib/libcaffe2_nvrtc.so failed: missing input file 'external/libtorch/lib/libcaffe2_nvrtc.so', owner: '@libtorch//:lib/libcaffe2_nvrtc.so'
ERROR: /build/output/external/libtorch/BUILD.bazel:17:11: SolibSymlink _solib_k8/_U@libtorch_S_S_Ctorch___Ulib/libtorch_cuda.so failed: missing input file 'external/libtorch/lib/libtorch_cuda.so', owner: '@libtorch//:lib/libtorch_cuda.so'
ERROR: /build/output/external/libtorch/BUILD.bazel:53:11: SolibSymlink _solib_k8/_U@libtorch_S_S_Cc10_Ucuda___Ulib/libc10_cuda.so failed: missing input file 'external/libtorch/lib/libc10_cuda.so', owner: '@libtorch//:lib/libc10_cuda.so'

Anyone familiar with bazel able to shed some light on the process here?

torch-tensorrt derivation

nix-shell derivation to build the torch-tensorrt derivation 
{ pkgs ? import ./. {
    overlays = [
      (final: prev: {
        cudaPackages = prev.cudaPackages_11_3.overrideScope' (final: prev: {
          cudnn = final.cudnn_8_2_1;
          tensorrt = final.tensorrt_8_2_4;
        });
      })
    ];
  }
}:
pkgs.mkShell {
  packages = with pkgs; [
    (python310.withPackages (pypkgs: [
      pypkgs.torch-tensorrt
    ]))
  ];
}
Full nix builder log 
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/awknl3d8ch8mcd8mqr2smr8f3k4kkqih-source
source root is source
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
applying patch /nix/store/qzkkn98vys8vzw4x8avawi0rd846x9za-workspace-local-deps.patch
patching file WORKSPACE
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
no configure script, doing nothing
@nix { "action": "setPhase", "phase": "buildPhase" }
building
patching script interpreter paths in .
./docker/dist-test.sh: interpreter directive changed from "#!/bin/bash" to "/nix/store/zwjm0gln1vk7x1akpyz0yxjsd1yc46gi-bash-5.1-p16/bin/bash"
./docker/dist-build.sh: interpreter directive changed from "#!/bin/bash" to "/nix/store/zwjm0gln1vk7x1akpyz0yxjsd1yc46gi-bash-5.1-p16/bin/bash"
./py/build_whl.sh: interpreter directive changed from "#!/bin/bash" to "/nix/store/zwjm0gln1vk7x1akpyz0yxjsd1yc46gi-bash-5.1-p16/bin/bash"
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Loading:
Loading: 0 packages loaded
Analyzing: target //:libtorchtrt (1 packages loaded, 0 targets configured)
Analyzing: target //:libtorchtrt (57 packages loaded, 292 targets configured)
INFO: Analyzed target //:libtorchtrt (65 packages loaded, 8877 targets configured).
INFO: Found 1 target...
[0 / 159] [Prepa] BazelWorkspaceStatusAction stable-status.txt ... (9 actions, 0 running)
ERROR: /build/output/external/libtorch/BUILD.bazel:88:11: SolibSymlink _solib_k8/_U@libtorch_S_S_Ccaffe2___Ulib/libcaffe2_nvrtc.so failed: missing input file 'external/libtorch/lib/libcaffe2_nvrtc.so', owner: '@libtorch//:lib/libcaffe2_nvrtc.so'
ERROR: /build/output/external/libtorch/BUILD.bazel:17:11: SolibSymlink _solib_k8/_U@libtorch_S_S_Ctorch___Ulib/libtorch_cuda.so failed: missing input file 'external/libtorch/lib/libtorch_cuda.so', owner: '@libtorch//:lib/libtorch_cuda.so'
ERROR: /build/output/external/libtorch/BUILD.bazel:53:11: SolibSymlink _solib_k8/_U@libtorch_S_S_Cc10_Ucuda___Ulib/libc10_cuda.so failed: missing input file 'external/libtorch/lib/libc10_cuda.so', owner: '@libtorch//:lib/libc10_cuda.so'
ERROR: /build/output/external/libtorch/BUILD.bazel:88:11: SolibSymlink _solib_k8/_U@libtorch_S_S_Ccaffe2___Ulib/libcaffe2_nvrtc.so failed: 1 input file(s) do not exist
ERROR: /build/output/external/libtorch/BUILD.bazel:53:11: SolibSymlink _solib_k8/_U@libtorch_S_S_Cc10_Ucuda___Ulib/libc10_cuda.so failed: 1 input file(s) do not exist
ERROR: /build/output/external/libtorch/BUILD.bazel:17:11: SolibSymlink _solib_k8/_U@libtorch_S_S_Ctorch___Ulib/libtorch_cuda.so failed: 1 input file(s) do not exist
Target //:libtorchtrt failed to build
ERROR: /build/output/external/libtorch/BUILD.bazel:17:11 SolibSymlink _solib_k8/_U@libtorch_S_S_Ctorch___Ulib/libtorch_cuda.so failed: 1 input file(s) do not exist
INFO: Elapsed time: 4.438s, Critical Path: 0.05s
INFO: 403 processes: 403 internal.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
1 Like
2

Before I answer any of your questions: bazel in nix is always a bit tricky and pain.

The files like external/libtorch/lib/libcaffe2_nvrtc.so are missing. I don’t know why either.

1 Like