I’m working on a research project for school where I need to deploy to an HPC system so I created a cuda container with Apptainer. It works on the HPC system flawlessly and I have a nix flake that provides the packages for my local system, but that same HPC container fails on my local system. The rocm version of the container gives the same issue, neither can find the library files required. But it seems like the same issue for both platforms. Any ideas?
❯singularity run \
--rocm \
-B /opt/rocm:/opt/rocm \
-B /dev/kfd:/dev/kfd \
-B /dev/dri:/dev/dri \
--env LD_LIBRARY_PATH=/opt/rocm/lib \
hpc/rocm.sif
WARNING: While finding ROCm bind points: could not retrieve ld cache: could not execute ldconfig: exit status 1
WARNING: Could not find any nv files on this host!
WARNING: Could not find any nv libraries on this host!
INFO: Mounting image with FUSE.
/opt/app/autollm: error while loading shared libraries: libhiprtc.so.6: cannot open shared object file: No such file or directory
❯ singularity exec -B /opt/rocm:/opt/rocm hpc/rocm.sif ldd /opt/app/autollm
INFO: Mounting image with FUSE.
linux-vdso.so.1 (0x00007f2d5920c000)
libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x00007f2d57223000)
libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007f2d56d10000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f2d56cf4000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f2d56ae2000)
/lib64/ld-linux-x86-64.so.2 (0x00007f2d5920e000)
libhiprtc.so.6 => not found
libamdhip64.so.6 => not found
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f2d56ab2000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f2d569c9000)