Unable to build Ollama in 24.05 stable

nPrevail · June 3, 2024, 1:50pm

I recently made this post yesterday, and came to conclusion that the issue lies in building Ollama: Unable to complete the NixOS 24.05 upgrade: Something to do with "Tensile" - #2 by Sybbian

Here’s what I have in my config thus far:

# enable Ollama
services.ollama = {
  #package = pkgs.unstable.ollama; # Uncomment if you want to use the unstable channel, see https://fictionbecomesfact.com/nixos-unstable-channel
  enable = true;  
};

When running sudo nixos-rebuild switch --upgrade, I encounter the issue mentioned in the aforementioned link from above: Unable to complete the NixOS 24.05 upgrade: Something to do with "Tensile" - #2 by Sybbian

[nprevail@nixos-RazerBlade142015:~]$ sudo nixos-rebuild switch 
building Nix...
building the system configuration...
trace: warning: The option `services.xserver.xkbVariant' defined in `/etc/nixos/configuration.nix' has been renamed to `services.xserver.xkb.variant'.
trace: warning: The option `services.xserver.layout' defined in `/etc/nixos/configuration.nix' has been renamed to `services.xserver.xkb.layout'.
trace: warning: The option `services.avahi.nssmdns' defined in `/etc/nixos/configuration.nix' has been renamed to `services.avahi.nssmdns4'.
trace: warning: The option `services.xserver.displayManager.sddm.enable' defined in `/etc/nixos/configuration.nix' has been renamed to `services.displayManager.sddm.enable'.
trace: warning: The option `services.locate.locate' defined in `/etc/nixos/configuration.nix' has been renamed to `services.locate.package'.
trace: warning: mlocate and plocate do not support the services.locate.localuser option. updatedb will run as root. Silence this warning by setting services.locate.localuser = null.
trace: warning: rustPlatform.rust.cargo is deprecated. Use cargo instead.
trace: warning: rustPlatform.rust.rustc is deprecated. Use rustc instead.
trace: warning: The option `services.xserver.xkbVariant' defined in `/etc/nixos/configuration.nix' has been renamed to `services.xserver.xkb.variant'.
trace: warning: The option `services.xserver.layout' defined in `/etc/nixos/configuration.nix' has been renamed to `services.xserver.xkb.layout'.
trace: warning: The option `services.avahi.nssmdns' defined in `/etc/nixos/configuration.nix' has been renamed to `services.avahi.nssmdns4'.
trace: warning: The option `services.xserver.displayManager.sddm.enable' defined in `/etc/nixos/configuration.nix' has been renamed to `services.displayManager.sddm.enable'.
trace: warning: The option `services.locate.locate' defined in `/etc/nixos/configuration.nix' has been renamed to `services.locate.package'.
trace: warning: mlocate and plocate do not support the services.locate.localuser option. updatedb will run as root. Silence this warning by setting services.locate.localuser = null.
trace: warning: rustPlatform.rust.cargo is deprecated. Use cargo instead.
trace: warning: rustPlatform.rust.rustc is deprecated. Use rustc instead.
trace: warning: cudaPackages.autoAddDriverRunpath is deprecated, use pkgs.autoAddDriverRunpath instead
trace: warning: The option `services.xserver.xkbVariant' defined in `/etc/nixos/configuration.nix' has been renamed to `services.xserver.xkb.variant'.
trace: warning: The option `services.xserver.layout' defined in `/etc/nixos/configuration.nix' has been renamed to `services.xserver.xkb.layout'.
trace: warning: The option `services.avahi.nssmdns' defined in `/etc/nixos/configuration.nix' has been renamed to `services.avahi.nssmdns4'.
trace: warning: The option `services.xserver.displayManager.sddm.enable' defined in `/etc/nixos/configuration.nix' has been renamed to `services.displayManager.sddm.enable'.
trace: warning: The option `services.locate.locate' defined in `/etc/nixos/configuration.nix' has been renamed to `services.locate.package'.
trace: warning: mlocate and plocate do not support the services.locate.localuser option. updatedb will run as root. Silence this warning by setting services.locate.localuser = null.
trace: warning: rustPlatform.rust.cargo is deprecated. Use cargo instead.
trace: warning: rustPlatform.rust.rustc is deprecated. Use rustc instead.
these 17 derivations will be built:
  /nix/store/7phv832r2c0sixvnbcvmhc3913b9s4kj-rocblas-6.0.2.drv
  /nix/store/gxmqynh260796zg1pgjbb8qg3166qha0-rocm-path.drv
  /nix/store/f6ayxd7fa3lpc5ljghgn5v9j7zg0zyhy-ollama-0.1.38.drv
  /nix/store/6fchpys894y7z19zvd2wdj4kyqqnrss1-unit-ollama.service.drv
  /nix/store/bs87b1gyb5qz8waairz7h0mlvw652caz-system-path.drv
  /nix/store/6g2hwwdbnh95l5dlpax5ph0j831ii8wc-X-Restart-Triggers-polkit.drv
  /nix/store/lk24vw75zl0b6y22sbzg4qhby0qzjcqs-unit-polkit.service.drv
  /nix/store/rp7vjmcfclklifdlq53qixhqfbx7cm5y-unit-accounts-daemon.service.drv
  /nix/store/iiag2zvnf9q0acy12v97brcvf1200y61-dbus-1.drv
  /nix/store/fk96y35kbjim8g0lkrrhlg19ays61g85-X-Restart-Triggers-dbus.drv
  /nix/store/wb6i2lfbk21avjfhpngwgxjran4dz0sg-unit-dbus.service.drv
  /nix/store/1h1piph65fiag09rsmcqnlxvg1ypv716-system-units.drv
  /nix/store/7infynb4va3324nqvblv5w4jddkkcmqw-unit-dbus.service.drv
  /nix/store/9l7sy1q272i56ycqb99dnhfki0ri2jxc-user-units.drv
  /nix/store/3v986ngwspzxj0lbaz8vbcaw8g07ifkx-etc.drv
  /nix/store/knxsj391gj83jhd2z1as6dy6w2d5sxwl-nixos-system-nixos-Razer_Blade14_2015-amd-gpu-24.05.675.805a384895c6.drv
  /nix/store/vj60npvzxcn10w2i0299169z8x8sbjmy-nixos-system-nixos-Razer_Blade14_2015-24.05.675.805a384895c6.drv
building '/nix/store/7phv832r2c0sixvnbcvmhc3913b9s4kj-rocblas-6.0.2.drv'...
Running phase: unpackPhase
unpacking source archive /nix/store/ihrai11dd7zmmvj6j9g1kyj4337f85hh-source
source root is source
Running phase: patchPhase
applying patch /nix/store/04x9sfq554jgzhafdkpyznpk0azglwp0-Extend-rocBLAS-HIP-ISA-compatibility.patch
patching file library/src/handle.cpp
patching file library/src/rocblas_auxiliary.cpp
patching file library/src/tensile_host.cpp
Running phase: updateAutotoolsGnuConfigScriptsPhase
Running phase: configurePhase
fixing cmake files...
cmake flags: -DCMAKE_FIND_USE_SYSTEM_PACKAGE_REGISTRY=OFF -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DCMAKE_INSTALL_LOCALEDIR=/nix/store/l7qm8j3ar5lcmd46g8mnzk35v6x4bdv1-rocblas-6.0.2/share/locale -DCMAKE_INSTALL_LIBEXECDIR=/nix/store/l7qm8j3ar5lcmd46g8mnzk35v6x4bdv1-rocblas-6.0.2/libexec -DCMAKE_INSTALL_LIBDIR=/nix/store/l7qm8j3ar5lcmd46g8mnzk35v6x4bdv1-rocblas-6.0.2/lib -DCMAKE_INSTALL_DOCDIR=/nix/store/l7qm8j3ar5lcmd46g8mnzk35v6x4bdv1-rocblas-6.0.2/share/doc/rocblas -DCMAKE_INSTALL_INFODIR=/nix/store/l7qm8j3ar5lcmd46g8mnzk35v6x4bdv1-rocblas-6.0.2/share/info -DCMAKE_INSTALL_MANDIR=/nix/store/l7qm8j3ar5lcmd46g8mnzk35v6x4bdv1-rocblas-6.0.2/share/man -DCMAKE_INSTALL_OLDINCLUDEDIR=/nix/store/l7qm8j3ar5lcmd46g8mnzk35v6x4bdv1-rocblas-6.0.2/include -DCMAKE_INSTALL_INCLUDEDIR=/nix/store/l7qm8j3ar5lcmd46g8mnzk35v6x4bdv1-rocblas-6.0.2/include -DCMAKE_INSTALL_SBINDIR=/nix/store/l7qm8j3ar5lcmd46g8mnzk35v6x4bdv1-rocblas-6.0.2/sbin -DCMAKE_INSTALL_BINDIR=/nix/store/l7qm8j3ar5lcmd46g8mnzk35v6x4bdv1-rocblas-6.0.2/bin -DCMAKE_INSTALL_NAME_DIR=/nix/store/l7qm8j3ar5lcmd46g8mnzk35v6x4bdv1-rocblas-6.0.2/lib -DCMAKE_POLICY_DEFAULT_CMP0025=NEW -DCMAKE_OSX_SYSROOT= -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_STRIP=/nix/store/j9zds0awk7qydhbmjq94mh2fc7gmfiy3-rocm-llvm-binutils-6.0.2/bin/strip -DCMAKE_RANLIB=/nix/store/4kh0a4njzcl36arivn752va8s00rfd47-rocm-llvm-clang-wrapper-6.0.2/bin/ranlib -DCMAKE_AR=/nix/store/4kh0a4njzcl36arivn752va8s00rfd47-rocm-llvm-clang-wrapper-6.0.2/bin/ar -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_INSTALL_PREFIX=/nix/store/l7qm8j3ar5lcmd46g8mnzk35v6x4bdv1-rocblas-6.0.2 -DCMAKE_C_COMPILER:STRING=hipcc -DCMAKE_CXX_COMPILER:STRING=hipcc -Dpython:STRING=python3 -DAMDGPU_TARGETS:STRING=gfx900;gfx906:xnack-;gfx908:xnack-;gfx90a:xnack+;gfx90a:xnack-;gfx942;gfx1010;gfx1030;gfx1100;gfx1101;gfx1102 -DBUILD_WITH_TENSILE:BOOL=TRUE -DROCM_SYMLINK_LIBS:BOOL=FALSE -DROCBLAS_TENSILE_LIBRARY_DIR:STRING=lib/rocblas -DBUILD_CLIENTS_TESTS:BOOL=FALSE -DBUILD_CLIENTS_BENCHMARKS:BOOL=FALSE -DCMAKE_INSTALL_INCLUDEDIR:STRING=include -DBUILD_WITH_PIP:BOOL=FALSE -DTensile_LOGIC:STRING=asm_full -DTensile_CODE_OBJECT_VERSION:STRING=default -DTensile_SEPARATE_ARCHITECTURES:BOOL=TRUE -DTensile_LAZY_LIBRARY_LOADING:BOOL=TRUE -DTensile_LIBRARY_FORMAT:STRING=msgpack -DTensile_PRINT_DEBUG:BOOL=TRUE 
-- The CXX compiler identification is Clang 17.0.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /nix/store/ddgj2bab8gr7mn7pvmma3s7q8i71myi7-clr-6.0.2/bin/hipcc - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Use hip-clang to build for amdgpu backend
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- OS detected is unknown
-- ROCM Platform Version:  is not set, using default supported list
-- Performing Test COMPILER_HAS_TARGET_ID_gfx900
-- Performing Test COMPILER_HAS_TARGET_ID_gfx900 - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx906_xnack_off
-- Performing Test COMPILER_HAS_TARGET_ID_gfx906_xnack_off - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx908_xnack_off
-- Performing Test COMPILER_HAS_TARGET_ID_gfx908_xnack_off - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx90a_xnack_on
-- Performing Test COMPILER_HAS_TARGET_ID_gfx90a_xnack_on - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx90a_xnack_off
-- Performing Test COMPILER_HAS_TARGET_ID_gfx90a_xnack_off - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx942
-- Performing Test COMPILER_HAS_TARGET_ID_gfx942 - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1010
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1010 - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1030
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1030 - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1100
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1100 - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1101
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1101 - Success
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1102
-- Performing Test COMPILER_HAS_TARGET_ID_gfx1102 - Success
-- The C compiler identification is Clang 17.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /nix/store/ddgj2bab8gr7mn7pvmma3s7q8i71myi7-clr-6.0.2/bin/hipcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS
-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS - Success
-- Using AMDGPU_TARGETS: gfx900;gfx906:xnack-;gfx908:xnack-;gfx90a:xnack+;gfx90a:xnack-;gfx942;gfx1010;gfx1030;gfx1100;gfx1101;gfx1102
-- Tensile script: /nix/store/1wj1wgs0jfh66qg1h10w8nzqa40fki1i-python3.11-tensile-6.0.2/lib/python3.11/site-packages/Tensile/bin/TensileCreateLibrary
-- Tensile_CREATE_COMMAND: /nix/store/1wj1wgs0jfh66qg1h10w8nzqa40fki1i-python3.11-tensile-6.0.2/lib/python3.11/site-packages/Tensile/bin/TensileCreateLibrary;--merge-files;--separate-architectures;--lazy-library-loading;--no-short-file-names;--library-print-debug;--code-object-version=default;--cxx-compiler=hipcc;--jobs=8;--library-format=msgpack;--architecture=gfx900_gfx906:xnack-_gfx908:xnack-_gfx90a:xnack+_gfx90a:xnack-_gfx942_gfx1010_gfx1030_gfx1100_gfx1101_gfx1102;/build/source/library/src/blas3/Tensile/Logic/asm_full;/build/source/build/Tensile;HIP
-- Tensile_MANIFEST_FILE_PATH: /build/source/build/Tensile/library/TensileManifest.txt
'/nix/store/1wj1wgs0jfh66qg1h10w8nzqa40fki1i-python3.11-tensile-6.0.2/lib/python3.11/site-packages/Tensile/bin/TensileCreateLibrary' '--merge-files' '--separate-architectures' '--lazy-library-loading' '--no-short-file-names' '--library-print-debug' '--code-object-version=default' '--cxx-compiler=hipcc' '--jobs=8' '--library-format=msgpack' '--architecture=gfx900_gfx906:xnack-_gfx908:xnack-_gfx90a:xnack+_gfx90a:xnack-_gfx942_gfx1010_gfx1030_gfx1100_gfx1101_gfx1102' '/build/source/library/src/blas3/Tensile/Logic/asm_full' '/build/source/build/Tensile' 'HIP' '--generate-manifest-and-exit'

################################################################################
# Tensile Create Library
Tensile::WARNING: Did not detect SupportedISA: [(8, 0, 3), (9, 0, 0), (9, 0, 6), (9, 0, 8), (9, 0, 10), (9, 4, 0), (9, 4, 1), (9, 4, 2), (10, 1, 0), (10, 1, 1), (10, 1, 2), (10, 3, 0), (10, 3, 1), (11, 0, 0), (11, 0, 1), (11, 0, 2)]; cannot benchmark assembly kernels.
# Found  hipcc version 6.0.32831-

Followed by this:

#   /build/source/library/src/blas3/Tensile/Logic/asm_full/vega20/vega20_Cijk_Alik_Bljk_HB_GB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/vega20/vega20_Cijk_Alik_Bljk_HHS_BH.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/vega20/vega20_Cijk_Alik_Bljk_HHS_BH_GB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/vega20/vega20_Cijk_Alik_Bljk_HSS_BH.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/vega20/vega20_Cijk_Alik_Bljk_SB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/vega20/vega20_Cijk_Alik_Bljk_SB_GB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/vega20/vega20_Cijk_Alik_Bljk_ZB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/vega20/vega20_Cijk_Alik_Bljk_ZB_GB.yaml
Reading logic files: Launching 8 threads for 765 tasks...
Reading logic files: Done.
[|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||] 100% (84.6 secs elapsed)
-- *** NOTE: blas2/rocblas_ger_kernels.cpp is compiled with the verbose flag -v for QC purposes.
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
-- Configuring done (1609.0s)
-- Generating done (0.1s)
CMake Warning:
  Manually-specified variables were not used by the project:

    BUILD_TESTING
    CMAKE_EXPORT_NO_PACKAGE_REGISTRY
    CMAKE_POLICY_DEFAULT_CMP0025


-- Build files have been written to: /build/source/build
cmake: enabled parallel building
cmake: enabled parallel installing
configurePhase completed in 26 minutes 50 seconds
Running phase: buildPhase
build flags: -j8 SHELL=/nix/store/306znyj77fv49kwnkpxmb0j2znqpa8bj-bash-5.2p26/bin/bash
[  0%] Generating /build/source/build/include/rocblas/internal/rocblas_device_malloc.hpp
[  1%] Generating prototypes from /build/source/library/src.
[  1%] Built target rocblas_device_malloc
[  1%] Generating Tensile Libraries
[  1%] Built target rocblas_proto_templates

################################################################################
# Tensile Create Library
Tensile::WARNING: Did not detect SupportedISA: [(8, 0, 3), (9, 0, 0), (9, 0, 6), (9, 0, 8), (9, 0, 10), (9, 4, 0), (9, 4, 1), (9, 4, 2), (10, 1, 0), (10, 1, 1), (10, 1, 2), (10, 3, 0), (10, 3, 1), (11, 0, 0), (11, 0, 1), (11, 0, 2)]; cannot benchmark assembly kernels.
# Found  hipcc version 6.0.32831-
                  cap gfx000 gfx803 gfx900 gfx906 gfx908 gfx90a gfx940 gfx941 gfx942 gfx1010 gfx1011 gfx1012 gfx1030 gfx1031 gfx1100 gfx1101 gfx1102 
      HasMFMA_bf16_1k      0      0      0      0      0      1      1      1      1       0       0       0       0       0       0       0       0 
       HasMFMA_i8_908      0      0      0      0      1      1      0      0      0       0       0       0       0       0       0       0       0 
       HasMFMA_i8_940      0      0      0      0      0      0      1      1      1       0       0       0       0       0       0       0       0 
           HasAddLshl      0      0      1      1      1      1      1      1      1       1       1       1       1       1       1       1       1 
         HasAtomicAdd      0      0      0      0      1      1      1      1      1       0       0       0       0       0       1       1       1 
   HasDirectToLdsDest      0      0      0      0      0      0      0      0      0       0       0       0       0       0       0       0       0 
 HasDirectToLdsNoDest      0      1      1      1      1      1      1      1      1       1       1       1       1       1       0       0       0 
        HasExplicitCO      0      0      1      1      1      1      1      1      1       1       1       1       1       1       1       1       1 
        HasExplicitNC      0      0      0      0      0      0      0      0      0       1       1       1       1       1       1       1       1 
       HasGLCModifier      0      1      1      1      1      1      0      0      0       1       1       1       1       1       1       1       1 
            HasLshlOr      0      0      1      1      1      1      1      1      1       1       1       1       1       1       1       1       1 
              HasMFMA      0      0      0      0      1      1      1      1      1       0       0       0       0       0       0       0       0 
            HasSMulHi      0      0      1      1      1      1      1      1      1       1       1       1       1       1       1       1       1 
              HasWMMA      0      0      0      0      0      0      0      0      0       0       0       0       0       0       1       1       1 
           MaxLgkmcnt      1      1      1      1      1      1      1      1      1       1       1       1       1       1       1       1       1 
             MaxVmcnt      0      1      1      1      1      1      1      1      1       1       1       1       1       1       1       1       1 
         SupportedISA      0      1      1      1      1      1      1      1      1       1       1       1       1       1       1       1       1 
      SupportedSource      1      1      1      1      1      1      1      1      1       1       1       1       1       1       1       1       1 
           HasMFMA_b8      0      0      0      0      0      0      1      1      1       0       0       0       0       0       0       0       0 
     HasMFMA_constSrc      0      0      0      0      0      1      1      1      1       0       0       0       0       0       0       0       0 
       v_dot2_f32_f16      0      0      0      1      1      1      1      1      1       0       1       1       1       1       1       1       1 
      v_dot2c_f32_f16      0      0      0      0      1      1      1      1      1       0       1       1       1       1       1       1       1 
            v_fma_f16      0      0      1      1      1      1      1      1      1       1       1       1       1       1       1       1       1 
           v_fmac_f16      0      0      0      0      0      0      0      0      0       0       0       0       0       0       0       0       0 
            v_mac_f16      0      1      1      1      1      1      1      1      1       0       0       0       0       0       0       0       0 
         v_pk_fma_f16      0      0      1      1      1      1      1      1      1       1       1       1       1       1       1       1       1 
        v_pk_fmac_f16      0      0      0      0      0      0      0      0      0       0       0       0       0       0       0       0       0 
            v_fma_f32      0      1      1      1      1      1      1      1      1       1       1       1       1       1       1       1       1 
        v_fma_mix_f32      0      0      0      1      1      1      1      1      1       1       1       1       1       1       1       1       1 
           v_fmac_f32      0      0      0      1      1      1      1      1      1       1       1       1       1       1       1       1       1 
            v_mac_f32      0      1      1      1      1      1      0      0      0       1       1       1       0       0       0       0       0 
        v_mad_mix_f32      0      0      1      0      0      0      0      0      0       0       0       0       0       0       0       0       0 
          HasMFMA_f64      0      0      0      0      0      1      1      1      1       0       0       0       0       0       0       0       0 
            v_fma_f64      0      1      1      1      1      1      1      1      1       1       1       1       1       1       1       1       1 
           HasMFMA_f8      0      0      0      0      0      0      1      1      1       0       0       0       0       0       0       0       0 
    VOP3v_dot4_i32_i8      0      0      0      1      1      1      1      1      1       0       1       1       1       1       0       0       0 
        v_dot4_i32_i8      0      0      0      0      0      0      0      0      0       0       0       0       0       0       0       0       0 
       v_dot4c_i32_i8      0      0      0      0      1      1      1      1      1       0       1       1       1       1       0       0       0 
HasMFMA_bf16_original      0      0      0      0      1      1      0      0      0       0       0       0       0       0       0       0       0 
         HasMFMA_vgpr      0      0      0      0      0      1      1      1      1       0       0       0       0       0       0       0       0 
         HasMFMA_xf32      0      0      0      0      0      0      1      1      1       0       0       0       0       0       0       0       0 
   ArchAccUnifiedRegs      0      0      0      0      0      1      1      1      1       0       0       0       0       0       0       0       0 
       CMPXWritesSGPR      1      1      1      1      1      1      1      1      1       0       0       0       0       0       0       0       0 
        CrosslaneWait      0      0      0      0      0      0      1      1      1       0       0       0       0       0       0       0       0 
        ForceStoreSC1      0      0      0      0      0      0      1      1      0       0       0       0       0       0       0       0       0 
             HasAccCD      0      0      0      0      0      1      1      1      1       0       0       0       0       0       0       0       0 
           HasEccHalf      0      0      0      1      1      1      1      1      1       0       0       0       0       0       0       0       0 
            HasWave32      0      0      0      0      0      0      0      0      0       1       1       1       1       1       1       1       1 
           InstRename      0      0      0      0      0      0      0      0      0       0       0       0       0       0       1       1       1 
        SeparateVscnt      0      0      0      0      0      0      0      0      0       1       1       1       1       1       1       1       1 
             VgprBank      0      0      0      0      0      0      0      0      0       1       1       1       1       1       1       1       1 
     Waitcnt0Disabled      0      0      0      0      1      1      1      1      1       0       0       0       0       0       0       0       0 
# CodeObjectVersion from TensileCreateLibrary: default
# CxxCompiler       from TensileCreateLibrary: hipcc
# Architecture      from TensileCreateLibrary: gfx900_gfx906:xnack-_gfx908:xnack-_gfx90a:xnack+_gfx90a:xnack-_gfx942_gfx1010_gfx1030_gfx1100_gfx1101_gfx1102
# LibraryFormat     from TensileCreateLibrary: msgpack
# LibraryLogicFiles:
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_BjlkC_CB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_BjlkC_CB_GB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_BjlkC_ZB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_BjlkC_ZB_GB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_Bjlk_4xi8II_BH.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_Bjlk_4xi8II_BH_GB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_Bjlk_BBS_BH.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_Bjlk_BBS_BH_GB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_Bjlk_BSS_BH.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_Bjlk_BSS_BH_GB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_Bjlk_CB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_Bjlk_CB_GB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_Bjlk_DB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_Bjlk_DB_GB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_Bjlk_HB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_Bjlk_HB_GB.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_Bjlk_HHS_BH.yaml
#   /build/source/library/src/blas3/Tensile/Logic/asm_full/aldebaran/aldebaran_Cijk_Ailk_Bjlk_HHS_BH_GB.yaml

And eventually, the computer freezes, and either terminal crashes, or I hard reset the system.

Has anyone else been able to build Ollama into their config?

drupol · June 3, 2024, 1:53pm

Can you post the issue you’re having while compiling it?

nPrevail · June 3, 2024, 1:55pm

I didn’t want to be redundant because it’s already mentioned in the link, but sure, I’ll add it to my original post.

APCodes · June 3, 2024, 3:05pm

Which AMD GPU are you running this with?

Maybe I just try Ollama with Nix for a change and see if it works out of the box now, so far I have been running it in docker.

nPrevail · June 3, 2024, 3:53pm

That’s a good question.

So I’m using ollama a little differently, because I’m actually using a specialization that allows me to switch from an AMD GPU and a Nvidia gpu.

I like to boot nixos from an external drive. I like to use different laptops and desktops, with various gpus. So I use a specialisation configuration that when I boot, I can choose which GPU to use.

Perhaps I should try building ollama the normal way first, and then enable the specializations.

Although nixos being a declarative system, it shouldn’t be the case.

rjpc · June 3, 2024, 4:13pm

I’m using a AMD GPU (7900 XT) I have a few lower end models that worked as well. I’ve beefed up my box for this crap - I am on unstable (Vicuña) and it took many an hour to compile it with a AMD Ryzen 9 5950X and 128GB RAM. But it does work. Takes some time

nPrevail · June 3, 2024, 6:21pm

So, I looked into it, and I think it has to do with my specialisation that I’m using in NixOS. It worked fine in 23.11.

I tried a simple config in 24.05, and it worked fine:

# Ollama
services.ollama = {
  #package = pkgs.unstable.ollama; # Uncomment if you want to use the unstable channel, see https://fictionbecomesfact.com/nixos-unstable-channel
  enable = true;
  acceleration = "cuda"; # Or "rocm"
  #environmentVariables = { # I haven't been able to get this to work myself yet, but I'm sharing it for the sake of completeness
    # HOME = "/home/ollama";
    # OLLAMA_MODELS = "/home/ollama/models";
    # OLLAMA_HOST = "0.0.0.0:11434"; # Make Ollama accesible outside of localhost
    # OLLAMA_ORIGINS = "http://localhost:8080,http://192.168.0.10:*"; # Allow access, otherwise Ollama returns 403 forbidden due to CORS
  #};
};

But when I tried my speclisation, it didn’t work. This is what I have in my config.

Ollama section:

# Ollama
services.ollama = {
  #package = pkgs.unstable.ollama; # Uncomment if you want to use the unstable channel, see https://fictionbecomesfact.com/nixos-unstable-channel
  enable = true;
  #environmentVariables = { # I haven't been able to get this to work myself yet, but I'm sharing it for the sake of completeness
    # HOME = "/home/ollama";
    # OLLAMA_MODELS = "/home/ollama/models";
    # OLLAMA_HOST = "0.0.0.0:11434"; # Make Ollama accesible outside of localhost
    # OLLAMA_ORIGINS = "http://localhost:8080,http://192.168.0.10:*"; # Allow access, otherwise Ollama returns 403 forbidden due to CORS
  #};
};

GPU specialisation section:

specialisation = {
	nvidia-gpu.configuration = {
	system.nixos.tags = [ "nvidia-gpu" ];
	#Enable video drivers ([ "nvidia" ] or [ "amdgpu" ]) (non-Hybrid)
		services.xserver.videoDrivers = ["nvidia"];
		 hardware.nvidia = {

		    # Modesetting is needed for most Wayland compositors
		    modesetting.enable = true;

		    # Nvidia power management. Experimental, and can cause sleep/suspend to fail.
		    powerManagement.enable = false;
		    # Fine-grained power management. Turns off GPU when not in use.
		    # Experimental and only works on modern Nvidia GPUs (Turing or newer).
		    powerManagement.finegrained = false;

		    # Use the NVidia open source kernel module (not to be confused with the
		    # independent third-party "nouveau" open source driver).
		    # Support is limited to the Turing and later architectures. Full list of 
		    # supported GPUs is at: 
		    # https://github.com/NVIDIA/open-gpu-kernel-modules#compatible-gpus 
		    # Only available from driver 515.43.04+
		    # Do not disable this unless your GPU is unsupported or if you have a good reason to.
		    open = false;

		    # Enable the Nvidia settings menu,
			# accessible via `nvidia-settings`.
		    nvidiaSettings = true;

		    # Optionally, you may need to select the appropriate driver version for your specific GPU.
		    package = config.boot.kernelPackages.nvidiaPackages.stable;

			forceFullCompositionPipeline = true;

		#Option A: Off-load
		prime = {
			offload = {
				enable = true;
				enableOffloadCmd = true;
			};
			# Make sure to use the correct Bus ID values for your system!
			intelBusId = "PCI:0:2:0";
			nvidiaBusId = "PCI:01:00:0";
				};
  
  
		#Option B: Sync  
		#    prime = { 
		#       sync.enable = true; 
		  
		       # Bus ID of the NVIDIA GPU. You can find it using lspci, either under 3D or VGA 
		#       nvidiaBusId = "PCI:01:00:0"; 
		  
		       # Bus ID of the Intel GPU. You can find it using lspci, either under 3D or VGA 
		#       intelBusId = "PCI:00:02:0"; 
		#     };	

		};

	#Ollama config
		services.ollama.acceleration = "cuda"; # Or "rocm"
	};

	amd-gpu.configuration = {
		system.nixos.tags = ["amd-gpu"];
				#Enable video drivers (["nvidia"] or ["amdgpu"]) (non-Hybrid)
				services.xserver.videoDrivers = ["amdgpu"];

				#AMD GPU config
				boot.initrd.kernelModules = [ "amdgpu" ];

				#systemd.tmpfiles.rules = [
				#    "L+    /opt/rocm/hip   -    -    -     -    ${pkgs.hip}"
				#  ];

				hardware.opengl.extraPackages = with pkgs; [
					rocm-opencl-icd
					rocm-opencl-runtime
					amdvlk
					];


				# For 32 bit applications
				#hardware.opengl.driSupport32Bit = true;

				# For 32 bit applications 
				# Only available on unstable
				#hardware.opengl.extraPackages32 = with pkgs; [
				#  driversi686Linux.amdvlk
				#];

		#Ollama config		
			services.ollama.acceleration = "rocm"; # Or "rocm" or "cuda"
			};
};

APCodes · June 4, 2024, 8:26am

Okay so my suspicion still is that it might have something to do with your AMD GPU configuration. Does ollama run with ROCm for you at all?

Because the error you posted “Did not detect SupportedISA” is clearly linked to different AMD GPU architectures. It almost looked like some parts of ROCm had to be compiled from source there and it didn’t find the correct GPU architecture target.

I like to boot nixos from an external drive. I like to use different laptops and desktops, with various gpus. So I use a specialisation configuration that when I boot, I can choose which GPU to use.

So you have various AMD GPUs? Which ones? Does it work on any of these or on none or on some yes, but on some no?

nPrevail · June 4, 2024, 4:25pm

It does not. In fact, I tried to build ollama with a AMD Ryzen 5 2500u, Vega 8, using rocm:

# Ollama
services.ollama = {
  enable = true;
  acceleration = "rocm"; # Or "cuda"
  #};
};

When running sudo nixos-rebuild switch, it also stalled and crashed the same way. It seems there’s a problem with rocm. The strange thing is that on 23.11, rocm worked fine with a Vega 8 GPU. But perhaps I was wrong(?)

On channel 23.11, when I ran Ollama, I would see the GPU thermal temperature rising, indicating that the GPU was working with Ollama.

APCodes · June 4, 2024, 4:52pm

It seems there’s a problem with rocm. The strange thing is that on 23.11, rocm worked fine with a Vega 8 GPU. But perhaps I was wrong(?)

It would be great to double check. You can also try to get ROCm 5.7 from nixpkgs and see if that is of any help. The problem is that your integrated APU is gfx902 when it comes to build targets. And that is not in the array that follows the Tensile::WARNING: Did not detect SupportedISA error you got.

For the ROCm build targets you can check this: User Guide for AMDGPU Backend — LLVM 19.0.0git documentation

Edit: Instead of looking at thermals, you might use a tool like that: GitHub - Umio-Yasuno/amdgpu_top: Tool to display AMDGPU usage

rjpc · June 4, 2024, 4:59pm

I don’t think the Vega 8 is gonna work? this doc lists the ones supported or tested ollama/docs/gpu.md at 60323e08057d36b617f11d3c4958d342a44d0342 · ollama/ollama · GitHub

APCodes · June 4, 2024, 6:26pm

yeah so the “official” support for ROCm by AMD is much much thinner than what actually works @rjpc … but you might be right in the end that the Vega 8 doesn’t even work unofficially.

One thing I do ask myself is: Why is it trying to build tensile from source?

rjpc · June 4, 2024, 7:06pm

I’d like to know that too! Honestly, it’s such a black box to me and the fact that with all my compute power it takes so long to compile. Will be nice to understand better if I could. Yes, though, I mention that link just because I didn’t see Vega 8 there so one might have more trouble

APCodes · June 4, 2024, 7:33pm

I mean you are right @rjpc, that APU doesn’t have any official support. And there is a fair chance, that it won’t work. But it just might work if you override the GPU architecture once you have ROCm available on your system.

That works for other “non-supported” GPUs just fine. for instance neither does a Radeon 6600 XT / Radeon Pro W6600 have official support. But I happen to own two of these, and they run ollama just fine. They are similar enough to gfx1030 or the Radeon 6800, which does have official support for a while.

But I should say: when I run ollama, I use docker to run it. Because when I tried with Nix I had to rebuild multiple ROCm libraries from source, which just isn’t practical. I am not totally sure why that rebuild happens, but it can take several hours to rebuild ROCm on a powerful CPU.

APCodes · June 5, 2024, 1:00pm

Maybe interesting: at least on the 5600G ollama appears to be working via podman.

And that one has a Vega 7 iGPU. So I see no reason why ollama shouldn’t work on the Vega 8, in principle. Maybe you can try and reproduce the podman / docker setup @nPrevail ?

https://blog.machinezoo.com/Running_Ollama_on_AMD_iGPU