Could someone please share their configuration to get rocm running on lmstudio for their amd graphcis card tried a few configuration but no luck, keeps showing the rocm is not compatible in lmstudio. Im using eGPU amd 5700XT (gfx1010) and vulkan drivers are working fine but want to use amd’s own drivers for better performance:
heres my present configuraiton for egpu that does not work for rocm:
{pkgs,...}:
{
systemd.packages = with pkgs; [ lact ];
systemd.services.lactd.wantedBy = ["multi-user.target"];
hardware.amdgpu.amdvlk.enable = true;
services.ollama.acceleration = true;
hardware.amdgpu.opencl.enable = true;
hardware.graphics = {
enable = true;
enable32Bit = true;
extraPackages = with pkgs; [
mesa # Mesa drivers for AMD GPUs
rocmPackages.clr # Common Language Runtime for ROCm
rocmPackages.clr.icd # ROCm ICD for OpenCL
rocmPackages.rocblas # ROCm BLAS library
rocmPackages.rpp # High-performance computer vision library
amdvlk # AMDVLK Vulkan drivers
nvtopPackages.amd # GPU utilization monitoring
];
};
# Configure /opt/rocm symlink for ROCm hardcoded paths
systemd.tmpfiles.rules = let
rocmEnv = pkgs.symlinkJoin {
name = "rocm-combined";
paths = with pkgs.rocmPackages; [
clr
clr.icd
rocblas
hipblas
rpp
];
};
in [
"L+ /opt/rocm - - - - ${rocmEnv}"
];
environment.variables = {
ROCM_PATH = "/opt/rocm"; # Set ROCm path
HIP_VISIBLE_DEVICES = "1"; # Use only the eGPU (ID 1)
ROCM_VISIBLE_DEVICES = "1"; # Optional: ROCm equivalent for visibility
LD_LIBRARY_PATH = "/opt/rocm/lib"; # Add ROCm libraries
HSA_OVERRIDE_GFX_VERSION = "10.3.0"; # Set GFX version override
};
}
rocm’s manifest json:
{
"name": "llama.cpp-linux-x86_64-amd-rocm-avx2",
"version": "1.8.0",
"domains": [
"llm",
"embedding"
],
"engine": "llama.cpp",
"target_libraries": [
{
"name": "llm_engine_rocm.node",
"type": "llm_engine",
"version": "0.1.2"
},
{
"name": "liblmstudio_bindings_rocm.node",
"type": "liblmstudio",
"version": "0.2.26"
}
],
"platform": "linux",
"cpu": {
"architecture": "x86_64",
"instruction_set_extensions": [
"AVX2"
]
},
"gpu": {
"make": "AMD",
"framework": "ROCm",
"targets": [
"gfx1030",
"gfx1100",
"gfx1101",
"gfx1102"
]
},
"supported_model_formats": [
"gguf"
],
"vendor_lib_package_name": "linux-llama-rocm-vendor-v1",
"manifest_version": "3"
}
I can also confirm that rocm module is loaded as rocminfo results in:
*******
Agent 2
*******
Name: gfx1030
Uuid: GPU-XX
Marketing Name: AMD Radeon RX 5700 XT
Vendor Name: AMD
Feature: KERNEL_DISPATCH
Profile: BASE_PROFILE
Float Round Mode: NEAR
Max Queue Number: 128(0x80)
Queue Min Size: 64(0x40)
Queue Max Size: 131072(0x20000)
Queue Type: MULTI
Node: 1
Device Type: GPU
Cache Info:
L1: 16(0x10) KB
L2: 4096(0x1000) KB
Chip ID: 29471(0x731f)
ASIC Revision: 2(0x2)
Cacheline Size: 128(0x80)
Max Clock Freq. (MHz): 2100
BDFID: 1536
Internal Node ID: 1
Compute Unit: 40
SIMDs per CU: 2
Shader Engines: 2
Shader Arrs. per Eng.: 2
WatchPts on Addr. Ranges:4
Coherent Host Access: FALSE
Features: KERNEL_DISPATCH
Fast F16 Operation: TRUE
Wavefront Size: 32(0x20)
Workgroup Max Size: 1024(0x400)
Workgroup Max Size per Dimension:
x 1024(0x400)
y 1024(0x400)
z 1024(0x400)
Max Waves Per CU: 40(0x28)
Max Work-item Per CU: 1280(0x500)
Grid Max Size: 4294967295(0xffffffff)
Grid Max Size per Dimension:
x 4294967295(0xffffffff)
y 4294967295(0xffffffff)
z 4294967295(0xffffffff)
Max fbarriers/Workgrp: 32
Packet Processor uCode:: 151
SDMA engine uCode:: 35
IOMMU Support:: None
Pool Info:
Pool 1
Segment: GLOBAL; FLAGS: COARSE GRAINED
Size: 8372224(0x7fc000) KB
Allocatable: TRUE
Alloc Granule: 4KB
Alloc Alignment: 4KB
Accessible by all: FALSE
Pool 2
Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED
Size: 8372224(0x7fc000) KB
Allocatable: TRUE
Alloc Granule: 4KB
Alloc Alignment: 4KB
Accessible by all: FALSE
Pool 3
Segment: GROUP
Size: 64(0x40) KB
Allocatable: FALSE
Alloc Granule: 0KB
Alloc Alignment: 0KB
Accessible by all: FALSE
ISA Info:
ISA 1
Name: amdgcn-amd-amdhsa--gfx1030
Machine Models: HSA_MACHINE_MODEL_LARGE
Profiles: HSA_PROFILE_BASE
Default Rounding Mode: NEAR
Default Rounding Mode: NEAR
Fast f16: TRUE
Workgroup Max Size: 1024(0x400)
Workgroup Max Size per Dimension:
x 1024(0x400)
y 1024(0x400)
z 1024(0x400)
Grid Max Size: 4294967295(0xffffffff)
Grid Max Size per Dimension:
x 4294967295(0xffffffff)
y 4294967295(0xffffffff)
z 4294967295(0xffffffff)
FBarrier Max Size: 32