I’m trying to use ollama with GPU acceleration on my system, following the documentation here using the ollama-rocm package. This is not working. The stock ollama package does not attempt to access the hardware through the ROCm libraries.
Here are some lines of interest from the strace, which make me think there’s some sort of linking bug in the package:
strace.txt:144:openat(AT_FDCWD, "/nix/store/v8bm9phvgnqn3yacx25snf0k43sjf815-rocm-path/lib/glibc-hwcaps/x86-64-v3/libresolv.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
strace.txt:145:newfstatat(AT_FDCWD, "/nix/store/v8bm9phvgnqn3yacx25snf0k43sjf815-rocm-path/lib/glibc-hwcaps/x86-64-v3/", 0x7ffe3f18cc70, 0) = -1 ENOENT (No such file or directory)
strace.txt:146:openat(AT_FDCWD, "/nix/store/v8bm9phvgnqn3yacx25snf0k43sjf815-rocm-path/lib/glibc-hwcaps/x86-64-v2/libresolv.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
strace.txt:147:newfstatat(AT_FDCWD, "/nix/store/v8bm9phvgnqn3yacx25snf0k43sjf815-rocm-path/lib/glibc-hwcaps/x86-64-v2/", 0x7ffe3f18cc70, 0) = -1 ENOENT (No such file or directory)
strace.txt:148:openat(AT_FDCWD, "/nix/store/v8bm9phvgnqn3yacx25snf0k43sjf815-rocm-path/lib/libresolv.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
strace.txt:149:newfstatat(AT_FDCWD, "/nix/store/v8bm9phvgnqn3yacx25snf0k43sjf815-rocm-path/lib/", {st_mode=S_IFDIR|0555, st_size=1094, ...}, 0) = 0
strace.txt:165:openat(AT_FDCWD, "/nix/store/v8bm9phvgnqn3yacx25snf0k43sjf815-rocm-path/lib/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
strace.txt:176:openat(AT_FDCWD, "/nix/store/v8bm9phvgnqn3yacx25snf0k43sjf815-rocm-path/lib/libstdc++.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
strace.txt:193:openat(AT_FDCWD, "/nix/store/v8bm9phvgnqn3yacx25snf0k43sjf815-rocm-path/lib/librt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
strace.txt:204:openat(AT_FDCWD, "/nix/store/v8bm9phvgnqn3yacx25snf0k43sjf815-rocm-path/lib/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
strace.txt:215:openat(AT_FDCWD, "/nix/store/v8bm9phvgnqn3yacx25snf0k43sjf815-rocm-path/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
strace.txt:227:openat(AT_FDCWD, "/nix/store/v8bm9phvgnqn3yacx25snf0k43sjf815-rocm-path/lib/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
strace.txt:239:openat(AT_FDCWD, "/nix/store/v8bm9phvgnqn3yacx25snf0k43sjf815-rocm-path/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
[anadon@puttimon:~]$ ROCR_VISIBLE_DEVICES="1,2" OLLAMA_DEBUG=1 ollama run deepseek
>>> This is a test. Please only respond with 'acknowledged.'
<think>
Okay, so I just received this message that says, "Please only respond with 'acknowledged.'" Hmm, let me think about how to approach this.
First off, the user sent a test message and specified they want only an acknowledgment in response. So, the main point here is to follow their instructions precisely. They probably want
to make sure that I can understand and execute simple commands, or maybe it's just for testing purposes.
I need to make sure that my reply strictly adheres to what they asked. That means no extra information, just "acknowledged." But wait, should I say anything else? Maybe I should also
consider the context. Since it's a test, perhaps there's an expectation of accuracy and quick response.
Also, I wonder if there's any underlying purpose to this test. Are they checking my ability to follow directions, or is it part of a larger interaction? Either way, sticking to their
exact request seems safe.
I should also think about the structure of the response. They want only "acknowledged," so no additional text or explanations. It needs to be concise and exactly as specified. Maybe I
can just type that without any formatting, keeping it simple.
Is there a chance they might expect something else after this? Probably not, since the instruction is clear. But just in case, I'll make sure my reply doesn't include anything beyond
"acknowledged."
Another thing to consider: sometimes in tests, they look for consistency and adherence to rules. So by strictly following their command, I'm demonstrating that I can be reliable.
I should also think about possible variations. What if they had said something slightly different? But no, the instruction is straightforward here. Just respond with "acknowledged."
Alright, so putting it all together, my response should simply be "acknowledged" without any extra words or explanations.
</think>
acknowledged
>>> Send a message (/? for help)
ollama run does not run the model itself, it connects to a service that runs ollama serve, try setting services.ollama.environmentVariables.OLLAMA_DEBUG=1 and looking at the logs for ollama.service
This worked. However, I would still like to pursue this a little further because it seems like it should be correctly detected by default. Can you help me with this? I’m still a little too green with the Nix ecosystem to be confident in directing my issue here.
am of the same opinion, the override should only be used as an exception, quite strange. Wasted alot of time on this previously to determine why ROCM was not detected by default