G’day,
Special thanks to whoever got ollama working nicely via Nix! I’m using it happily with rocm mi50.
( nixpkgs/nixos/modules/services/misc/ollama.nix at 20075955deac2583bb12f07151c2df830ef346b4 · NixOS/nixpkgs · GitHub )
One minor thing I noticed:
Anyone had any luck with services.ollama.loadModels please?
I’m using a config like this, where I’m listing a bunch of loadModels:
#
# ollama
#
# HIP_VISIBLE_DEVICES=GPU-0e54792172da5eeb OLLAMA_CONTEXT_LENGTH=131072 ollama serve
#
# https://github.com/NixOS/nixpkgs/blob/20075955deac2583bb12f07151c2df830ef346b4/nixos/modules/services/misc/ollama.nix
#
{ config, lib, pkgs, ... }:
let
mi50euid = "GPU-0e54792172da5eeb";
ctxLength = toString 131072; # 128k
in {
services.ollama = {
enable = true;
host = "[::]";
#port = 11434; # default
acceleration = "rocm";
environmentVariables = {
HIP_VISIBLE_DEVICES = mi50euid;
OLLAMA_CONTEXT_LENGTH = ctxLength;
# OLLAMA_DEBUG = "1";
# AMD_LOG_LEVEL = "3";
};
# https://github.com/ollama/ollama/blob/d3450dd52efcbca0348e9e8ac42fad6263ec9f91/docs/troubleshooting.md#amd-gpu-discovery
loadModels = [
"nomic-embed-text:latest"
"codellama:34b"
"llama3.2:latest"
"gpt-oss:20b"
"deepseek-r1:32b"
"llama3-groq-tool-use:70b-q2_K"
"qwen2.5-coder:32b"
];
};
}
The service starts happily:
[das@l:~/nixos/desktop/l]$ systemctl status ollama
● ollama.service - Server for local large language models
Loaded: loaded (/etc/systemd/system/ollama.service; enabled; preset: ignored)
Active: active (running) since Sun 2025-08-24 12:45:38 PDT; 18min ago
Invocation: 0788c480967442c5a458dd785cef25c2
Main PID: 58347 (.ollama-wrapped)
IP: 52.6G in, 1.4G out
IO: 0B read, 49.7G written
Tasks: 75 (limit: 154186)
Memory: 51.8G (peak: 51.8G)
CPU: 10min 19.412s
CGroup: /system.slice/ollama.service
└─58347 /nix/store/hmyi7sxhf3vylqkmll4d2gj08i7cfiab-ollama-0.11.4/bin/ollama serve
Aug 24 12:55:36 l ollama[58347]: [GIN] 2025/08/24 - 12:55:36 | 200 | 30.817µs | ::1 | HEAD "/"
Aug 24 12:55:36 l ollama[58347]: [GIN] 2025/08/24 - 12:55:36 | 200 | 431.734295ms | ::1 | POST "/api/pull"
Aug 24 12:55:37 l ollama[58347]: [GIN] 2025/08/24 - 12:55:37 | 200 | 446.382935ms | ::1 | POST "/api/pull"
Aug 24 12:55:37 l ollama[58347]: [GIN] 2025/08/24 - 12:55:37 | 200 | 453.218384ms | ::1 | POST "/api/pull"
Aug 24 12:55:37 l ollama[58347]: time=2025-08-24T12:55:37.107-07:00 level=INFO source=download.go:177 msg="downloading ac3d1ba8aa77 in 20 1 GB part(s)"
Aug 24 12:55:37 l ollama[58347]: time=2025-08-24T12:55:37.173-07:00 level=INFO source=download.go:177 msg="downloading b112e727c6f1 in 16 861 MB part(s)"
Aug 24 12:55:37 l ollama[58347]: time=2025-08-24T12:55:37.182-07:00 level=INFO source=download.go:177 msg="downloading 5a6a1816e05f in 27 1 GB part(s)"
Aug 24 12:55:37 l ollama[58347]: time=2025-08-24T12:55:37.194-07:00 level=INFO source=download.go:177 msg="downloading 6150cb382311 in 20 1 GB part(s)"
Aug 24 13:01:33 l ollama[58347]: [GIN] 2025/08/24 - 13:01:33 | 200 | 22.282µs | 127.0.0.1 | HEAD "/"
Aug 24 13:01:33 l ollama[58347]: [GIN] 2025/08/24 - 13:01:33 | 200 | 490.321µs | 127.0.0.1 | GET "/api/tags"
[das@l:~/nixos/desktop/l]$
However, the models don’t seem to come down. I have manually done “ollama pull blah” to get these ones, but as you can see this list doesn’t match the loadModels list.
[das@l:~/nixos]$ ollama list
NAME ID SIZE MODIFIED
llama3.2:latest a80c4f17acd5 2.0 GB 9 minutes ago
nomic-embed-text:latest 0a109f422b47 274 MB 9 minutes ago
codellama:34b 685be00e1532 19 GB 9 minutes ago
[das@l:~/nixos]$
This isn’t a big deal, because obviously the models are easy to pull manually.
Oh, actually, looks like “systemctl restart ollama-model-loader.service” did the trick!
…
[das@l:~/nixos]$ systemctl restart ollama-model-loader.service
[das@l:~/nixos]$ systemctl status ollama-model-loader.service
○ ollama-model-loader.service - Download ollama models in the background
Loaded: loaded (/etc/systemd/system/ollama-model-loader.service; enabled; preset: ignored)
Active: inactive (dead) since Sun 2025-08-24 13:09:08 PDT; 1s ago
Duration: 424ms
Invocation: acc81e4c960e43af844cfe640a174b3a
Process: 74537 ExecStart=/nix/store/qxxlnhf3b7b53ck66ccbfhkbn47wq1dy-unit-script-ollama-model-loader-start/bin/ollama-model-loader-start (code=exited, status=0/SUCCESS)
Main PID: 74537 (code=exited, status=0/SUCCESS)
IP: 14.6K in, 11.2K out
IO: 0B read, 0B written
Mem peak: 54.1M
CPU: 175ms
Aug 24 13:09:08 l ollama-model-loader-start[74545]: [122B blob data]
Aug 24 13:09:08 l ollama-model-loader-start[74545]: [122B blob data]
Aug 24 13:09:08 l ollama-model-loader-start[74545]: [122B blob data]
Aug 24 13:09:08 l ollama-model-loader-start[74545]: [122B blob data]
Aug 24 13:09:08 l ollama-model-loader-start[74545]: [122B blob data]
Aug 24 13:09:08 l ollama-model-loader-start[74545]: [27B blob data]
Aug 24 13:09:08 l ollama-model-loader-start[74545]: [20B blob data]
Aug 24 13:09:08 l ollama-model-loader-start[74545]: [25B blob data]
Aug 24 13:09:08 l systemd[1]: ollama-model-loader.service: Deactivated successfully.
Aug 24 13:09:08 l systemd[1]: ollama-model-loader.service: Consumed 175ms CPU time, 54.1M memory peak, 14.6K incoming IP traffic, 11.2K outgoing IP traffic.
[das@l:~/nixos]$ ollama list
NAME ID SIZE MODIFIED
deepseek-r1:32b edba8017331d 19 GB 9 seconds ago
codellama:34b 685be00e1532 19 GB 10 seconds ago
gpt-oss:20b aa4295ac10c3 13 GB 10 seconds ago
llama3-groq-tool-use:70b-q2_K dab8a158f092 26 GB 10 seconds ago
llama3.2:latest a80c4f17acd5 2.0 GB 10 seconds ago
nomic-embed-text:latest 0a109f422b47 274 MB 10 seconds ago
qwen2.5-coder:32b b92d6a0bd47e 19 GB 10 seconds ago
[das@l:~/nixos]$
I guess the challenge is that ollama-model-loader.service isn’t being restarted if/when the loadModels list changes. Should I file an issue for this?
Kind regards,
Dave Seddon