I cheeked systemd-oomd
logs and it did not kill anything…
I have set services.ollama.acceleration
to "cuda"
and comething compiled.
I think the acceleration works now.
I have decided to try and enable nixpkgs.config.cudaSupport
.
Here’s what happens:
# nixos-rebuild test
building Nix...
building the system configuration...
trace: warning: cudaPackages.autoAddDriverRunpath is deprecated, use pkgs.autoAddDriverRunpath instead
trace: warning: cudaPackages.autoAddDriverRunpath is deprecated, use pkgs.autoAddDriverRunpath instead
trace: warning: cudaPackages.autoFixElfFiles is deprecated, use pkgs.autoFixElfFiles instead
trace: warning: cudaPackages.autoAddOpenGLRunpathHook is deprecated, use pkgs.autoAddDriverRunpathHook instead
activating the configuration...
setting up /etc...
reloading user units for egycobra...
restarting sysinit-reactivation.target
the following new units were started: dev-disk-by\x2duuid-<UUID>.device, srv-nfs-outer.mount
If I disable cudaSupport
the warnings disappear.
Also I have possibly encountered another problem:
# nixos-option nixpkgs.config.cudaSupport
error: error: At 'cudaSupport' in path 'nixpkgs.config.cudaSupport': error: Attribute not found
An error occurred while looking for attribute names. Are you sure that 'nixpkgs.config.cudaSupport' exists?
When I set nixpkgs.config.cudaSupport = true
then ollama.service.acceleration
seems to not detect it:
# nixos-option services.ollama.acceleration
Value:
null
Default:
null
Type:
"null or one of false, \"rocm\", \"cuda\""
Example:
"rocm"
Description:
''
What interface to use for hardware acceleration.
- `null`: default behavior
if `nixpkgs.config.rocmSupport` is enabled, uses `"rocm"`
if `nixpkgs.config.cudaSupport` is enabled, uses `"cuda"`
otherwise defaults to `false`
- `false`: disable GPU, only use CPU
- `"rocm"`: supported by most modern AMD GPUs
- `"cuda"`: supported by most modern NVIDIA GPUs
''
Declared by:
[ "/nix/var/nix/profiles/per-user/root/channels/nixos/nixos/modules/services/misc/ollama.nix" ]
Defined by:
[ "/nix/var/nix/profiles/per-user/root/channels/nixos/nixos/modules/services/misc/ollama.nix" ]