How to run ollama service from unstable channel

Help
1

The question is how to get loadModels to access the unstable channnel? services.ollama.loadModels? may or may not be related)

sudo nix-channel --add  unstablesudo
nix-channel --update

configuration.nix
...
 #https://wiki.nixos.org/wiki/Ollama
  services.ollama = {
    enable = true;
    unstable.loadModels = ["gemma3:27b-it-qat"]; 
    acceleration = "rocm";
  };
  environment.systemPackages = with pkgs; [
...
   unstable.ollama-rocm
...
];
2

I would like to add a feature request to enhance the documenation in https://wiki.nixos.org/wiki/Ollama as most Ollama models are not running with the “stable” version. Any proposals how to file such a request? Thanks.

3

There is no reason to do this, and it is not possible. The models are defined and pulled from an external service, not a nixpkgs cache.

4

What errors do you see?

5

[nixos:~]$ ollama --version
ollama version is 0.12.11
Warning: client version is 0.17.7

In the first line, we have the “stable” version. In the second, it is the “unstable” version due to unstable.ollama-rocm

6

Set this:

services.ollama.package = unstable.ollama-rocm;
7

To sum up:

sudo nix-channel --add https://nixos.org/channels/nixos-unstable unstable
sudo nix-channel --update

and in configuration.nix

…#https://wiki.nixos.org/wiki/Ollama
services.ollama.package = unstable.ollama-rocm;
services.ollama = {enable = true;
loadModels = [“gemma3:27b-it-qat”];
acceleration = “rocm”;
};
environment.systemPackages = with pkgs; [
…
unstable.ollama-rocm
…
];
8

You can simplify your configuration.

You do not need to explicitly set environment.systemPackages as the service does it for you.

Also, acceleration is deprecated in nixpkgs-unstable and in nixpkgs-stable it is recommended that you use package instead.

Thus your config becomes this:

services.ollama = {
  enable = true;
  package = unstable.ollama-rocm;
  loadModels = [“gemma3:27b-it-qat”];
};
1 Like
9

To finally complete sum up:

sudo nix-channel --add https://nixos.org/channels/nixos-unstable unstable
sudo nix-channel --update

configuration.nix:

{ config, pkgs, ... }:

let
  # Import the unstable channel
  unstable = import <unstable> { config = { allowUnfree = true; }; };
in
{

 #https://wiki.nixos.org/wiki/Ollama
  services.ollama = {
    enable = true;
    package = unstable.ollama-rocm;
    loadModels = ["gemma3:27b-it-qat"];
  };
10

Sorry, the frequently used “preformatted” text option does not seem very user friendly.

11

Fine. I propose to update https://wiki.nixos.org/wiki/Ollama as this is a very concise and neat way of using an up-to-date version other users will benefit from.

12

You can switch it in the top left to just use pure markdown.

1 Like
13

Looking good now :slight_smile: Appreciate the hint.

1 Like
14

You should be able to make the update yourself if you register for an account with the wiki.

15

If have created an account but get round to contributing only in about one year’s time when retired. I will not forget :slight_smile:

1 Like
16

I’ve stopped using ollama because it is very slow, but here’s my config.

This was described in the YouToob video: https://youtu.be/mUrVC7oo2_g?si=z2vC84h5K3-a5S7D

I’m using llama.cpp instead now, which is mostly because I have crappy old AMD MI60 card and so vllm doesn’t support it. There is a pending PR to add the multiple card support.

1 Like