Getting the server running on NixOS

I wanted to try and run a github copilot-like setup locally on my nixos machines but ran into some packaging issues.

The way to do it right now seems to be the extensions for VSCode and Jetbrains IDEs.
The way it works is you install the extension, that runs the server, which connects to a language model to generate all the completions.
The way it connects to a locally running language model is through (Ollama | Continue).

Now I managed to do the following:

  1. ollama is packaged in nixpkgs
  2. a module for running ollama as a service is available (e.g. GitHub - havaker/ollama-nix: CUDA-enabled ollama nix flake)
  3. you can install the extension into vscode

Running the server packaged with the extension fails on NixOS.

 [error] Error: spawn /home/xxx/.vscode-oss/extensions/continue.continue-0.7.29-linux-x64/exe/continue_server ENOENT
	at ChildProcess._handle.onexit (node:internal/child_process:283:19)
	at onErrorNT (node:internal/child_process:476:16)
	at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

To solve issues like this you can run the server on your own and connect to it, docs here:

So I started a quest to first package the server and then run it as a service.
However I’m not very experienced in packaging (complex ai) python packages with poetry2nix, so I got stuck after a few ours of trying.

Here is what I got so far, probably including a lot of mistakes. I feel like I’m missing something fundamental, since it’s doing so many package overrides.

Would appreciate if someone could give me some pointers, as I feel like it would be a cool capability for everyone to have this running on nixos.

1 Like

Bump, would like to know if a solution is possible

How come I overlooked this, I find this really interesting! The combination of continue + ollama looks really promising. I will have to check later if there is anything that can be done. I won’t promise anything though.

Seems like the latest version of the continue extension does indeed work on nixos!

Would still be nice to have the server packaged though.

So having it packaged means having it packaged as a vscode extension, yes?