Getting the continue.dev server running on NixOS

Help
1

Hey,
I wanted to try and run a github copilot-like setup locally on my nixos machines but ran into some packaging issues.

The way to do it right now seems to be the https://continue.dev/ extensions for VSCode and Jetbrains IDEs.
The way it works is you install the extension, that runs the continue.dev server, which connects to a language model to generate all the completions.
The way it connects to a locally running language model is through https://ollama.ai/ (Ollama | Continue).

Now I managed to do the following:

  1. ollama is packaged in nixpkgs
  2. a module for running ollama as a service is available (e.g. GitHub - havaker/ollama-nix: CUDA-enabled ollama nix flake)
  3. you can install the continue.dev extension into vscode

HOWEVER
Running the continue.dev server packaged with the extension fails on NixOS.

 [error] Error: spawn /home/xxx/.vscode-oss/extensions/continue.continue-0.7.29-linux-x64/exe/continue_server ENOENT
	at ChildProcess._handle.onexit (node:internal/child_process:283:19)
	at onErrorNT (node:internal/child_process:476:16)
	at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

To solve issues like this you can run the continue.dev server on your own and connect to it, docs here: https://continue.dev/docs/walkthroughs/manually-run-continue

So I started a quest to first package the server and then run it as a service.
However I’m not very experienced in packaging (complex ai) python packages with poetry2nix, so I got stuck after a few ours of trying.

Here is what I got so far, probably including a lot of mistakes. I feel like I’m missing something fundamental, since it’s doing so many package overrides.

Would appreciate if someone could give me some pointers, as I feel like it would be a cool capability for everyone to have this running on nixos.

1 Like
2

Bump, would like to know if a solution is possible

3

How come I overlooked this, I find this really interesting! The combination of continue + ollama looks really promising. I will have to check later if there is anything that can be done. I won’t promise anything though.

4

Seems like the latest version of the continue extension does indeed work on nixos!

Would still be nice to have the server packaged though.

5

So having it packaged means having it packaged as a vscode extension, yes?

6

the continue extension does in fact work, however, it is broken due to the server not being packaged, leading to a currently non working install of continue.dev

7

As the VSCode page on wiki.nixos.org suggests, using vscode.vhs allows running VSCode & this Continue extension without other tinkering about. (I was able to get it running with ollama).

With vscode (not the FHS one), the extension runs into problems before it can set up. e.g. I get this error:

2024-05-01 15:13:58.996 [info] ExtensionService#_doActivateExtension Continue.continue, startup: false, activationEvent: 'onWalkthrough:continue'
2024-05-01 15:14:00.427 [error] Error: libstdc++.so.6: cannot open shared object file: No such file or directory
	at process.func [as dlopen] (node:electron/js2c/node_init:2:2214)
	at Module._extensions..node (node:internal/modules/cjs/loader:1356:18)
...

Which AFAICT relates to the out/build/Release/node_sqlite3.node. (New to me: the .node files are nodejs addons).

For a more declarative vscode-with-extensions.override { ... }, I guess the next step would be to look at how the other VSCode extensions in nixpkgs work.

FWIW… the links from @lenny’s original post are now dead. (The Python server got moved to continuedev/legacy-python-server, there’s no mention in the continue docs about running a continue server).

8

I created overlay to solve this issue

self: super: {
  vscode = super.vscode.overrideAttrs (oldAttrs: rec {
    nativeBuildInputs = oldAttrs.nativeBuildInputs or [] ++ [self.makeWrapper];
    postInstall =
      oldAttrs.postInstall
      or ""
      + ''
        wrapProgram $out/bin/code --set LD_LIBRARY_PATH "${self.stdenv.cc.cc.lib}/lib/"
      '';
  });
}