Local AI agent for NixOS configuration — looking for experience reports

Help
1

Hi all,

I’m new to NixOS — installed it recently and still wrapping my head around modules and flakes. I use AI coding assistants daily for other work, and I’d like to combine the two fully locally, without sending my config off to a cloud provider.

The setup I want to explore: running an agent like opencode (an open-source Claude Code alternative) against a local model via Ollama, to help me iterate on my configuration.nix — propose changes, run nixos-rebuild build or nix flake check to validate, and roll back when things break.

What I’ve already found on this forum

A few threads touch the topic without really covering it:

What I couldn’t find is a concrete report on the combination opencode (or similar) + local model + actually editing a NixOS config. Hence this post.

My context

  • Nearly fresh NixOS install, some flakes
  • MSI B760 / i7-13700K / RTX 4060 Ti 8 GB VRAM
  • Beginner level: I want the agent to help me learn, not to do everything in a black box

Questions for the community

  1. Has anyone actually set up a local AI agent (opencode, Aider, Continue, anything else) to work on their NixOS config? Concrete reports very welcome.
  2. Which local model do you find most useful for Nix specifically? My impression is that Nix syntax may be underrepresented in training data and small models hallucinate a lot.
  3. How do you feed the model context about NixOS options — a hand-written AGENTS.md, an MCP server like mcp-nixos, RAG over the nixpkgs docs?
  4. Does nixai compose with an agent like opencode, or is it one or the other?
  5. For a beginner: is jumping into this now a good idea, or better to learn Nix “by hand” for a few months before relying on an agent?

Any feedback welcome, including negative (“tried it, the local model hallucinated too much, gave up” is a useful answer). If something works out, I’ll post a follow-up write-up with the working setup.

Thanks!

1 Like
2

I think it is because many people are using NixOS because they are in control of every little detail.

Using an agent to do this kind of defeats the purpose, doesn’t it? I mean why not use the agent to configure your Ubuntu directly without the nix step?

3

On point 5, I would be fine with LLMs as personal tutor that never gets bored. I only use paid models because work pays a subscription. opencode is ok, I haven’t kicked the tires much because I can’t use it without it being more expensive than claude code. I don’t have a good enough gpu to run a model that I think is better than me.

I genuinely think that learning how to load your config into the repl, and learning how to explore nix via the repl is a far more useful level to pull before reaching for AI. AI easy button is something you hit after you’ve earnt the right. LLMs Corrupt Your Documents When You Delegate otherwise this.

4

A month ago I dabbled a bit with local AI and was disappointed. (I’ve never used cloud AI.)

I used opencode + ollama, BUT I don’t want/trust these anymore.

  • opencode does connect to the internet, But I can’t trust a project that does such things/changes/..
  • ollama is slow,late,…
    I’m not going to give the references, you can find them online.

I have a beefy laptop with:

  • cpu: Intel i9-13950HX
  • ram: 128GB ECC
  • gpu: NVIDIA RTX 3500 Ada
  • vram: 12GB

I’m thinking/trying to accomplish the following. I could be missing something, so feedback is appreciated.

Everything in a nix controlled vm. I could eventually split the vm in: ai, agent1, agent2, agent3, …

  • microvm.nix:
    – Shared dir: This will be the only connection with the host system. I’m not sure how I’d like to share the project dir. I don’t want the AI to make changes to the .git without me knowing it. Should I use a local git, and let the AI make pr to it…
    – Offline, no internet access. But it needs internet for: direnv , mix(downloading elixir deps) , …
  • pi-mono (instead of opencode)
  • llama-swap
  • llama.cpp (instead of ollama)
  • models? Not sure yet: qwen coder was the one I used in the past.
    – I’d like the host to download/store the model. Then link it to the shared dir.

I don’t have experience with microvm, so I hope it’s possible to do the following in the VM:

  • Use GPU(Nvidia)
  • Use excess system ram
    This will allow me to run bigger models, that don’t fit in vram.
5

I think in the time you got all of this working, you probably could have read all of https://nix.dev and nix pills, and be in a better position to make all of this work on nixos, but that’s just like my opinion, man.

If you just want to sandbox the disk, maybe bubblewrap is easier? Maybe if you want to dynamic firewall you can run opensnitch, and then you can cut internet access a bit more dynamically? agent sandboxing is something I think there is lots of space to do great things with nix, again, just haven’t had the time to invest. Partial rootless overlay store in container? - #2 by tehmatt might be interesting to you?

6
  1. Has anyone actually set up a local AI agent (opencode, Aider, Continue, anything else) to work on their NixOS config? Concrete reports very welcome.

While technically not an “agent” I have enjoyed using the aichat Home Manager - Option Search module from this project: GitHub - sigoden/aichat: All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents, with access to OpenAI, Claude, Gemini, Ollama, Groq, and more. · GitHub

I use it as a CLI for both local models running in Ollama and online with Claude. I have integrated it with shell aliases and some wrapper scripts to quickly give role/context, for example:

  programs.bash = {
    shellAliases = {
      "??" = "aichat-ctx nix_env --role nix-env";
      "???" = "aichat-ctx make_nix --role nix-env --session-ctx ~/.config/aichat/contexts/make-nix.md";
    }
    // lib.optionalAttrs hasLocalAi {
      "?qc" = "aichat-ctx local_env --model ollama:qwen3-coder:latest --role local-env";
      "?q9" = "aichat-ctx local_env --model ollama:qwen3.5:9b --role local-env";
      "?qu" = "aichat-ctx local_env --model ollama:jaahas/qwen3.5-uncensored:latest --role local-env";
      "?doc" = "aichat-ctx doc_search --rag documents --role local-env";
    };

Ollama is running as a system level service and aichat uses the Home manager module with these configs:

  1. Which local model do you find most useful for Nix specifically? My impression is that Nix syntax may be underrepresented in training data and small models hallucinate a lot.

I don’t get the impression that Nix is underrepresented in training data. I have definitely noticed improvement in the quality of Nix outputs at least in frontier models over the past year. With smaller models I think you just have to temper expectations and provide more limited tasks. Qwen3 coder runs OK on my RTX-5070 (max-q) with 8gb VRAM. Gemma4-e4b runs fine as well, but I haven’t tested its performance much with Nix.

  1. How do you feed the model context about NixOS options — a hand-written AGENTS.md, an MCP server like mcp-nixos, RAG over the nixpkgs docs?

I have some limited MD files that are written as part of my Home manager configuration to provide some environment context and preferences. I have rules to try to prevent the biggest syntax mistakes I see even frontier models constantly make, mostly related to forgetting Nix character escaping rules for injected code:

  home.file.".config/aichat/contexts/make-nix.md".text = # markdown
    ''
      ### Escape Variable References
      All shell variable references inside Nix multiline strings (''' ... ''') 
      MUST be escaped with '''''${var} and not referenced with ''${var}.

      ### Package References
      Any binary not part of pkgs.coreutils should be referenced from nixpkgs:
      grep -> ''${pkgs.gnugrep}/bin/grep

      ### Heredoc Restriction
      Heredoc syntax (<<EOF ... EOF) must not be used in Nix injected shell strings.
      The heredoc delimiter conflicts with Nix multiline string parsing, causing
      interpolation errors and breaking LSP highlighting from the heredoc onward.

      ### Nix Multiline String Quote Escaping
      To output literal ''' in a Nix multiline string (.text = ''' ... '''):
      - Use 5x single quotes when the ''' must precede a dollar sign to also
      prevent interpolation, producing literal ''${
      - Use 3x single quotes in all other contexts where a literal ''' is needed.
      Note: These rules cannot be demonstrated with literal examples inside a Nix
      multiline string without triggering the very escaping they describe.
      ...
    '';
  1. Does nixai compose with an agent like opencode, or is it one or the other?

I have not used nixai.

  1. For a beginner: is jumping into this now a good idea, or better to learn Nix “by hand” for a few months before relying on an agent?

There is no way to offload learning. If your intent is to learn, I think an LLM could be a tool that helps facilitate that, but it can also be a crutch that detracts from it. With NixOS, I’m more sympathetic to using AI to help get things working, because if you are using it as a daily driver, and you can’t get the basic software and features that you need working, then that also detracts from learning.

I think the atomic nature of the Nix build system would lend itself to an agentic coding loop, because at a minimum you will know if it built a valid configuration. I don’t think the time and energy to implement it would be worth it, but I haven’t tried it myself, so don’t know for sure.

7

Check out my repo: GitHub - Azteczek/LocalAI-On-NixOS · GitHub

1 Like