Nixd: nix language server

inclyc · June 8, 2023, 6:35pm

Recently I wrote a brand new nix language server, which links with official NixOS/nix library.

Home page: GitHub - nix-community/nixd: Nix language server, based on nix libraries [maintainer=@inclyc]

About

Nixd is a Nix language server that directly uses (i.e., is linked with) the official Nix library (GitHub - NixOS/nix: Nix, the purely functional package manager).

Some notable features provided by linking with the Nix library include:

Nixpkgs option support, for all option system (NixOS/home-manager/flake-parts).
Diagnostics and evaluation that produce identical results as the real Nix command.
Shared eval caches (flake, file) with your system’s Nix.
Native support for cross-file analysis.
Precise Nix language support. We do not maintain “yet another parser & evaluator”.
Support for built-ins, including Nix plugins.

Features

We provide native options auto-completion system, by lazily evaluated merged option set. (Goto declarations also work)
hm-options-decl

Note that not only home-manager options, generally all nixpkgs option system are supported. We have tested nixos options, home-manager options, and even flake-parts.

And, we know the lambda arguments passed to your nix file, e.g. by callPackage

Also, we complete variables inside with expressions (and also complete stdenv, lib, with hover support). Note that the diagnostic missing “name” is reported.

package

Installation & User Guide

We support all kinds of evaluation & cross-file analysis by allow our user explicitly specify a nix installable. Which will be used for cross-file analysis pivot. For more information, please see:

github.com

nix-community/nixd/blob/b5079c4d79905048d3c0b39e1a2a6a66067f1111/docs/user-guide.md

## User Guide

### Installation

Our flake.nix provides a package named `nixd`, and an overlay to nixpkgs that add the `nixd` package.
So, if you would like to use `nixd` in your flake:

```nix
{
  description = "My configuration";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    nixd.url = "github:nix-community/nixd";
  };

  outputs = { nixpkgs, nixd, ... }:
    {
      nixosConfigurations = {
        hostname = nixpkgs.lib.nixosSystem

This file has been truncated. show original

Developers’ Manual

So how do these things work? I’ve written a brief explanation here. TLDR: we use callbacks and inject custom AST nodes to nix evaluator.

Limitations

Currently critical features like “rename” & “semantic tokens” are not yet supported yet. Because we cannot retrieve range information from nix ASTs. Many language features like AST-based editing do expect ASTs preserving enough information.
Source Tree Abstraction. We do need provide a VFS to nix evaluator, and provide file contents from the memory. Because buffers you are editing does nothing to do with real filesystems, however, we need to tell nix do not access filesystems directly. This prevents us collecting language information from pure-evaluated nix flakes.

Status

I just released the first version (which means all trivial features are implemented and tested). And the code base is in rapid development. The status page could be found here. I just submitted a PR: nixpkgs#236675

Feedback

Any questions & feature request please comment here

Update v1.1.0:

Main topic: Nixd 1.1.0 released

Completely New Features

Goto Definition - Values (e.g. lambdas, attrs)

goto-def

Goto Definition - Derivations

goto-def-2

Document Symbol

symbol

Document Link

Rename

rename

Static Analysis

Provide static bindings (such as `rec`) without any evaluation.

Improvements

Automatic Evaluation

You don’t need to manually set “eval depth”. Nixd will evaluate where you touched lazily.

Diagnostic

Show nix stack trace (completely consistent with official implementation)

Option System

The nixpkgs option auto-completion subsystem will show you example if specified.

Write a package using nixd!

write-package

Nebucatnetzer · June 9, 2023, 5:45am

Do you know how it compares to Nil?

jamesbrock · June 9, 2023, 7:55am

Very nice! Is anyone working on a Visual Studio Code extension?

inclyc · June 9, 2023, 8:02am

Currently we do not need a dedicated vscode extension

VSCode extension “Nix IDE” works out of box for nixd. GitHub - nix-community/vscode-nix-ide: Nix language support for VSCode editor [maintainer: @jnoortheen]

Just use:

{
  "nix.enableLanguageServer": true,
  "nix.serverPath": "<path-to-nixd>",
  "nix.serverSettings": {
     // Nixd stuff goes here
     "nixd": { }
  }
}

inclyc · June 9, 2023, 8:07am

You can find more static analysis & more AST-based editing features provided by nil. Instead, for nixd you can find more “dynamic” features that inspect what happens in nix evaluator it self. (e.g. nixd provide accurate attrset fields in lib, pkgs, and even complete with expressions, that is impossible without evaluation).

Actually nixd has a completely different way to implement completion. For example, in nixd, we lazily evaluated a merged option set.

Currently, some important LSP features cannot be trivially implemented, for example, just variable renaming & symbol hierarchy. I post the topic here to see if upstream nix authors would like to co-operate with this language server, or we have to write our own parser (for preserving language information).

mightyiam · June 9, 2023, 11:20am

Thank you for your work.

NobbZ · June 9, 2023, 12:28pm

An alternative approach that I used for nil a long time, was to create a wrapping derivation that provided both bin/nil and bin/rnix-lsp, where the latter was a script running the former (base idea).

I still have the drv around and use it occasionally, but in the meantime I also explicitely configured VScode to use nil from PATH.

If anyone is interested, the derivation is publicly available:

github.com

NobbZ/nixos-config/blob/c9753278a89272384150d543fb2f20e600ae3903/packages/default.nix#L20-L32


      
          nilBase =
            if upkgs.stdenv.isLinux
            then inputs'.nil.packages.nil
            else upkgs.nil;
          
          
rnil-lsp = upkgs.writeShellScriptBin "rnix-lsp" ''
            exec ${nilBase}/bin/nil "$@"
          '';
          
          
nil = upkgs.symlinkJoin {
            name = "nil";
            paths = [nilBase rnil-lsp];
          };

iFreilicht · June 9, 2023, 10:09pm

That is amazing! This sort of discoverability was always the biggest annoyance with current Nix language servers for me. Huge win in usability.

Ah that’s interesting, maybe this might be a good way for nixd to quickly offer these features without having to re-implement them?

It seems like you can’t get everything with a single approach.
You need to actually partially evaluate nix expressions to offer the completion features of nixd, but the features it’s currently missing seem to be pretty well implemented in nil from my experience.

@inclyc, what is your plan for implementing these features?

iFreilicht · June 9, 2023, 10:17pm

The link to regression tests in your readme directs to a 404.

inclyc · June 10, 2023, 2:02am

That’s a good point for new contributors

Regression tests: https://github.com/nix-community/nixd/tree/main/tools/nixd/test

inclyc · June 10, 2023, 2:08am

Yes, I was also considering whether we should use IPC with nil to implement functions such as rename and code action. In fact, the evaluation of nixd internal implementation is achieved through IPC with some child processes.

As mentioned earlier, we actually lack a lot of position information in the AST. The parser in NixOS/nix does not save this information for us. I think the best approach would be for us to work together with upstream. (similar to Source Tree Abstraction).

inclyc · June 10, 2023, 2:16am

In fact, implementing AST-based editing is essential for a language server. There are several options:

Upstream Nix parser saves position information for us, at least for the beginning and end of each expression.
Use IPC with nil. Text Document Sync events need to be sent to nil, and features such as completion can be implemented by merging results.
We need to write our own parser and connect the “Static AST” with the “Dynamic AST” provided by Nix.

inclyc · June 10, 2023, 2:26am

At the beginning of the design, I wanted to provide a language server with a more consistent experience with NixOS/nix. Similar to LSP’s diagnostic information, error reporting, and completion, all should be the same as NixOS/nix. This has already been achieved in the C++ world (clangd/ccls and clang). So I may be more inclined to work with upstream. In fact, it is not difficult to re-implement a Nix parser (in fact, I have been working on clang front-end and some private compiler work for a long time), but doing so requires:

Extensive testing to ensure that your parser is consistent with upstream.
Update all syntax owned by Nix with upstream updates (will your LSP report an error for ExprPos? (i.e. __curPos)

In fact, currently there is no parser & evaluator in nixd, and we completely share the upstream infrastructure (i.e. libexpr, libcmd, etc.).

iFreilicht · June 10, 2023, 6:39am

I also think this is the best long-term solution. As you say, this guarantees 100% consistency between LSP and actual evaluation. I think it might also result in the least maintenance effort.

To actually do this, it might make most sense for you to just create a fork of nix, implement the features you need, and them submit them as a PR to upstream.

Potentially, this could be a faster solution to quickly get the most functionality possible into the hands of users.

I can’t judge the amount of effort either solution requires, though.

inclyc · June 12, 2023, 7:12am

Our new feature: goto definition for packages, really.

goto-def-package

inclyc · June 12, 2023, 11:04am

And new feature: goto definition on more generic values. (e.g. where is mkOrder defined?)

lambda-location

Sandro · June 13, 2023, 9:19am

Are you going to remove the syntax error as soon as you start to type anything?

inclyc · June 13, 2023, 9:46am

Fine, that might be annoying.

Freed-Wu · July 6, 2023, 3:35pm

How can we complete nixos options on /etc/nixos/configuration.nix and package names on environment.systemPackages of /etc/nixos/configuration.nix? And can we have document hover for nixos options and package description on /etc/nixos/configuration.nix? I always cost many time to switch between browser which open https://search.nixos.org/options and https://search.nixos.org/packages and vim which open /etc/nixos/configuration.nix. If there are a language server can complete and hover nixos options and package names, it will be nice.

inclyc · July 7, 2023, 5:19am

How can we complete nixos options

Nixd do these kind of completion now, but it is based on string manipulation, not for ASTs, (the AST-based implementation is working in progress).

See Nixd 1.1.0 released

package names

It is already implemented. See this example: Nixd 1.1.0 released

document hover

The documentation + example are provided with the completion list. “Hover” documentation is working in progress.

package description

v1.1.0 release has support “goto definition” for packages. The description & version is somehow easy to implemented, how about submit a formal issue (for feature-request) on github?