Handy scripts for fuzzy searching nixpkgs and nixos-options


#1

Hey y’all, I cooked up a few little scripts to help make searching for package names and options a bit easier; I thought I would share the results in case it was useful to others, or to get feedback.

This script can be adapted for a ton of other uses, but here is how I have it wired together:
an XMonad hotkey launches a utrvt terminal, running fzf, this provides fuzzy searching on packages or options, using an fzf preview window to see additional information (current settings or package descriptions).
Here are the results:


First we need to generate 2 text files full of all the system’s packages and options:

  1. Packages are real easy, simply do nix-env -qa -P | awk '{print $1}' > allpackages.txt
  2. Options are a bit harder, I eventually made a short haskell program to wrap around the “nixos-option” command to dump every possible option name, I am sure there is a much better solution to this. But here is my script:
{-# LANGUAGE OverloadedStrings #-}

module Main where

import qualified Data.Text.Lazy as T
import Data.Text.Lazy (Text)
import qualified Data.Text.Lazy.IO as TIO
import Data.Tree
import System.IO
import System.Process
import Control.Monad
import Data.Monoid

-- Builds option list from local nixpkgs (including new modules)

nixQuery :: Text -> IO Text
nixQuery q = withFile "/dev/null" WriteMode (\null -> do
  (_, Just hout, _, _) <- createProcess (proc "nixos-option" [T.unpack q])
    { std_out = CreatePipe, std_err = UseHandle null }
  TIO.hGetContents hout)

nixBranch :: Text -> IO (Text, [Text])
nixBranch q = do
  ls <- nixQuery q
  if T.isPrefixOf "Value:" ls then return (q, []) else
    let ls'  = T.lines ls
        lsd  = if T.null q then ls' else map ((q<>) . T.cons '.') ls'
    in return (q, lsd)

mkTree :: Text -> IO (Tree Text)
mkTree = unfoldTreeM nixBranch

leaves :: Tree a -> [a]
leaves (Node x xs) | null xs   = [x]
                   | otherwise = concatMap leaves xs

options :: Text -> IO [Text]
options q = leaves <$> mkTree q

main = TIO.writeFile "alloptions.txt" . T.unlines =<< options ""

even looking at that script again I see a few things that would make it significantly faster. Piping directly to the file instead of using “writeFile” would be a big improvement. But in any case this is functional. Compiling with “-threaded” helps a bit, but full disclosure: this is slow as hell and needs to be rebuilt from the ground up. When I set out I had imagined I was going to to hold onto this tree structure, but in actuallity it shouldn’t construct a tree at all. Instead it should just crawl through the options and immediately write them to the text file when it reaches a dead end. IN ANY CASE IT WORKS haha

Edit: I have made a solution with Nix that is a million times faster. It can be found here for anybody who wants to use it.

to run fzf I use:
packages

cat allpackages.txt | fzf \
    --reverse \
    --prompt="NixOS Packages> " \
    --preview="echo -e \"{1}\n\"; nix-env -qa --description -A {1]" \
    --preview-window=wrap

and for options

cat alloptions.txt | fzf \
    --reverse \
    --prompt="NixOS Options> " \
    --preview="echo -e \"{1}\n\"; nixos-option {1}" \
    --preview-window=wrap

for folks who want to wrap all of is a urxvt launcher:
urxvt -e zsh -ic 'cat allfoo.txt | fzf ...'
Just remember to add extra escapes and what not where necessary.

Extensions:

  1. Obviously improving the haskell program
  2. Add fzf actions to open files that define packages and options.
  3. Colorize/otherwise improve fzf previews

#2

Cool. Your using Haskell to do this reminded me of how nixpkgs-update looks up the attr path given a package name. It also is pretty slow, I bet I could speed it up by caching the entire list of packages into a HashMap, like how you save them to allpackages.txt.


#3

Thank Ryan. Last night I caved and gutted the a bunch of the documentation code out of <nixpkgs/nixos/docs/manual> to accomplish this same task, which runs miiiiiles faster. It has the added advantage of dumping descriptions and other info as well so I don’t have to use nixos-options with fzf (which was a little sluggish).
I do prefer writing scripts in Haskell though, Nix is great for packages but it can be so frustrating when you want to borrow a function thats floating around some .nix file’s “let” section.
Nixly Solution


#4

Your screenshots look seductive. FZF is much more interesting than I expected. I imagine combined dictionary:

  • options (current value, default, defintion file, declaration file)
  • packages (description, defintion file, latest version, current version)
  • tests (defintion file)
  • filenames in nixpkgs source tree
  • … maybe some other runtime information (for current system)? …

#5

That sounds like a solid plan. Be sure to check out my other post where I have a vastly improved nix script for pulling the options list.
And yeah I totally agree, FZF is such an awesome tool. Over the years I always find new uses for it; I know it’s basically just fancy AG, but the interface is just so nice :slight_smile:


Franken-Script to generate NixOS Options docs : with custom modules