Lazy Apps – On demand realization

Turns out I made a thing called Lazy Apps a year and a half ago that maybe is of general interest. Basically it allows you to wrap a package so that it won’t be realized until you run a specific executable within the package.

For example, if you install lazy-app.override { pkg = pkgs.hello; } to your profile then pkgs.hello will not be downloaded into your profile. Instead a small executable hello will be installed, which when run transparently realizes pkgs.hello and runs ${getExe pkgs.hello} with the arguments you provided to the wrapper.

I find this very useful for tools and applications that I use very rarely but still be available ASAP when I need them.

Caveats

By its nature this won’t work when you are offline.

Some packages, especially GUI ones, don’t seem to like being wrapped in this way so your mileage may vary.

I won’t develop this package further so if you find it interesting and would like to improve it then feel free to take it over.

Example usage

In my Emacs configuration I use a bunch of LSP servers but they will only be realized when I need them:

# Make LSP servers available in exec-path and PATH. We need to do both
# since envrc seem to recreate exec-path from PATH.
let
  tree =
    with pkgs;
    symlinkJoin {
      name = "lazy-language-servers";
      paths = map lazy-app.override [
        { pkg = bash-language-server; }
        { pkg = shellcheck; }

        {
          pkg = clang-tools;
          exe = "clangd";
        }
        { pkg = cmake-language-server; }
        { pkg = dockerfile-language-server-nodejs; }
        { pkg = kotlin-language-server; }
        { pkg = nil; }
        {
          pkg = pyright;
          exe = "pyright-langserver";
        }
        { pkg = rust-analyzer; }
        { pkg = terraform-ls; }
        { pkg = yaml-language-server; }
      ];
    };
in
''
  (add-to-list 'exec-path "${tree}/bin")
  (setenv "PATH" (concat "${tree}/bin" path-separator (getenv "PATH")))
''

Occasionally I want to do a quick OCR of a piece of text on screen. For this I have a small shell script that will only download Tesseract when needed (the Tesseract package is a bit hefty so running this when needing to download may take a little while):

pkgs.writeShellScriptBin "shot-ocr" (
  let
    grim = lib.getExe pkgs.grim;
    slurp = lib.getExe pkgs.slurp;
    tesseract = lib.getExe (pkgs.lazy-app.override { pkg = pkgs.tesseract; });
    wlClip = lib.getExe pkgs.wl-clipboard-rs;
  in
  ''
    ${grim} -g "$(${slurp})" - | ${tesseract} stdin stdout | ${wlClip}
  ''
)

You can find some more examples in my configuration.

24 Likes

How does this compare with comma?

Comma will pull an application down on demand, and the app name must be prefixed with “,” each time it’s launched, and doesn’t have to be specified in your config in advance.

This will pull an application down on demand (it does so via a launcher shortcut or by just typing the command in the terminal) and it does need to be specified in you config in advance.

So, they’re subtly different, but both have advantages depending on the app and your workflow when you occasionally need it.

4 Likes

Indeed, that is pretty much it. I use lazy-apps mainly when I want the the command to be available through tab-completion or launcher or when I use the command inside some script like in the two examples I gave. I use comma for purely ad-hoc commands and it is really cool.

Since the derivation is provided when you build the wrapper it has a much quicker startup than comma, which needs to look up the derivation that provides the executable. You also avoid the risk of multiple packages providing the same executable.

1 Like

This seems like a really awesome and cool concept. Quick question though, do you happen to have any insights as to why GUI apps seem to behave problematically? Why would being wrapped in this way cause problems vs. the usual wrapping that certain apps have to make them work in NixOS?

1 Like

I can’t recall the exact error but I think it mainly had to do with not finding resources or so. Actually, in my configuration I had these applications commented out and when I just now tried turning them into lazy-apps they all worked! So I guess the problem must have resolved in Nixpkgs. See the diff. Even Chromium worked OK :slightly_smiling_face:

3 Likes

Awesome! This concept seemed kinda cool to me especially for some apps that I know I download but use exceedingly rarely, and it’d be great to make them lazier so that I only need to worry about em in those rare cases . Knowing it works without any issues (hopefully at least, you never know) is really great.

@rycee back at it :stuck_out_tongue:

Pretty fun and simple idea; just create 1-off scripts that nix realise and exec it lol.

I haven’t thought this through; but you probably lose any introspection through nix query or do they show up since the /nix/store path is in the script?

You also probably delete it all everytime you gc which is fine.

It kind of has the wrapProgram type of vibe – this might be a nice addition to wrapProgram

The “use-case” i can see here is that you can create “full-featured” NixOS configurations for tiny-hardware (RaspberryPi) and only execute/pay for what you need without sacrificing being judicious in the NixOS config itself.

I imagine you could enforce a gc root through using a nix command (like profile maybe?) but i can imagine how that’ll quietly lead to junk entries that may end up eating tons of storage over time. That is actually a good question though, what would be the best/most reasonable manner of creating roots for these?

As-is it will indeed be immediately garbage collected. But, as I mention in the future improvements section, it should be pretty easy to simultaneously install the package into a special lazy-apps profile. Then something like nix-collect-garbage --delete-older-than 30d should leave the realized derivation alone.

Unrelated, another difference from comma is that if you set up a good desktop file then tools like xdg-open should work as expected when trying to open a file that is handled by a lazy application. For example, if you run xdg-open capture.pcap (or, if you are GUI inclined, double-click the capture.pcap file) then Wireshark would be started even if it wasn’t on your system previously.

2 Likes

You could also replace this as well with nix run as a wrapProgram I think maybe;
(I’d have to test it) – but I like the use of nix realize more.

Is there a way to use this approach for unfree/custom packages that can’t be simply nix-store --realise’d?

Say for e.g. lazy-app.override { pkg = pkgs.spotify; }, it cannot be substituted from cache.nixos.org.

I have tried to realise the drvPath of the pkg with unsafeDiscardStringContext as a fallback.

diff --git a/default.nix b/default.nix
index 7228404..d3f1364 100644
--- a/default.nix
+++ b/default.nix
@@ -16,6 +16,7 @@
           (
             let
               exePath = if exe != null then lib.getExe' pkg exe else lib.getExe pkg;
+              drvPath = builtins.unsafeDiscardStringContext pkg.drvPath;
 
               notify-send = lib.getExe pkgs.libnotify;
             in
@@ -35,6 +36,7 @@
 
                 app='${exe}'
                 path='${builtins.unsafeDiscardStringContext exePath}'
+                drv='${drvPath}'
 
                 if [[ -e $path ]]; then
                     exec $path "$@"
@@ -44,7 +46,7 @@
                     noteId=$(${notify-send} -t 0 -p "Realizing $app …")
                     trap "${notify-send} -r '$noteId' 'Canceled realization of $app'" EXIT
                     SECONDS=0
-                    nix-store --realise "$path" > /dev/null 2>&1
+                    nix-store --realise "$path" > /dev/null 2>&1 || nix-store --realise "$drv" > /dev/null 2>&1
                     trap - EXIT
                     ${notify-send} -r "$noteId" "Realized $app in $SECONDS s"
                 fi

but now, how can I retain the pkg.drvPath? it needs to exist first, so should I nix-instantiate the pkg’s derivation to ensure its drvPath file gets created? keep-derivations is set to true by default but they still get gc’d in this case because no gc-roots tied to pkg.

is there a simple solution I am missing here?

if it is more appropriate to discuss this elsewhere please let me know