Add runtime dependencies to a service

Help
1

I’m using lighttpd in NixOS. I have a CGI script that needs to be able to invoke tools such as pandoc. How can I make these tools available to lighttpd at runtime?

#!/usr/bin/env bash

echo 'Content-Type: text/html'
echo ''
echo '<h1>Hello CGI world</h1>'

echo "The next line fails because pandoc isn't available to lighttpd."
pandoc -v

Here’s the relevant portion of my NixOS configuration:

  services.lighttpd = {
    enable = true;
    port = 3000;
    document-root = "/www";
    enableModules = [ "mod_auth" "mod_cgi" ];
    extraConfig = ''
        $HTTP["url"] =~ "/cgi-bin/" {
        	cgi.assign = ( "" => "" )
        }
        
        debug.log-request-handling = "enable"    
      '';
  };
2

I also tried using a nix-shell shebang…

#!/usr/bin/env nix-shell
#!nix-shell -i bash -p pandoc

echo 'Content-Type: text/html'
echo ''
echo '<h1>Hello CGI world</h1>'

echo "added the nix-shell line<br/>"

echo "The next line fails because pandoc isn't available to lighttpd."
pandoc -v

However…

Aug 01 20:50:33 wombat11k lighttpd[3764112]: error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)
Aug 01 20:50:33 wombat11k lighttpd[3764112]:        at «string»:1:25:
Aug 01 20:50:33 wombat11k lighttpd[3764112]:             1| {...}@args: with import <nixpkgs> args; (pkgs.runCommandCC or pkgs.runCommand) "shell" { buildInputs = [ (pandoc) ]; } ""
Aug 01 20:50:33 wombat11k lighttpd[3764112]:              |                         ^
Aug 01 20:50:33 wombat11k lighttpd[3764112]: (use '--show-trace' to show detailed location information)
Aug 01 20:50:33 wombat11k lighttpd[3762864]: 127.0.0.1 localhost:3000 - [01/Aug/2022:20:50:33 +0100] "GET /cgi-bin/test.sh HTTP/1.1" 500 365 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:103.0) Gecko/20100101 Firefox/103.0"
3

Probably best to package your script so it includes all dependencies and reference this from lighttpd. From here you can put all your cgi scripts into a single derivation and point to it with lighttpd I assume.

Does that make sense?

4

I can package up my program easily enough, but how would I make it available to lighttp? Whether it’s pandoc, or a custom program I write, I need some way that lighttpd can invoke it. The lighttpd doesn’t “see” anything installed by Nix. It can’t even invoke basic stuff like ls or cat.

5

The shebang error is saying roughly what you can do (if you want to approach it that way). You may be able to use something like -I nixpkgs=channel=nixos-unstable-small or -I nixpkgs=https://github.com/nixos/nixpkgs/archive/<some-commit-rev>.tar.gz.

I imagine the performance of that wouldn’t be amazing, though; not sure if that matters in your case?

6

Yeah, I wouldn’t like that. Even though I guess it would cache the build the first time, I imagine it would keep checking the nixpkgs repo to see if there’s an updated copy or something.

I’d rather figure out the “right way” to do stuff like this.

7

I haven’t really fiddled with services like this so I’m not sure about the first (and most important…) part here.

That said, if your scripts are all bash and are more than a few lines long or change a lot, you might benefit from using resholve (nix API README, project) to do the packaging step. It’ll try to ensure all of the dependencies are found, specified, and substituted into the script so that it can run without worrying about its environment.

8

It will cache, but nix-shell has some per-invocation overhead; if the server runs this script a lot, it might be prohibitive. If it’s once in a blue moon, it may be fine.

9

If you package your script make sure you include all dependencies required so the program can execute in any environment and it will have everything it needs (like coreutils, pandoc, etc…).

Just read up on how to package bash scripts with nix.

10

The tests, at least, model how to supply the document root from a package: https://github.com/NixOS/nixpkgs/blob/90b3a5b927570da950a7aec66820cf2b0316ba64/nixos/tests/lighttpd.nix#L8-L11.

I’m not super sure, but it smells like https://github.com/jekor/jen/blob/62cc2f7a75177181d85da07102d54c698e619ff6/jen.nix may also be doing something similar to what you need.

11

@mhwombat I wrote a post about the main approaches to packaging Shell with Nix a couple weeks ago: no-look, no-leap Shell script dependencies. It addresses the shebang approach, and three “packaged” approaches (including resholve, which I mentioned a couple comments up).

3 Likes
12

Wow that is awesome @abathur! I hadn’t taken the time to read through your work until now and I regret that I waited this long. What a great reference post among other things.

:heart:

1 Like
13

I’m using lighttpd in NixOS. I have a CGI script that needs to be able to invoke tools such as pandoc. How can I make these tools available to lighttpd at
runtime?

┌────
│ {
│ systemd.services.lighttpd.path = with pkgs; [ pandoc ];
│ }
└────

This injects pandoc into the path of the systemd unit, so anything invoking pandoc should just work.

1 Like
14

Yeah, resholve is awesome!

1 Like
15

Ooh! I now see there’s a whole bunch of options I didn’t know about, of the form:

systemd.services.<name>.<something>
16

Ooh! I now see there’s a whole bunch of options I didn’t know about, of the form:
systemd.services..

Sure, but those are for the systemd configuration for the application. systemd.services.lighttpd vs services.lighttpd.

In most cases, you shouldn’t need to touch anything there. Injecting applications into the path is an anti-pattern IMHO.

17

I tried systemd.services.lighttpd.path = with pkgs; [ pandoc ];, but the problem remains. I guess my best option at this point is to write my own derivation/NixOS service that bundles lighttpd and my app together. Which is kind of more trouble than I wanted to go to.

Aug 02 13:45:29 wombat11k lighttpd[3809461]: (response.c.513) Basedir      : /www
Aug 02 13:45:29 wombat11k lighttpd[3809461]: (response.c.515) Rel-Path     : /cgi-bin/test.sh
Aug 02 13:45:29 wombat11k lighttpd[3809461]: (response.c.517) Path         : /www/cgi-bin/test.sh
Aug 02 13:45:29 wombat11k lighttpd[3809461]: (response.c.536) -- handling subrequest
Aug 02 13:45:29 wombat11k lighttpd[3809461]: (response.c.538) Path         : /www/cgi-bin/test.sh
Aug 02 13:45:29 wombat11k lighttpd[3809461]: (response.c.540) URI          : /cgi-bin/test.sh
Aug 02 13:45:29 wombat11k lighttpd[3809461]: (response.c.542) Pathinfo     :
Aug 02 13:45:29 wombat11k lighttpd[3809651]: /www/cgi-bin/test.sh: line 10: pandoc: command not found
Aug 02 13:45:29 wombat11k lighttpd[3809461]: 127.0.0.1 localhost:3000 - [02/Aug/2022:13:45:29 +0100] "GET /cgi-bin/test.sh HTTP/1.1" 200 119 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:103.0) Gecko/20100101 Firefox/103.0"
Aug 02 13:45:35 wombat11k lighttpd[3809461]: (connections.c.1456) connection closed - keep-alive timeout: 8

The current version of my CGI script is

#!/usr/bin/env bash

echo 'Content-Type: text/html'
echo ''
echo '<h1>Hello CGI world</h1>'

echo "The next line fails because pandoc isn't available to lighttpd."
pandoc -v

My current NixOS config:

  systemd.services.lighttpd.path = with pkgs; [ pandoc ];
  services.lighttpd = {
    enable = true;
    port = 3000;
    document-root = "/www";
    enableModules = [ "mod_auth" "mod_cgi" "mod_rewrite" ];
    mod_status = true;
    extraConfig = ''
        url.rewrite-once = ( "^/maths/(.+)" => "/cgi-bin/md2html?$1" )
        
        $HTTP["url"] =~ "/cgi-bin/" {
        	cgi.assign = ( "" => "" )
        }
        
        debug.log-request-handling = "enable"    
      '';
  };
1 Like
18

I tried systemd.services.lighttpd.path = with pkgs; [ pandoc ];, but the problem remains.

Assuming your `test.sh’ script is also managed by nix, you should be able to inject the path there or even better use resholve to ensure all dependencies are satisfied.