Trying to get $out directory from Nix derivation

Help
1

Hi! Struggling a bit with some Nix syntax, and I am not able to properly figure it after lots of googling.

I have a derivation ihp-build.nix looking like this:

{ stdenv, pkgs, ... }:

stdenv.mkDerivation rec {
  name = "blog";
  src = /home/lillo/kode/ihp-test;
  buildInputs = [ ];
  buildPhase = '' 
    nix-shell - -run "make build/bin/RunUnoptimizedProdServer"
    nix-shell - -run "make static/prod.css static/prod.js"
    '';
  installPhase = ''
    mkdir - p $out
    cp -rv $src/* $out
  '';
  shellHook = ''set - o allexport; source .env; set +o allexport'';
}

I want to run a systemd process and access a build script that will exist in the $out folder of the derivation above and treat it like a set.

Here is the initial configuration, where I optimisticly want to just get ${ihpApp.out}, even though I understand it’s not the way. But I suppose this is in some way possible?

{ config, lib, pkgs, stdenv, ... }:
let
  ihpApp = import ./ihp-build.nix;
in
{

  systemd.services.ship = {
    description = "IHP server on ship-nix";
    enable = true;

    after = [ "network.target" "postgresql.service" ];
    wantedBy = [ "multi-user.target" ];
    environment = {
      DATABASE_URL = "postgres://lillo:userpassword@localhost:5432/ship";
      SYSTEMD_LOG_LEVEL = "debug";
    };
    serviceConfig = {
      Type = "exec";
      WorkingDirectory = /home/lillo/kode/ihp-test;
      User = "lillo";
      ExecStart = ''
        ${ihpApp.src}/build/bin/RunProdServer
      '';
    };
  };
}

This gets me the error message:

building the system configuration...
error: value is a function while a set was expected

       at /nix/store/bvf99a2npxhn2zjz24j6n7hx01iqin2b-source/ihp-test.nix:70:11:

           69|       ExecStart = ''
           70|         ${ihpApp.src}/build/bin/RunProdServer
             |           ^
           71|       '';

So I think I get what I need to unblock myself next.

So I try to add pkgs and stdenv as arguments to the import statement to hopefully be able to access the derivation as a set.

{ config, lib, pkgs, stdenv, ... }:
let
  ihpApp = import ./ihp-build.nix { inherit pkgs stdenv; };
in

It appears logical to me that the top statement exposes pkgs and stdenv, so I am not getting why I can’t pass these attributes to the ihpApp value.

But I’m then getting stuck on this error message:

building the system configuration...
error: attribute 'stdenv' missing

       at /nix/store/9dmig1pv9njj5kswvs8yvw3qp6b81zkd-source/lib/modules.nix:365:28:

          364|         builtins.addErrorContext (context name)
          365|           (args.${name} or config._module.args.${name})
             |                            ^
          366|       ) (lib.functionArgs f);
(use '--show-trace' to show detailed location information)

I guess the let in statement are out of scope, but how would I get in inside the scope of the let..in? Both help for unblocking and relevant reading resources will be highly appreciated :pray:

2

Why run a nix-shell inside a buildPhase?

The $out in the simplest case is just “${inhApp}/build…” but you neet to copy the build dir to out, but it doesn’t seem you do it:

  installPhase = ''
    mkdir - p $out
    cp -rv $src/* $out
  '';
1 Like
3

stdenv is an attribute of pkgs, so it’s pkgs.stdenv, it’s not passed in to the configuration modules

You seem to have some big dark areas… maybbe you need to study the docs a bit more?

4

If this is to get at some dependencies in a shell.nix, it’d be much better to just create a default.nix in that directory that does the actual build and import it. Then you can share the dependencies between your shell.nix and default.nix. Calling nix recursively to manage build dependencies is probably not advisable, at the very least because it’ll be very slow.

If you share the whole example, including the nix files in that project, maybe we can help work through something a bit nicer?

5

Thanks for help so far :slight_smile:

build dir is built inside src and is certainly present, I checked by creating it standalone and inspecting nix show-derivation, so it should be there :slight_smile: This is how the framework builds it by default.

Ah, allright, hard to keep condensate all the info in the docs into my brain, and process to practical knowledge, but yeah, I’m learning as best I can, although possibly a bit slowly :slight_smile: I promise I will re-read the docs more thoroughly. I’m very motivated to get this knowledge as it’s very relevant to my coming project which will do lots of nix-stuff.

Rewriting to only have pkgs as an argument and using pkgs.stdenv works well on the imports.

nix-shell might or might not be necessary as far as I know, but did it because it was necessary to do this performing it from my own terminal.

Yes, that would be awesome :slight_smile: I think the current nix files covers all the necessary declarations for that part, as the issue is getting the systemd to build and run properly.

The project is basically the IHP starter, a Haskell based web framework, and yes it has a default.nix:
I pushed it to a public repo for reference GitHub - kodeFant/ihp-test

I’ll post more complete nix files in coming reply so this chunk is not becoming to long :slight_smile:

6

This is the ihp-test.nix, which is basically just imported into configuration.nix which is connected to a flake.nix if that is relevant.

It’s a bit messy now as I tried to iterate, but wanted to reply quickly, so please forgive that and read the intention behind it :wink:

{ config, lib, pkgs, ... }:
let
  ihpApp = import ./ihp-build.nix { inherit pkgs; };
in
{

  systemd.services.nginx.serviceConfig.ProtectHome = "read-only";
  # systemd.services.nginx.serviceConfig.ReadWritePaths = [ "/home/lillo/kode/ihp-test" ];

  # Uncomment and edit to enable https
  # security.acme.certs = {
  # "#{domain}".email = "#{userEmail}";
  # };
  # Accept terms and conditions of LetsEncrypt
  # security.acme.acceptTerms = true;

  services.nginx = {
    enable = true;
    user = "lillo";
  };

  services.nginx.virtualHosts = {
    localhost = {
      default = true;
      enableACME = false;
      forceSSL = false;
      locations."/" = {
        proxyPass = "http://localhost:8000";
      };
    };
  };

  services.postgresql = {
    enable = true;
    package = pkgs.postgresql_11;
    ensureDatabases = [ "ship" ];
    ensureUsers = [
      {
        name = "lillo";
        ensurePermissions = {
          "DATABASE ship" = "ALL PRIVILEGES";
        };
      }
    ];
    enableTCPIP = true;
    authentication = ''
      host    all             all             0.0.0.0/0            md5
    '';
    initialScript = pkgs.writeText "backend-initScript" ''
      ALTER ROLE ship WITH PASSWORD 'userpassword';
    '';
  };


  systemd.services.ship = {
    description = "IHP server on ship-nix";
    enable = true;

    after = [ "network.target" "postgresql.service" ];
    wantedBy = [ "multi-user.target" ];
    environment = {
      DATABASE_URL = "postgres://lillo:userpassword@localhost:5432/ship";
      SYSTEMD_LOG_LEVEL = "debug";
    };
    serviceConfig = {
      Type = "exec";
      WorkingDirectory = ihpApp;
      User = "lillo";
      ExecStart = ''
        ${ihpApp}/build/bin/RunProdServer
      '';
    };
  };
}

And this is the ihp-build.nix file.

{ pkgs, ... }:

pkgs.stdenv.mkDerivation rec {
  name = "blog";
  src = ./ihp-test;
  buildInputs = [ ];
  buildPhase = '' 
    make build/bin/RunUnoptimizedProdServer
    make static/prod.css static/prod.js
    '';
  installPhase = ''
    mkdir -p $out
    cp -rv $src/* $out
  '';
  shellHook = ''set - o allexport; source .env; set +o allexport'';
}

I think the example is complete enough to see what I’m doing, I have confirmed that the nginx stuff works, so it’s basically the systemd and derivation part I haven’t quite figured out yet.

7

I got it working!

The issue was that I needed direnv, so I added buildInputs = [ pkgs.direnv ]; and it builds and runs perfectly now.

Thanks for the pointers :slight_smile: :pray:

1 Like
8

Glad to see it’s working now and that you got rid of the nested nix build, but still I’m a bit suspicious of some of this. I’ve not tested it, and I imagine your actual project has changed a bit from your GitHub repository at this point, so don’t take my word as gospel. But still, let me point out two things:

  • direnv is a tool to manage environment variables in interactive shells, and should absolutely never be allowed to become a build dependency. Your repository also doesn’t have a .envrc (nor does it have the .env that is referred to in your derivation), so if you depend on direnv somehow it’s importing variables from your local machine - possibly an uncommitted .envrc?
  • You’re not declaring any build dependencies at the moment, while your Makefile depends on a Makefile.dist that should come from the IHP project - is this coming from an uncommitted git submodule?

I think the idiomatic way of doing this would be to make your default.nix look like this:

{...}@args:

let
    ihp = builtins.fetchGit {
        url = "https://github.com/digitallyinduced/ihp.git";
        ref = "refs/tags/v0.18.0";
    };
    haskellEnv = import "${ihp}/NixSupport/default.nix" {
        ihp = ihp;
        haskellDeps = p: with p; [
            cabal-install
            base
            wai
            text
            hlint
            p.ihp
        ];
        # Cleaning up an anti-pattern:
        # https://nix.dev/anti-patterns/language#reproducibility-referencing-top-level-directory-with
        projectPath = builtins.path { path = ./.; name = "ihp-test"; };
    } // args;
in
    haskellEnv

And then just to call it from your ihp-build.nix file with the optimized arg set to true:

import ./ihp-test { optimized = true; }

Or instead of making a file just for that, in configuration.nix:

{ config, lib, pkgs, ... }:
let
  ihpApp = import ./ihp-test { optimized = true; };
<snip>

The upstream build function will build the optimized server target (RunOptimizedProdServer) if you do that, which already includes the css and js targets, so that should work.

This has the huge benefit that it syncs up the native dependencies of your dev and production environments, and automagically updates with any changes to the build scripts in your dev environment, too.

1 Like
9

Thank you so much for taking the time for this answer, @TLATER !

It turned out to be a false positive, that successfully built for some reason. Your feedback is much appreciated! I’m having some growing pains into advanced Nix :laughing:

I also got a tip from the IHP creator himself here: Running a IHP app build on github actions · Discussion #1292 · digitallyinduced/ihp · GitHub

Tomorrow will be an educational day reading thoroughly your post and the other snippet. And I guess the weekend goes to reading the docs through word for word :grinning_face_with_smiling_eyes:

I will post an update to how it’s going when I have tested this!

10

We all did at one point :wink: You’re doing something tricky here, and frankly I find the use of nix in IHP a bit fishy. It’s no wonder you’re struggling with it, especially since the upstream IHP docs don’t explain very much. But then I’ve not done web dev with Haskell, so what do I know.

If you’re planning to maintain your server long-term and fully with nix, I would dissuade you from doing that. It’s adding imperative stuff where it’s really not required; I like your current approach better, adding CI to that will require thinking on the server rather than the application level.

If you’re planning on spinning up AWS servers with non-NixOS operating systems that’s probably a better approach, and that server model suits the “I want to deploy my web server and don’t care what it runs on” model better.

1 Like