After successful mkDerivation, system cannot find executable

Hello Nix community,

I am a beginner in NixOS, and would like to ask a question.

I build a package (magic-vlsi) from Github, using the stdenv.mkDerivation routing. It installed successfully, and I can see the magic shell script in my /nix/store path. If I execute the script in /nix/store/$out/bin/, it runs well. ($out is the install dir of magic)

However, when I run which magic, NixOS return that there is no magic in /nix/store/$out/bin/magic. I am not sure what the problem is. Maybe its because bin/magic is a shell script and not an executable?

Thanks :wink:

How exactly did you install it? Did you add it to environment.systemPackages?

Hello TLATER, thanks for replying.

No, I did not add magic to systemPackages. I used the stdenv mkDerivation method. Here is my script:

let 
  pkgs = import <nixpkgs> { };
in 
  pkgs.stdenv.mkDerivation {
    name = "magic";
    buildInputs = with pkgs;[
        # some packages
];
    src = pkgs.fetchFromGitHub {
      owner = "RTimothyEdwards";
      repo = "magic";
      rev = "8.3.440";
      sha256 = "1c94m6pcbvs3ppvrkbbccwh30lmyssvzm6hylf4jwrbyqll078sg";
    };
  }

Then i use nix-build to build the package.

Right, so that’s just a package definition. You can use nix-build to build it, and then run the application from the result directory.

This does not mean that you’ve installed the package, though. To install a package, you either need to use nix-env/nix profile, or tell NixOS to include it in your configuration by adding it to environment.systemPackages in configuration.nix (or use home-manager).

To turn that into a package you can install from NixOS, first rewrite it as such:

{
  pkgs ? import <nixpkgs> { }
}:
pkgs.stdenv.mkDerivation {
  name = "magic";
  buildInputs = with pkgs;[
    # some packages
  ];
  src = pkgs.fetchFromGitHub {
    owner = "RTimothyEdwards";
    repo = "magic";
    rev = "8.3.440";
    sha256 = "1c94m6pcbvs3ppvrkbbccwh30lmyssvzm6hylf4jwrbyqll078sg";
  };
}

Then, copy that file to /etc/nixos/magic.nix or such, and add it to your environment.systemPackages like so:

# configuration.nix
{pkgs, ...}: {
  # <snip>
  environment.systemPackages = [
    (pkgs.callPackage ./magic.nix { })
  ];
  # <snip>
}

There’s still some bad practice lurking in here (shouldn’t really use pkgs directly in package definitions). If you want a good, full guide for writing packages, nix.dev has a really good one, though it does not cover the actual installation step for the moment, so if you can’t figure out how to turn this into an environment.systemPackages entry feel free to ask: Packaging existing software with Nix — nix.dev documentation

2 Likes

Wow, thank you for the thorough and friendly answer! And thanks for the link, I hadn’t seen a compendium of all stuff nix related in one place. For now I will probably use the executable in result, but in the future if I run into trouble, I will follow the package installation you described.

Okay, I have a new follow-up question (not sure if it should be in a separate thread). Would it be possible to install the package using home-manager and not the systemPackages?

Yes, you can install it using home-manager. it works exactly the same (you just need to put the file in the home-manager configuration folder instead).

Okay, so I read the guide and now have the following structure

├─- magic
│   ├── default.nix
│   ├── magic.nix
├─- modules
|    ├─- system.nix

If needed I can provide the files default.nix and magic.nix, but after running nix-build inside magic/, the package is successfully built.

In modules/system.nix, I have the following line:

{ config, pkgs, ...}:
{
  # <snip>
  environment.systemPackages = with pkgs; [
    # Some other packages, like helix, curl, etc.
    (callPackage ../magic {})
  ];
  # <snip>
}

However, when I try to rebuild my system I get an error:

error: 'functionArgs' requires a function

       at /nix/store/263ppmlj4sk7q3znfkxhlazxh9n998il-source/lib/trivial.nix:443:10:

          442|     then f.__functionArgs or (lib.furnctionArgs (f.__functor f))
          443|     else builtins.functionArgs f;
             |          ^
          444|

I think its because default.nix is not a function, and thats why callPackage fails. I am not sure at this moment how that can be solved.

It would be easier if you’d share the contents of default.nix and magic.nix :wink:

# default.nix
let 
  pkgs = import <nixpkgs> { };
  tclPath = pkgs.tcl;
  tkPath = pkgs.tk;
in
{
  magic = pkgs.callPackage ./magic.nix {  
    tclStorePath = builtins.storePath tclPath;
    tkStorePath = builtins.storePath tkPath;
 };
}
# magic.nix
{ lib
, stdenv
, fetchFromGitHub
, tcl
, git
, cairo
, tcsh
, python3
, tk
, m4
, ncurses
, libGLU
  # undocumented packages required
, freeglut
, mesa
, tclStorePath
, tkStorePath
}:
stdenv.mkDerivation {
  name = "magic";

  buildInputs = [
    tcl
    git
    cairo
    tcsh
    python3
    tk
    m4
    ncurses
    libGLU
    # undocumented packages required
    freeglut
    mesa
  ];

  makeFlags = [];

  # TODO fix hardcoded paths  
  configureFlags = [
    "--with-tcl=${tclStorePath}/lib/"
    "--with-tk=${tkStorePath}/lib/"
  ];

  src = fetchFromGitHub {
    owner = "RTimothyEdwards";
    repo = "magic";
    rev = "8.3.440";
    sha256 = "1c94m6pcbvs3ppvrkbbccwh30lmyssvzm6hylf4jwrbyqll078sg";
  };
}

Maybe an experience user of NixOS will cringe at this, but I tried changing default.nix to a function (just to try if it would work), like so:

{
  pkgs ? import <nixpkgs> { },
  tclPath ? pkgs.tcl,
  tkPath ? pkgs.tk
}:
{
  magic = pkgs.callPackage ./magic.nix {  
    tclStorePath = builtins.storePath tclPath;
    tkStorePath = builtins.storePath tkPath;
 };
}

nix-build works, but now I have just shifted the issue when rebuilding my system, and get this error:

$ sudo nixos-rebuild --flake . switch
building the system configuration...
error: A definition for option `environment.systemPackages."[definition 1-entry 6]"' is not of type `package'. Definition values:
       - In `/nix/store/4fqp3aw12nf5bxvj1ndy2s0yvnxrzqnd-source/modules/system.nix':
           {
             magic = <derivation magic>;
             override = <function, args: {pkgs?, tclPath?, tkPath?}>;
             overrideDerivation = <function>;
           }
(use '--show-trace' to show detailed location information)

Try changing the package definition to this: (notice how you don’t need builtins.storePath, you can simply interpolate the packages in a string, which behind the scenes will use their .outPath attribute)

# magic.nix
{ lib
, stdenv
, fetchFromGitHub
, tcl
, git
, cairo
, tcsh
, python3
, tk
, m4
, ncurses
, libGLU
  # undocumented packages required
, freeglut
, mesa
}:
stdenv.mkDerivation (finalAttrs: {
  pname = "magic";
  version = "8.3.440";

  buildInputs = [
    tcl
    git
    cairo
    tcsh
    python3
    tk
    m4
    ncurses
    libGLU
    # undocumented packages required
    freeglut
    mesa
  ];

  makeFlags = [];

  # TODO fix hardcoded paths  
  configureFlags = [
    "--with-tcl=${tcl}/lib/"
    "--with-tk=${tk}/lib/"
  ];

  src = fetchFromGitHub {
    owner = "RTimothyEdwards";
    repo = "magic";
    rev = finalAttrs.version;
    sha256 = "1c94m6pcbvs3ppvrkbbccwh30lmyssvzm6hylf4jwrbyqll078sg";
  };
})

And then in your home-manager config, you use callPackage to call this function, something like this:

{ pkgs, ... }:
{
  home.packages = [
    (pkgs.callPackage ../magic.nix { })
  ];
}
2 Likes

If you really want a default.nix you can have that too, as a side note.

It’s mostly used by nix.dev because they don’t have a NixOS config to get pkgs from, because nix-build is easier to use for demonstration and because they can show off composing multiple packages this way.

Completely overkill for your use case, but here’s how you’d do it:

# home.nix
{ pkgs, ... }: let
  # Use `import` and pass arguments explicitly when not importing a
  # package definition.
  #
  # Technically `callPackage` works, since it just assigns arguments
  # to an imported function, and `pkgs` is one of the potential
  # assignments, but we're not calling a package, so let's not
  # pretend.
  mypkgs = import ./magic { inherit pkgs; };
in {
  home.packages = [
    # The output of our function is an attribute set whose single
    # attribute is a package. We need to take that package, not the
    # whole set.
    mypkgs.magic
  ];
}
# magic/default.nix
{
  # This sets a default for `pkgs` if unset.
  #
  # We do this so `nix-build` still works (it does not by default pass a
  # `pkgs` argument), but we can also pass in a `pkgs` from our
  # NixOS config that has potentially been modified with overlays.
  #
  # It also avoids re-evaluating the entirety of nixpkgs when we don't
  # need to, which saves a fair bit of memory and some compute
  # time.
  pkgs ? import <nixpkgs>
}: {
  # Ideally you don't leak logic to the callsite like you did previously,
  # not having those variables here makes everything easier.
  magic = pkgs.callPackage ./magic.nix { };
}

With @R-VdP 's version of magic.nix.

The advantage of this is that you can now add additional packages, and you can neatly make them depend on each other as well as use nix-build to test them instead of having to rebuild your system just for that.

It’s more effort than necessary right now, but if you’re planning to write a number of downstream packages it might be worth it.

1 Like

Thanks @R-VdP and @TLATER, you guys are MVPs! :wink:

This thread has been super helpful to me as a NixOS newbie!

I have set up my directory structure line this:

apps--
     | -- app1
     |       |-- default.nix
     |       |-- app1.nix
     |
     | -- app2
     |       |-- default.nix
     |       |-- app2.nix

Is there a way to alter the mypkgs = ... line in home.nix and/or the individual default.nix files such that it can accommodate this structure?

If I write mypkgs = import ./apps{ inherit pkgs; };, I believe that a default.nix file would have to go directly under the apps directory. If write mypkgs_app1 = import ./apps/app1 { inherit pkgs; };, I’ll have to repeat that for each app. Clearly, I’m a novice.

Thanks!

The clean way IMO to do that would be to create an apps/default.nix which you import with { inherit pkgs; }:

# apps/default.nix
{ pkgs }: {
  app1 = pkgs.callPackage ./app1/app1.nix { };
  app2 = pkgs.callPackage ./app2/app2.nix { };
}
# elsewhere
let
  mypkgs = import ./apps { inherit pkgs; };
in
  mypkgs.app1

I’m generally not a fan of parsing directory names into attributes, but from there you could then hide that detail if explicitly callPackageing all individual package directories is too much effort:

# apps/default.nix

# Note: untested, probably completely nonfunctional
{ pkgs }: let
  dirs = pkgs.filterAttrs (_: kind: kind == "directory") (builtins.readDir .);
in
  pkgs.mapAttrs (dir: _: pkgs.callPackage "./${dir}/${dir}.nix" { }) dirs;

This is much less clear than the more manual option though, and things get messy with UTF-8 dir names, directories without default.nix, re-importing the package set itself, overrides, etc. You’ll need to keep things very regular if you do this.

Manual is fine for most cases, even nixpkgs only recently started migrating away from manual callPackages for most packages.

If you use flakes + nix build you can get away from the awkward indirection with the duplicated entrypoints and just have a default.nix for each application too, but let me not push that too much :wink: