Nix does not link files properly for dependencies

adithyaov · September 16, 2020, 1:49am

Hi, I’m a novice nix user. I would highly appreciate it if someone
could help me with my problem.
I have 2 packages, myEmacs, and myEmacsFonts
The expressions are as follows:

myEmacs:

with (import <nixpkgs> {});
stdenv.mkDerivation {
  name = "myEmacs";
  srcs = [ ./src/package-init.el ./src/init.el ./src/custom.el ] ;
  builder = "${bash}/bin/bash";
  system = builtins.currentSystem;
  args = [ ./builder.sh ];
  inherit coreutils;
  buildInputs =
    [ emacs26 ] ++
    [ myEmacsFonts ] ++
    (with emacs26Packages;
       [ ahk-mode
         column-enforce-mode
         eshell-git-prompt
         csharp-mode
         highlight-indent-guides
         solarized-theme
         impatient-mode
         ivy
         swiper
         counsel
         magit
         projectile
         counsel-projectile
         multiple-cursors
         avy
         hydra
         rust-mode
         highlight-function-calls
         org-tree-slide
         org
         expand-region
         ox-reveal
         s
         dash
         forge
         doom-themes
         ivy-posframe
         doom-modeline
	       nix-mode
       ]);
  pathsToLink = [ "/share" ];
}

myEmacsFonts:

with (import <nixpkgs> {});
stdenv.mkDerivation {
  name = "myEmacsIcons";
  src = builtins.fetchGit {
    url = "https://github.com/domtronn/all-the-icons.el.git";
    rev = "d363bb3e73909be013fcf35e1458bb654ec5bbaa";
  };
  builder = "${bash}/bin/bash";
  system = builtins.currentSystem;
  args = [ ./builder.sh ];
  inherit coreutils;
  pathsToLink = [ "/share" ];
}

When I just install myEmacs via nix-env -iA nixpkgs.myEmacs, I
dont see a ~/.nix-profile/share/fonts.

But when I install myEmacsFonts via nix-env -iA nixpkgs.myEmacsFonts,
I see a ~/.nix-profile/share/fonts.

Thanks in advance!

adithyaov · September 16, 2020, 1:58am

The builders are rather trivial,

For myEmacs it is,

export PATH="$coreutils/bin"
siteLisp="$out/share/emacs/site-lisp"
content=""
mkdir -p $siteLisp
for i in $srcs
do
  value=`cat $i`
  content="$content\n$value"
done    
echo -e "$content" >> "$siteLisp/default.el"

and for myEmacsFonts it is,

export PATH="$coreutils/bin"
mkdir -p $out/share/fonts
cp $src/fonts/*.ttf $out/share/fonts/

Ninlives · September 16, 2020, 6:58am

myEmacs does not produce “$out/share/fonts” thus you won’t see it in the profile.

I think you do not want to set builder and args, because buildInputs and pathsToLink will become useless. Actually those arguments are used by the default-builder defined by stdenv, which is override by args here. (I believe you are confused by the difference between the builtin derivation function and the stdenv.mkDerivation)

You may wish to use runCommand to build your myEmacs package:

runCommand "myEmacs" { srcs = [ ./src/package-init.el ./src/init.el ./src/custom.el ] ; } ''
    siteLisp="$out/share/emacs/site-lisp"
    content=""
    mkdir -p $siteLisp
    for i in $srcs
    do
      value=`cat $i`
      content="$content\n$value"
    done    
    echo -e "$content" >> "$siteLisp/default.el"
''

then use buildEnv to build the actual package:

buildEnv {
    name = "emacsEnv";
    buildInputs = [ emacs 26 myEmacs myEmcasFonts ]; # ++ those emacs packages
}

adithyaov · September 16, 2020, 9:05am

Hi, Thank you for your response!

It seems I’ve understood a lot of things incorrectly. I have a few questions though.

Consider a package a that depends on b.
My assumption is that the state of the machine after running,

nix-env -i a

is equivalent to the state of the machine after running,

nix-env -i b
nix-env -i a

Apparently, this isn’t correct. Maybe I need to go through the nix-manual again but if it is possible can briefly explain how I can understand this better?

I will go through the documentation once more regarding derivation and mkDerivation.

I could possibly use runCommand for both, myEmacs and myEmacsFonts. Is that the proper way to go?
I believe runCommand is a wrapper over (simplified) stdenv.mkDerivation. So, why not use stdenv.mkDerivation itself?

From what I’ve read, I believe stdenv.mkDerivation is a wrapper over derivation. Can you briefly point out what you think my confusion is about?

adithyaov · September 16, 2020, 9:06am

It might be worth mentioning that both myEmacs and myEmacsFonts are exposed via overlays.

NobbZ · September 16, 2020, 9:07am

No!

If you “install” a, then only as output will be linked into your profile. That way, another package c, can have a dependency on b in a different version, without any problems.

adithyaov · September 16, 2020, 9:13am

I see, that makes a lot of sense!

I think I kind of get the idea. I probably have to read up a little more. Thank you for the quick response.

NobbZ · September 16, 2020, 9:16am

And as a side-note, unless you really have no alternatives, you shouldn’t rely on nix-env, but use a more declarative approach of managing your user environment.

adithyaov · September 16, 2020, 9:23am

The only other alternative I know is nix-instantiate + nix-store --realise which is basically nix-env in a way I guess.

Could you please let me know other alternatives? And workflow in general if possible?

If the answers to those questions are available elsewhere please let me know. By any chance are you referring to the method in the nix-packages manual which covers the declarative way to install & configure packages?

NobbZ · September 16, 2020, 9:33am

I’m basically speaking about either using /etc/nixos/configuration.nix for system configuration or home-manager for user configuration.

nix-env just doesn’t work as one would expect and it is hard to stop it from updating things that are required explicitely in an old version.

There are two common scenarios I have observed with nix-env and updates:

a depends on b, it is known that there was an update on b. nix-env won’t update a unless you enforce it, as as version hasn’t changed. And as a isn’t updated, b isn’t as well, due to the way how nix works.
There are 2 (or more versions) available of a tool in different attributes. You use nix-env -iA tool_1, as you know tool_2 has a incompatibility with what you need it for. The next time you run nix-env to update all packages, it will recognize that there is another derivation with the same name-attribute as tool_1 but a higher version. So it will update to the new version.

Declarative package management only ever considers the current channel state as input and the specified attributes. They are re-evaluated and rebuild as necessary on every “switch”.

Neither of the described problems can arise with declarative packagemanagement either using the configuration.nix or home-manager.

adithyaov · September 16, 2020, 9:41am

I only use the nix package manager but the idea is to move to nixos in the future.home-manager is an option though but it looks like I have to put in some effort to understand how it works. Since I’m only starting out with nix I choose not to have too much on my plate but once I’m comfortable with it, I’ll definitely move towards home-manager, or at least I’d like to move towards a configuration that does something similar.
Again, thanks for replying, Have a good day!

adithyaov · September 16, 2020, 10:42am

This is probably a typo, rewriting it for newbies like myself.
buildInputs should be paths instead.

buildEnv {
    name = "emacsEnv";
    buildInputs = [ emacs 26 myEmacs myEmcasFonts ]; # ++ those emacs packages
}

should be,

buildEnv {
    name = "emacsEnv";
    paths = [ emacs26 myEmacs myEmcasFonts ]; # ++ those emacs packages
}