Home-manager & spacemacs

Help
1

The usual way of installing spacemacs is

git clone https://github.com/syl20bnr/spacemacs ~/.emacs.d

(And my personal choice is then to switch to the develop branch.)

During the normal course of usage, the contents of ~/.emacs.d will then be mutated by emacs (package installations, caches commands and visited buffers, etc.).

I’m trying to find a way of getting home-manager to take care of installing spacemacs, but using things like fetchGit is incompatible with the expectation that the fetched directory (.emacs.d) be mutable.

Can you suggest a suitable approach?

1 Like
2

I used a link farm to allow spacemacs to clone it’s dependencies (note it was a while ago that I cloned this):

  home.file.".emacs.d" = {
    # don't make the directory read only so that impure melpa can still happen
    # for now
    recursive = true;
    source = pkgs.fetchFromGitHub {
      owner = "syl20bnr";
      repo = "spacemacs";
      rev = "26b8fe0c317915b622825877eb5e5bdae88fb2b2";
      sha256 = "00cfm6caaz85rwlrbs8rm2878wgnph6342i9688w4dji3dgyz3rz";
    };
  };
5 Likes
3

Interesting, initial trials suggest that this is working.

This inspires a few questions:

  • Where in the documenation should I have found that what fetchFromGitHub creates, is not read-only?
  • Where can I read up on what a nix link farm is? (google-fu failing me, as it so often does when it comes to nix: plenty of code which uses pkgs.linkFarm but no explanation or documentation)
  • Initial emacs package installation (based on my existing .spacemacs) blocked a couple of times on things like
    File evil-unimpaired-20200708.180707.el is write-protected; try to save anyway? (y or n) y`
    which never happened before. Here I’m using the latest commit on develop; previously, I re-installed spacemacs from scratch two days ago, and I’m pretty sure it didn’t happen? Could it be nix-related? (I just accepted with y and it seems to have worked, but it’s annoying that the installation process now blocks waiting for human input.)
4

Where in the documenation should I have found that what fetchFromGitHub creates, is not read-only?

It’s still read only. recursive = true; causes home-manager to create a writable directory tree with a symlink for each file in the derivation given to source. This allows melpa to write new files into the .emacs.d directory and files from the derivation are still read-only.

Where can I read up on what a nix link farm is?

nixpkgs/pkgs/build-support/trivial-builders.nix at 6e63209f28ccbc95b6286822347840894cdaef14 · NixOS/nixpkgs · GitHub is probably what you’re looking for. Perhaps that doc-comment should be added to the nixpkgs manual. Or maybe we should include the phrase “link farm” in there somewhere.

Could it be nix-related?

Possibly. I think I ran into this as well, but it only asked once, and I was not bothered enough to figure out why it happened.

Edit: but → and

5

Most of the time I ended up reading the source, although even that is sometimes hard enough to find. Unfortunately Google is of little use these days - most convenient is to search directly on GitHub or locally in the repo.

I can’t really judge if this is good or bad. Reading the source makes me more proficient with the language and the libraries, it introduces me to ideas how to solve certain types of problems and brings my way of thinking closer to the development culture. Nothing is more informative than seeing how exactly something is built. Having to dig it up, or even find pointers to something, on the other hand is tedious at best.

That being said, it sure is sad that the ”official“ documentation is lacking many of these fine details, even though it’s already huge. But this project is a community effort - that’s why “official” is an inappropriate concept - and therefore this forum is just as well part of the documentation. We can simply accrete information we find valuable and try to present it in a discoverable and reproducible way - by nicely linking to commit hashes for source references, or hard-copying relevant pieces, for example.

Here is what I’ve found in
nixpkgs/pkgs/build-support/trivial-builders.nix on the master branch:

 /*
  * Quickly create a set of symlinks to derivations.
  *
  * This creates a simple derivation with symlinks to all inputs.
  *
  * entries is a list of attribute sets like
  * { name = "name" ; path = "/nix/store/..."; }
  *
  * Example:
  *
  * # Symlinks hello and stack paths in store to current $out/hello-test and
  * # $out/foobar.
  * linkFarm "myexample" [ { name = "hello-test"; path = pkgs.hello; } { name = "foobar"; path = pkgs.stack; } ]
  *
  * This creates a derivation with a directory structure like the following:
  *
  * /nix/store/qc5728m4sa344mbks99r3q05mymwm4rw-myexample
  * |-- foobar -> /nix/store/6lzdpxshx78281vy056lbk553ijsdr44-stack-2.1.3.1
  * `-- hello-test -> /nix/store/qy93dp4a3rqyn2mz63fbxjg228hffwyw-hello-2.10
  *
  * See the note on symlinkJoin for the difference between linkFarm and symlinkJoin.
*/
 linkFarm = name: entries: runCommand name { preferLocalBuild = true; allowSubstitutes = false; }
   ''mkdir -p $out
     cd $out
     ${lib.concatMapStrings (x: ''
         mkdir -p "$(dirname ${lib.escapeShellArg x.name})"
         ln -s ${lib.escapeShellArg x.path} ${lib.escapeShellArg x.name}
     '') entries}
   '';

It turns out to be really simple, it just links multiple packages into one, each under its given name. The interesting bit is how it relates to symlinkJoin, which is declared right before in the same file.

6

We can simply accrete information we find valuable and try to present it in a discoverable and reproducible way

Oh, yeah. I used to ask “where did you first look to find this information?”. Then I would put documentation there, under the assumption that they first looked is a place that others would think too look too.

So, @jacg Where did you first look for an explanation of a link farm?

7

Yes, this frequently turns out to be the only resource I can find. But there is a significant cost to reading the source in order to understand something, compared to having a high-level explanation. This becomes especially important when you are on a deep stack of yak-shaving efforts. When I am trying to find a solution to a problem, which itself is an attempt to find a solution to a problem, which itself is … it would be good to get some idea of whether any specific thing might be relevant or not, QUICKLY.

Exploring the exponentially growing tree of yak-shaving components by having to read the source of each item, is, for me at least, an intractable problem. But maybe I’m just too dim.

A search engine.

More often then not, such searches suggest something, more or less relevant, in the Nix (or nixpkgs) manual, on the Nix pills. But I rarely go to those resources myself directly, because they are so vast, that you need a search engine to find what you want in them, anyway. Well, the manuals are a on a single (huge) page, so sometimes (rarely) I try those directly, but the pills, being spread over multiple pages, I almost always acces via a search engine.

Indeed.

I can’t quite put my finger on the reason, but I struggle to find Nix-related answers much more than in other technical areas, but I’m sure that part of it is this:

… the documentation is both huge and yet sparse.

You mention missing fine details, but I also find that the big picutre is often absent.

One might think of the documentation as being at four different levels

  1. Global overview of Nix.
  2. High-level description of sub-topics. Strategies for achieving different kids of goals.
  3. How to do some specific things.
  4. Detailed and comprehensive documentation of individual components.

In my eyes, level 1 is covered adequately.

Most of Nix documentation seems to be on level 3, and if what you’re trying to do happens to be covered, you’re in luck, if not …

Level 2 is where I feel the pain most acutely. Here I usually end up finding something vaguely related on level 3 and frequently find myself trying to fit a square peg into a round hole, thinking that there must be a better and more direct way of doing what I’m trying to do.

Level 4 appers to be mostly non-existent, in relative rather than absolute terms: there may be much that is documented, but there is much more that is not. [Personally, I guess I just have to develop the habit of reaching for ripgrep or Emacs helm in my local clone of the nix repos, rather than web search engines, and just accept that the documentation is the source.]

I see Nix as a crucial tool for taming complexity … unfortunately it’s failing miserably at taming its own complexity.

I would like to contribute to mitigating the situation, but it’s difficult to see the best approach, or even an adequate one.

Bah, I’ve written too much already, so I’ll stop here.

3 Likes
8

I would like to contribute to mitigating the situation, but it’s difficult to see the best approach, or even an adequate one.

I think you could submit changes to the manual, and this discussion could already be helpful to others in your shoes.

I agree with the 1 & 3 are covered statement. I think 4 could be best served by something like rustdoc for nixpkgs, giving function-by-function documentation (and location in the attribute tree (attrsets of attrsets)). 2 is where the manuals have failed me too. I don’t know what to do there.

9

I had the exact same experience in the first few months of what essentially amounted to setting up my personal Linux distro.

Npw that I deal with nix projects regularly, I noticed two things:

  1. nix itself has a fairly narrow interface, and works very reliably. I only once bumped into it for some fringe cosmetic aspect, which even I could have easily patched with enough motivation. Error messages are not good, but fortunately there is work in progress on that. The rest is libraries and NixOS modules, which are in essence sugar and code reuse. If something does not exist that I need - well, it’s a fairly young project still. It’s easy enough to add your own function or module, and even submit a patch. Getting into a discussion on how to solve your local problem helps both yourself and the community.
  2. The real fight is with the software or configuration of interest. Pieces don’t fit together nicely, programs have implicit or undocumented assumptions for build or runtime environments. Often, as with your example, there are conceptual mismatches between the software’s design and how you would usually manage it under nix. Then of course you get maximal friction.

As you said, nix is the tool to tame complexity… to me it feels more like confining plasma in a stellarator. It looks weird and all, but once it’s done correctly, it stays in place.

The mere fact that my own little endeavor worked out even remotely satisfactory is telling how powerful the tool and the existing ecosystem is.

To get back on topic, I think your confusion over mutability of that configuration directory is firmly in the domain of home-manager. As @theotherjimmy wrote, that is which creates the symlinks, and I remember it was pretty clearly documented somewhere obvious, but can’t find it any more. Hm… Looks like it’s back to reading the source again.

10

My current approach is to use git checkout in my nix-daemon script: https://github.com/Mic92/dotfiles/blob/ba7841ebb86860d364c0dc467b6f880fd03b65ac/nixpkgs-config/modules/emacs/default.nix#L31

However the home-manager approach looks also interesting.

11

Would it be possible to use a similar conditional git clone/checkout in home-manager, somehow?

12

Yes. The code you are looking at also is part of my home-manager setup. It is a systemd user service. It could be moved home.activation in theory. Actually now I switched to home.file.".emacs.d" too. However I don’t deep symlink everything because spacemacs contains a lot of files. Instead I only symlink all top-level directories of spacemacs: https://github.com/Mic92/dotfiles/blob/3c7dad7b342570a18e35f6972f0f954cb263bd64/nixpkgs-config/modules/emacs/default.nix#L47 This should be faster at activation time.