It’s not just this PR; #310014 was merged just before and is in the same state. There simply hasn’t been a manual merge from staging to staging-next since May 31, it looks like.
I am with some refactors that do not cause mass rebuild but depend on this PR.
And it is bothersome to rebase it while busy-waiting.
Per curiousity, I have found 40 files (via rg) citing memcached. However it generates a mass rebuild.
I believe this package is too near the root of bootstrap.
On the other hand, Melpa modifications affects … emacs plugins. Nothing else.
My bad. It is probably better to merge your PR into master first.
That being said, you only have to rebase once after the next staging cycle finishes.
It seems the bottle-neck of rebuilding 6k (24k for four platforms) small Emacs lisp packages is compression (probably xz -9). This can be improved if cache.nixos.org switches to a faster compressor.
About the elisp packages, I have noticed that I never cache-hit them.
They are always built from sources in my local machine, and I don’t know if it is because I use emacs-gtk3 instead of merely emacs.
I am not sure why we decide to only provide cache for Emacs packages of one flavour.
The need for cache of different flavours is real. For example, I personally want cache for pkgs.emacs-nox for my servers. Others probably want cache for pkgs.emacs-gtk or pkgs.emacs-pgtk.
Since updating Emacs packages (6k each) goes through staging cycles, it is probably valid to provide cache for more flavours. At least there are precedents: cache is provided for two versions of python packages (8k each) and two versions of perl packages (2k each).
What about selecting some flavours and adding cache for them now?
Technically they should make no difference.
I don’t know any package that acts differently when in different emacs flavors. Ideally both Emacsen flavors can run the same “bytecode”.
This idea is doable. Emacs looks for comp-native-version-dir in native-comp-eln-load-path. comp-native-version-dir is different for different Emacsen. comp-native-version-dirconsists ofversion and Vcomp_abi_hash. However, we can patch Vcomp_abi_hash (Vsystem_configuration and Vsystem_configuration_options) to make sure it is unchanged.
I am not sure if it is a good idea though. Upstream must have a reason to put Vsystem_configuration and Vsystem_configuration_options into consideration.
Well… I don’t know. This whole thread is because building the emacs packages isn’t that cheap. By multiplying the numbers this aspect would get even worse.
I should mention that this month things are slower than usual, as the infra is building two stable releases (including their staging* cycles) instead of the usual one stable.
I think that’s mainly a question for emacs(Packages*) users – what they consider better. Maybe there could be a compromise of selecting just some (very popular?) packages to build and do that without the staging* slow-down and perhaps even for multiple variants (flavors/versions?)
Well, I can’t talk for the whole emacs-nix community, but…
First, I don’t believe it would be a bad idea to let the end user compile the packages locally.
I am living this way since I am using pgtk.
Also I do not believe it will be easy to select the “hottest” Emacs Lisp packages.
Second, some time ago it was suggested something like a detached, low-priority Hydra batch job dedicated to these. It looks doable to me.
P.P.S.: wee, a new Emacs version with security updates arrived!
To provide elisp packages cache for more Emacs flavors, two proposals are as follows.
We only provide cache for a limited number of elisp packages. We can select them by their popularity (both GNU Elpa and MELPA have this info) and build time.
We patch Emacs source code and disentangle elisp packages from Emacs in nix so that one elisp package can be used by all Emacs flavors of the same version. Guix does a similar thing. Their motivation is different though. Here is their patch.
Note that these two proposals can be implemented at the same time.
A “popularity” selection does not sound like a good idea to me. It creates inconsistent behaviour and is another data source that needs to be kept up to date.
If the patch is safe, it should be easy enough to use. You’d simply emacs29-gtk.pkgs.withPackages (_: with emacs29.pkgs; [ magit foo bar ]) instead of taking the emacs packages provided to the lambda.