Apply a patch to a specific file inside a package?

How do I apply modify a file within a package? I’m trying to add a patch that modifies a specific file that is included in a package but I can’t seem to get the patch to actually apply. In my case I’m trying to modify a file named tramp.el inside of emacsPackages.tramp; which is located in the store at /nix/store/h680qia5ilk0c71nj4ylsvf5sjg1djb6-emacs-tramp-

The patch I’m trying to apply

diff --git a/lisp/tramp.el b/lisp/tramp.el
index bd556753..a97d8e37 100644
--- a/lisp/tramp.el
+++ b/lisp/tramp.el
@@ -688,7 +688,7 @@ The `sudo' program appears to insert a `^@' character into the prompt."
 (defcustom tramp-otp-password-prompt-regexp
   (rx bol (* nonl)
       ;; JumpCloud.
-      (group (| "Verification code"))
+      (group (| "Verification code" "Enter PIN"))
       (* nonl) (any "::៖") (* blank))
   "Regexp matching one-time password prompts.
 The regexp should match at end of buffer."

Overlay file

{ outputs, inputs }:
  addPatches = pkg: patches:
    (oldAttrs: { patches = (oldAttrs.patches or [ ]) ++ patches; });
in {
  nh = inputs.nh.overlays.default;
  emacs = inputs.emacs-overlay.overlay;

  # For every flake input, aliases 'pkgs.inputs.${flake}' to
  # 'inputs.${flake}.packages.${pkgs.system}' or
  # 'inputs.${flake}.legacyPackages.${pkgs.system}'
  flake-inputs = final: _: {
    inputs = builtins.mapAttrs (_: flake:
        legacyPackages = ((flake.legacyPackages or { }).${final.system} or { });
        packages = ((flake.packages or { }).${final.system} or { });
      in if legacyPackages != { } then legacyPackages else packages) inputs;

  # This one brings our custom packages from the 'pkgs' directory
  additions = final: prev:
    import ../pkgs { pkgs = final; } // {
      #formats = prev.formats // import ../pkgs/formats { pkgs = final; };
      tmuxPlugins = prev.tmuxPlugins
        // final.callPackage ../pkgs/tmux-plugins { };

  # This one contains whatever you want to overlay
  modifications = final: prev: {
    vivaldi = prev.vivaldi.override {
      proprietaryCodecs = true;
      enableWidevine = true;

    tramp-fix = addPatches prev.tramp [ ./emacs-tramp.patch ];

    papirus-icon-theme = prev.papirus-icon-theme.override { color = "violet"; };

    firefox = prev.firefox.override { cfg.nativeMessagingHosts.fxCast = true; };

    discord = prev.discord.override { withOpenASAR = true; };
1 Like

Looks correct at a glance, what’s the peoblem?

The code looks very overengineered to me, by the way. Unless you have a good reason not to, this kind of stuff (especially for personal configs, like presumably this one) is best applied at the point of use - wherever you set services.emacs.package, for example.

That’s the thing… I’m running nixos-rebuild but the patch isn’t being applied at all. I look at the file inside the nix store and it’s just the same as before (without the changes the patch makes).

The overlay file was something I borrowed from this nix-config since it looked the most sensible at the time when I was first learning about flakes. The patch wasn’t made by me, it was made by the creator of tramp itself. He sent it to me to test if adding the additional text fixes an issue with the program not recognizing PIN input.

Where are you referencing this? Assuming modifications is one of the overlays you apply over your Nixpkgs, this would just add pkgs.tramp-fix.


:exploding_head: Oh goodness I hadn’t realized I was referring to the package as emacsPackages.tramp inside the Home-Manager programs.emacs module instead of pkgs.tramp-fix. I thought the overlay would create override emacsPackages.tramp with the tramp-fix definition.

  programs = {
    emacs = {
      enable = true;
      package = pack;
      extraPackages = epkgs: with epkgs; [ 
           tramp # Originally how I was referring to it
    pyenv.enable = true;

    packages = with pkgs; [ tramp-fix ]; # just added this

I’m recompiling the flake right now, will update once it finishes!

You can change that to:

tramp = addPatches prev.tramp [ ./emacs-tramp.patch ];

to actually change the tramp attribute - that said, all emacs packages are under emacsPackages.*, so that would not override what you end up depending on.

To change the epkgs you’re using, you need something a bit more involved than a top-level nixpkgs overlay.

1 Like

I know I’m answering a different question than what is being asked here, but if you are just trying out patches to packages, I would probably just stick the patched file into a directory further down in the load-path (and most importantly outside the nix store) and mess around with things there, rather than deal with patching tramp.el from emacs.

1 Like

Good news! I got it to finally at least try to apply the patch (thanks @jtojnar !), but the bad news is now I’m getting this error as soon as it tries to apply the patch

tramp> building '/nix/store/slxi80z27xxdai91ihx1g7q2d0af5xj3-emacs-tramp-'
tramp> Running phase: patchPhase
tramp> applying patch /nix/store/maarhpqad13ra6lwqiazp1rnaiijr246-emacs-tramp.diff
tramp> can't find file to patch at input line 5
tramp> Perhaps you used the wrong -p or --strip option?
tramp> The text leading up to this was:
tramp> --------------------------
tramp> |diff --git a/lisp/tramp.el b/lisp/tramp.el
tramp> |index bd556753..a97d8e37 100644
tramp> |--- a/lisp/tramp.el
tramp> |+++ b/lisp/tramp.el
tramp> --------------------------
tramp> File to patch:
tramp> Skip this patch? [y]
tramp> Skipping patch.
tramp> 1 out of 1 hunk ignored

Unsure what’s going on here because I even tried to add the path that the files appears in the nix store but even that didn’t fix the issue.

Is it possible to do that with Emacs packages? IIRC, Tramp specifically is built along with Emacs and thus is always in the nix store, so I can’t really override it with a load-path either. I might be mistaken though.

Ah ok! That makes sense.

So I include L27 and then append the tramps override line with something like this?
tramp = addPatches emacsPkgs.tramp [ ./emacs-tramp.patch ]
I may end up keeping it at the top level for now at least until I can figure out why the patch file isn’t working

Yeah, though you might need to change down the line (or just add pkgs.tramp-fix to extraPackages rather than to - presumably - home.packages).

As a hack, I usually override the patchPhase to add some ls and other things to inspect and use exit 1 to force a failure so I can check what’s actually in the build path. E.g.:

tramp-fix = prev.tramp.overrideAttrs (old: {
  patchPhase = ''
    cat lisp/tramp.el
    exit 1

There are better ways with nix-shell and nix develop, too, but somewhere along the way the new CLI made it too cumbersome to use, and IME the build env and whatever nix develop creates rarely match… Probably should learn how to do that properly sometime, but the above is a simple escape hatch.

1 Like

This is really odd… there’s nothing in the build path at all except for env-vars (which I also did ls env-vars to see what was in it

┃ error: builder for '/nix/store/dsmr3d0sb5273ql4sfpj4gx8lb1x4wp6-emacs-tramp-' failed with exit code 1;
┃        last 8 log lines:
┃        > Running phase: patchPhase
┃        > total 16
┃        > drwx------ 2 nixbld nixbld 4096 Jan 29 22:48 .
┃        > drwxr-x--- 9 nobody nixbld 4096 Jan 29 22:48 ..
┃        > -rw-r--r-- 1 nixbld nixbld 7526 Jan 29 22:48 env-vars
┃        > env-vars
┃        > bin  build  dev  etc  nix  proc  tmp
┃        > cat: lisp/tramp.el: No such file or directory
┃        For full logs, run 'nix log /nix/store/dsmr3d0sb5273ql4sfpj4gx8lb1x4wp6-emacs-tramp-'.

Hm, ok. What are you applying that overlay to? I have to admit I’m mildly surprised there even is a prev.tramp, given that looks like an overlay intended to be applied to the top-level nixpkgs.

Currently I’m trying to over over emacsPackages.tramp v2.6.2.0 that’s from emacs-overlay (which also overlays over the nixpkgs emacsPackages.tramp) specifically since it has the lines that the patch modifies while the current one that’s on nixpkgs doesn’t. Maybe that might be the issue?

emacs-overlay has an example of an override on top of itself here: GitHub - nix-community/emacs-overlay: Bleeding edge emacs overlay [maintainer=@adisbladis]

I’ve never used emacs-overlay, personally, but their package structure probably differs quite a bit from the one in nixpkgs, so I’d stick to their documentation. Either way, your changes would probably not carry over to an emacs from emacs-overlay if you make them with a normal nixpkgs overlay.

1 Like

:cold_sweat: I’m almost completely out of ideas now. I tried using emacs-overlay’s docs for setting up the override and it just doesn’t work at all. It’s not picking up on the override once again. And another rather big issue just reared its ugly head: because of how emacs-overlay sets up Emacs… Doom Emacs completely breaks (because all of the files that Doom Emacs accesses gets set to read-only).

  programs = {
    emacs = {
      enable = true;
      package = (pkgs.emacsWithPackagesFromUsePackage {
           #config = "${config.home.sessionVariables.EMDOTDIR}/init.el";
           config = ./init.el;
           #config = "${config.home.sessionVariables.FLAKE}/home/novaviper/dots/doom/config.el";
           package = pack;
           override = final: prev: {
             tramp = prev.elpaPackages.tramp.overrideAttrs (oldAttrs: {
               # patches = [ ./emacs-tramp.patch ];
               patchPhase = ''
                 cat lisp/tramp.el
                 exit 1
           #extraEmacsPackages = epkgs: [ epkgs.tramp ];
      extraPackages = epkgs: with epkgs; [ tramp pdf-tools ];
    pyenv.enable = true;

just stick the patched file into a directory further down in the load-path (and most importantly outside the nix store) and mess around with things there, rather than deal with patching tramp.el from emacs.

Is it possible to do that with Emacs packages? IIRC, Tramp specifically is built along with Emacs and thus is always in the nix store, so I can’t really override it with a load-path either. I might be mistaken though.

Sure you can - it’s emacs, so you don’t technically need a file to do this from:

  1. open tramp.el
  2. copy the contents of the function you are patching into a scratch buffer
  3. make the changes you need
  4. run eval-defun on the patched function
  5. test that tramp does what you need
  6. repeat steps 3 to 5 until everything is good
  7. stick your new and modified function somewhere in your emacs config and guard it with (with-eval-after-load 'tramp) so that your function overwrites the one shipped with tramp.

I gave this a try but for some reason, the variable that’s in tramp.el keeps overriding the one I made
The code in my file

(with-eval-after-load 'tramp
(defcustom tramp-otp-password-prompt-regexp
  (rx bol (* nonl)
      ;; JumpCloud.
      (group (| "Verification code" "Enter PIN"))
      (* nonl) (any "::៖") (* blank))
  "Regexp matching one-time password prompts.
The regexp should match at end of buffer."
  :version "29.2"
  :type 'regexp))

What the describe-variable buffer says what the variable is

It took me a bit but I eventually figured it out! I ended up setting the lisp code like this and that worked! Thank you guys for helping!

(with-eval-after-load 'tramp
(customize-set-variable 'tramp-otp-password-prompt-regexp
  (rx bol (* nonl)
      ;; JumpCloud.
      (group (| "Verification code" "Enter PIN"))
      (* nonl) (any "::៖") (* blank))))

Correct, you just need to set the variable to the correct value. I hadn’t looked at what the patch you were trying to apply actually did, but you definitely shouldn’t defcustom again. This is really just setting the value of a variable so forget all about patching and local files.

If you are on emacs 29, just use setopt instead of customize-set-variable.

1 Like

Uh… I just got a new patch from the main developer of Tramps to test out but this time it’s new functions being declared. How do I override the default function with the new changes?

diff --git a/lisp/tramp-sh.el b/lisp/tramp-sh.el
index 44c0bdc7..3e6fb384 100644
--- a/lisp/tramp-sh.el
+++ b/lisp/tramp-sh.el
@@ -544,6 +544,7 @@ shell from reading its init file."
     (tramp-terminal-prompt-regexp tramp-action-terminal)
     (tramp-antispoof-regexp tramp-action-confirm-message)
     (tramp-security-key-confirm-regexp tramp-action-show-and-confirm-message)
+    (tramp-security-key-pin-regexp tramp-action-otp-password)
     (tramp-process-alive-regexp tramp-action-process-alive))
   "List of pattern/action pairs.
 Whenever a pattern matches, the corresponding action is performed.
@@ -563,6 +564,7 @@ corresponding PATTERN matches, the ACTION function is called.")
     (tramp-wrong-passwd-regexp tramp-action-permission-denied)
     (tramp-copy-failed-regexp tramp-action-permission-denied)
     (tramp-security-key-confirm-regexp tramp-action-show-and-confirm-message)
+    (tramp-security-key-pin-regexp tramp-action-otp-password)
     (tramp-process-alive-regexp tramp-action-out-of-band))
   "List of pattern/action pairs.
 This list is used for copying/renaming with out-of-band methods.
diff --git a/lisp/tramp.el b/lisp/tramp.el
index 61f51272..f3da56e7 100644
--- a/lisp/tramp.el
+++ b/lisp/tramp.el
@@ -788,6 +788,13 @@ The regexp should match at end of buffer."
   :version "28.1"
   :type 'regexp)

+(defcustom tramp-security-key-pin-regexp
+  (rx bol (* "\r") (group "Enter PIN for " (* nonl)) (* (any "\r\n")))
+  "Regular expression matching security key PIN prompt.
+The regexp should match at end of buffer."
+  :version "29.3"
+  :type 'regexp)
 (defcustom tramp-operation-not-permitted-regexp
   (rx (| (: "preserving times" (* nonl)) "set mode") ":" (* blank)
       "Operation not permitted")
@@ -5589,7 +5596,7 @@ of."
       (goto-char (point-min))
       (tramp-check-for-regexp proc tramp-process-action-regexp)
-      (setq prompt (concat (match-string 1) " "))
+      (setq prompt (concat (string-trim (match-string 1)) " "))
       (tramp-message vec 3 "Sending %s" (match-string 1))
       ;; We don't call `tramp-send-string' in order to hide the
       ;; password from the debug buffer and the traces.
@@ -5665,14 +5672,17 @@ Wait, until the connection buffer changes."
       (ignore set-message-function clear-message-function)
       (tramp-message vec 6 "\n%s" (buffer-string))
       (tramp-check-for-regexp proc tramp-process-action-regexp)
-      (with-temp-message
-	  (replace-regexp-in-string (rx (any "\r\n")) "" (match-string 0))
+      (with-temp-message (concat (string-trim (match-string 0)) " ")
 	;; Hide message in buffer.
 	(narrow-to-region (point-max) (point-max))
 	;; Wait for new output.
 	(while (not (tramp-compat-ignore-error file-error
-		       proc 0.1 tramp-security-key-confirmed-regexp)))
+		       proc 0.1
+		       (tramp-compat-rx
+			(| (regexp tramp-security-key-confirmed-regexp)
+			   (regexp tramp-security-key-pin-regexp)
+			   (regexp tramp-security-key-timeout-regexp))))))
 	  (when (tramp-check-for-regexp proc tramp-security-key-timeout-regexp)
 	    (throw 'tramp-action 'timeout))
 	  (redisplay 'force)))
@@ -6726,12 +6736,13 @@ Consults the auth-source package."
 		   (tramp-get-connection-property key "login-as")))
 	 (host (tramp-file-name-host-port vec))
-	  (or prompt
-	      (with-current-buffer (process-buffer proc)
-		(tramp-check-for-regexp proc tramp-password-prompt-regexp)
-		(if (string-match-p "passphrase" (match-string 1))
-		    (match-string 0)
-		  (format "%s for %s " (capitalize (match-string 1)) key)))))
+	  (string-trim-left
+	   (or prompt
+	       (with-current-buffer (process-buffer proc)
+		 (tramp-check-for-regexp proc tramp-password-prompt-regexp)
+		 (if (string-match-p "passphrase" (match-string 1))
+		     (match-string 0)
+		   (format "%s for %s " (capitalize (match-string 1)) key))))))
 	 (auth-source-creation-prompts `((secret . ,pw-prompt)))
 	 ;; Use connection-local value.
 	 (auth-sources (buffer-local-value 'auth-sources (process-buffer proc)))

Hey I actually found out it’s not because of emacs-overlay, but because tramp is being build with elpaBuild, which apparently does not expose any of the contents of the package to the system (hence not allowing me to apply the patch). This explains why the built package wasn’t showing anything on the file system.