Copying Custom EDID

ross · August 11, 2023, 6:08am

Hello,

I’m trying to get a custom EDID for my monitor copied over to the /lib/firmware/ folder (under an EDID subdirectory). I am then using this EDID as part of my kernel params.

I have this in my nix.conf:

  hardware.firmware = [
    (pkgs.runCommandNoCC "firmware-custom-edid" {} ''
      mkdir -p $out/lib/firmware/edid/
      cp "${./firmware/QX2710Patched.bin}" $out/lib/firmware/edid/QX2710Patched.bin
    '')
  ];

  boot.kernelParams = ["quiet" "drm_kms_helper.edid_firmware=DP-3:edid/QX2710Patched.bin"];

This builds fine but when I check the result\firmware directory there is no edid directory. On reboot the EDID file can’t be loaded:

[    0.033829] Kernel command line: initrd=\efi\nixos\iz83zwfddldfkygy992wvyp0v5qp0b1z-initrd-linux-6.1.43-initrd.efi init=/nix/store/a8c33i6l67251xp11i6b23q366rzdqyw-nixos-system-nixos-23.11pre511168.5a8e9243812b/init quiet drm.edid_firmware=DP-3:edid/QX2710Patched.bin splash loglevel=4
[    4.327066] platform DP-3: Direct firmware load for edid/QX2710Patched.bin failed with error -2
[    4.327077] [drm:edid_load [drm]] *ERROR* Requesting EDID firmware "edid/QX2710Patched.bin" failed (err=-2)

Does anyone have any idea on what I’m doing wrong?

Thanks!
Ross

ross · August 11, 2023, 6:40am

UPDATE: I can find the file in /run/current-system/firmware/edid/ but it has a .bin.xz extension.

[ross@nixos:/run/current-system/firmware/edid]$ cd /run/current-system/firmware/edid

[ross@nixos:/run/current-system/firmware/edid]$ ls -la
total 12
dr-xr-xr-x 2 root root 4096 Jan  1  1970 .
dr-xr-xr-x 3 root root 4096 Jan  1  1970 ..
-r--r--r-- 1 root root  272 Jan  1  1970 QX2710Patched.bin.xz

I updated the kernel params with the correct name but still it will not load.

[    0.032613] Kernel command line: initrd=\efi\nixos\arfny9j2qcblwfizjqz5y7g2k5y5xdx5-initrd-linux-6.1.43-initrd.efi init=/nix/store/flanlfzxvpvrpp5s7b417sd4jxmpyssx-nixos-system-nixos-23.11pre511168.5a8e9243812b/init quiet drm.edid_firmware=edid/QX2710Patched.bin.xz splash loglevel=4
[    4.300052] platform DP-3: Direct firmware load for edid/QX2710Patched.bin.xz failed with error -2
[    4.300062] [drm:edid_load [drm]] *ERROR* Requesting EDID firmware "edid/QX2710Patched.bin.xz" failed (err=-2)

cmm · August 11, 2023, 6:51am

maybe the extension does not lie and the firmware blob really is gzipped, in which case you’d probably want to gunzip it when copying? EDIT xz’d not gzipped, argh

ross · August 11, 2023, 7:02am

Thanks

I can confirm it is compressed:

[ross@nixos:~]$ xz -d -v QX2710Patched.bin.xz
QX2710Patched.bin.xz (1/1)
  100 %               272 B / 256 B = 1.062

What I don’t understand is that the original file is not compressed. It seems to me that the firmware is being compressed as part of pkgs.runCommandNoCc?

building '/nix/store/11ya5vxmqgm7bddd7nz1ff1gvdyknr3v-firmware-custom-edid.drv'...
QX2710Patched.bin.xz (1/1)

ross · August 11, 2023, 7:18am

It seems to me that every file in the firmware folder is xz’d. Is there a way to prevent this for one file?

ross · August 11, 2023, 12:37pm

I’ve found where the compression takes place but I don’t know how to stop it.

github.com

NixOS/nixpkgs/blob/650596759b8b38399a0c4d5e366847d190360e55/nixos/modules/services/hardware/udev.nix#L169


      
              done
          
          
    echo "Generating hwdb database..."
              # hwdb --update doesn't return error code even on errors!
              res="$(${pkgs.buildPackages.systemd}/bin/systemd-hwdb --root=$(pwd) update 2>&1)"
              echo "$res"
              [ -z "$(echo "$res" | egrep '^Error')" ]
              mv etc/udev/hwdb.bin $out
            '';
          
          
compressFirmware = firmware: if (config.boot.kernelPackages.kernelAtLeast "5.3" && (firmware.compressFirmware or true)) then
            pkgs.compressFirmwareXz firmware
          else
            id firmware;
          
          
# Udev has a 512-character limit for ENV{PATH}, so create a symlink
          # tree to work around this.
          udevPath = pkgs.buildEnv {
            name = "udev-path";
            paths = cfg.path;
            pathsToLink = [ "/bin" "/sbin" ];

How do I override firmware.compressFirmware from the above?

ross · August 11, 2023, 12:45pm

This commit adds the compressFirmware option but I’m pretty new to this. Is compressFirmware a variable I can set?

This commit adds compression originally:

github.com/NixOS/nixpkgs

nixos/udev: compress all firmware if supported

committed 02:36PM - 13 May 22 UTC

alyssais

+30 -3

This should be a significant disk space saving for most NixOS installations. Th…is method is a bit more complicated than doing it in the postInstall for the firmware derivations, but this way it's automatic, so each firmware package doesn't have to separately implement its compression. Currently, only xz compression is supported, but it's likely that future versions of Linux will additionally support zstd, so I've written the code in such a way that it would be very easy to implement zstd compression for those kernels when they arrive, falling back to xz for older (current) kernels. I chose the highest possible level of compression (xz -9) because even at this level, decompression time is negligible. Here's how long it took to decompress every firmware file my laptop uses: i915/kbl_dmc_ver1_04.bin 2ms regulatory.db 4ms regulatory.db.p7s 3ms iwlwifi-7265D-29.ucode 62ms 9d71-GOOGLE-EVEMAX-0-tplg.bin 22ms intel/dsp_fw_kbl.bin 65ms dsp_lib_dsm_core_spt_release.bin 6ms intel/ibt-hw-37.8.10-fw-22.50.19.14.f.bseq 7ms And since booting NixOS is a parallel process, it's unlikely (but difficult to measure) that the time to user interaction was held up at all by most of these. Fixes (partially?) #148197

iFreilicht · August 11, 2023, 2:43pm

Awesome progress man, you’ve gotten so close!

The important bit is a little further down the file you linked:

    hardware.firmware = mkOption {
      type = types.listOf types.package;
      default = [];
      description = "[...]";
      apply = list: pkgs.buildEnv {
        name = "firmware";
        paths = map compressFirmware list;
        pathsToLink = [ "/lib/firmware" ];
        ignoreCollisions = true;
      };
};

This is the definition of the option you’re setting in nix.conf. The option is applied by calling pkgs.buildEnv with the list you set the option to (that’s the apply = list: pkgs.buildEnv { part).

In there, you’ll notice paths = map compressFirmware list;. This is what calls the compressFirmware option on your firmware file, which you found to be defined as:

  compressFirmware = firmware: if (config.boot.kernelPackages.kernelAtLeast "5.3" && (firmware.compressFirmware or true)) then
    pkgs.compressFirmwareXz firmware
  else
    id firmware;

The firmware argument in this call is exactly the result of the call in your nix.conf, which is a derivation:

nix-repl> p = pkgs.runCommandNoCC "firmware-custom-edid" {} ''
            mkdir -p $out/lib/firmware/edid/
            cp "${./firmware/QX2710Patched.bin}" $out/lib/firmware/edid/QX2710Patched.bin
          ''
nix-repl> p
«derivation /nix/store/l8ccac39dll1qpfxyqihqwpb91bv7fcv-firmware-custom-edid.drv»

This might be a bit confusing, because the paths argument of buildEnv can also take strings, but when derivations are treated as strings in Nix, they are automatically converted to their output path:

nix-repl> "OutPath: ${p}"
"OutPath: /nix/store/fp0q6cw3vcmp649pd5jf25y3nnb455j2-firmware-custom-edid"

nix-repl> builtins.toString(p)
"/nix/store/fp0q6cw3vcmp649pd5jf25y3nnb455j2-firmware-custom-edid"

Now, a derivation inside the Nix language is just an attribute set, so you could add the compressFirmware attribute like so:

nix-rep> p // { compressFirmware = false; }

But looking at the definition of the arguments of runCommandNoCC, there’s a better option:

pkgs.runCommandNoCC "firmware-custom-edid" { compressFirmware = false; } ''
  mkdir -p $out/lib/firmware/edid/
  cp "${./firmware/QX2710Patched.bin}" $out/lib/firmware/edid/QX2710Patched.bin
''

This adds the compressFirmware attribute to the derivation, and while this is mostly to add environment variables for the script to read, the attribute can also be read directly from the derivation within the Nix language.

ross · August 13, 2023, 4:44am

Thank you so much - that’s really helpful and it answers so many questions I had
I am working my way through the language spec to really get my head around this.

One last question on the above: Why does the firmware appear in the /run/current-system/firmware directory but not in the result/firmware directory?

Also I have realised that this isn’t actually what I need - I believe I actually need to copy the firmware into initrd’s /lib/firmware using something like this:

  boot.initrd.extraFiles =  
  { "/lib/firmware/edid/qnix.bin" = 
   { 
     source = "${out}/lib/firmware/edid/qnix.bin";
   }; 
  };

However although this compiles it does not work. The extra files are copied in stage1.nix where it appears that the source file should be symlinked into the target directory.

However I don’t get any logging to show me why this isn’t working. I have tried using lsinitrd to list the files in the generated initrd and my file is not symlinked (although other firmware files are). Am I missing something simple?

Thanks again for helping me

    boot.initrd.extraFiles = mkOption {
      default = { };
      type = types.attrsOf
        (types.submodule {
          options = {
            source = mkOption {
              type = types.package;
              description = lib.mdDoc "The object to make available inside the initrd.";
            };
          };
        });
      description = lib.mdDoc ''
        Extra files to link and copy in to the initrd.
      '';
    };

++ (lib.mapAttrsToList
        (symlink: options:
          {
            inherit symlink;
            object = options.source;
          }
        )
        config.boot.initrd.extraFiles);

iFreilicht · August 14, 2023, 12:55pm

nixos-rebuild doesn’t create a result directory by default. How are you invoking it? Or did you run nix build manually? If so, how did you invoke that?

In my case, the paths are exactly identical.

$ sudo nixos-rebuild switch --flake path:$HOME/.dotfiles
$ readlink /run/current-system
/nix/store/8ba2c0phwcgd3pzi8hdi94r1k8handz1-nixos-system-junction-23.11.20230812.f045184
$ nix build path:$HOME/.dotfiles#nixosConfigurations.junction.config.system.build.toplevel
$ readlink result
/nix/store/8ba2c0phwcgd3pzi8hdi94r1k8handz1-nixos-system-junction-23.11.20230812.f045184

The .config.system.toplevel attribute is what nixos-rebuild builds by default.

About the second question:

What is ${out} referring to here? It could be that this option never checks whether the path actually exists and just silently ignores non-existing files? I would consider that a bug, though.

ross · August 15, 2023, 6:56am

Awesome thanks I will look into the symlink issue further and report back.

I am using nixos-rebuild switch but interestingly my symlinks point to different places. I did switch to unstable a while back and this looks to be the difference. I’m not sure why though

[ross@nixos:/etc/nixos]$ readlink /run/current-system
/nix/store/10mql9m4icxilb59ybliyawv7kcay0pi-nixos-system-nixos-23.11pre511168.5a8e9243812b

[ross@nixos:/etc/nixos]$ readlink result
/nix/store/mbfvxskrn4ib60rhg88574hf7dd12i4q-nixos-system-nixos-23.05.2162.6da4bc6cb07

ross · August 15, 2023, 7:03am

Seems like it I am seeing this bug with regards to the result folder: nixos-rebuild pollutes current directory with "result" symlink · Issue #12665 · NixOS/nixpkgs · GitHub.

I’ve cleaned it up now and a nixos-rebuild did not re-create it. That’s helpful because I was looking at the ‘result’ folder for a while.

iFreilicht · August 15, 2023, 6:44pm

Glad to hear it! Do the contents of run/current-system match what you expect now?

snork · September 2, 2023, 6:51pm

Hey @ross, I was wondering if you were ever able to figure this out? The previous comments on the subject were helpful but I’m ultimately still unable to properly load the firmware.