Logic behind Kernel modules added to conf by nixos-generate-config

blue · July 1, 2023, 4:11pm

Hi everybody,

I’m trying to understand logic for Kernel modules configuration generated by nixos-generate-config

On my laptop I got:

boot.initrd.availableKernelModules = [ "xhci_pci" "thunderbolt" "nvme" "usb_storage" "sd_mod" "sdhci_pci" ];
boot.initrd.kernelModules = [ ];
boot.kernelModules = [ "kvm-intel" ];
boot.extraModulePackages = [ ];

and virtual machines with two different configurations:

boot.initrd.availableKernelModules = [ "ata_piix" "uhci_hcd" "virtio_pci" "sr_mod" "virtio_blk" ];
boot.initrd.kernelModules = [ ];
boot.kernelModules = [ ];
boot.extraModulePackages = [ ];

boot.initrd.availableKernelModules = [ "ata_piix" "uhci_hcd" "virtio_pci" "virtio_scsi" "sd_mod" "sr_mod" ];
boot.initrd.kernelModules = [ ];
boot.kernelModules = [ ];
boot.extraModulePackages = [ ];
# + imported qemu-guest.nix
boot.initrd.availableKernelModules = [ "virtio_net" "virtio_pci" "virtio_mmio" "virtio_blk" "virtio_scsi" "9p" "9pnet_virtio" ];
boot.initrd.kernelModules = [ "virtio_balloon" "virtio_console" "virtio_rng" ];

From that I can see that initramfs gets “can boot from” modules like storage, PCI and USB / Thunderbolt.

What doesn’t make sense to me:

VM RAM balloning, VM console and VM random number generatator is forced to load instead of being just “available”.
kvm-intel is forced to load evan there is no tell that I want to run QEMU/libvirt stack to run VMs. And even if I was I’m pretty sure that QEMU daemon would modprobe it.

Can somebody please explain?

Thank you.

jan · July 10, 2023, 1:07pm

I’d be also interested understand the logic and hardware-configuration.nix is not supposed to be changed but maybe some tweaks make sense.

On my laptop I got:

  boot.initrd.availableKernelModules = [ "xhci_pci" "thunderbolt" "nvme" "usbhid" "usb_storage" "uas" "sd_mod" "sdhci_pci" ];
  boot.initrd.kernelModules = [ ];
  boot.kernelModules = [ "kvm-intel" ];
  boot.extraModulePackages = [ ];

And I’m pretty sure I can remove thunderbolt since I don’t boot from external disk and I probably could remove kvm-intel since why should I keep this loaded if I’m not running virt-manager…

peterhoeg · July 10, 2023, 3:06pm

The installer just injects as many modules as it can find hardware for to be sure that the OS actually boots after the installation is complete. Better safe than sorry.

hardware-configuration.nix is not supposed to be changed

No, you can change it all you want (or delete it you are so inclined). As long as the modules necessary to boot are loaded, it doesn’t matter where they come from.

jan · July 11, 2023, 8:28am

I know I can do that technically since all imported files are merged into a single large derivation. My understanding is that the best practice is to have a unique file for each unique device and have it generated by installer so in theory a new installer in future might do something else / better but people don’t need to worry about that since they don’t edit the file.

That said I want to change it - maybe just for learning reasons to remove “kvm-intel” and “thunderbolt” and see what happens. I also want to create a slightly bulkier list of boot.initrd.availableKernelModules so assuming that disks have same labels then I can reuse all configurations including hardware-configuration across multiple laptops. The only thing that will be different will be hostname and networking.hostId.

peterhoeg · July 11, 2023, 8:54am

My understanding is that the best practice is to have a unique file for each unique device

You can, but you don’t have to. You can just pass arguments to the builder. Personally (and at $JOB) for static hosts, I have a file per device, but this file just sets a hardware profile, a role and a hostname and then everthing gets pulled in from that - there are no hardware specific entries ever in the device-specific file.

and have it generated by installer so in theory a new installer in future might do something else / better but people don’t need to worry about that since they don’t edit the file.

I have used the installer for 2 things:

the very first fresh install, and
for recovery

I cannot imagine running the installer just to see if it does something differently. The config generated by the installer really is just to get you started.

That said I want to change it - maybe just for learning reasons to remove “kvm-intel” and “thunderbolt” and see what happens.

I can answer that for you: not much if you’re not using it…

I also want to create a slightly bulkier list of boot.initrd.availableKernelModules so assuming that disks have same labels then I can reuse all configurations including hardware-configuration across multiple laptops.

Don’t do that. Create your own module with an option per model and then set that, because there are far more things that will differ than just kernel modules.

uep · July 11, 2023, 9:59am

Some virtual devices don’t have any other ‘trigger’, some (virtual) hardware probe that can be detected for them to handle and make them actually load. So they get added here. I’m not sure about whether these specific ones fall into that category, perhaps they can be discovered? It’s an easy test…

jan · July 12, 2023, 7:07am

I can imagine that since I’m not expert on kernel modules. I can recognize things like thunderbolt and kvm-intel but the rest of that is just mumbo jumbo for me and I need the installer to tell me what this random piece of hardware need to work.

And I can imagine that in future Linux Kernel v7 will be able to work for example with some audio device that didn’t work before since drivers were written only for Windows.

I’m assuming by module you mean some block of configuration in a separate file (or not) that is enabled for specific devices by custom config.myCustomStuff.oldHpLaptop = true; in hardware-configuration.nix and then options.myCustomStuff.oldHpLaptop = mkOption ... and mkIf config.myCustomStuff.oldHpLaptop <hardware specific config>; in the actual configuration.

Now it makes more sense. Thank you. But I bet that kvm-intel is not something that has to be forced to run since QEMU-KVM will modprobe it for you.

ius · July 12, 2023, 8:00am

You’re right on the first part, but in fact it should be loaded by udev as soon as it’s available based on cpu properties because of a modalias.

$ modinfo kvm_amd
alias:          cpu:type:x86,ven*fam*mod*:feature:*00C2*

$ modinfo kvm_intel
alias:          cpu:type:x86,ven*fam*mod*:feature:*0085*

As for why it’s forcibly loaded, a git blame shows this was added to nixos-hardware-scan.pl 16 years ago, in 2008.

The Linux kernel introduced the modalias for AMD in 2012, I guess we would’ve been fine without the nix bit for the past 10 years…

jan · July 12, 2023, 4:51pm

Thank you for the research. For a Nix novice, it amazes me the long history of the project since it feels so current and so modern . Fantastic project and even better community.

peterhoeg · July 14, 2023, 3:25am

I realize I left out an “again” which changes my answer quite radically.

I cannot imagine running the installer just to see if it does something differently. The config generated by the installer really is just to get you started.

This should have been “running the installer AGAIN” because once the machine is up, there really is nothing gained from doing it again.

I can imagine that since I’m not expert on kernel modules.

Most kernel modules will auto-load.

And I can imagine that in future Linux Kernel v7 will be able to work for example with some audio device that didn’t work before since drivers were written only for Windows.

Audio drivers should auto-load. The installer is not the tool for identifying them.

I’m assuming by module you mean some block of configuration in a separate file (or not) that is enabled for specific devices by custom config.myCustomStuff.oldHpLaptop = true; in hardware-configuration.nix and then options.myCustomStuff.oldHpLaptop = mkOption and mkIf config.myCustomStuff.oldHpLaptop ; in the actual configuration.

Exactly that, yes.

jan · July 14, 2023, 8:19am

I got your point even without AGAIN. What I meant is that I can imagine myself running it after a year or two when I decide to completely wipe the system and re-install it. Not that NixOS needs a wipe but I like to do re-install time to time.

So your point is that installer will help you to boot. In other words anything outside of boot.initrd.* is not important and modules in boot.initrd.* will unlikely to change even with Kernel v7 in two years since you shouldn’t care about new audio devices for booting.

Great explanation.

You changed my mind about this topic. I’m going to run the installer across all devices that I want to Nixify from the current Ubuntu/Debian fleet in my house and I’ll keep those generated hardware-configuration for “research” purposes nicely organized by name of the device (HP Lattitude 123, No-Name Chinese Mini-PC, … ) and then I can see differences and start generating the “modules” that can be enabled via mkIf. In other words my final configuration for all my Nixified devices will not have a file called hardware-configuration but I’ll keep those files to make sure that I understand needs for different devices.

And I think in a couple of years when I reinstall something, I’ll re-run the nixos installer anyway for “research” purposes so I can diff the new hardware-configuration with the old one and most likely see no changes

Thank you Peter and thank you everybody!