Hard drive(s) change what /dev/sda is upon reboot?

Help
1

I suspect there is some race condition happening?

My hardware is a mix of SSD (sata), HD’s (sata) and HD (IDE)…

I had posted problems with a version uplift were I discovered the root problem I hit was my configuration.nix was pointed (incorrectly) at `/dev/sdc)

I’ve since noticed that it seems the mapping of /dev/sda changes on reboot - and my boot drive, which was /dev/sdc changed to /dev/sda after a reboot… and has now again flipped back to /dev/sdc

I do understand that using /dev/disk/by-id/ or other options under /dev/disk which are more static than the legacy /dev/sdX are safer. However, I wonder why the generated configuration.nix decided to use /dev/sda.

So a few questions

  1. Why does configuration.nix end up with /dev/sda in the generated file?
  2. Why does NixOS seems to renumber my drives on reboot? Historically with other Linux distros, the name would only change if hardware changes happened.
  3. What is the best practice for specifying boot.loader.grub.device = "/dev/sda";?
2

Do your filesystems have labels? That is how the generated config usually refers to them no? Or maybe it does ids :thinking:

3

I’ve got 3 systems with NixOS currently. Two are a bit more ‘legacy’ - as evidenced that one of them has an IDE drive in it.

Only the 2 systems that have grub defined in the configuration.nix have this issue - because the problem is with the grub configuration (unsurprisingly).

Looking at the hardware-configuration.nix file - all 3 systems appear to describe the root filesystem by-id which is a very sane way to manage talking about hard disks.

I suspect that either - in the initial setup - I chose /dev/sda when I was going through the installer (but if I was presented with a better option I would have) - or the default(s) are done with /dev/sda for grub.

I don’t yet know enough about NixOS to figure out where this might be - and how we might affect change.

I’m still puzzled by why the numbering is changing (from /dev/sda to /dev/sdc) on this particular system. Again, this could be a race condition - and only happening because I have both SATA and IDE devices connected.

4

It is very much not supposed to do that.

No, this is pretty normal. The kernel makes absolutely no guarantees about the order in which drives will be brought up. It can be anything from hardware components starting up faster or slower, to software like loading kernel modules happening at different times. On any distro, this should not be considered unusual.

Pretty sure you can point to the root partition with something like /dev/by-uuid/asdf and grub will automatically find the correct physical device to install onto, IIRC.

1 Like
5

Hmm… isn’t that the code that generates /etc/nixos/hardware--configuration.nix?

In my /etc/nixos/hardware--configuration.nix - I absolutely have a /by-id/ mapping for /

  fileSystems."/" =
    { device = "/dev/disk/by-uuid/637c3c7b-3b64-4b95-a597-70326e3fd934";
      fsType = "ext4";
    };

This is not the issue I’m raising. It is the grub configuration lines that exist in /etc/nixos/configuration.nix

  # Use the GRUB 2 boot loader.
  boot.loader.grub.enable = true;
  # boot.loader.grub.efiSupport = true;
  # boot.loader.grub.efiInstallAsRemovable = true;
  # boot.loader.efi.efiSysMountPoint = "/boot/efi";
  # Define on which hard drive you want to install Grub.
  boot.loader.grub.device = "/dev/sda"; # or "nodev" for efi only
  boot.loader.grub.configurationLimit = 10;

Oh, wait - so that file looks like it is generated by the same file… but later – starting here: nixpkgs/nixos/modules/installer/tools/nixos-generate-config.pl at 908087cf870ad68aecffe59d8d9e7ef82b482584 · NixOS/nixpkgs · GitHub

It almost appears like the grub configuration is hard coded to be /dev/sda and left up to the user to enable/fix when they want to run their first rebuild? (and I probably naively just used that?) – if this is true - maybe this is a simple matter of fixing this script to use the same /by-id/ helper function to fill in the value more correctly

6

I’ve tagged the post by @ElvishJerricco as the ‘solution’ because it captures all 3 of the things I was asking about.

I think there is still some discussion on how we might improve the NixOS install when a user has ‘grub’ setup … because it does look like that’s hard coded.

I’ve run Linux a long time - and this is the first time that I’ve noticed boot to boot with no hardware changes having the devices re-order… but… the explanation also makes a lot of sense.

As for using the right specifier - I’m personally a fan of /dev/disk/by-id/ because often those names are easier to map to the actual physical device in my experience. This helps when you go pull a drive and whatnot.

Thanks everyone for being a great community

7

hm, well that code has the grub device commented out

8

Well - that’s what I saw too - but is there another place in the installer that doesn’t comment out the grub code? It’s been a while since I did the install - I can re-do it one day an rediscover what’s gone on here… I’m sorry but I do not have good evidence as to what I did to get to this state…

But, I have two machines that I installed a month or so apart … both with grub… and both with the same /dev/sda value… which seems suspicious.

9

I don’t think so? This is the only place that sets grub.device and it’s commented nixpkgs/nixos/modules/installer/tools/nixos-generate-config.pl at 908087cf870ad68aecffe59d8d9e7ef82b482584 · NixOS/nixpkgs · GitHub

The nixos manual also explicitly tells you that you need to set grub.device.

I wonder if the graphical calamares installer is doing this wrong? It looks like it asks you where to install grub to, but I don’t know if it would use a stable path or not.

10

It is very likely user error - but maybe future users who have a similar problem… will find this post and have the same face-palm moment I did.