Stuck in grub rescue after an upgrade. How to reliably recover / reinstall grub?

I did upgrade to 21.05 (System freezes (only during big updates) - #29 by maxdevjs for details).

Now:

  • at boot time, I am stuck in grub rescue
  • nothing that I tried really worked (all the listed filesystems are reported as unknown)

boot in dev/sda1, not encrypted
root, home in dev/sda2, encrypted

  • I booted with a live USB (20.03, older grub)
  • mounted my encrypted partitions

I did not find an eloquent walk-through on grub rescue and not sure how to continue:

  • should I chroot from the live USB as in a normal installation?
  • in this case, should strictly be a version equal to the installed one?
  • I see that the Minimal ISO image contains “a number of rescue tools”, how can I know what tools are those?
  • then directly issue a nixos-install (configuration is already set up, used the system for several months)?

Suggestions will be extremely welcomed.

2 Likes

This is a ‘bit’ messy and not exhaustive, but… So far I:

  • booted a 21.05 live USB
  • chrooted into my system
    • all seems to be there
      • I can also access my user and my configuration

tried nix-store --verify --repair --check-contents and it:

Accuses several errors (apparently, mostly kde/plasma things) and it fails with

reading the Nix store...
checking path existence...
path '/nix/store/09x01ijmyi9nw9d1srk18g5ilipvflf2-kpackage-5.81.0-dev' disappeared, but it still has valid referrers!
copying path '/nix/store/09x01ijmyi9nw9d1srk18g5ilipvflf2-kpackage-5.81.0-dev' from 'https://cache.nixos.org'...
warning: unable to download 'https://cache.nixos.org/nar/005nymvisr2vicximf8my02yicjjxds6922jrphn12c3ih37fawi.nar.xz': Couldn't resolve host name (6); retrying in 267 ms
warning: unable to download 'https://cache.nixos.org/nar/005nymvisr2vicximf8my02yicjjxds6922jrphn12c3ih37fawi.nar.xz': Couldn't resolve host name (6); retrying in 532 ms
warning: unable to download 'https://cache.nixos.org/nar/005nymvisr2vicximf8my02yicjjxds6922jrphn12c3ih37fawi.nar.xz': Couldn't resolve host name (6); retrying in 1315 m

...

warning: unable to download 'https://cache.nixos.org/nar/0iz8p1gv6fiji05nm9cppa1any48mmj5bq752gpw2gx5ya071wj3.nar.xz': Couldn't resolve host name (6); retrying in 325 ms
warning: unable to download 'https://cache.nixos.org/nar/0iz8p1gv6fiji05nm9cppa1any48mmj5bq752gpw2gx5ya071wj3.nar.xz': Couldn't resolve host name (6); retrying in 574 ms
warning: unable to download 'https://cache.nixos.org/nar/0iz8p1gv6fiji05nm9cppa1any48mmj5bq752gpw2gx5ya071wj3.nar.xz': Couldn't resolve host name (6); retrying in 1238 ms
warning: unable to download 'https://cache.nixos.org/nar/0iz8p1gv6fiji05nm9cppa1any48mmj5bq752gpw2gx5ya071wj3.nar.xz': Couldn't resolve host name (6); retrying in 2589 ms
unable to download 'https://cache.nixos.org/nar/0iz8p1gv6fiji05nm9cppa1any48mmj5bq752gpw2gx5ya071wj3.nar.xz': Couldn't resolve host name (6)
warning: cannot repair path '/nix/store/lgg8zgjxjb1v4syi43hvqa7r9fjjnkd9-akonadi-mime-21.04.0'
path '/nix/store/1i5amp4gpy1l2mnka1hvsrilmfpbmynr-akonadi-21.04.0' disappeared, but it still has valid referrers!
copying path '/nix/store/1i5amp4gpy1l2mnka1hvsrilmfpbmynr-akonadi-21.04.0' from 'https://cache.nixos.org'...

...

warning: unable to download 'https://cache.nixos.org/nar/1spia5h0s3srkcc40rq5an2y4hxmys6x5jn1yz4fzf4jzj75fcd5.nar.xz': Couldn't resolve host name (6); retrying in 328 ms
warning: unable to download 'https://cache.nixos.org/nar/1spia5h0s3srkcc40rq5an2y4hxmys6x5jn1yz4fzf4jzj75fcd5.nar.xz': Couldn't resolve host name (6); retrying in 547 ms
warning: unable to download 'https://cache.nixos.org/nar/1spia5h0s3srkcc40rq5an2y4hxmys6x5jn1yz4fzf4jzj75fcd5.nar.xz': Couldn't resolve host name (6); retrying in 1113 ms
warning: unable to download 'https://cache.nixos.org/nar/1spia5h0s3srkcc40rq5an2y4hxmys6x5jn1yz4fzf4jzj75fcd5.nar.xz': Couldn't resolve host name (6); retrying in 2317 ms
unable to download 'https://cache.nixos.org/nar/1spia5h0s3srkcc40rq5an2y4hxmys6x5jn1yz4fzf4jzj75fcd5.nar.xz': Couldn't resolve host name (6)
warning: cannot repair path '/nix/store/l7ivczkp3krvbk3cmv9p0fdnl0i4b2p5-kwalletmanager-21.04.0'
path '/nix/store/smly5lipl0fs5l9ypyh65yyb10rzc78x-kdeconnect-kde-21.04.0' disappeared, but it still has valid referrers!
copying path '/nix/store/smly5lipl0fs5l9ypyh65yyb10rzc78x-kdeconnect-kde-21.04.0' from 'https://cache.nixos.org'...

warning: unable to download 'https://cache.nixos.org/nar/0camwcg27ghlxml1sq0d2xrc7g6bq3anjwvs0ridsahd44yrfxyg.nar.xz': Couldn't resolve host name (6); retrying in 253 ms
warning: unable to download 'https://cache.nixos.org/nar/0camwcg27ghlxml1sq0d2xrc7g6bq3anjwvs0ridsahd44yrfxyg.nar.xz': Couldn't resolve host name (6); retrying in 669 ms
warning: unable to download 'https://cache.nixos.org/nar/0camwcg27ghlxml1sq0d2xrc7g6bq3anjwvs0ridsahd44yrfxyg.nar.xz': Couldn't resolve host name (6); retrying in 1071 ms
warning: unable to download 'https://cache.nixos.org/nar/0camwcg27ghlxml1sq0d2xrc7g6bq3anjwvs0ridsahd44yrfxyg.nar.xz': Couldn't resolve host name (6); retrying in 2045 ms
unable to download 'https://cache.nixos.org/nar/0camwcg27ghlxml1sq0d2xrc7g6bq3anjwvs0ridsahd44yrfxyg.nar.xz': Couldn't resolve host name (6)kdeApplications
path '/nix/store/sh86mqjlijwprh2vqk3jyzx4qkni38lr-kdegraphics-thumbnailers-21.04.0' disappeared, but it still has valid referrers!
copying path '/nix/store/sh86mqjlijwprh2vqk3jyzx4qkni38lr-kdegraphics-thumbnailers-21.04.0' from 'https://cache.nixos.org'...
warning: unable to download 'https://cache.nixos.org/nar/1acfbmp766gnk4vz70znn2w4435k5vmaa5fxx0c3cap4mvijqd5h.nar.xz': Couldn't resolve host name (6); retrying in 284 ms
warning: unable to download 'https://cache.nixos.org/nar/1acfbmp766gnk4vz70znn2w4435k5vmaa5fxx0c3cap4mvijqd5h.nar.xz': Couldn't resolve host name (6); retrying in 638 ms
warning: unable to download 'https://cache.nixos.org/nar/1acfbmp766gnk4vz70znn2w4435k5vmaa5fxx0c3cap4mvijqd5h.nar.xz': Couldn't resolve host name (6); retrying in 1357 ms
warning: unable to download 'https://cache.nixos.org/nar/1acfbmp766gnk4vz70znn2w4435k5vmaa5fxx0c3cap4mvijqd5h.nar.xz': Couldn't resolve host name (6); retrying in 2228 ms
unable to download 'https://cache.nixos.org/nar/1acfbmp766gnk4vz70znn2w4435k5vmaa5fxx0c3cap4mvijqd5h.nar.xz': Couldn't resolve host name (6)
warning: cannot repair path '/nix/store/sh86mqjlijwprh2vqk3jyzx4qkni38lr-kdegraphics-thumbnailers-21.04.0'

...

tried nixos-rebuild switch --upgrade (nixos-rebuild switch --upgrade --show-trace) and it fails:

warning: unable to download 'https://nixos.org/channels/nixos-21.05': Couldn't resolve host name (6); retrying in 294 ms
warning: unable to download 'https://nixos.org/channels/nixos-21.05': Couldn't resolve host name (6); retrying in 654 ms
warning: unable to download 'https://nixos.org/channels/nixos-21.05': Couldn't resolve host name (6); retrying in 1310 ms
warning: unable to download 'https://nixos.org/channels/nixos-21.05': Couldn't resolve host name (6); retrying in 2031 ms
warning: warning: unable to download 'https://nixos.org/channels/nixos-21.05': Couldn't resolve host name (6); using cached result
error: unable to download '/nixexprs.tar.bz2': URL using bad/illegal format or missing URL (3)

I also tried to add stalled-download-timeout = 99999999 to ~/.config/nix/nix.conf (from HTTP error 200 (curl error: Timeout was reached) - unable to update and install #3025 )

1 Like

I downloaded yesterday the 21.05 iso (that I used in the previous attempts).

Trying an install (with a newly created configuration) from it:

...
  /nix/store/xganmmrd8baw0xcn9dsr6psgi90sgqf6-xserver.conf
  /nix/store/z08fdca1b7fbmbvizv5yrmc19hyr75wl-fc-cache
building '/nix/store/wijf8c13yipnpn6vh4fa4srp5rpkzga5-kernel-modules.drv'...
error: getting attributes of path '/nix/store/2r9lrcxgs9wlbxhzqcvz78jqjpjb88fy-kdeclarative-5.81.0-bin': No such file or directory

By the way:

sudo NIXOS_INSTALL_BOOTLOADER=1 chroot /mnt /nix/var/nix/profiles/system/bin/switch-to-configuration boot
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = "",
        LC_ALL = (unset),
        LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
updating GRUB 2 menu...
read_file '/proc/self/mountinfo' - open: No such file or directory at /nix/store/xh1sd9ms5b2ixhz61qqiz07j2b9xv4z9-install-grub.pl line 128.