I swapped both the nixos and the home-manager channels with their newer counterparts and fixed any configuration errors in my configuration.nix that showed up during upgrading. However, the final time I ran sudo nixos-rebuild switch --upgrade-all something went wrong. It continued with the installation since my config file didn’t give me any errors, but sometime during the upgrade it stopped, saying that I was missing GLIBC 2.38.
At this point I noticed that my commands also stopped working, giving me the same GLIBC 2.38 missing error. I figured rebooting might fix this, since (at least this was my understanding) installations/upgrades are atomic.
However, rebooting gave me a kernel panic. First during shutdown. Right at the end my capslock started flashing, so I had to hold down the power button. I’m also getting kernel panics during boot of any generation, not just the latest one.
OK I got it working again I had to do a system rescue using a nixos live usb
(If your disk is not encrypted you can replace the cryptsetup and mounting of /dev/mapper/my_volume with your normal partitions)
nvme0n1p1 is my boot partition, nvme0n1p2 is my root partition.
sudo cryptsetup luksOpen /dev/nvme0n1p2 my_volume
sudo mount /dev/mapper/my_volume /mnt
sudo mount /dev/nvme0n1p1 /mnt/boot
sudo nixos-enter
nixos-rebuild switch --upgrade-all
At the end of the rebuild it gave me a bunch of errors saying that the older generations didn’t have GLIBC 2.38 or something, but I can now boot into my system again!
(Please reply with what I could’ve done to prevent this and/or what I could’ve done better during the rescue. Those errors during the rebuild surely mean that something didn’t go right…)
They are, at least if you run nixos-rebuild boot. If you use switch, the new configuration will be immediately activated, at which point things may break (especially if you have graphics driver updates that make your driver libs no longer match up with the running kernel). It’s better practice to use nixos-rebuild boot in general, but probably especially for full version upgrades (in fact, the release notes recommend this in the highlights).
Either way, if the nixos-rebuild boot part of nixos-rebuild switch completed successfully, a reboot should boot you into the same (presumably broken) generation unless you pick another one from your boot loader. The evaluation was successful, so the same thing would happen every time from there on out.
That’s a kernel panic, just not visible when your console is in graphical mode. Kernel panics should not happen because of random software updates, let alone atomic nix ones. You sure this isn’t a hardware issue?
You sure this isn’t corruption caused by a hardware issue? Something went very wrong here.
What channels are you running on your system? Have you tried verifying your store?
I’ve never had this happen before. It would be a big coincidence (yet still a possibility) that the first time a hardware error occurred was when I upgraded to 23.11.
Lesson learned. I’ll use boot instead of switch for 24.05. So far, I haven’t had a problem with switch for updates within 23.05.
I think so, if you’re still using old nix commands maybe you need to use some variant of nix-store --verify, but I’ve lost track whether there were any changes to the store format.
It didn’t give me much info, just that a bunch of stuff is untrusted. No idea what that means.
nix store verify --all
❯ sudo nix store verify --all
[sudo] password for sylvester:
path '/nix/store/jd99cyc0251p0i5y69w8mqjcai8mcq7h-xgcc-12.2.0-libgcc' is untrusted
path '/nix/store/567zfi9026lp2q6v97vwn640rv6i3n4c-libunistring-1.1' is untrusted
path '/nix/store/4563gldw8ibz76f1a3x69zq3a1vhdpz9-libidn2-2.3.4' is untrusted
path '/nix/store/q74idm55v5km2pp9yh5qhzc4cw639kp4-cantarell-fonts-0.303.1' is untrusted
path '/nix/store/bzjd66sdi85rwj7kd3zcdsi52xzhx69c-nix-wallpaper-simple-dark-gray.png' is untrusted
path '/nix/store/bl4f68x3l9dkkr7l2blbj4jsdhvm81bs-nix-wallpaper-simple-blue.png' is untrusted
path '/nix/store/qs605c91wv5fr9rny4mmhpdna6i59kdk-linux-headers-6.2' is untrusted
path '/nix/store/01pvlzh6kwi0yw2m29s277bmgrrmi0ld-libcpuid-0.6.3' is untrusted
path '/nix/store/46m4xx889wlhsdj72j38fnlyyvvvvbyb-glibc-2.37-8' is untrusted
path '/nix/store/02s99g788b8ifb6csz73zb3i4m253y0m-ncurses-6.4-man' is untrusted
path '/nix/store/69jpyha5zbll6ppqzhbihhp51lac1hrp-zlib-1.2.13' is untrusted
path '/nix/store/03grn1z9pl9m8702wkshj5pwnvknazim-zlib-1.2.13-dev' is untrusted
path '/nix/store/xqblj2djqj4as6s5i00rkybcm4hmfldr-keyutils-1.6.3-lib' is untrusted
path '/nix/store/8fv91097mbh5049i9rglc73dx6kjg3qk-bash-5.2-p15' is untrusted
path '/nix/store/08bbg98v5l4aalxbr583mhpxasrwsf41-xz-5.4.3-bin' is untrusted
path '/nix/store/h9rmj5nqwm89aj8zlm0cf4fxbbxdd8zm-xz-5.4.3' is untrusted
path '/nix/store/09vhdg4r0z9rzabhcadxapcm5p9jhwzq-diffutils-3.9-info' is untrusted
path '/nix/store/i1jf9af43mf4jpnz7angp62bfndm8ksd-jq-1.6-bin' is untrusted
path '/nix/store/q6w2m8ds74j6gwxlbqdq6c6jn7yqjzz1-jq-1.6-lib' is untrusted
path '/nix/store/sdk9yz647zsl7bpnkijs5zkq9xqwz0mp-oniguruma-6.9.8-lib' is untrusted
path '/nix/store/0cw69kwj62xnid3bkcd3lylnwz7w6gn8-patch-2.7.6' is untrusted
path '/nix/store/h3lwhsmmnw3n7f3ryn55xi9igl6c28ha-ed-1.19' is untrusted
path '/nix/store/81d13il7plchw65gz8y9ywcxrngq149c-gcc-12.2.0-libgcc' is untrusted
path '/nix/store/3kk0nlrfq90qfljsrk23nnlxjj06djfl-editline-1.17.1' is untrusted
path '/nix/store/127cavf9wx401az6hciz2v2172xnzbz2-boehm-gc-8.2.2' is untrusted
path '/nix/store/9jwr6p0ix32qa5wkvjh66lvsj617bn9j-lowdown-1.0.2-lib' is untrusted
path '/nix/store/b19c558b7c21a72hxr6cq61jhzz3h8hq-libsodium-1.0.18' is untrusted
path '/nix/store/2av8x1ipwfm5n8nrx1i18bygza6v8b6g-brotli-1.0.9-lib' is untrusted
path '/nix/store/jwvvq9nyfrjvj10pl533my6d3gpn9nq0-zstd-1.5.5' is untrusted
path '/nix/store/kaa2mgjp0zjg3s5s75rdi3fk90d4yxzd-libxml2-2.10.4' is untrusted
path '/nix/store/whkpnh2db07rn1b7hcvfjfwag8imygvp-attr-2.5.1' is untrusted
path '/nix/store/v0iir4ksr0vb1px00yd80hip9yy5p4hk-bzip2-1.0.8' is untrusted
path '/nix/store/6yp0l9mdrwqkd1a33ps6ykk2mnlyzx3v-acl-2.3.1' is untrusted
path '/nix/store/4iabmjjq95069myjsrid8pk2ib3yz4nn-openssl-3.0.10' is untrusted
path '/nix/store/mrg3c5a3b0395nfzwikja0f1cn5hm8wc-busybox-static-x86_64-unknown-linux-musl-1.36.1' is untrusted
path '/nix/store/cq0j3c5kilkh3dgqn0v6g94kjwhcxd15-libarchive-3.6.2-lib' is untrusted
path '/nix/store/n12a68qch9s85k6ni4m4r4xxr8lwys1i-sqlite-3.41.2' is untrusted
path '/nix/store/843dqq10jdkalr2yazaz6drx334visrb-gcc-12.2.0-lib' is untrusted
path '/nix/store/gwsf52ljpp3pi13fflmclmn71nwgds12-aws-c-compression-0.2.16' is untrusted
path '/nix/store/fm7a66v8l2rjmfass0i7d4fqllxzs09z-aws-c-cal-0.5.21' is untrusted
path '/nix/store/hj1caixhxrldx5jrx7x7axlcghn3r82n-s2n-tls-1.3.43' is untrusted
path '/nix/store/2nvsyg7qp6vixv5bpwvvafy9kxi6qklz-aws-c-sdkutils-0.1.9' is untrusted
path '/nix/store/qp4q2wd52fs4q48gdmd5i2c04xp8j9cl-aws-checksums-0.1.14' is untrusted
path '/nix/store/0kmx26k1zk1gclfswwk0hbvkv6l2ahfx-aws-c-http-0.7.6' is untrusted
path '/nix/store/ykg2prfglnifc947a9n44py79mlyqmhw-aws-c-io-0.13.19' is untrusted
path '/nix/store/5wwvy6bam5wa0w3cfbghazck90jyxaxn-aws-c-event-stream-0.2.20' is untrusted
path '/nix/store/v4r4x88y91nc72sjbi9791hqsziiz4ns-aws-crt-cpp-0.19.8' is untrusted
path '/nix/store/s3l67dahg8lj8zppyq851bkn97f4i9r0-nlohmann_json-3.11.2' is untrusted
path '/nix/store/m63030yga75q2xhs6g016anlsr3afpc3-aws-c-auth-0.6.26' is untrusted
path '/nix/store/nnla37mnr2nimvysvvmpn25vip3qiglc-aws-c-s3-0.2.8' is untrusted
path '/nix/store/7lci7yd3nqsn79j8zrz9nl7y4vpma078-aws-c-mqtt-0.8.8' is untrusted
path '/nix/store/0lvgi2bnwi8cs0wx2242g1275kqx8bba-libssh2-1.11.0' is untrusted
path '/nix/store/7aw32rkwgbim37x6kgza7qvr35czjysa-nghttp2-1.51.0-lib' is untrusted
path '/nix/store/zp5f1mi3diiqx68c2cnca4cv1zchw6v4-libseccomp-2.5.4-lib' is untrusted
path '/nix/store/y2qmx9f9bskxks88l3h6klhzxxwb10ca-curl-8.1.1' is untrusted
path '/nix/store/jj38aw52c2rddpknmi3ifmlx4ygn3flb-aws-c-common-0.8.15' is untrusted
path '/nix/store/wlacmv33nsrf2vnfa2akcym13zh6zzrx-libkrb5-1.20.1' is untrusted
path '/nix/store/ixrv7gpjdp5lrb8b9x89sww6b7i6xhfm-libuv-1.44.2' is untrusted
path '/nix/store/m0inyjcw7p1h9677mv9gah92ksz1mg03-expat-2.5.0' is untrusted
path '/nix/store/lyri4viji7lx2jwp5xy5fi1i1y0wx8z0-glibc-2.37-8-bin' is untrusted
path '/nix/store/hkj250rjsvxcbr31fr1v81cv88cdfp4l-glibc-2.37-8-dev' is untrusted
path '/nix/store/7ihbibsv0svmdwmnxhvhfzm5m9fvxsmh-curl-8.1.1-man' is untrusted
path '/nix/store/f954k4xbkjk6xkv4k7py5y2ys0y0r4ql-curl-8.1.1-bin' is untrusted
path '/nix/store/5x0cf6xglh7h6qszkflll6fspgl39jrb-openssl-3.0.10-bin' is untrusted
path '/nix/store/m5vynwrm18blksb171g5l5m55phb6pkb-pcre-8.45' is untrusted
path '/nix/store/7gqmifamblx3zb25qcrxdsmsi3az7an2-zstd-1.5.5-bin' is untrusted
path '/nix/store/8mzvz6kk57p9aqdk72pq1adsl38bkzi6-gnugrep-3.7' is untrusted
path '/nix/store/sss6wmnvh82nyhf9024bdxgqi6k0ygm2-openssl-3.0.10-dev' is untrusted
path '/nix/store/shc9sqfjkpfr6rw69080vgz59cbn4jk9-gmp-with-cxx-6.2.1' is untrusted
path '/nix/store/j4fwy5gi1rdlrlbk2c0vnbs7fmlm60a7-coreutils-9.1' is untrusted
path '/nix/store/1x4qg9ap1x3b50z7j4mbdvqlf1sjkbf8-diffutils-3.9' is untrusted
path '/nix/store/2d4sykcfjj2kc0968fym92p4yb67lk7x-which-2.21' is untrusted
path '/nix/store/2dd5ix229kycm3g3bpx36yfhxs9jknqc-openssl-3.0.10' is untrusted
path '/nix/store/2jgnizza14z2mz2jfm098gbvvl4aysil-binutils-2.40-lib' is untrusted
path '/nix/store/5p2mnbvabkg3lxm3cs9cfqx3nmnalpw0-libxcrypt-4.4.33' is untrusted
path '/nix/store/bzd9s7qjjwvmgcyli9cqwrfds5xx2k3r-db-4.8.30' is untrusted
path '/nix/store/c09ywak92jyy25yfv6cfqqmfzybb1b7l-tcb-1.2' is untrusted
path '/nix/store/xz9n01grlbbqf8mfmjcnvsvza6k5n67z-audit-3.1' is untrusted
path '/nix/store/2la3yai9gk0i7fl4lnqkzhv5rww7nff9-shadow-4.13' is untrusted
path '/nix/store/8px21amlnlknrkgmw4zmff4mzma81zxj-linux-pam-1.5.2' is untrusted
path '/nix/store/2q1x5hazpr8yf5hh0pj09nmz94sc7q81-file-5.44' is untrusted
path '/nix/store/2v3m2kpfwvkik81k0hqfh4hv8zddbakh-e2fsprogs-1.46.6-man' is untrusted
path '/nix/store/rqrgs48l9knris8r86b7zlg6lnkc57fj-find-xml-catalogs-hook' is untrusted
path '/nix/store/b634rxg4s74sj9ac6dijy8v6ssfrf390-gmp-6.2.1' is untrusted
path '/nix/store/lp72fysfjmrid1zsgk4k43lg3smycgn7-mpfr-4.2.0' is untrusted
path '/nix/store/gqs234r2zdmlxvs6jzk8w70v68mxd62f-libmpc-1.3.1' is untrusted
path '/nix/store/i9cq7v56xmpiyzw1dvdfs6gbzma0c787-libtool-2.4.7-lib' is untrusted
path '/nix/store/kv3p3lh74azm5z1s97xp08p6x35zq9qd-expand-response-params' is untrusted
path '/nix/store/7lirwch44w8ym9ap47px6qlrr5km8s8b-gnum4-1.4.19' is untrusted
path '/nix/store/cwl4ylj9rps9alf2z51s8l32ivvc50yv-isl-0.20' is untrusted
path '/nix/store/qnjzh4b0zgdkpb9x3r3h3bnc3rhdysbx-binutils-wrapper-2.40' is untrusted
path '/nix/store/lcf37pgp3rgww67v9x2990hbfwx96c1w-gcc-wrapper-12.2.0' is untrusted
path '/nix/store/yqa6z204wc1qwrac9sjdsnp2cdjay6bg-file-5.44' is untrusted
path '/nix/store/vqj2w8rqghmmp4wkn9lkcym5kzlqk372-gnused-4.9' is untrusted
path '/nix/store/zkjq96ik8cbv6ijh1lylnkk2bni9qvas-binutils-2.40' is untrusted
path '/nix/store/p42fvwnx9pcp5ijrj5n5qqqfbqimhrkv-readline-8.2p1' is untrusted
path '/nix/store/i6y3vfw18aygsivl29fk7y0pb153j571-ncurses-6.4' is untrusted
path '/nix/store/b7hvml0m3qmqraz1022fwvyyg6fc1vdy-gcc-12.2.0' is untrusted
path '/nix/store/3cs2pdd202bl1vz5z92323c2jzv8jcm1-linux-pam-1.5.2-doc' is untrusted
path '/nix/store/3r6w393z8kmg9r0z4jqdkc0yff4kfxmv-mpfr-4.2.0' is untrusted
path '/nix/store/c6c3jfh0abzp8rw8x7g03mv7qypcslq3-gmp-with-cxx-6.2.1' is untrusted
path '/nix/store/3vxbq3ih3rjfrwwh8324cx23wn4y1b88-mailcap-2.1.53' is untrusted
path '/nix/store/4a6iv3pl9npf4iwm72dskwl89hckdcdj-libffi-3.4.4' is untrusted
path '/nix/store/49q1zfhkan4k0rqyxfhj512nw2wwngsf-gawk-5.2.1' is untrusted
path '/nix/store/3yx6fa7gxgp4p6d79skvscvdd21alclp-tzdata-2023c' is untrusted
path '/nix/store/x6rwgp1jl5sgzwbsaigqkdbdc7krzwj7-findutils-4.9.0' is untrusted
path '/nix/store/bsca0mm8zc66gnibdvp5wwpfqh5c91xa-bzip2-1.0.8-bin' is untrusted
path '/nix/store/5rbqiz11vagap5fsk3vw8fgznw6vxyrh-patchelf-0.15.0' is untrusted
path '/nix/store/5rzhg243yff6n5hw9v4h2ws8p3ms8j8k-nix-2.13.5-doc' is untrusted
path '/nix/store/9p6qqynyba4lpjdscqxz423hbfb2zm3d-shadow-4.13-su' is untrusted
path '/nix/store/rnpkycdw46g2yrdaigy295i2ypv0ghmz-util-linux-minimal-2.38.1-lib' is untrusted
path '/nix/store/z9jh87pcwwh93cikdqh8s8azggg33bvj-libcap-ng-0.8.3' is untrusted
path '/nix/store/6ivxlbrn6sxaqwnlihhfidmpl97smpf1-fuse-2.9.9' is untrusted
path '/nix/store/vmm9fdvcw59s6yr8imjpkhs1b0spjnfc-util-linux-minimal-2.38.1-bin' is untrusted
path '/nix/store/6rcgb5zvkjz7h1yxx93rwaa6anzzfwk9-perl-5.36.0-man' is untrusted
path '/nix/store/dpyzh5m8qaavscjk5k7d8q505hzwd5l8-libxslt-1.1.37' is untrusted
path '/nix/store/6zrssd9yygjznj44q7irgq61g7cjkrbp-gzip-1.12-info' is untrusted
path '/nix/store/798wjzmkdxczl9jlvbsi95f0vfhd33fq-libffi-3.4.4-dev' is untrusted
path '/nix/store/7azb0ilkr43hy8x1a42ljsrykfl7yxnq-bzip2-1.0.8-man' is untrusted
path '/nix/store/7ffyq3fgbb2jb8y9y7brnrjsad1z04p9-pkg-config-0.29.2' is untrusted
path '/nix/store/7m3dhim5mnsb92yfpfxr245p3my8n120-cpio-2.14' is untrusted
path '/nix/store/9sl9vgmpz7wry8l1zyxsmjwhjv1dphm6-aws-sdk-cpp-1.11.37' is untrusted
path '/nix/store/z0950pbn0246r2ibgwrpr97lh7j10164-nix-2.13.5-man' is untrusted
path '/nix/store/8vk4w1fiy2lr84xqkaqi0rjlw2s7882h-nix-2.13.5' is untrusted
path '/nix/store/8yil1h65napjd9z4jlz79xl2p1x644yq-gawk-5.2.1-man' is untrusted
path '/nix/store/yvwaw5wlzkpdxyczlmz6kqljd0j2vmp7-gdbm-1.23' is untrusted
path '/nix/store/93fy4p3vrjpxkzrliql5lh51flcanx04-pkg-config-wrapper-0.29.2' is untrusted
path '/nix/store/a2qh67rrhczccdax10jrgp86p1xafrzi-attr-2.5.1-man' is untrusted
path '/nix/store/bc45k1n0pkrdkr3xa6w84w1xhkl1kkyp-python3-3.10.12' is untrusted
path '/nix/store/avc5q7gnlrp0kg70aiw4d12xa5034ai2-shadow-4.13-man' is untrusted
path '/nix/store/zphqjmf8n6f3bws7m1sy6nk5jhhp5l5i-attr-2.5.1-bin' is untrusted
path '/nix/store/avl37x32aq90xy5lq1c1b0nnqj8nc5lp-llvm-15.0.7-lib' is untrusted
path '/nix/store/c9dqnnhwdy64zqwpjkzk1s81zvz0infj-expat-2.5.0-dev' is untrusted
path '/nix/store/ch1wp2pp4scp65izfr0cx59ikn451qky-acl-2.3.1-bin' is untrusted
path '/nix/store/chl43hal884g2g9yr8yjbq5gn58g11zy-e2fsprogs-1.46.6' is untrusted
path '/nix/store/ik9ibmq0rbznwyj6472lv2ab0c397lhy-gnumake-4.4.1' is untrusted
path '/nix/store/s9dzv58651xf1d6m5i48s17vkbipg5ww-gzip-1.12' is untrusted
path '/nix/store/cj9q2w7z4m207fi0d10isx6g3ss0mmrb-stdenv-linux' is untrusted
path '/nix/store/mddiy1w0ikb2pd48c7fciw044csv536y-gnutar-1.35' is untrusted
path '/nix/store/ckxyp83780r81pcnldqkkyfcb0hayzih-fuse-3.11.0' is untrusted
path '/nix/store/dzq6j9pq9ihr4cazairbr4p6xnidbmcr-findutils-4.9.0-info' is untrusted
path '/nix/store/f53cpkpxqfysbz1dwk77jifpjflzqls9-attr-2.5.1-doc' is untrusted
path '/nix/store/f8rqxdsam9finfgj9in6ggxqbsrslixk-gnugrep-3.7-info' is untrusted
path '/nix/store/fgr89gfh5if2x12qm0wz2s996jyn5qx9-gettext-0.21' is untrusted
path '/nix/store/gcblz9l9l55jdbjrn8s9zi1n03q40wy7-gnused-4.9-info' is untrusted
path '/nix/store/i1igrl7nfjxk30cmv3vn5wrvb0iil45i-icu4c-73.1' is untrusted
path '/nix/store/ijhb7ysrzhk8p43ycdpdirc9y8salrbh-e2fsprogs-1.46.6-bin' is untrusted
path '/nix/store/ivpn7nw456naz6sx7ascxpry79v5qrnh-cracklib-2.9.8' is untrusted
path '/nix/store/iw1ww8s0ggh6m51r30l7mqamzblb0m2h-meson-1.1.0' is untrusted
path '/nix/store/k5pwxymp487xdyhbc5hrf9p53igbszr2-xz-5.4.3-man' is untrusted
path '/nix/store/vrav2nnbyl26bwfq42sr1amjfsjxhdi7-perl-5.36.0' is untrusted
path '/nix/store/l0m423h6hjy25x04m5hz9smnwfhbcgff-linux-pam-1.5.2-man' is untrusted
path '/nix/store/lj2ax4rz029fibiv1av8dx0q6gy9wjlr-bison-3.8.2' is untrusted
path '/nix/store/mfmql9lmb0bdydrkh1i2zrzsgmqgj8jv-boost-1.79.0' is untrusted
path '/nix/store/mqrlr7jrg66zgrqdb86mca0ra2mm7xag-unzip-6.0' is untrusted
path '/nix/store/mw0p4sr553wcd5gpyi06lnw6fyan1crb-libmpc-1.3.1' is untrusted
path '/nix/store/mz2shzp38dv31daacqj4gmbfn1shkrm5-gnutar-1.35-info' is untrusted
path '/nix/store/nya6zsqrc6rwfc101wdfkj7jhivy4six-e2fsprogs-1.46.6-info' is untrusted
path '/nix/store/pkfz1773whxl4gnih56mzk1xavh02pmx-xz-5.4.3-doc' is untrusted
path '/nix/store/qgamcky4sy168nn6lbd3dnfgv61gaarm-nss-cacert-3.92-p11kit' is untrusted
path '/nix/store/s86wnrhyzvcvwwgnbzzc6pi1hsm42dky-gawk-5.2.1-info' is untrusted
path '/nix/store/sh0zc6qfyqbz7njcvnj5xvk66wapihlx-gzip-1.12-man' is untrusted
path '/nix/store/sxrh37b22ga4q42kcapfh1746jk887ky-lzo-2.10' is untrusted
path '/nix/store/vfdg65hiv4bwls48588msw8la7452w2q-stdenv-linux' is untrusted
path '/nix/store/wd2678n4jhz1clp28d2niyliv4smjhbm-glibc-iconv-2.37' is untrusted
path '/nix/store/xrj2gdhdkhk90l5jg96ziimhip5bzwjd-audit-3.1-bin' is untrusted
path '/nix/store/yf0l4b1pidv1yyqpyfv4xsg2ifkyg4kv-zstd-1.5.5-man' is untrusted
path '/nix/store/ys6g8wvr5rwxyfqrlvwd7vipj9ysqn70-nss-cacert-3.92' is untrusted
path '/nix/store/zzwm1rrdign5f94wsqwwgh8yzsziwz9b-libarchive-3.6.2' is untrusted
path '/nix/store/z132c8g33b6z3m5m1zc3cj0kasv4rqxm-acl-2.3.1-doc' is untrusted
path '/nix/store/z7dl4wk8afc6k16hr2bjpl9nj44an1nx-acl-2.3.1-man' is untrusted
That means that those paths are not signed by a key your nix trusts, in theory means they weren’t built by hydra (nor locally). Where on earth they came from if that’s not it I have no idea - nix should not even download them if that’s the case.
More likely the signatures were corrupted, or that they were signed with a key that is no longer trusted for some reason. Or that this is a bug in nix store verify.
Either way, doesn’t seem like this was caused by some kind of drive failure.
In absence of any other data, I’m going to guess you fell victim to the rare “missing mount unit” thing the release notes were referring to, and that that somehow caused the bootloader not to be written to your boot partition correctly, resulting in the subsequent boot failures.
Hard to debug without a time machine, but it matches the symptoms (except the mysterious glibc warnings, but I’ll write that off as some kind of stable/unstable incompatibility mixed with overlays) and the fact that just a rebuild fixed it.
I had to do a system rescue using a nixos live usb