I can’t find the boot time fsck logs for the root file system after the system starts. Where are they located? There were some ext4 errors, after restarting the problem was resolved. But I still can’t find the logs. Found nothing:
journalctl -u systemd-fsck -xe
dmesg | grep fsck
cat /var/log/syslog | grep fsck
I expect that fsck worked at the beginning because boot.initrd.checkJournalingFS = true; by default.
If you just do journalctl -b 0 to see the full log of the current boot, you’ll see the initrd log near the beginning, including fsck messages. -u systemd-fsck would be relevant for systemd-based initrd (which is not enabled by default) and for stage 2 file systems.
Yes thank you very much it worked. I don’t know much about fsck, so I’ll ask this at the same time. Do these logs mean that the errors have been resolved and there are no more problems with ext4 at least at the moment?
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:18 UTC 2023] starting device mapper and LVM...
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:18 UTC 2023] checking /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8...
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:18 UTC 2023] fsck (busybox 1.36.1)
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:18 UTC 2023] [fsck.ext4 (1) -- /mnt-root/] fsck.ext4 -a /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:18 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8 contains a file system with errors, check forced.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 655552 extent tree (at level 2) could be narrower. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 655578 extent tree (at level 1) could be narrower. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 655580 extent tree (at level 1) could be narrower. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 655605 extent tree (at level 1) could be narrower. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 655628 extent tree (at level 1) could be narrower. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 655704 extent tree (at level 1) could be narrower. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 656588 extent tree (at level 1) could be narrower. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 656663 extent tree (at level 1) could be narrower. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 657262 extent tree (at level 1) could be narrower. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 657620 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 657640 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 657685 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 657728 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 657807 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 657824 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 657826 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 667783 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 667785 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 667788 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 667791 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 668665 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 668678 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 668680 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 668726 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 728759 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 728813 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 757286 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:25 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 757287 extent tree (at level 1) could be shorter. IGNORED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:32 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 2906739 passes checks, but checksum does not match inode. FIXED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:32 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 2906743 passes checks, but checksum does not match inode. FIXED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:32 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 2906745, i_size is 4339, should be 4096. FIXED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:32 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 2906749 passes checks, but checksum does not match inode. FIXED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:32 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: Inode 2906751, i_size is 4148, should be 4096. FIXED.
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:50 UTC 2023] /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8: 1261198/3932160 files (0.3% non-contiguous), 14597430/15727379 blocks
Kas 06 00:06:52 fridge stage-1-init: [Sun Nov 5 21:06:50 UTC 2023] mounting /dev/disk/by-uuid/fef36210-e7f9-45b1-9c7f-e2f82d9ffae8 on /...
Kas 06 00:06:52 fridge kernel: EXT4-fs (sda1): mounted filesystem with ordered data mode. Quota mode: none.
Kas 06 00:06:52 fridge kernel: EXT4-fs (sda1): re-mounted. Quota mode: none.
Kas 06 00:06:52 fridge kernel: EXT4-fs (sda1): re-mounted. Quota mode: none.
That certainly seems like you’re ok, but it’s impossible to say with a file system that doesn’t do data checksumming like ext4. Definitely run nix-store --verify --check-contents to ensure the integrity of the Nix store.
Yep, some broken files inside nix store was the first reason I started looking into FS. There were 3 different hash mismatches. nix-store command fixed everything after fsck. Also, subsequential fsck command returned clean result. All good, again thank you.