Failed to update systemd-boot on NixOS due to 'Resource temporarily unavailable'

Hi everyone! I’m new to the NixOS Discourse community and would greatly appreciate your patience and guidance. Thank you in advance for any help!

Issue

During nixos-rebuild switch on nixos-unstable, upgrading systemd 257.2 to 257.3 , systemd-boot update fails with Resource temporarily unavailable when copying EFI files to the /boot partition (vFAT-formatted).

$ ssh -o ControlMaster=auto -o ControlPath=/tmp/nixos-rebuild.2CIU0o/ssh-%n -o ControlPersist=60 -t azurcrystal@nixos sudo systemd-run -E LOCALE_ARCHIVE -E NIXOS_INSTALL_BOOTLOADER= --collect --no-ask-password --pipe --quiet --service-type=exec --unit=nixos-rebuild-switch-to-configuration --wait /nix/store/0ijv5mrhgd0xpm6fl55cxcxgidh9xa4m-nixos-system-NixOS-25.05.20250318.b6eaf97/bin/switch-to-configuration boot
updating systemd-boot from 257.2 to 257.3
Failed to copy data from "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/systemd/.#systemd-bootx64.efi4716ff31e75287f2": Resource temporarily unavailable
Failed to copy data from "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/BOOT/.#BOOTX64.EFI9a7e5d75ea1862f6": Resource temporarily unavailable
Traceback (most recent call last):
  File "/nix/store/654xnpff3rl6brn73nnw7j06r1lvfi9l-systemd-boot/bin/systemd-boot", line 431, in <module>
    main()
  File "/nix/store/654xnpff3rl6brn73nnw7j06r1lvfi9l-systemd-boot/bin/systemd-boot", line 414, in main
    install_bootloader(args)
  File "/nix/store/654xnpff3rl6brn73nnw7j06r1lvfi9l-systemd-boot/bin/systemd-boot", line 348, in install_bootloader
    run(
  File "/nix/store/654xnpff3rl6brn73nnw7j06r1lvfi9l-systemd-boot/bin/systemd-boot", line 58, in run
    return subprocess.run(cmd, check=True, text=True, stdout=stdout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/wz0j2zi02rvnjiz37nn28h3gfdq61svz-python3-3.12.9/lib/python3.12/subprocess.py", line 573, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/bin/bootctl', '--esp-path=/boot', 'update']' returned non-zero exit status 1.
Failed to install bootloader
Shared connection to nixos closed.
warning: error(s) occurred while switching to the new configura

Manual attempts (via bootctl update ) reproduce the same error.

[root@NixOS:~]# /nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/bin/bootctl --esp-path=/boot update
Failed to copy data from "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/systemd/.#systemd-bootx64.efi70651a65779fe95e": Resource temporarily unavailable
Failed to copy data from "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/BOOT/.#BOOTX64.EFI68876412343aef63": Resource temporarily unavailable

And so do nixos-rebuild switch --install-bootloader even with the bootloader I locked last month.

nixos-rebuild switch --install-bootloader with systemd 257.2 (version by now)
$ ssh -o ControlMaster=auto -o ControlPath=/tmp/nixos-rebuild.Jjvcge/ssh-%n -o ControlPersist=60 -t azurcrystal@nixos sudo systemd-run -E LOCALE_ARCHIVE -E NIXOS_INSTALL_BOOTLOADER=1 --collect --no-ask-password --pipe --quiet --service-type=exec --unit=nixos-rebuild-switch-to-configuration --wait /nix/store/5538pga92q679v6j0zf3wpad9qhhajdr-nixos-system-NixOS-25.05.20250307.36fd87b/bin/switch-to-configuration switch
Failed to copy data from "/nix/store/83a0wfzmb0v5bzr5bv0zvx14g733x3dv-systemd-257.2/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/systemd/.#systemd-bootx64.efid56ff724770b3466": Resource temporarily unavailable
Failed to copy data from "/nix/store/83a0wfzmb0v5bzr5bv0zvx14g733x3dv-systemd-257.2/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/BOOT/.#BOOTX64.EFI2dc066fa25b339b2": Resource temporarily unavailable
Traceback (most recent call last):
  File "/nix/store/j1d03dl4zjf038zjzyrk7r5r6ch5ik1s-systemd-boot/bin/systemd-boot", line 431, in <module>
    main()
  File "/nix/store/j1d03dl4zjf038zjzyrk7r5r6ch5ik1s-systemd-boot/bin/systemd-boot", line 414, in main
    install_bootloader(args)
  File "/nix/store/j1d03dl4zjf038zjzyrk7r5r6ch5ik1s-systemd-boot/bin/systemd-boot", line 307, in install_bootloader
    run(
  File "/nix/store/j1d03dl4zjf038zjzyrk7r5r6ch5ik1s-systemd-boot/bin/systemd-boot", line 58, in run
    return subprocess.run(cmd, check=True, text=True, stdout=stdout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/26yi95240650jxp5dj78xzch70i1kzlz-python3-3.12.9/lib/python3.12/subprocess.py", line 573, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/nix/store/83a0wfzmb0v5bzr5bv0zvx14g733x3dv-systemd-257.2/bin/bootctl', '--esp-path=/boot', '--graceful', 'install']' returned non-zero exit status 1.
Failed to install bootloader
Shared connection to nixos closed.
warning: error(s) occurred while switching to the new configuration

Details

I suffered this a month ago 2025-02-10T00:00:00Z when I first try installing nixos via nixos-unstable iso, while I’m using disko to declare my disk paratitions.

part of related disko-config.nix
# disko-config.nix
{
 disko.devices = {
    disk = {
      main = {
        device = "/dev/disk/by-id/nvme-CT1000P3PSSD8_241247EE3C3D";
        type = "disk";
        content = {
          type = "gpt";
          partitions = {
            ESP = {
              size = "1G";
              type = "EF00";
              content = {
                type = "filesystem";
                format = "vfat";
                mountpoint = "/boot";
                mountOptions = [ "umask=0077" ];
              };
            };
            zfs = {
              size = "100%";
              content = {
                type = "zfs";
                pool = "rpool";
              };
            };
          };
        };
      };
    };
   # zpool configuration
   ...
}
part of related configuration.nix
# configuration.nix
  ...
  boot.loader = {
    systemd-boot = {
      enable = true;
    };
    efi.canTouchEfiVariables = true;
  };

  boot.initrd.systemd.enable = true;
  boot.initrd.network.enable = true;
  system.stateVersion = "24.11";
  ...
  • At that time I got the same error like now (systemd 257.2).
  • I tried to format mannually without disko, but bootloader installation also failed.
  • I tried to install nixos-24.11 with disko instead, and it works.
  • I did upgrade flake inputs to unstable and nixos-rebuild switch to switch system from 24.11 to unstable, and it succeed without error.
  • However, this occured while I upgrading systemd this month, with a force-install systemd-boot.
  • boot.loader.systemd-boot.graceful = true cannot solve this.

More details below here.

mount & df & fsck
[root@NixOS:~]# mount | grep /boot
/dev/nvme0n1p1 on /boot type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)

[root@NixOS:~]# df -h /boot
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p1 1022M  476M  547M  47% /boot

[root@NixOS:~]# fsck.vfat -a /dev/nvme0n1p1
fsck.fat 4.2 (2021-01-31)
There are differences between boot sector and its backup.
This is mostly harmless. Differences: (offset:original/backup)
  65:01/00
  Not automatically fixing this.
Dirty bit is set. Fs was not properly unmounted and some data may be corrupt.
 Automatically removing dirty bit.

*** Filesystem was changed ***
Writing changes.
/dev/nvme0n1p1: 178 files, 121627/261600 clusters

There’s strace for bootctl update.

strace details
[root@NixOS:~]# strace -f -e trace=file /nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/bin/bootctl --esp-path=/boot update
...
...
openat(AT_FDCWD, "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/kernel/install.conf", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 3
openat(3, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
openat(3, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
openat(3, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
openat(4, "etc", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 6
openat(6, "kernel", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = -1 ENOENT (No such file or directory)
openat(3, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
openat(3, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
openat(3, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
openat(4, "usr", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 6
openat(6, "lib", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/sys/firmware/efi/fw_platform_size", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 3
openat(3, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
openat(3, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
openat(4, "nix", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 6
openat(6, "store", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 4
openat(4, "57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 6
openat(6, "lib", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 4
openat(4, "systemd", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 6
openat(6, "boot", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 4
openat(4, "efi", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 6
openat(6, ".", O_RDONLY|O_NONBLOCK|O_NOFOLLOW|O_CLOEXEC|O_DIRECTORY) = 3
faccessat2(3, "systemd-bootx64.efi.signed", F_OK, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
openat(4, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
statx(5, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
openat(4, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 6
openat(5, "nix", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
openat(7, "store", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 5
openat(5, "57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
openat(7, "lib", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 5
openat(5, "systemd", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
openat(7, "boot", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 5
openat(5, "efi", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
openat(7, "systemd-bootx64.efi", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 5
openat(AT_FDCWD, "/boot", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
openat(4, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=4096, ...}) = 0
statx(5, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
openat(AT_FDCWD, "/boot", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
openat(4, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=4096, ...}) = 0
statx(5, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
openat(4, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=4096, ...}) = 0
statx(5, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
openat(4, ".", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
openat(4, ".", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 6
openat(5, "EFI", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
openat(7, "systemd", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 5
openat(5, "systemd-bootx64.efi", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
openat(AT_FDCWD, "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi", O_RDONLY|O_NOCTTY|O_CLOEXEC) = 4
openat(AT_FDCWD, "/boot/EFI/systemd/systemd-bootx64.efi", O_RDONLY|O_NOCTTY|O_CLOEXEC) = 5
openat(AT_FDCWD, "/boot/EFI/systemd/.#systemd-bootx64.efi354b2388d162cb72", O_WRONLY|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0644) = 5
newfstatat(5, "", {st_mode=S_IFREG|0755, st_size=0, ...}, AT_SYMLINK_NOFOLLOW|AT_EMPTY_PATH) = 0
unlink("/boot/EFI/systemd/.#systemd-bootx64.efi354b2388d162cb72") = 0
Failed to copy data from "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/systemd/.#systemd-bootx64.efi354b2388d162cb72": Resource temporarily unavailable
openat(AT_FDCWD, "/boot", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
openat(4, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=4096, ...}) = 0
statx(5, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
openat(AT_FDCWD, "/boot", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
openat(4, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=4096, ...}) = 0
statx(5, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
openat(4, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=4096, ...}) = 0
statx(5, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
openat(4, ".", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
openat(4, ".", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 6
openat(5, "EFI", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
openat(7, "BOOT", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 5
openat(5, "BOOTX64.EFI", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
openat(AT_FDCWD, "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi", O_RDONLY|O_NOCTTY|O_CLOEXEC) = 4
openat(AT_FDCWD, "/boot/EFI/BOOT/BOOTX64.EFI", O_RDONLY|O_NOCTTY|O_CLOEXEC) = 5
openat(AT_FDCWD, "/boot/EFI/BOOT/.#BOOTX64.EFI54d6859dd11ba632", O_WRONLY|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0644) = 5
newfstatat(5, "", {st_mode=S_IFREG|0755, st_size=0, ...}, AT_SYMLINK_NOFOLLOW|AT_EMPTY_PATH) = 0
unlink("/boot/EFI/BOOT/.#BOOTX64.EFI54d6859dd11ba632") = 0
Failed to copy data from "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/BOOT/.#BOOTX64.EFI54d6859dd11ba632": Resource temporarily unavailable
+++ exited with 1 +++
  • I’m currently refactoring my entire NixOS configuration modules, so I’m unable to share the full configuration.nix on github (it contains some plain secrets) . If specific configuration snippets or details are needed, please let me know, and I’ll provide them as best as I can.
  • The system works perfectly (switch and boot) if there’s no bootloader updates.
  • This machine is critical for daily use, and I currently lack access to replacement hardware for reinstallation. Thus, solutions requiring a full reinstall are impractical for now.

Thank you all in advance for your time and expertise!

1 Like

did you just fsck a file system while it was mounted?

Do you use Windows? Every time I boot into Windows, the boot partition gets corrupted, so I deleted Windows…

No, I just umount it and did fsck.
The results of these commands are extracted from my previous debug notes, and I’m sure that the filesystem is not corrupted.
Anyway, I have wiped the whole vfat filesystem when I first met this issue…

No, this is a pure NixOS installation, without any other distributions and windows.

can you try SYSTEMD_LOG_LEVEL=debug bootctl install to get debug output?

[root@NixOS:/nix/store]# env SYSTEMD_LOG_LEVEL=debug /nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/bin/bootctl update --esp-path /boot
Found container virtualization none.
Using EFI System Partition at /boot.
Checking whether /boot/EFI/systemd/ contains any files…
/dev/nvme0n1p1: Partition has wrong PART_ENTRY_TYPE=c12a7328-f81f-11d2-ba4b-00a0c93ec93b for XBOOTLDR partition.
Didn't find an XBOOTLDR partition, using the ESP as $BOOT.
Loaded machine ID c2d741d4cee0460997435fd29fffa435 from /etc/machine-id.
Using entry token: c2d741d4cee0460997435fd29fffa435
EFI binary LoaderInfo marker: "systemd-boot 257.3"
EFI binary LoaderInfo marker: "systemd-boot 257.2"
Comparing versions: "systemd-boot 257.3" > "systemd-boot 257.2
Failed to copy data from "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/systemd/.#systemd-bootx64.efiafa890ee72ab0301": Resource temporarily unavailable
EFI binary LoaderInfo marker: "systemd-boot 257.3"
EFI binary LoaderInfo marker: "systemd-boot 257.2"
Comparing versions: "systemd-boot 257.3" > "systemd-boot 257.2
Failed to copy data from "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/BOOT/.#BOOTX64.EFI6e3c84a612134f00": Resource temporarily unavailable

[root@NixOS:/nix/store]# env SYSTEMD_LOG_LEVEL=debug /nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/bin/bootctl install --esp-path /boot
Found container virtualization none.
Using EFI System Partition at /boot.
/dev/nvme0n1p1: Partition has wrong PART_ENTRY_TYPE=c12a7328-f81f-11d2-ba4b-00a0c93ec93b for XBOOTLDR partition.
Didn't find an XBOOTLDR partition, using the ESP as $BOOT.
Loaded machine ID c2d741d4cee0460997435fd29fffa435 from /etc/machine-id.
Using entry token: c2d741d4cee0460997435fd29fffa435
Failed to copy data from "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/systemd/.#systemd-bootx64.efi913be74eec949f5b": Resource temporarily unavailable
Failed to copy data from "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/BOOT/.#BOOTX64.EFI42a50781d8ade3a6": Resource temporarily unavailable

use strace -f, don’t filter out other syscalls.

I’m sorry that due to the limitation of discourse (32000 characters), I cannot push full strace log on it. There’s last several lines of strace log that refer to the error.

openat(3, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
fstat(4, {st_mode=S_IFDIR|0755, st_size=19, ...}) = 0
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
openat(4, "usr", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 6
fstat(6, {st_mode=S_IFDIR|0755, st_size=3, ...}) = 0
close(4)                                = 0
openat(6, "lib", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = -1 ENOENT (No such file or directory)
close(5)                                = 0
close(6)                                = 0
close(3)                                = 0
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 3
openat(3, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
close(4)                                = 0
openat(3, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
fstat(4, {st_mode=S_IFDIR|0755, st_size=19, ...}) = 0
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
openat(4, "nix", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 6
fstat(6, {st_mode=S_IFDIR|0755, st_size=5, ...}) = 0
close(4)                                = 0
openat(6, "store", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 4
fstat(4, {st_mode=S_IFDIR|S_ISVTX|0775, st_size=19959, ...}) = 0
close(6)                                = 0
openat(4, "57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 6
fstat(6, {st_mode=S_IFDIR|0555, st_size=8, ...}) = 0
close(4)                                = 0
openat(6, "lib", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 4
fstat(4, {st_mode=S_IFDIR|0555, st_size=21, ...}) = 0
close(6)                                = 0
openat(4, "kernel", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = -1 ENOENT (No such file or directory)
close(5)                                = 0
close(4)                                = 0
close(3)                                = 0
openat(AT_FDCWD, "/etc/kernel/install.conf", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/run/kernel/install.conf", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/local/lib/kernel/install.conf", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/kernel/install.conf", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/kernel/install.conf", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 3
openat(3, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
close(4)                                = 0
openat(3, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
close(4)                                = 0
openat(3, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
fstat(4, {st_mode=S_IFDIR|0755, st_size=19, ...}) = 0
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
openat(4, "etc", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 6
fstat(6, {st_mode=S_IFDIR|0755, st_size=70, ...}) = 0
close(4)                                = 0
openat(6, "kernel", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = -1 ENOENT (No such file or directory)
close(5)                                = 0
close(6)                                = 0
openat(3, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
close(4)                                = 0
openat(3, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
close(4)                                = 0
openat(3, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
fstat(4, {st_mode=S_IFDIR|0755, st_size=19, ...}) = 0
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
openat(4, "usr", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 6
fstat(6, {st_mode=S_IFDIR|0755, st_size=3, ...}) = 0
close(4)                                = 0
openat(6, "lib", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = -1 ENOENT (No such file or directory)
close(5)                                = 0
close(6)                                = 0
close(3)                                = 0
openat(AT_FDCWD, "/sys/firmware/efi/fw_platform_size", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=4096, ...}) = 0
read(3, "64\n", 4096)                   = 3
ioctl(3, TCGETS, 0x7ffccfa6fee0)        = -1 ENOTTY (Inappropriate ioctl for device)
read(3, "", 4096)                       = 0
close(3)                                = 0
umask(002)                              = 022
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 3
openat(3, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
close(4)                                = 0
openat(3, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
fstat(4, {st_mode=S_IFDIR|0755, st_size=19, ...}) = 0
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
openat(4, "nix", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 6
fstat(6, {st_mode=S_IFDIR|0755, st_size=5, ...}) = 0
close(4)                                = 0
openat(6, "store", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 4
fstat(4, {st_mode=S_IFDIR|S_ISVTX|0775, st_size=19959, ...}) = 0
close(6)                                = 0
openat(4, "57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 6
fstat(6, {st_mode=S_IFDIR|0555, st_size=8, ...}) = 0
close(4)                                = 0
openat(6, "lib", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 4
fstat(4, {st_mode=S_IFDIR|0555, st_size=21, ...}) = 0
close(6)                                = 0
openat(4, "systemd", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 6
fstat(6, {st_mode=S_IFDIR|0555, st_size=97, ...}) = 0
close(4)                                = 0
openat(6, "boot", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 4
fstat(4, {st_mode=S_IFDIR|0555, st_size=3, ...}) = 0
close(6)                                = 0
openat(4, "efi", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 6
fstat(6, {st_mode=S_IFDIR|0555, st_size=5, ...}) = 0
close(4)                                = 0
close(5)                                = 0
close(3)                                = 0
openat(6, ".", O_RDONLY|O_NONBLOCK|O_NOFOLLOW|O_CLOEXEC|O_DIRECTORY) = 3
fstat(3, {st_mode=S_IFDIR|0555, st_size=5, ...}) = 0
fcntl(3, F_GETFL)                       = 0x38800 (flags O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_NOFOLLOW|O_DIRECTORY)
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
close(6)                                = 0
getdents64(3, 0x55b92682c640 /* 5 entries */, 32768) = 168
faccessat2(3, "systemd-bootx64.efi.signed", F_OK, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
openat(4, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
statx(5, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
close(5)                                = 0
openat(4, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
fstat(5, {st_mode=S_IFDIR|0755, st_size=19, ...}) = 0
openat(AT_FDCWD, "/", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 6
openat(5, "nix", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
fstat(7, {st_mode=S_IFDIR|0755, st_size=5, ...}) = 0
close(5)                                = 0
openat(7, "store", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 5
fstat(5, {st_mode=S_IFDIR|S_ISVTX|0775, st_size=19959, ...}) = 0
close(7)                                = 0
openat(5, "57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
fstat(7, {st_mode=S_IFDIR|0555, st_size=8, ...}) = 0
close(5)                                = 0
openat(7, "lib", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 5
fstat(5, {st_mode=S_IFDIR|0555, st_size=21, ...}) = 0
close(7)                                = 0
openat(5, "systemd", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
fstat(7, {st_mode=S_IFDIR|0555, st_size=97, ...}) = 0
close(5)                                = 0
openat(7, "boot", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 5
fstat(5, {st_mode=S_IFDIR|0555, st_size=3, ...}) = 0
close(7)                                = 0
openat(5, "efi", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
fstat(7, {st_mode=S_IFDIR|0555, st_size=5, ...}) = 0
close(5)                                = 0
openat(7, "systemd-bootx64.efi", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 5
fstat(5, {st_mode=S_IFREG|0444, st_size=138752, ...}) = 0
close(7)                                = 0
close(6)                                = 0
close(5)                                = 0
close(4)                                = 0
openat(AT_FDCWD, "/boot", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
openat(4, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=4096, ...}) = 0
statx(5, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
close(5)                                = 0
close(4)                                = 0
openat(AT_FDCWD, "/boot", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
openat(4, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=4096, ...}) = 0
statx(5, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
close(5)                                = 0
openat(4, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=4096, ...}) = 0
statx(5, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
close(5)                                = 0
openat(4, ".", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
fstat(5, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
openat(4, ".", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 6
openat(5, "EFI", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
fstat(7, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
close(5)                                = 0
openat(7, "systemd", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 5
fstat(5, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
close(7)                                = 0
openat(5, "systemd-bootx64.efi", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
fstat(7, {st_mode=S_IFREG|0755, st_size=138240, ...}) = 0
close(5)                                = 0
close(6)                                = 0
close(7)                                = 0
close(4)                                = 0
openat(AT_FDCWD, "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi", O_RDONLY|O_NOCTTY|O_CLOEXEC) = 4
openat(AT_FDCWD, "/boot/EFI/systemd/systemd-bootx64.efi", O_RDONLY|O_NOCTTY|O_CLOEXEC) = 5
fstat(4, {st_mode=S_IFREG|0444, st_size=138752, ...}) = 0
mmap(NULL, 138752, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7fcf865f5000
munmap(0x7fcf865f5000, 138752)          = 0
fstat(5, {st_mode=S_IFREG|0755, st_size=138240, ...}) = 0
mmap(NULL, 138240, PROT_READ, MAP_PRIVATE, 5, 0) = 0x7fcf865f5000
munmap(0x7fcf865f5000, 138240)          = 0
lseek(4, 0, SEEK_SET)                   = 0
close(5)                                = 0
getrandom("\x00\xd1\x0d\x61\xbc\x7b\x73\xb2", 8, GRND_INSECURE) = 8
umask(000)                              = 002
openat(AT_FDCWD, "/boot/EFI/systemd/.#systemd-bootx64.efib2737bbc610dd100", O_WRONLY|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0644) = 5
umask(002)                              = 000
fcntl(4, F_GETFL)                       = 0x8000 (flags O_RDONLY|O_LARGEFILE)
fcntl(5, F_GETFL)                       = 0x28001 (flags O_WRONLY|O_LARGEFILE|O_NOFOLLOW)
lseek(4, 0, SEEK_CUR)                   = 0
lseek(5, 0, SEEK_CUR)                   = 0
newfstatat(5, "", {st_mode=S_IFREG|0755, st_size=0, ...}, AT_SYMLINK_NOFOLLOW|AT_EMPTY_PATH) = 0
ioctl(5, BTRFS_IOC_CLONE or FICLONE, 4) = -1 EXDEV (Invalid cross-device link)
copy_file_range(4, NULL, 5, NULL, 9223372036854775807, 0) = -1 EXDEV (Invalid cross-device link)
sendfile(5, 4, NULL, 9223372036854775807) = 131072
sendfile(5, 4, NULL, 9223372036854644735) = -1 EAGAIN (Resource temporarily unavailable)
unlink("/boot/EFI/systemd/.#systemd-bootx64.efib2737bbc610dd100") = 0
ioctl(1, TCGETS, {c_iflag=ICRNL|IXON, c_oflag=NL0|CR0|TAB0|BS0|VT0|FF0|OPOST|ONLCR, c_cflag=B38400|CS8|CREAD, c_lflag=ISIG|ICANON|ECHO|ECHOE|ECHOK|IEXTEN|ECHOCTL|ECHOKE, ...}) = 0
ioctl(2, TCGETS, {c_iflag=ICRNL|IXON, c_oflag=NL0|CR0|TAB0|BS0|VT0|FF0|OPOST|ONLCR, c_cflag=B38400|CS8|CREAD, c_lflag=ISIG|ICANON|ECHO|ECHOE|ECHOK|IEXTEN|ECHOCTL|ECHOKE, ...}) = 0
ioctl(2, TCGETS, {c_iflag=ICRNL|IXON, c_oflag=NL0|CR0|TAB0|BS0|VT0|FF0|OPOST|ONLCR, c_cflag=B38400|CS8|CREAD, c_lflag=ISIG|ICANON|ECHO|ECHOE|ECHOK|IEXTEN|ECHOCTL|ECHOKE, ...}) = 0
writev(2, [{iov_base="\33[0;1;31m", iov_len=9}, {iov_base="Failed to copy data from \"/nix/s"..., iov_len=220}, {iov_base="\33[0m", iov_len=4}, {iov_base="\r\n", iov_len=2}], 4Failed to copy data from "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/systemd/.#systemd-bootx64.efib2737bbc610dd100": Resource temporarily unavailable
) = 235
close(5)                                = 0
close(4)                                = 0
openat(AT_FDCWD, "/boot", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
openat(4, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=4096, ...}) = 0
statx(5, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
close(5)                                = 0
close(4)                                = 0
openat(AT_FDCWD, "/boot", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
openat(4, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=4096, ...}) = 0
statx(5, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
close(5)                                = 0
openat(4, "..", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=4096, ...}) = 0
statx(5, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_TYPE|STATX_INO|STATX_MNT_ID, {stx_mask=STATX_TYPE|STATX_MODE|STATX_NLINK|STATX_UID|STATX_GID|STATX_MTIME|STATX_CTIME|STATX_INO|STATX_SIZE|STATX_BLOCKS|STATX_MNT_ID, stx_attributes=STATX_ATTR_MOUNT_ROOT, stx_mode=S_IFDIR|0755, stx_size=19, ...}) = 0
close(5)                                = 0
openat(4, ".", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 5
fstat(5, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
openat(4, ".", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 6
openat(5, "EFI", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
fstat(7, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
close(5)                                = 0
openat(7, "BOOT", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 5
fstat(5, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
close(7)                                = 0
openat(5, "BOOTX64.EFI", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_PATH) = 7
fstat(7, {st_mode=S_IFREG|0755, st_size=138240, ...}) = 0
close(5)                                = 0
close(6)                                = 0
close(7)                                = 0
close(4)                                = 0
openat(AT_FDCWD, "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi", O_RDONLY|O_NOCTTY|O_CLOEXEC) = 4
openat(AT_FDCWD, "/boot/EFI/BOOT/BOOTX64.EFI", O_RDONLY|O_NOCTTY|O_CLOEXEC) = 5
fstat(4, {st_mode=S_IFREG|0444, st_size=138752, ...}) = 0
mmap(NULL, 138752, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7fcf865f5000
munmap(0x7fcf865f5000, 138752)          = 0
fstat(5, {st_mode=S_IFREG|0755, st_size=138240, ...}) = 0
mmap(NULL, 138240, PROT_READ, MAP_PRIVATE, 5, 0) = 0x7fcf865f5000
munmap(0x7fcf865f5000, 138240)          = 0
lseek(4, 0, SEEK_SET)                   = 0
close(5)                                = 0
getrandom("\x3a\x0c\x89\xb7\xab\x98\x1f\xa2", 8, GRND_INSECURE) = 8
umask(000)                              = 002
openat(AT_FDCWD, "/boot/EFI/BOOT/.#BOOTX64.EFIa21f98abb7890c3a", O_WRONLY|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0644) = 5
umask(002)                              = 000
fcntl(4, F_GETFL)                       = 0x8000 (flags O_RDONLY|O_LARGEFILE)
fcntl(5, F_GETFL)                       = 0x28001 (flags O_WRONLY|O_LARGEFILE|O_NOFOLLOW)
lseek(4, 0, SEEK_CUR)                   = 0
lseek(5, 0, SEEK_CUR)                   = 0
newfstatat(5, "", {st_mode=S_IFREG|0755, st_size=0, ...}, AT_SYMLINK_NOFOLLOW|AT_EMPTY_PATH) = 0
ioctl(5, BTRFS_IOC_CLONE or FICLONE, 4) = -1 EXDEV (Invalid cross-device link)
copy_file_range(4, NULL, 5, NULL, 9223372036854775807, 0) = -1 EXDEV (Invalid cross-device link)
sendfile(5, 4, NULL, 9223372036854775807) = 131072
sendfile(5, 4, NULL, 9223372036854644735) = -1 EAGAIN (Resource temporarily unavailable)
unlink("/boot/EFI/BOOT/.#BOOTX64.EFIa21f98abb7890c3a") = 0
writev(2, [{iov_base="\33[0;1;31m", iov_len=9}, {iov_base="Failed to copy data from \"/nix/s"..., iov_len=209}, {iov_base="\33[0m", iov_len=4}, {iov_base="\r\n", iov_len=2}], 4Failed to copy data from "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi" to "/boot/EFI/BOOT/.#BOOTX64.EFIa21f98abb7890c3a": Resource temporarily unavailable
) = 224
close(5)                                = 0
close(4)                                = 0
getdents64(3, 0x55b92682c640 /* 0 entries */, 32768) = 0
close(3)                                = 0
umask(0170022)                          = 002
exit_group(1)                           = ?
+++ exited with 1 +++

systemd-bootctl has trouble copying some files.

The syscall sendfile returned EAGAIN after copying 128KB of data. I’m unsure why it returned EAGAIN.

sendfile(5, 4, NULL, 9223372036854775807) = 131072
sendfile(5, 4, NULL, 9223372036854644735) = -1 EAGAIN (Resource temporarily unavailable)

From sendfile(2):

EAGAIN Nonblocking I/O has been selected using O_NONBLOCK and the write would block.

Since it’s blocking, bootctl doesn’t check this error code and retry the operation.

Try copying with cp:

$ cp /nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi /boot/EFI/BOOT/test0
$ cp /nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi /boot/EFI/systemd/test0

If copying files to a different filesystem, cp will use the read and write syscalls (I have verified this with strace).

If cp succeeds, this issue might be a problem with bootctl.

1 Like

well that certainly sounds like an upstream systemd bug for us to report, I guess

1 Like

Directly run cp to copy from /nix/store reports no error:

[root@NixOS:~]# cp -v /nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi
 /boot/EFI/BOOT/test0
'/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi' -> '/boot/EFI/BOOT/test0'

[root@NixOS:~]# ls -la /boot/EFI/BOOT/
total 280
drwxr-xr-x 2 root root   4096 Mar 23 12:42 .
drwxr-xr-x 6 root root   4096 Feb 13 19:22 ..
-rwxr-xr-x 1 root root 138240 Jan  1  1980 BOOTX64.EFI
-rwxr-xr-x 1 root root 138752 Mar 23 12:44 test0
[root@NixOS:~]# cp -v /nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi
 /boot/EFI/systemd/test0
'/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi' -> '/boot/EFI/systemd/test0'

[root@NixOS:~]# ls -lah /boot/EFI/systemd
total 280K
drwxr-xr-x 2 root root 4.0K Mar 23 12:50 .
drwxr-xr-x 6 root root 4.0K Feb 13 19:22 ..
-rwxr-xr-x 1 root root 135K Jan  1  1980 systemd-bootx64.efi
-rwxr-xr-x 1 root root 136K Mar 23 12:50 test0
strace -f cp ...
[root@NixOS:~]# strace -f cp /nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi /boot/EFI/BOOT/test0
execve("/run/current-system/sw/bin/cp", ["cp", "/nix/store/57rayvb28n66prgyavcbz"..., "/boot/EFI/BOOT/test0"], 0x7ffcddd1a898 /* 39 vars */) = 0
brk(NULL)                               = 0x29521000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9781194000
access("/etc/ld-nix.so.preload", R_OK)  = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/jpbbkpwhgvdr0sgf6x9iv9ysrhd5416x-acl-2.3.2/lib/glibc-hwcaps/x86-64-v3/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/jpbbkpwhgvdr0sgf6x9iv9ysrhd5416x-acl-2.3.2/lib/glibc-hwcaps/x86-64-v3/", 0x7fff100a0550, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/jpbbkpwhgvdr0sgf6x9iv9ysrhd5416x-acl-2.3.2/lib/glibc-hwcaps/x86-64-v2/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/jpbbkpwhgvdr0sgf6x9iv9ysrhd5416x-acl-2.3.2/lib/glibc-hwcaps/x86-64-v2/", 0x7fff100a0550, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/jpbbkpwhgvdr0sgf6x9iv9ysrhd5416x-acl-2.3.2/lib/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/jpbbkpwhgvdr0sgf6x9iv9ysrhd5416x-acl-2.3.2/lib/", {st_mode=S_IFDIR|0555, st_size=6, ...}, 0) = 0
openat(AT_FDCWD, "/nix/store/xqx1wf2ycqam9mcs80i7xdj4j65hqd34-attr-2.5.2/lib/glibc-hwcaps/x86-64-v3/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/xqx1wf2ycqam9mcs80i7xdj4j65hqd34-attr-2.5.2/lib/glibc-hwcaps/x86-64-v3/", 0x7fff100a0550, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/xqx1wf2ycqam9mcs80i7xdj4j65hqd34-attr-2.5.2/lib/glibc-hwcaps/x86-64-v2/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/xqx1wf2ycqam9mcs80i7xdj4j65hqd34-attr-2.5.2/lib/glibc-hwcaps/x86-64-v2/", 0x7fff100a0550, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/xqx1wf2ycqam9mcs80i7xdj4j65hqd34-attr-2.5.2/lib/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/xqx1wf2ycqam9mcs80i7xdj4j65hqd34-attr-2.5.2/lib/", {st_mode=S_IFDIR|0555, st_size=6, ...}, 0) = 0
openat(AT_FDCWD, "/nix/store/yr1w7yzbqqj30a7qnn1yhzn2a2jb6h7h-gmp-with-cxx-6.3.0/lib/glibc-hwcaps/x86-64-v3/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/yr1w7yzbqqj30a7qnn1yhzn2a2jb6h7h-gmp-with-cxx-6.3.0/lib/glibc-hwcaps/x86-64-v3/", 0x7fff100a0550, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/yr1w7yzbqqj30a7qnn1yhzn2a2jb6h7h-gmp-with-cxx-6.3.0/lib/glibc-hwcaps/x86-64-v2/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/yr1w7yzbqqj30a7qnn1yhzn2a2jb6h7h-gmp-with-cxx-6.3.0/lib/glibc-hwcaps/x86-64-v2/", 0x7fff100a0550, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/yr1w7yzbqqj30a7qnn1yhzn2a2jb6h7h-gmp-with-cxx-6.3.0/lib/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/yr1w7yzbqqj30a7qnn1yhzn2a2jb6h7h-gmp-with-cxx-6.3.0/lib/", {st_mode=S_IFDIR|0555, st_size=10, ...}, 0) = 0
openat(AT_FDCWD, "/nix/store/z2g8g76mw8xkyniz5yxnrn8r0gpdckzy-openssl-3.4.1/lib/glibc-hwcaps/x86-64-v3/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/z2g8g76mw8xkyniz5yxnrn8r0gpdckzy-openssl-3.4.1/lib/glibc-hwcaps/x86-64-v3/", 0x7fff100a0550, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/z2g8g76mw8xkyniz5yxnrn8r0gpdckzy-openssl-3.4.1/lib/glibc-hwcaps/x86-64-v2/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/z2g8g76mw8xkyniz5yxnrn8r0gpdckzy-openssl-3.4.1/lib/glibc-hwcaps/x86-64-v2/", 0x7fff100a0550, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/z2g8g76mw8xkyniz5yxnrn8r0gpdckzy-openssl-3.4.1/lib/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0555, st_size=6630688, ...}) = 0
mmap(NULL, 5957328, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9780be5000
mmap(0x7f9780cb2000, 3534848, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xcd000) = 0x7f9780cb2000
mmap(0x7f9781011000, 1146880, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x42c000) = 0x7f9781011000
mmap(0x7f9781129000, 425984, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x543000) = 0x7f9781129000
mmap(0x7f9781191000, 9936, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f9781191000
close(3)                                = 0
openat(AT_FDCWD, "/nix/store/jpbbkpwhgvdr0sgf6x9iv9ysrhd5416x-acl-2.3.2/lib/libacl.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0555, st_size=44416, ...}) = 0
mmap(NULL, 41008, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9780bda000
mmap(0x7f9780bdc000, 20480, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f9780bdc000
mmap(0x7f9780be1000, 8192, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x7f9780be1000
mmap(0x7f9780be3000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x8000) = 0x7f9780be3000
close(3)                                = 0
openat(AT_FDCWD, "/nix/store/jpbbkpwhgvdr0sgf6x9iv9ysrhd5416x-acl-2.3.2/lib/libattr.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/xqx1wf2ycqam9mcs80i7xdj4j65hqd34-attr-2.5.2/lib/libattr.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0555, st_size=30632, ...}) = 0
mmap(NULL, 28696, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9780bd2000
mmap(0x7f9780bd4000, 12288, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f9780bd4000
mmap(0x7f9780bd7000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x5000) = 0x7f9780bd7000
mmap(0x7f9780bd8000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x5000) = 0x7f9780bd8000
close(3)                                = 0
openat(AT_FDCWD, "/nix/store/jpbbkpwhgvdr0sgf6x9iv9ysrhd5416x-acl-2.3.2/lib/libgmp.so.10", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/xqx1wf2ycqam9mcs80i7xdj4j65hqd34-attr-2.5.2/lib/libgmp.so.10", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/yr1w7yzbqqj30a7qnn1yhzn2a2jb6h7h-gmp-with-cxx-6.3.0/lib/libgmp.so.10", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0555, st_size=746072, ...}) = 0
mmap(NULL, 696840, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9780b27000
mmap(0x7f9780b39000, 520192, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x12000) = 0x7f9780b39000
mmap(0x7f9780bb8000, 94208, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x91000) = 0x7f9780bb8000
mmap(0x7f9780bcf000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xa8000) = 0x7f9780bcf000
close(3)                                = 0
openat(AT_FDCWD, "/nix/store/jpbbkpwhgvdr0sgf6x9iv9ysrhd5416x-acl-2.3.2/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/xqx1wf2ycqam9mcs80i7xdj4j65hqd34-attr-2.5.2/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/yr1w7yzbqqj30a7qnn1yhzn2a2jb6h7h-gmp-with-cxx-6.3.0/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/z2g8g76mw8xkyniz5yxnrn8r0gpdckzy-openssl-3.4.1/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/glibc-hwcaps/x86-64-v3/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/glibc-hwcaps/x86-64-v3/", 0x7fff100a04d0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/glibc-hwcaps/x86-64-v2/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/glibc-hwcaps/x86-64-v2/", 0x7fff100a04d0, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\243\2\0\0\0\0\0"..., 832) = 832
pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784
fstat(3, {st_mode=S_IFREG|0555, st_size=2388416, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9780b25000
pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784
mmap(NULL, 2121176, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f978091f000
mmap(0x7f9780947000, 1527808, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x28000) = 0x7f9780947000
mmap(0x7f9780abc000, 352256, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x19d000) = 0x7f9780abc000
mmap(0x7f9780b12000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1f2000) = 0x7f9780b12000
mmap(0x7f9780b18000, 52696, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f9780b18000
close(3)                                = 0
openat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0555, st_size=15696, ...}) = 0
mmap(NULL, 16400, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f978091a000
mmap(0x7f978091b000, 4096, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7f978091b000
mmap(0x7f978091c000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f978091c000
mmap(0x7f978091d000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f978091d000
close(3)                                = 0
openat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\0\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0555, st_size=16544, ...}) = 0
mmap(NULL, 16400, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9780915000
mmap(0x7f9780916000, 4096, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7f9780916000
mmap(0x7f9780917000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f9780917000
mmap(0x7f9780918000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f9780918000
close(3)                                = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9780913000
arch_prctl(ARCH_SET_FS, 0x7f9780913b80) = 0
set_tid_address(0x7f9780913e50)         = 57279
set_robust_list(0x7f9780913e60, 24)     = 0
rseq(0x7f97809144a0, 0x20, 0, 0x53053053) = 0
mprotect(0x7f9780b12000, 16384, PROT_READ) = 0
mprotect(0x7f9780918000, 4096, PROT_READ) = 0
mprotect(0x7f978091d000, 4096, PROT_READ) = 0
mprotect(0x7f9780bcf000, 8192, PROT_READ) = 0
mprotect(0x7f9780bd8000, 4096, PROT_READ) = 0
mprotect(0x7f9780be3000, 4096, PROT_READ) = 0
mprotect(0x7f9781129000, 413696, PROT_READ) = 0
mprotect(0x53d000, 53248, PROT_READ)    = 0
mprotect(0x7f97811d2000, 8192, PROT_READ) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
prctl(PR_SET_NAME, "cp")                = 0
prctl(PR_SET_MM, PR_SET_MM_ARG_START, 0x7fff100a18f9, 0, 0) = 0
getrandom("\xe7\x60\x13\x06\x39\xa4\x98\xb7", 8, GRND_NONBLOCK) = 8
brk(NULL)                               = 0x29521000
brk(0x29542000)                         = 0x29542000
openat(AT_FDCWD, "/run/current-system/sw/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=3061520, ...}) = 0
mmap(NULL, 3061520, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f9780627000
close(3)                                = 0
geteuid()                               = 0
openat(AT_FDCWD, "/boot/EFI/BOOT/test0", O_RDONLY|O_PATH|O_DIRECTORY) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi", {st_mode=S_IFREG|0444, st_size=138752, ...}, 0) = 0
openat(AT_FDCWD, "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=138752, ...}) = 0
openat(AT_FDCWD, "/boot/EFI/BOOT/test0", O_WRONLY|O_CREAT|O_EXCL, 0444) = 4
ioctl(4, BTRFS_IOC_CLONE or FICLONE, 3) = -1 EXDEV (Invalid cross-device link)
fstat(4, {st_mode=S_IFREG|0755, st_size=0, ...}) = 0
lseek(3, 0, SEEK_DATA)                  = 0
fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0
lseek(3, 0, SEEK_HOLE)                  = 138752
lseek(3, 0, SEEK_SET)                   = 0
uname({sysname="Linux", nodename="NixOS", ...}) = 0
copy_file_range(3, NULL, 4, NULL, 138752, 0) = -1 EXDEV (Invalid cross-device link)
mmap(NULL, 270336, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f97805e5000
read(3, "MZ\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 138752) = 138752
write(4, "MZ\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 138752) = 138752
lseek(3, 138752, SEEK_DATA)             = -1 ENXIO (No such device or address)
close(4)                                = 0
close(3)                                = 0
munmap(0x7f97805e5000, 270336)          = 0
lseek(0, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
close(0)                                = 0
close(1)                                = 0
close(2)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++

I doubt about a few things when first encounting this issue:

  • Disko
    • This was my first time using Disko for partitioning and installing NixOS.
  • SSD (Crucial P3 Plus)
    • This was also my first time formatting 4k sector for my NVMe disk.
  • systemd
smartctl -a /dev/nvme0n1
[nix-shell:~]# smartctl -a /dev/nvme0n1
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.12.17] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       CT1000P3PSSD8
Serial Number:                      241247EE3C3D
Firmware Version:                   P9CR413
PCI Vendor/Subsystem ID:            0x1344
IEEE OUI Identifier:                0x00a075
Controller ID:                      0
NVMe Version:                       1.4
Number of Namespaces:               1
Namespace 1 Size/Capacity:          1,000,204,886,016 [1.00 TB]
Namespace 1 Formatted LBA Size:     4096
Namespace 1 IEEE EUI-64:            00a075 0147ee3c3d
Local Time is:                      Sun Mar 23 13:04:27 2025 CST
Firmware Updates (0x14):            2 Slots, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x00d7):     Comp Wr_Unc DS_Mngmt Sav/Sel_Feat Timestmp Verify
Log Page Attributes (0x1e):         Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg Pers_Ev_Lg
Maximum Data Transfer Size:         64 Pages
Warning  Comp. Temp. Threshold:     83 Celsius
Critical Comp. Temp. Threshold:     85 Celsius
Namespace 1 Features (0x08):        No_ID_Reuse

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     5.50W       -        -    0  0  0  0        0       0
 1 +     3.00W       -        -    1  1  1  1        0       0
 2 +     1.50W       -        -    2  2  2  2        0       0
 3 -   0.0300W       -        -    3  3  3  3     5000    2500
 4 -   0.0025W       -        -    4  4  4  4     8000   40000

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 -     512       0         1
 1 +    4096       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        36 Celsius
Available Spare:                    100%
Available Spare Threshold:          5%
Percentage Used:                    0%
Data Units Read:                    1,012,823 [518 GB]
Data Units Written:                 1,476,221 [755 GB]
Host Read Commands:                 9,214,954
Host Write Commands:                25,180,858
Controller Busy Time:               117
Power Cycles:                       67
Power On Hours:                     728
Unsafe Shutdowns:                   19
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               36 Celsius

Error Information (NVMe Log 0x01, 16 of 255 entries)
No Errors Logged

Read Self-test Log failed: Invalid Field in Command (0x002)

I’d rather believe this is a pure systemd issue… but the most weird things are:

  • NixOS unstable systemd-boot failed, so I returned to 24.11
  • When I first install NixOS by 24.11 iso, nothing happened
  • The first upgrade from 24.11 to nixos unstable (systemd 257.2) succeed with no error
    • I cannot even reinstall the bootloader then (systemd 257.2)

I haven’t tested any other distribution and hardware yet… I’ll try to reproduce it on other device, and I’m also wondering is there any bypass solution to mannually upgrade systemd-boot without triggering NixOS force update?

Anyway, really thanks for your help in this tricky issue!

Does the issue happen every time now, or only sometimes?

Sometimes you get errors, and sometimes you don’t. This might be because sendfile sometimes succeeds without returnEAGAIN, but other times it fails with EAGAIN, systemd doesn’t check for this error code and thinks it’s a failure.

But the thing strange is: why would sendfile return EAGAIN? The file is opened in a default blocking mode (conformed by the source code and the outputs of strace), so it should not return EAGAIN(according to the man page). I think because the documentation doesn’t mention it will return EAGAIN in blocking mode, systemd doesn’t check this error code and thinks it’s a failure.

Thinking about broadly, if the documentation is not accurate, we might have bugs everywhere.

I wrote a test program to reproduce the issue. Its logical is similar to systemd-bootctl, but it checks EAGAIN error and retries the operation.

$ gcc test.c -o test
$ sudo strace ./test

Note that it will delete the file /boot/EFI/BOOT/test0 if it exists.

test.c
#include <errno.h>
#include <fcntl.h>
#include <limits.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/sendfile.h>
#include <sys/stat.h>
#include <unistd.h>

int main() {

  char *from = "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi";
  char *t = "/boot/EFI/BOOT/test0";

  int r = remove(t);
  if (r < 0) {
    if (errno != ENOENT) {
      fprintf(stderr, "Error removing %s: %s\n", t, strerror(errno));
      return 1;
    }
  }

  int fd_from = open(from, O_RDONLY | O_CLOEXEC | O_NOCTTY);
  if (fd_from < 0) {
    fprintf(stderr, "Error opening %s: %s\n", from, strerror(errno));
    return 1;
  }
  int fd_to =
      open(t, O_WRONLY | O_CREAT | O_CLOEXEC | O_EXCL | O_NOFOLLOW, 0644);
  if (fd_to < 0) {
    fprintf(stderr, "Error opening %s: %s\n", t, strerror(errno));
    close(fd_from);
    return 1;
  }

  long m = LONG_MAX;
  int retry = 3;
  for (;;) {
    ssize_t n = sendfile(fd_to, fd_from, NULL, m);
    if (n < 0) {
      if (errno == EAGAIN) {
        // usleep(100000);
        if (retry == 0) {
          fprintf(stderr, "Tried %d times\n", retry);
          close(fd_from);
          close(fd_to);
          return 1;
        }
        retry--;
        fprintf(stderr, "Warning: sendfile(): %s\n", strerror(errno));
        continue;
      }
      fprintf(stderr, "Error using sendfile() from %s to %s: %s\n", from, t,
              strerror(errno));
      close(fd_from);
      close(fd_to);
      return 1;
    } else if (n == 0) {
      close(fd_from);
      close(fd_to);
      return 0;
    }
    m -= n;
  }

  return 0;
}
1 Like

Every time.

Unfortunately, it breaks:

[nix-shell:~]# gcc test.c -o test

[nix-shell:~]# ls
test  test.c

[nix-shell:~]# sudo strace ./test
execve("./test", ["./test"], 0x7ffd3ae034d0 /* 27 vars */) = 0
brk(NULL)                               = 0x38a6f000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7faea8b18000
access("/etc/ld-nix.so.preload", R_OK)  = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/3f2mhd2w0a5r7ii310fbxxm6v1av2s1x-shell/lib/glibc-hwcaps/x86-64-v3/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/3f2mhd2w0a5r7ii310fbxxm6v1av2s1x-shell/lib/glibc-hwcaps/x86-64-v3/", 0x7fff0ddcff10, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/3f2mhd2w0a5r7ii310fbxxm6v1av2s1x-shell/lib/glibc-hwcaps/x86-64-v2/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/3f2mhd2w0a5r7ii310fbxxm6v1av2s1x-shell/lib/glibc-hwcaps/x86-64-v2/", 0x7fff0ddcff10, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/3f2mhd2w0a5r7ii310fbxxm6v1av2s1x-shell/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/3f2mhd2w0a5r7ii310fbxxm6v1av2s1x-shell/lib/", 0x7fff0ddcff10, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/glibc-hwcaps/x86-64-v3/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/glibc-hwcaps/x86-64-v3/", 0x7fff0ddcff10, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/glibc-hwcaps/x86-64-v2/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/glibc-hwcaps/x86-64-v2/", 0x7fff0ddcff10, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\243\2\0\0\0\0\0"..., 832) = 832
pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784
fstat(3, {st_mode=S_IFREG|0555, st_size=2388416, ...}) = 0
pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784
mmap(NULL, 2121176, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7faea8912000
mmap(0x7faea893a000, 1527808, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x28000) = 0x7faea893a000
mmap(0x7faea8aaf000, 352256, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x19d000) = 0x7faea8aaf000
mmap(0x7faea8b05000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1f2000) = 0x7faea8b05000
mmap(0x7faea8b0b000, 52696, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7faea8b0b000
close(3)                                = 0
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7faea890f000
arch_prctl(ARCH_SET_FS, 0x7faea890f740) = 0
set_tid_address(0x7faea890fa10)         = 10309
set_robust_list(0x7faea890fa20, 24)     = 0
rseq(0x7faea8910060, 0x20, 0, 0x53053053) = 0
mprotect(0x7faea8b05000, 16384, PROT_READ) = 0
mprotect(0x403000, 4096, PROT_READ)     = 0
mprotect(0x7faea8b56000, 8192, PROT_READ) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
unlink("/boot/EFI/BOOT/test0")          = 0
openat(AT_FDCWD, "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi", O_RDONLY|O_NOCTTY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/boot/EFI/BOOT/test0", O_WRONLY|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0644) = 4
sendfile(4, 3, NULL, 9223372036854775807) = -1 EAGAIN (Resource temporarily unavailable)
write(2, "Warning: sendfile(): Resource te"..., 54Warning: sendfile(): Resource temporarily unavailable
) = 54
sendfile(4, 3, NULL, 9223372036854775807) = -1 EAGAIN (Resource temporarily unavailable)
write(2, "Warning: sendfile(): Resource te"..., 54Warning: sendfile(): Resource temporarily unavailable
) = 54
sendfile(4, 3, NULL, 9223372036854775807) = -1 EAGAIN (Resource temporarily unavailable)
write(2, "Warning: sendfile(): Resource te"..., 54Warning: sendfile(): Resource temporarily unavailable
) = 54
sendfile(4, 3, NULL, 9223372036854775807) = -1 EAGAIN (Resource temporarily unavailable)
write(2, "Tried 0 times\n", 14Tried 0 times
)         = 14
close(3)                                = 0
close(4)                                = 0
exit_group(1)                           = ?
+++ exited with 1 +++

[nix-shell:~]# ./test
Warning: sendfile(): Resource temporarily unavailable
Warning: sendfile(): Resource temporarily unavailable
Warning: sendfile(): Resource temporarily unavailable
Tried 0 times

uncomment this line?

Also breaks…

[nix-shell:~]# sudo strace ./test
execve("./test", ["./test"], 0x7ffd7adc3130 /* 27 vars */) = 0
brk(NULL)                               = 0x340b1000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f09b9c0b000
access("/etc/ld-nix.so.preload", R_OK)  = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/3f2mhd2w0a5r7ii310fbxxm6v1av2s1x-shell/lib/glibc-hwcaps/x86-64-v3/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/3f2mhd2w0a5r7ii310fbxxm6v1av2s1x-shell/lib/glibc-hwcaps/x86-64-v3/", 0x7fff586b1a00, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/3f2mhd2w0a5r7ii310fbxxm6v1av2s1x-shell/lib/glibc-hwcaps/x86-64-v2/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/3f2mhd2w0a5r7ii310fbxxm6v1av2s1x-shell/lib/glibc-hwcaps/x86-64-v2/", 0x7fff586b1a00, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/3f2mhd2w0a5r7ii310fbxxm6v1av2s1x-shell/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/3f2mhd2w0a5r7ii310fbxxm6v1av2s1x-shell/lib/", 0x7fff586b1a00, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/glibc-hwcaps/x86-64-v3/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/glibc-hwcaps/x86-64-v3/", 0x7fff586b1a00, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/glibc-hwcaps/x86-64-v2/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/glibc-hwcaps/x86-64-v2/", 0x7fff586b1a00, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\243\2\0\0\0\0\0"..., 832) = 832
pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784
fstat(3, {st_mode=S_IFREG|0555, st_size=2388416, ...}) = 0
pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784
mmap(NULL, 2121176, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f09b9a05000
mmap(0x7f09b9a2d000, 1527808, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x28000) = 0x7f09b9a2d000
mmap(0x7f09b9ba2000, 352256, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x19d000) = 0x7f09b9ba2000
mmap(0x7f09b9bf8000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1f2000) = 0x7f09b9bf8000
mmap(0x7f09b9bfe000, 52696, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f09b9bfe000
close(3)                                = 0
mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f09b9a02000
arch_prctl(ARCH_SET_FS, 0x7f09b9a02740) = 0
set_tid_address(0x7f09b9a02a10)         = 10590
set_robust_list(0x7f09b9a02a20, 24)     = 0
rseq(0x7f09b9a03060, 0x20, 0, 0x53053053) = 0
mprotect(0x7f09b9bf8000, 16384, PROT_READ) = 0
mprotect(0x403000, 4096, PROT_READ)     = 0
mprotect(0x7f09b9c49000, 8192, PROT_READ) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
unlink("/boot/EFI/BOOT/test0")          = 0
openat(AT_FDCWD, "/nix/store/57rayvb28n66prgyavcbz2kk1qv3zydw-systemd-257.3/lib/systemd/boot/efi/systemd-bootx64.efi", O_RDONLY|O_NOCTTY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/boot/EFI/BOOT/test0", O_WRONLY|O_CREAT|O_EXCL|O_NOFOLLOW|O_CLOEXEC, 0644) = 4
sendfile(4, 3, NULL, 9223372036854775807) = -1 EAGAIN (Resource temporarily unavailable)
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=100000000}, NULL) = 0
write(2, "Warning: sendfile(): Resource te"..., 54Warning: sendfile(): Resource temporarily unavailable
) = 54
sendfile(4, 3, NULL, 9223372036854775807) = -1 EAGAIN (Resource temporarily unavailable)
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=100000000}, NULL) = 0
write(2, "Warning: sendfile(): Resource te"..., 54Warning: sendfile(): Resource temporarily unavailable
) = 54
sendfile(4, 3, NULL, 9223372036854775807) = -1 EAGAIN (Resource temporarily unavailable)
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=100000000}, NULL) = 0
write(2, "Warning: sendfile(): Resource te"..., 54Warning: sendfile(): Resource temporarily unavailable
) = 54
sendfile(4, 3, NULL, 9223372036854775807) = -1 EAGAIN (Resource temporarily unavailable)
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=100000000}, NULL) = 0
write(2, "Tried 0 times\n", 14Tried 0 times
)         = 14
close(3)                                = 0
close(4)                                = 0
exit_group(1)                           = ?
+++ exited with 1 +++

An unfortunate event for me, but a lucky one for this issue, is that this machine encountered even more serious problems (though unrelated to this topic), forcing me to take it down from my router setup and replace it with a backup router.

Now I can run more tests and debugging on this machine…