Failure when creating (big) .qcow file

Hello everyone!

I am trying to build >100gb .qcow files using Nix, but I get reliable failures.

When trying to build a VM for the mysystem configuration:

❯ nix build .#nixosConfigurations.mysystem.config.system.build.images.qemu

…it reliably fails:

error: builder for '/nix/store/a5jmljz6wk45k62kg2ff6pck8fjj549h-nixos-disk-image.drv' failed with exit code 1;
       last 25 log lines:
       > [    2.597122] Mem-Info:
       > [    2.597125] active_anon:0 inactive_anon:0 isolated_anon:0
       > [    2.597125]  active_file:8549 inactive_file:8549 isolated_file:0
       > [    2.597125]  unevictable:0 dirty:68 writeback:0
       > [    2.597125]  slab_reclaimable:4099 slab_unreclaimable:2852
       > [    2.597125]  mapped:0 shmem:0 pagetables:0
       > [    2.597125]  sec_pagetables:0 bounce:0
       > [    2.597125]  kernel_misc_reclaimable:0
       > [    2.597125]  free:1041 free_pcp:0 free_cma:0
       > [    2.597131] Node 0 active_anon:0kB inactive_anon:0kB active_file:34196kB inactive_file:34196kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:272kB writeback:0kB shmem:0kB writeback_tmp:0kB kernel_stack:116kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? yes
       > [    2.597136] Normal free:4164kB boost:4096kB min:5364kB low:5680kB high:5996kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:34196kB inactive_file:34196kB unevictable:0kB writepending:272kB present:102400kB managed:100768kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
       > [    2.597144] lowmem_reserve[]: 0 0
       > [    2.597148] Normal: 1*4kB (M) 2*8kB (U) 155*16kB (UE) 46*32kB (UE) 3*64kB (U) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4164kB
       > [    2.597158] 17098 total pagecache pages
       > [    2.597161] 25600 pages RAM
       > [    2.597164] 0 pages HighMem/MovableOnly
       > [    2.597167] 408 pages reserved
       > [    2.597169] Tasks state (memory values in pages):
       > [    2.597172] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
       > [    2.597175] Out of memory and no killable processes...
       > [    2.597178] Kernel panic - not syncing: System is deadlocked on memory
       > [    2.597182] ---[ end Kernel panic - not syncing: System is deadlocked on memory ]---
       > cptofs: lib/posix-host.c:448: panic: Assertion `0' failed.
       > /nix/store/j5l019m38wgq2lp8p1knk5k5rj4xwx7n-vm-run: line 132:   706 Aborted                    (core dumped) cptofs -p -P 1 -t ext4 -i $diskImage $root/* /
       > ERROR: cptofs failed. diskSize might be too small for closure.

Despite the cptofs error, it seems like the failure comes a little earlier (Out of memory and no killable processes...).

I then tried to override the memSize parameter, which I notice had been an issue in the past (I found a similar problem whose fixed was already merged) — then I try to build the qcow file as a package (the only way I found to override the memSize parameter) and I get a similar error, even though the memSize is much higher.

nix build #nixosConfigurations.mysystem.config.system.build.images.qemu

Here’s the flake I am using, for reference.

{
  inputs = {
    nixpkgs.url = "path:/home/alex/prog/nixpkgs";
  };

  outputs =
    {
      self,
      nixpkgs,
      ...
    }@inputs:
    let
      system = "x86_64-linux";
      pkgs = nixpkgs.legacyPackages.${system};
    in
    {
      packages.${system} = {
        qemu-image = pkgs.callPackage "${nixpkgs}/nixos/lib/make-disk-image.nix" {
          config = self.nixosConfigurations.mysystem.config;
          format = "qcow2";
          diskSize = 150 * 1024;
          memSize = 4096;
        };
      };
      nixosConfigurations = {
        mysystem = nixpkgs.lib.nixosSystem {
          system = "x86_64-linux";
          modules = [
            {
              virtualisation.diskSize = 150 * 1024;
              fileSystems."/" = {
                device = "/dev/disk/by-label/nixos";
                fsType = "ext4";
              };
              boot.loader.grub.device = "/dev/vda";
              boot.loader.grub.enable = true;
              boot.kernelModules = [ "virtio_rng" ];
            }
          ];
        };
      };
    };
}

The .qcow seems to be generated correctly — I see the success in the logs, and the partitioning is also done — but then the copying of the system fails:

Disk /build/nixos.raw: 161GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start   End    Size   Type     File system  Flags
 1      1049kB  161GB  161GB  primary  ext4

mke2fs 1.47.3 (8-Jul-2025)
Discarding device blocks: done                            
Creating filesystem with 39321344 4k blocks and 9830400 inodes
Filesystem UUID: 7ceb1f49-b7cb-4c4f-818a-ae435f894b0e
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
        4096000, 7962624, 11239424, 20480000, 23887872

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done

copying staging root to image...
[    0.000000] Linux version 6.6.0 (nixbld@localhost) (gcc (GCC) 14.3.0, GNU ld (GNU Binutils) 2>
[    0.000000] memblock address range: 0x7fffe9c00000 - 0x7ffff0000000
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x00007fffe9c00000-0x00007fffefffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x00007fffe9c00000-0x00007fffefffffff]
[    0.000000] Initmem setup node 0 [mem 0x00007fffe9c00000-0x00007fffefffffff]
[    0.000000] Kernel command line:  mem=100M virtio_mmio.device=352@0x1000000:1
[    0.000000] Dentry cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.000000] Inode-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 25250
[    0.000000] mem auto-init: stack:all(zero), heap alloc:off, heap free:off
[    0.000000] Memory: 100768K/102400K available (6981K kernel code, 1733K rwdata, 1518K rodata,>
[    0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[...]
[    2.597161] 25600 pages RAM
[    2.597164] 0 pages HighMem/MovableOnly
[    2.597167] 408 pages reserved
[    2.597169] Tasks state (memory values in pages):
[    2.597172] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
[    2.597175] Out of memory and no killable processes...
[    2.597178] Kernel panic - not syncing: System is deadlocked on memory
[    2.597182] ---[ end Kernel panic - not syncing: System is deadlocked on memory ]---
cptofs: lib/posix-host.c:448: panic: Assertion `0' failed.

I can work around this problem by doing it manually, but I would appreciate any hints or help in understanding what really is going on here!

Thank you very much!

1 Like