Best practice for sandbox builds?

Hi all,

Currently struggling to not modify the sandbox settings for builds while using openwrt-imagebuilder to generate a VM image for openwrt. Previously I couldn’t get it to build, now I am getting permissions errors in the build I presume due to the build happening in a sandbox environment, where permissions cannot be properly managed. To mitigate this, (through lots of AI prompting and googling) I was presented with the following solution:

nix.settings = {
    experimental-features = ["nix-command" "flakes"];
# Below is the recommended solution
    extra-sandbox-paths = ["/var/tmp/openwrt-build"];
    sandbox = "relaxed";

It works, and I am able to print the path to the image in the nix store for later use in my system:

evaluation warning: OpenWRT image path: /nix/store/93h2bb7pbj4c40gpc06imbbpdv9p00pa-openwrt-24.10.4-x86-64-generic
updating GRUB 2 menu...
activating the configuration...
setting up /etc...
# commented out my user in this line
reloading user units for root...
restarting sysinit-reactivation.target
restarting the following units: nix-daemon.service
the following new units were started: libvirtd.service, NetworkManager-dispatcher.service
Done. The new configuration is /nix/store/s2v7bmnzawq79sbqy324c1k8b5pv7lkz-nixos-system-nixos-25.05.20251205.00d2457

Original Error I am Receiving:

error: builder for '/nix/store/yzqy78zm7jcxsmiglyjqi3fw6lyqllw6-openwrt-24.10.4-x86-64-generic.drv' failed with exit code 2;
       last 25 log lines:
       >  * copy_file: unable to preserve ownership of `/build/openwrt-imagebuilder-24.10.4-x86-64.Linux-x86_64/build_dir/target-x86_64_musl/root-x86/tmp/opkg-ocDYkP/mkf2fs_1.16.0-r3_x86_64.ipk': Invalid argument.
       >  * copy_file: unable to preserve ownership of `/build/openwrt-imagebuilder-24.10.4-x86-64.Linux-x86_64/build_dir/target-x86_64_musl/root-x86/tmp/opkg-ocDYkP/libf2fs6_1.16.0-r3_x86_64.ipk': Invalid argument.
       >  * copy_file: unable to preserve ownership of `/build/openwrt-imagebuilder-24.10.4-x86-64.Linux-x86_64/build_dir/target-x86_64_musl/root-x86/tmp/opkg-ocDYkP/mtd_26_x86_64.ipk': Invalid argument.
       >  * copy_file: unable to preserve ownership of `/build/openwrt-imagebuilder-24.10.4-x86-64.Linux-x86_64/build_dir/target-x86_64_musl/root-x86/tmp/opkg-ocDYkP/odhcp6c_2024.09.25~b6ae9ffa-r1_x86_64.ipk': Invalid argument.
       >  * copy_file: unable to preserve ownership of `/build/openwrt-imagebuilder-24.10.4-x86-64.Linux-x86_64/build_dir/target-x86_64_musl/root-x86/tmp/opkg-ocDYkP/odhcpd-ipv6only_2025.10.02~b14cf98c-r2_x86_64.ipk': Invalid argument.
       >  * copy_file: unable to preserve ownership of `/build/openwrt-imagebuilder-24.10.4-x86-64.Linux-x86_64/build_dir/target-x86_64_musl/root-x86/tmp/opkg-ocDYkP/opkg_2024.10.16~38eccbb1-r1_x86_64.ipk': Invalid argument.
......

My goal is to use best practices to make this built image made available to me in Nix as an input or variable to capture the path to pass to my nixvirt module.

Is relaxed sandbox with allowed users okay long term? Or does that remove some safety and security from the system? Is there a way to accomplish what I am trying?

More context, here is my flake.nix:

{
  description = "A simple NixOS flake";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05";
        
    openwrt-imagebuilder = {
      url = "github:astro/nix-openwrt-imagebuilder";
      inputs.nixpkgs.follows = "nixpkgs";
    };
    
    nixvirt = {
      url = "https://flakehub.com/f/AshleyYakeley/NixVirt/*.tar.gz";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = { self, nixpkgs, openwrt-imagebuilder, nixvirt, ... }@inputs: {
    
    packages.x86_64-linux.my-router = import ./network/openwrt.nix {
      pkgs = nixpkgs.legacyPackages.x86_64-linux;
      inherit inputs;
      inherit openwrt-imagebuilder;
    };
    
    nixosConfigurations.nixos = nixpkgs.lib.nixosSystem {
      specialArgs = {inherit inputs;};
      modules = [
        ./configuration.nix
        ./network/nixvirt.nix
      ];
    };
  };
}

Here is my nixvirt.nix:

{ config, pkgs, lib, inputs, ... }:

{
  virtualisation.libvirtd.enable = true;
  users.users.dalen.extraGroups = [ "libvirtd" ];
  
  # Force evaluation and show path
  warnings = [
    "OpenWRT image path: ${inputs.self.packages.x86_64-linux.my-router}"
  ];
}

In a prior post a couple days ago I was recommended to use openwrt-imagebuilder as a package and run in directly in the terminal to produce an image. I really want to be able to make it a part of my configuration in a way that I can pass it to nixvirt as a location in the store of the image. Is this possible?

edit: I have also tried using: openwrt-imagebuilder.override {

buildRoot = “/var/tmp/openwrt-build“;

}

this unfortunately did not work

Can you add a -L to that so you can actually see what’s going wrong?

I would definitely not recommend disabling the sandbox for this. Figure out what permissions are attempted to be set and why, then either fix that or find a more appropriate workaround. .ipk is an archive format of some sort, those files almost certainly do not need some weird permissions that aren’t available in a sandbox.

What the sandbox does is documented quite well in the manual: https://nix.dev/manual/nix/2.32/command-ref/conf-file.html?highlight=Relaxed#conf-sandbox

The result is largely that your builds become non-reproducible; if you’re hitting the walls of the sandbox, you’re probably doing something that can cause reproducibility issues. Disabling it permanently strikes me as pretty bad, since you’re removing those guardrails for yourself.

It hardly contributes to security, the sandbox isn’t designed to keep naughty code out, just make it hard to accidentally do stupid things.

1 Like

Thank you for the guidance, Seems I was mistaken. I only get issues when trying to build the image separately with nix build manually in the command line. When running the entire nixos-rebuild switch –flake it seems to complete the derivation and show the output in the log as the path to the image in the nix-store:

sudo nixos-rebuild switch --flake .#nixos
warning: Git tree '/etc/nixos' is dirty
building the system configuration...
warning: Git tree '/etc/nixos' is dirty
evaluation warning: OpenWRT image path: /nix/store/zzdkdyywfpns6hgjc6v6czm8l9af8pf8-openwrt-24.10.4-x86-64-generic # this is the line I was looking for
updating GRUB 2 menu...
removing obsolete file /boot/kernels/67cqm2jg3i1ay718q2kpqz51vx6il2i8-initrd-linux-6.12.55-initrd
removing obsolete file /boot/kernels/y71qpdjr6h699hxnjqgq4v8pklgmhxpq-linux-6.12.55-bzImage
activating the configuration...
setting up /etc...
reloading user units for root...
restarting sysinit-reactivation.target
reloading the following units: dbus.service
restarting the following units: polkit.service, systemd-udevd.service
the following new units were started: libvirtd.service, NetworkManager-dispatcher.service
Done. The new configuration is /nix/store/jgapy90g71gx0z02k8jvz8z3d8l6yl2a-nixos-system-nixos-25.05.20251210.d2b1213