Why can my builds access the network?

Running nix-build on the code below finishes without error, whereas I would expect it to fail to fetch from the internet.

{ pkgs ? import <nixpkgs> { } }:

pkgs.runCommand "foo" { } ''
  mkdir $out
  ${pkgs.curl}/bin/curl http://example.com -o $out/foo
''

In order to trigger a sandboxing error, I have to run nix-build --sandbox:

❯ nix-build --sandbox
this derivation will be built:
  /nix/store/d9wjpaxd0d2hfdla9kqhsshcp37k2mr2-foo.drv
building '/nix/store/d9wjpaxd0d2hfdla9kqhsshcp37k2mr2-foo.drv'...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: example.com
error: builder for '/nix/store/d9wjpaxd0d2hfdla9kqhsshcp37k2mr2-foo.drv' failed with exit code 6;
       last 3 log lines:
       >   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
       >                                  Dload  Upload   Total   Spent    Left  Speed
       >   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: example.com
       For full logs, run 'nix log /nix/store/d9wjpaxd0d2hfdla9kqhsshcp37k2mr2-foo.drv'.

My /etc/nix/nix.conf:

❯ cat /etc/nix/nix.conf
# WARNING: this file is generated from the nix.* options in
# your NixOS configuration, typically
# /etc/nixos/configuration.nix.  Do not edit it!
build-users-group = nixbld
max-jobs = 4
cores = 0
sandbox = true
extra-sandbox-paths = 
substituters = https://cache.nixos.org/
trusted-substituters = 
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
auto-optimise-store = false
require-sigs = true
trusted-users = root asymmetric
allowed-users = *
builders =

system-features = nixos-test benchmark big-parallel kvm
sandbox-fallback = false

experimental-features = nix-command flakes

(note the sandbox = true value).

Does trusted-users have anything to do with this?

This is a NixOS unstable system.

2 Likes

Here is my /etc/nix/nix.conf:

# WARNING: this file is generated from the nix.* options in
# your NixOS configuration, typically
# /etc/nixos/configuration.nix.  Do not edit it!
build-users-group = nixbld
max-jobs = auto
cores = 0
sandbox = true
extra-sandbox-paths = 
substituters = https://cache.nixos.org/
trusted-substituters = 
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
auto-optimise-store = false
require-sigs = true
trusted-users = root
allowed-users = *
builders =

system-features = nixos-test benchmark big-parallel kvm
sandbox-fallback = false

experimental-features = nix-command flakes
keep-build-log = false
keep-outputs = true

When I try to build that derivation with my own user, which is not specified as a trusted-user, the build fails:

error: builder for '/nix/store/435h2y7nfsdb370as56xi3pv7iz7qp3w-foo.drv' failed with exit code 6;
       last 3 log lines:
       >   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
       >                                  Dload  Upload   Total   Spent    Left  Speed
       >   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: example.com
       For full logs, run 'nix log /nix/store/435h2y7nfsdb370as56xi3pv7iz7qp3w-foo.drv'.
1 Like

I removed asymmetric from trusted-users and the issue persists, so it doesn’t seem to be the problem.

Could you try adding your user to trusted-users and seeing if anything changes? (You first need to remove the already built path in the store, otherwise it would be a no-op)

By the way, thinking more about it, it doesn’t seem likely that trusted-users should affect hermeticity. IIUC trusted-users regulates access to the daemon.

Even after I added my user to the trusted-users and rebooted, the build still fails as above.

I don’t understand which path should be removed. Building the derivation fails in both cases.

@toraritte @ryantm do you have the same issue?

OK, then I think we agree that trusted-users is not doing anything related to this issue.

Sorry, that didn’t make sense in your case (since the build failed in the first place)

Here’s the output of nix-info:

❯ nix-info -m
 - system: `"x86_64-linux"`
 - host os: `Linux 5.10.89, NixOS, 22.05 (Quokka)`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.5.1`
 - channels(root): `"nixos-21.05.4442.b86b6bc5e75"`
 - channels(asymmetric): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

This doesn’t look right. Here is mine:

❯ nix-info -m
 - system: `"x86_64-linux"`
 - host os: `Linux 5.10.90, NixOS, 21.11 (Porcupine)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.4`
 - nixpkgs: `/nix/store/8hd2049cr4dz9myjcjjh6cjjapz1p107-source`

I wonder if this has something to do with my filesystem:

❯ mount | grep btrfs
/dev/mapper/cryptoroot on / type btrfs (rw,noatime,compress=lzo,ssd,space_cache,subvolid=257,subvol=/@)
/dev/mapper/cryptoroot on /nix/store type btrfs (ro,noatime,compress=lzo,ssd,space_cache,subvolid=257,subvol=/@)
/dev/mapper/cryptoroot on /.snapshots type btrfs (rw,noatime,compress=lzo,ssd,space_cache,subvolid=259,subvol=/@snapshots)
/dev/mapper/cryptoroot on /mnt/btrfs type btrfs (rw,noatime,compress=lzo,ssd,space_cache,subvolid=5,subvol=/)
/dev/mapper/cryptoroot on /home type btrfs (rw,noatime,compress=lzo,ssd,space_cache,subvolid=258,subvol=/@home)
/dev/mapper/cryptoroot on /var/lib/docker/btrfs type btrfs (rw,noatime,compress=lzo,ssd,space_cache,subvolid=257,subvol=/@)

In particular, my configuration.nix doesn’t define a mount-point for /nix/store, but one is present in the output of mount.

If you are on nixos, that’s probably the “read only bind mount”, that is created to avoid accidental changes not intented by nix tooling.

1 Like

Thanks for the tip! Do you know if this is documented anywhere?

Anyway, seems like that was another red herring, the search for the root cause continues :thinking:

In the corresponding nixos option.

It is possible your nix configuration is being overwritten from somewhere else. Not only is there the user config file to consider, but there also the NIX_CONFIG variable which can be set to arbitrary config values. Perhaps by some strange accident. sandbox = no is being set by mistake?

You can review man nix.conf to check all the possible places this could be set.

give us your configuration.nix (if you dare!) .

The same behaviour happens to both users of my system (normal and superuser).

Nope, it’s empty for all users.

  • NIX_CONF_DIR is empty
  • I posted the contents of /etc/nix/nix.conf
  • NIX_USER_CONF_FILES is empty
  • NIX_CONFIG is empty
  • XDG_CONFIG_DIRS is /etc/xdg:/home/asymmetric/.nix-profile/etc/xdg:/etc/profiles/per-user/asymmetric/etc/xdg:/nix/var/nix/profiles/default/etc/xdg:/run/current-system/sw/etc/xdg, and paths in this list either don’t exist, or don’t have anything relevant

AFAIU that’s all the places I could’ve looked.

I found it!

$ cat /root/.config/nix/nix.conf 
sandbox = false

The little fucker was turning the sandbox off for all users on the system! I guess it makes sense.

I have no idea when I created that file. Wish I never did, and that there weren’t so many different places to look at to debug this behaviour!

Thanks everyone, and especially @nrdxp for prompting me to look around!

2 Likes

Probably a point in favour of flakes…

Edit: wait. That might not have helped either.

Yep, flakes are not relevant here (and the system does use them).

Thinking more about this, it doesn’t seem intuitive that a user setting (even though that user is root) can influence other users of the system.

Created an issue.