Weird error when running nixos/nix under podman

Development
1

Hey everyone,

I’ve been a very happy nix user for development environments for a while now, and I’m currently investigating whether I can use the nixos/nix docker image in CI, mainly to make sure I’m using the same version of ansible.

First of all, the reason why I’m posting this here and not (yet) in a github issue is because I need help localizing this. From the error message, I first assumed that niv’s sources.nix was the culprit, but I could reduce the problem to this. I’m here looking for any and all ideas on what could cause add-20-2 to show.

On my local machine, I prefer to run podman, and it’s usually very compatible, but I’ve encountered this issue that I don’t understand at all. Basically:

> docker run --rm -ti nixos/nix nix-build -E \
    "builtins.fetchTarball { name = \"nixpkgs-src\"; url = \"https://github.com/NixOS/nixpkgs/archive/59ee4cee790d077072dc1392e94fa6ac0099760d.tar.gz\"; sha256 = \"1k7qkpr7ay4qvkzmljl1kalpvj28ni36pmyy15nyfylrb18q3d9b\"; }"
error: expression does not evaluate to a derivation (or a set or list of those)

This is good and fine, because it is not actually a derivation.

Meanwhile, on podman:

> podman run --rm -ti nixos/nix nix-build -E \
    "builtins.fetchTarball { name = \"nixpkgs-src\"; url = \"https://github.com/NixOS/nixpkgs/archive/59ee4cee790d077072dc1392e94fa6ac0099760d.tar.gz\"; sha256 = \"1k7qkpr7ay4qvkzmljl1kalpvj28ni36pmyy15nyfylrb18q3d9b\"; }"
error: renaming '/nix/store/add-1-2/x' to '/nix/store/9liwm7rnx4357qzgz3bk8v37nkbsj07z-nixpkgs-src'

The number counts up:

> podman run --rm -ti nixos/nix
bash-4.4> for I in `seq 5`; do nix-build -E "builtins.fetchTarball { name = \"nixpkgs-src\"; url = \"https://github.com/NixOS/nixpkgs/archive/59ee4cee790d077072dc1392e94fa6ac0099760d.tar.gz\"; sha256 = \"1k7qkpr7ay4qvkzmljl1kalpvj28ni36pmyy15nyfylrb18q3d9b\"; }"; done
error: renaming '/nix/store/add-3-2/x' to '/nix/store/9liwm7rnx4357qzgz3bk8v37nkbsj07z-nixpkgs-src'
error: renaming '/nix/store/add-13-2/x' to '/nix/store/9liwm7rnx4357qzgz3bk8v37nkbsj07z-nixpkgs-src'
error: renaming '/nix/store/add-20-2/x' to '/nix/store/9liwm7rnx4357qzgz3bk8v37nkbsj07z-nixpkgs-src'
error: renaming '/nix/store/add-27-2/x' to '/nix/store/9liwm7rnx4357qzgz3bk8v37nkbsj07z-nixpkgs-src'
error: renaming '/nix/store/add-34-2/x' to '/nix/store/9liwm7rnx4357qzgz3bk8v37nkbsj07z-nixpkgs-src'

Image versions:

> (docker image ls; podman image ls) | grep nixos
nixos/nix                                           latest         858f475577cf   52 years ago   457MB
docker.io/nixos/nix                                                 latest      858f475577cf  52 years ago   549 MB
2

Is it not? I could have sworn fetch* were “trivial builders” that make derivations of the files they download.

Have you tried this with raw nix, without the podman/docker CLI? I wonder if docker handles escapes differently. Then again, that should be the shell’s job.

You’d probably be better served using single quotes than escaping internal double quotes, since you don’t use substitution.

Edit: Ah, not builders, but they do make derivations: NixOS - Nixpkgs 21.11 manual

3

Thanks for you answer. Yeah, the raw nix works perfectly. I’m using this in a project’s shell.nix and want to know if I can re-use that in my deployment pipeline.

In the meantime, I made a few more observations:

  • It only happens when running podman as a regular (i.e. non-root) user
  • It does happens when using kernel overlayfs (podman info | yq '.store.graphOptions prints {})
  • It does not happen when using fuse-overlayfs (podman info | yq '.store.graphOptions' prints stuff including fuse-overlayfs)
    Configurable using this ~/.config/containers/storage.conf:
    [storage]
driver = "overlay"

[storage.options.overlay]
mount_program = "/usr/bin/fuse-overlayfs"
    and podman system reset (which deletes everything including volumes, be warned)

I’ve looked into the nix sourcecode a bit and tried more stuff, now I’m pretty certain it’s in the additional work that downloadTarball does over downloadFile in this file. I’m more than just a little bit out of my depth in that code, though, and I don’t have the energy to pursue it further.

1 Like
4

I’ve opened an issue on the podman repo, and we’re making some progress, so if anybody is interested best look there.

And we’ve identified a possible fix: nix fails due to `rename` returning `EXDEV` in podman (non-root overlayfs) · Issue #6262 · NixOS/nix · GitHub

1 Like