Using mill in sandbox builds fails even though it works in pure shell

Help
1

I am using NixOS so nix by default has the sandbox enabled.

For a scala project I am using the mill build tool (which is in nixpkgs). With the following shell.nix file I can run nix-shell --pure --run build and that works fine:

{ pkgs ? import <nixpkgs> {} }:

pkgs.mkShell {
  buildInputs = with pkgs; [ scala mill graalvm11-ce gcc ];
  shellHook = ''
    build () {
      mill ulang.assembly
      native-image --no-fallback -H:IncludeResources='.*' -H:Log=registerResource: -jar out/ulang/assembly/dest/out.jar ulang.exe
    }
    '';
}

But if I try to rewrite this as a proper derivation like so

{ pkgs ? import <nixpkgs> {} }:
pkgs.stdenv.mkDerivation {
  name = "ulang";
  src = pkgs.lib.cleanSourceWith {
    src = pkgs.lib.cleanSource ./.;
    filter = name: type: let baseName = baseNameOf (toString name); in baseName != "out";
  };
  buildInputs = with pkgs; [graalvm11-ce gcc mill];
  buildPhase = ''
    mill ulang.assembly
    native-image --no-fallback -H:IncludeResources='.*' -H:Log=registerResource: -jar out/ulang/assembly/dest/out.jar ulang.exe
    '';
  installPhase = ''
    mkdir -p $out/bin
    cp ulang.exe $out/bin/ulang
  '';
}

and then run nix-build it fails when executing mill in the sandboxed build env:

these derivations will be built:
  /nix/store/db9kd0m8ivm9mfn3xppc15jp0lszahzf-ulang.drv
building '/nix/store/db9kd0m8ivm9mfn3xppc15jp0lszahzf-ulang.drv'...
unpacking sources
unpacking source archive /nix/store/m4088rvsq92jhgn4yizm2lz3m3h731r5-source
source root is source
patching sources
configuring
no configure script, doing nothing
building
mkdir: cannot create directory '/homeless-shelter': Permission denied
builder for '/nix/store/db9kd0m8ivm9mfn3xppc15jp0lszahzf-ulang.drv' failed with exit code 1
error: build of '/nix/store/db9kd0m8ivm9mfn3xppc15jp0lszahzf-ulang.drv' failed

I searched through the github repo of mill but could not find any reference of this homeless-shelter thing.

Does anyone know how to get mill working in the sandboxed build env? Or does anyone know if I can disable the sandbox for just one nix-build?

2

It tries to write or read to/from $HOME on disk.

Set HOME to a temporary location in your build folder and it should work.

export HOME=$(PWD)/home; mkdir -p $HOME or something similar often helps to mitigate.

The preferred way though is to find more specific variables that would control the behaviour of your tool to use non-default cache, config or temporary locations.

3

Thanks that gets me one step further. Now mill tries to download stuff with curl. I will have to go upstream and ask how to use mill in a sandbox.

4

Iā€™m not used to mill, nor do I know what exactly is possible and what not.

But for the erlang and elixir eco systems we have some ā€œpre fetchersā€ which use fixed output derivations to download dependencies (but not build them yet). Which we then copy/link over to a place in the actual softwares source tree where it could be found for the build.

Perhaps you can similarily split up the build with mill and separate fetching dependencies from building them?

5

I asked upstream what to download and where to put it: thread

Where do look for these pre-fetchers for erlang and elexir you are talking about?

6
7

This is the classic ā€˜clashā€™ between the nix/os way (which is a build system) and the per language build systems. Usually they get written because many of todays build systems are so so bad, confusing , heavy weight and start down a path that aways leads to the same result. Complex, unmaintainable, unreproducible build systems.

They all start our as ā€˜the package manager is dead, long live the new package managerā€™, but as more edge and corner cases get added to work around problems, they get to a stage where they become just as problematic as what they are trying to replace. Build system evolution - start off fantastic, and end up become a monster it was trying to replace! Itā€™s a hard problem and it shows.

Iā€™m not pointing the finger directly at mill, iā€™m sure itā€™s fantastic and solve problems better than the other build systems, but even in the mill documentation it statesā€¦

Mill is your shiny new Java/Scala build tool! Scared of SBT? Melancholy over Maven? Grumbling about Gradle? Baffled by Bazel? Give Mill a try!

Fair play to convert their frustration into a real code and project, but nix/os is alway in a paradigm war between what role the operating system has in placing code onto the system against applications having this function. Debian apt, has done a venerable job for yearsā€¦ windows I donā€™t think ever has had package manager, copy it to filesystem by handā€¦and your done! . Itā€™s just the way things worked out.

Thereā€™s all ways a line between the build system and OS package managerā€¦ nix solves this by being both. Some now solve these build/install/packaging problems with containers, but i think in the long run that will lead to ā€˜containermageddonā€™. Containers donā€™t address build problems, but sometime they are used in that wayā€¦ itā€™s just a temporary band aide. Docker fans will probably disagree with meā€¦

Nix tried to address problems all the other build system donā€™t, where things are, how they can be found, where they should be put. It gets more reliability through restricting user choice, a happy medium that some libraries are shared, and some are statically linked but not how you think.

Should a build system be able to just reach-out and download code that is needs? or should the operating system be the governor of this. You cant have a reproducible build if the build system has determinism in it, the network being big no-no. Hence the war between dependencies being full nix packages, or fetching dependencies with nix ā€˜fetchersā€™. This still confuses meā€¦ and it varies to language ecosystem to ecosystem.

I write this as iā€™m interested what reaction and replies you get from the mill peopleā€¦ the usual response is that building the developers code is their problem and packaging the resulting artefacts is for the OS package maintainer. ā€˜The packagerā€™ .

Iā€™m not sure if nix can win this war between letting the operating system control the chaos of adding code to a system, or letting individual developers make those decisionsā€¦ at least more traditional approach developers can point the finger at package maintainers, and package maintainers can point the finger at developers, and kernel developers donā€™t care, because itā€™s a user space problem LOL. ;-).

1 Like
8

I got half an answer from upstream. I can execute mill __.prepareOffline and that will load all dependencies to ~/.cache/coursier.

Is it possible to mount that folder into the build context?

9

To ~/.cache/coursier or $HOME/.cache/coursier or even some XDG_*?

~ does not necessarily resolve to $HOME. Anyway, you can try a single FOD which has HOME=$(pwd) and another one that sets export HOME=${dependencies}.

Thats the first steps quickā€™nā€™dirty solution.

Next step would be to create either an abstraction that is split up into two functions similar how we do it in the BEAM ecosystem or that is ā€œjoint togetherā€ as in the rust and go ecosystems.

10

This should be fixed on master with mill: 0.9.8 -> 0.9.9 by jonringer Ā· Pull Request #130823 Ā· NixOS/nixpkgs Ā· GitHub. The mill package before was using the release script, which would try to download the mill assembly to ~/.cache/mill. Which is probably why you were getting those errors.

2 Likes