I’m trying to get the Unity Licensing Server running on nixos. (I know).
Unfortunately it has a dumb filesystem check within it where it first checks to make sure the file “/bin/bash” exists in the filesystem before trying to execute bash. You can tell from the stack trace that they manually added the check with a custom error message.
Unhandled exception. System.IO.FileNotFoundException: Bash cannot be found
File name: '/bin/bash'
at Unity.Licensing.Platform.CommonUtils.RunInBash(String cmd, String& output)
at Unity.Licensing.Server.Services.ServerPermissionService.ApplyCommand(String command)
at Unity.Licensing.Server.Services.ServerPermissionService.IsUserInGroup()
at Unity.Licensing.Server.Services.ServerPermissionService.IsUserInGroupInCurrentSession()
at Unity.Licensing.Server.ServerImportExecutor.ValidateUserPermission()
at Unity.Licensing.Server.ServerImportExecutor.OnExecute(CommandLineApplication app, IConsole console, String archivePath)
at Unity.Licensing.Server.ServerImport.Execute(ServerImportExecutor commandService, CommandLineApplication app, IConsole console)
at Unity.Licensing.Server.InjectedCommand`1.OnExecute(CommandLineApplication app, IConsole console)
at Unity.Licensing.Server.InjectedCommand`1.OnExecute(CommandLineApplication app, IConsole console)
at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.InvokeAsync(MethodInfo method, Object instance, Object[] arguments)
at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.OnExecute(ConventionContext context, CancellationToken cancellationToken)
at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.<>c__DisplayClass0_0.<<Apply>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync(String[] args, CancellationToken cancellationToken)
at Program.<Main>$(String[] args)
at Program.<Main>(String[] args)
fish: Job 1, 'sudo ./Unity.Licensing.Server i…' terminated by signal SIGABRT (Abort)
Simultaneously, it requires write access to the directory /usr/share/unity3d.
I first made /usr/share/unity3d available via tmpfiles.d in my config (with only lax permissions because I’m trying to get it to work).
# configuration.nix
systemd.tmpfiles.rules = [
"d /usr/share/unity3d/ 7777 root root - -"
];
Then I use a shell.nix to set up a sandbox to run the server.
# shell.nix
{ pkgs ? import <nixpkgs> {} }:
(pkgs.buildFHSEnv {
targetPkgs = pkgs: (with pkgs; [
bash
];
runScript = "./Unity.Licensing.Server"
}).env
Well great, now I’m in a catch-22:
- If I use buildFHSEnv, this doesn’t mount the
/usr/share/unity3d/ to the sandbox and it’s seemingly not possible to. (/usr is read-only)
- If I use mkShell to run it, it fails to find
/bin/bash and I cannot create it. (/bin is read-only)
So my question is, is there a way to make /bin/bash available in mkShell? OR
Is there a way to make the host’s /usr/share/unity3d available to buildFHSEnv?
1 Like
Sounds like a job for bubblewrap?
1 Like
← SNIP →
NEVERMIND it didn’t work.
I’d like bubblewrap, though I couldn’t find much reading material about the subject. I’ll keep looking.
Bubblewrap is only present in buildFHSDev, but since /usr is mounted read only, you can’t add to it with extra bubble wrap mounts. Bubblewrap doesn’t appear to be available with mkShell, and even if it were, /bin is also read-only.
I tried the following:
extraBwrapArgs = [
"--bind ./unity3d /usr/share/unity3d"
];
bwrap: Can’t mkdir /usr/share/unity3d: Read-only file system
Looking at the other bubble wrap arguments, I’m not finding any other tools at my disposal.
What do you mean? Just add pkgs.bubblewrap to your shell, or run nix-shell -p bubblewrap, or use any other technique for getting an application on Nix.
If you don’t bind /, you don’t have to worry about /bin or /usr being read-only. Just bind in the bits you need—/nix and /dev, any user directories you want access to, and populate /bin and /usr with whatever mix of binds, writable directories, and overlays you want.
1 Like
Oh okay!
Manually binding everything I need sucks, the darn app requires a dozen packages. I’ll do my best though.
Create a package that adds a symlink from /usr/share/unity3d to something that gets bind-mounted? Or even create a package with this directory and do extraBwrapArgs with --bind?
You can buildEnv all the stuff you need, then bind-mount /nix and use all-the-stuff-env for /bin and /usr/bin
1 Like
Unfortunately envfs didn’t work, it appears that Unity’s file-exists check bypasses envfs’s “magic” mount. (I did run sudo envfs /bin before trying)
bwrap seems really promising, struggling to get it to run properly in a shell.nix though. I’ll keep hacking at it.
I’ll try making a package that symlinks if I can’t find a way to easily bwrap.
buildFHSEnv is already Bubblewrap so you can add the options there
1 Like
Tried bwrap. Ran into various nix-ld panics, and extremely undescriptive errors. I kept mounting more stuff and just kept getting new unhelpful errors. I don’t have the prerequisite knowledge to reconstruct the filesystem state from scratch without /bin or /usr. I was reading how buildFHSEnv does it and think it might be easier for me to try 7c6f434c’s suggestion.
Here’s a few errors I got. started with mounting proc, and dev.
bwrap: execvp bash: No such file or directory
realized I forgot /nix
bash-5.2# ./Unity.Licensing.Server import /home/naelstrof/Downloads/kobold.zip
bash: ./Unity.Licensing.Server: cannot execute: required file not found
tried adding /lib and /lib64
bash-5.2# ./Unity.Licensing.Server import /home/naelstrof/Downloads/kobold.zip
[nix-ld] FATAL: panicked at src/main.rs:185:55:
called `Result::unwrap()` on an `Err` value: Posix(2)
Aborted (core dumped)
Moving on to trying to construct a package that contains a symlink to use in buildFHSEnv.
Unfortunately, both /bin and /usr are already read-only by the time my arguments get added. Unless you can overlay on top of a read-only directory-- I didn’t try that. I’m not exactly sure how it works. I’ll look into it.
You can overlay on top of a read-only directory, but it needs to exist. However, I believe that share gets aggregated into /usr/share/ so you can just add a quasi-package that creates an empty directory or a symlink to an RW-mounted location at /usr/share/unity3d as a dependency of your FHS env.
1 Like
Nice, yeah this is very close. I created a quasi-package that looks like this:
#./unity-folder/default.nix
{writeTextFile}:
writeTextFile {
name = "testing";
text = "";
destination = "/share/unity3d/testing";
}
And I included it at the top of my shell.nix with:
{ pkgs ? import <nixpkgs> {}, unity-folder ? pkgs.callPackage ./unity-folder { } }:
including unity-folder as a package ensures that /usr/share/unity3d exists by the time my bwrapArgs are evaluated.
There’s no documentation for --overlays though, all I’ve got is the help. Not even present within the manpages.
extraBwrapArgs = [
"--overlay-src /usr/share/unity3d"
"--overlay /usr/share/unity3d /usr/share/unity3d /usr/share/unity3d"
];
bwrap: Can’t make overlay mount on /newroot/usr/share/unity3d with options upperdir=/oldroot/usr/share/unity3d,workdir=/oldroot/usr/share/unity3d,lowerdir=/oldroot/usr/share/unity3d,userxattr: Invalid argument
Guess I was naive thinking I could get get away with that. Going to try to finding reading material. This feature seems very new.
I’ll try some more after I get some sleep. I realize I can try making a package that tries to symlink to the working directory or similar instead of trying to use bwrap’s overlay like you suggested earlier.
Thank you all for the super helpful input and patience!
Why not just --bind or --ro-bind ?
1 Like
Oh I assumed that wouldn’t work due to it being read-only. I’ll try when I wake up in the morning.
This worked! Nice!
Now it has the proper file structure, and it can also be ran as a normal user. Sweet! I’ll share my script as soon as I get it working.
However it’s now trying to run “groupadd” within the sandbox, which isn’t allowed.
INFO - [Unity.Licensing.Server.Services.ServerPermissionService] Will run: groupadd unity-licensing-server
INFO - [Unity.Licensing.Server.Services.ServerPermissionService] cmd result: 10: groupadd: cannot open /etc/group: Too many levels of symbolic links
It also seems to check for the group with a id -nG root | grep unity-licensing-server which also fails, but it only does this when I try to run it with fakeroot.
Faking groups and sharing namespaces is not very well documented for bwrap, I’ll keep looking.
I think you can just hand-craft and /etc/groups and bind-mount it over, but not sure.
1 Like
I had the same thought, and I gave it a shot, but /etc/ needs to exist, so I bind mount /etc also and then we’re back to the really unhelpful panic messages. I can see that buildFHSEnv handles /etc/ extremely carefully. This has become a 3 day project and I realize I’m silly for even trying to run such scuffed software from NixOS haha. Not to mention every time I fail the verification I have to manually reach out to support to clear my “unhealthy license” status lmao.
I’ve submitted a bugreport to unity for trying to load /bin/bash as a file because if that is fixed then I’d be able to trivially load it with mkShell (at the cost of it being unsandboxed and running as root).
My solution for now is to spin up an Arch VM who’s job is to just run the licensing server. Where it can have as jank of a setup as it needs to run.
The package almost did work however, just failed to get permissions. I’ll post my script for posterity.
# shell.nix
{ pkgs ? import <nixpkgs> {}, unity-folder ? pkgs.callPackage ./unity-folder { } }:
(pkgs.buildFHSEnv {
name = "simple-x11-env";
targetPkgs = pkgs: (with pkgs; [
udev
alsa-lib
gtk3
udev
xorg.libXrandr
stdenv.cc.cc.lib
libglvnd
xorg.libX11
xorg.libXcursor
glib
gdk-pixbuf
libxml2
zlib
icu
git
ripgrep
openssl
bash
fakeroot
shadow
xorg.libXi
xorg.libXrender
gnome2.GConf
libcap
unity-folder
]);
multiPkgs = pkgs: (with pkgs; [
udev
alsa-lib
]);
extraInstallCommands = ''
mkdir -p ./unity3d
mkdir -p ./root-home
mkdir -p ./fakehome/licensing-server
'';
extraBwrapArgs = [
"--bind ./unity3d /usr/share/unity3d"
"--bind ./root-home /root"
"--bind ./fakehome /home/naelstrof"
"--bind ./ ./home/naelstrof/licensing-server"
#"--setenv FAKEROOTDONTTRYCHOWN 1"
#"--unshare-user"
#"--uid 0"
#"--gid 0"
];
runScript = pkgs.writeScript "run.sh" ''
./Unity.Licensing.Server setup /home/naelstrof/Downloads/kobold.zip
'';
}).env
And the package…
# ./unity-folder/default.nix
{writeTextFile}:
writeTextFile {
name = "testing";
text = "";
destination = "/share/unity3d/testing";
}
Most of the packages in there may or may not be required as I was mid-tinkering.
Running shell-nix in the folder properly creates a /usr/share/unity3d, ensures /bin/bash is on the path, and actually allows Unity.Licensing.Server to run setup. Even keeps files owned by a unprivileged user.
Just fails at creating a group on run, and switching to that group due to the sandbox.
tl;dr: I’m gonna fix it with a VM running a different OS and asking Unity to fix it. Thanks for all the help!
Just in case, you might want to know that you can bind-mount a file over a file.