Nix-minecraft: Making Minecraft More Declarative

Announcements
1

For a little bit now, I’ve been somewhat disappointed with how Minecraft (particularly servers) have been packaged with Nix and NixOS. So, since I run a couple servers of my own, I decided to spin off my module and packages into its own (flakes!) repository for the world to use: GitHub - Infinidoge/nix-minecraft: An attempt to better support Minecraft-related content for the Nix ecosystem

(Updated 2024-09-19)

I’ve tried to make sure that the README file (decently) comprehensive, so please check it out for more in-depth information, but in summary:

  • Packages all Vanilla versions
  • Packages all supported versions of Fabric, Quilt, Legacy Fabric, and Paper
  • Provides a module for managing multiple Minecraft servers at once, including the ability to symlink, copy, or generate arbitrary files (with support for environment file secrets) (declarative mods and config, anyone?) and do custom actions on start/stop/reload.
  • Includes a fetcher for Packwiz packs, and a prefetcher for Modrinth mods

In the future, I hope to package Forge! Let me know if you have any other loader or server suggestions!

If you have any ideas for improvements to the packages, module, or documentation, feel free to open an issue or a pull request!

25 Likes
2

nix-minecraft-servers/pkgs/tools/extract-forge-deps.py at d7271c7db69a7195fd5399f779299580a5fbea48 · TLATER/nix-minecraft-servers · GitHub may be worth something :slight_smile:

I’ve had some thoughts about curseforge before too: GitHub - TLATER/nix-minecraft-servers: Nix flake that provides utilities for hosting minecraft servers

Maybe I’ll try contributing a curseforge fetcher when I decide to pick this back up again. I theoretically have one in my personal server config, could do with maintaining it elsewhere :slight_smile:

1 Like
3

I have a semi working addition to the Minecraft worker module that allows you to install fabric mods and the different versions declaratively : https://github.com/legendofmiracles/nixpkgs/blob/3993955bdba38daac7ca7dc732ca37deec2a3215/nixos/modules/services/games/minecraft-server.nix

It’s quite old, but I would love to get your thoughts on it

4

Interesting, haven’t really thought of actually running the installer on the server. I’d probably say that using the packages like in my repo may be a bit easier to work with though, since you can just feed it into the package option of pretty much any Minecraft server service.

(As a side note, doing it like that technically breaks purity, since its giving the installer internet access. I believe Fabric still hasn’t made an offline installer, sadly)

With the symlinks option on mine, I’ve been able to get declarative mods already working, and it works quite well. In theory all I’d need to do is alias server.<name>.mods to server.<name>.symlinks.mods and have it use linkFarmFromDrvs. It may be neat to have a dedicated server.<name>.mods option though, for convenience.

Your technique of using the file to keep track of what has already been linked is interesting, and I could probably look into doing something similar for trying to do stuff like declarative configuration files.

5

:eyes: The dependency extractor may be exactly what I need to install Forge. That’s pretty much the same process used for packaging Fabric, except that the Fabric devs are much more reasonable and just provide the information you need through an API. (Feel free to check it out under /pkgs/fabric-servers, in the loader.nix and server.nix files. See update.py for how the API information is handled.) Though, could you link to where they said to not automate the installation process? Since their code appears to be under LGPL, I’m not sure if they can really prevent it. (And honestly, I’m fine slightly annoying a few people for purity, personally :p)

I agree with your thoughts on CurseForge, which is entirely why I’ve been trying my best to push for and use Modrinth whenever possible. (Their API is so nice, by the way.)

As for Modpacks, yeah that is another tricky thing. Using Packwiz is probably a good solution for most people, and unsup (if you’re willing to write the manifest) is another great solution. I was thinking of possibly using Nix to read these formats and take over for handling the mods, but that could quickly spiral into complexity.

6

This is neat! My brother and I also made something similar a while ago, modules/minecraft · 70a15531f1a1ab33754efb849120d9a21fee80ee · Coffee Tables / myrdd · GitLab. It was designed for papermc. With the configs option you can arbitrarily configire yaml files anywhere in the data directory. We also setup a waterfall proxy and wired servers up to it automatically. I’ve been meaning to revive it and factor it out into its own flake.

7

Very neat setup with the configs option, I may look into using that myself since it seems like a very slick way to setup files all in-Nix. Does that option support types other than YAML, like TOML, JSON, or INI?

Also the proxy setup is also interesting, I may derive a module into my repo based on it, though I will admit that I haven’t needed WaterFall as of recently.

8

Ours only supports yaml, because from what i remember papermc is entirely configured with yaml. Waterfall was nice for us, so we didn’t need to open mutiple ports and we could just tell our friends the domain to join and they could go to the appropriate server once in the lobby.

9

Ohh and another useful addition we had was an mcshell command that could put you into the shell of any server with just the name. The module auto setup rcon and the command would read the server properties.

1 Like
10

Yeah I noticed that. I could probably include it as a convenience, but I setup the server to use a tmux socket for the console as opposed to relying solely on RCON. (So then you have direct access to the console, mods affecting the console work, etc.)

Maybe I’ll add a similar command to open the tmux socket.

1 Like
11

Minecraft supports RCON natively these days, just use that instead of those input-redirection hacks. Hm, I clearly can’t read. But yes, I think this should be preferred over trying to get tmux running inside systemd. I think systemd itself also has some features for keeping stdin/stdout attached to a tty, but I’ve not found the need since RCON support was added (and giving a user op permissions is usually sufficient anyway).

1 Like
12

If you extract the zip and read the json (that my script parses):

{
    "_comment_": [
        "Please do not automate the download and installation of Forge.",
        "Our efforts are supported by ads from the download page.",
        "If you MUST automate this, please consider supporting the project through https://www.patreon.com/LexManos/"
    ],
    "id": "1.18.2-forge-40.1.25",
    "time": "2022-05-23T19:07:39+00:00",
    "releaseTime": "2022-05-23T19:07:39+00:00",
    "type": "release",
...

Even if their project is LGPL licensed, they may choose to exempt you from the license, so it being LGPL doesn’t really keep this from being relevant.

I found it in the source while trying to reverse engineer their installer at some point too, I think.

My plan was to use the nix-store --add feature or whatever it was instead, so that people need to download the initial tarball themselves (or have some other mechanism for getting it into their store). That way users have a chance to acknowledge this message, and can choose for themselves how they wish to support the project.

You still gain all the reproducibility benefits that way (and don’t need to relearn how to deploy a forge server to install it either), and with a little bit of auxiliary work a user can completely ignore that message too. I just don’t want to make that choice for users, since the project asks so nicely (and also because I don’t want to act as irresponsibly as curseforge does regarding these things) :slight_smile:

1 Like
13

Yeah I chose to use RCON because it was simpler and integrated better with systemd’s service management features.

2 Likes
14

From my experience tmux runs just fine under systemd with the forking type. See the Terraria service for another example of using tmux for the console.

15

My objection is conceptual and technical; lots of hacks can work but shouldn’t be used in practice. Lots of things that shouldn’t be used in practice actually are. This is one of them, and a major pet peeve of mine whenever I have to wade the game server industry :wink:

Let me try to explain where I’m coming from. It’s a bit long, but basically, the reason I dislike this is because it doesn’t inter-operate with the rest of the Linux ecosystem well. I recognize it’s sometimes necessary for the less well designed game servers out there, but Minecraft has a much better interface. Let’s use it!

Less conceptually, your service currently fails to properly interact with the systemd journal to my knowledge, so all system monitoring software that uses the journal as its source of truth fundamentally can’t know what’s going on with Minecraft. This means that graphing user metrics and log monitoring isn’t possible without modding the game or tmux and writing additional code. journalctl -xe --unit minecraft won’t give you useful logs.

By using forking you are also actively violating the type contract you’re setting up with systemd - the application that calls fork() is supposed to exit by itself shortly after the child is spawned. Systemd will not manage the tmux process, which means that we now have a service that isn’t actually controlled by our service manager. Most of the time this is fine, but there are edge cases, some of which you currently tape over in your module by setting the start/stop scripts.

This isn’t great - firstly, systemd should never consider the Minecraft service “started”, which means that you can’t use this service for composition (i.e., DependsOn, Requires and such won’t work). If it does consider the service started, then something else calls fork() and exits and we now are tracking the wrong PID with systemd, which is an entirely different problem.

Also, the SIGTERM signal that systemd normally sends is the standard way of asking Minecraft to shut down nicely, and it actually listens to this standard. Systemd has a variety of options for watchdogs that allow you to control and monitor this behavior, but by using tmux we throw it all out of the window, because tmux won’t receive the SIGTERM. You’re either relying on the fact that tmux will shut down by itself, or, as in this case, write scripts to talk to tmux interactively instead. In either case, if tmux has an issue but our server doesn’t, systemd can’t do its job correctly.

There are probably a variety of other issues that arise from this that I don’t see immediately off the top of my head - while it “works”, a lot of assumptions and standards are broken, and it’s a bit of a mess. A hack by definition.

Back to more conceptual things, tmux also comes with a fair bit of overhead; it’s a full GUI after all, designed to multiplex terminals as the name says. It’s not at all intended for this strange use case the game server community keeps abusing it for, and has a lot of features that go unused or can even interfere with this use case.

1 Like
16

Those are indeed good points. However, I would still like to be able to attach directly to the stdin/stdout of the server, though, so mods like JLine for Dedicated Server can still function properly.

Additionally, from my experience, RCON really doesn’t quite compare to the actual server terminal. Yes, you can execute commands, however you can’t do stuff like watch chat responses, etc, which you could do from an attached stdin/stdout.

1 Like
17

As an additional note, systemd doesn’t actually report the server as “started”, for me it reports as “active (running)” which appears to be what is wanted. Also if the server crashes, it properly recognizes that it did and will restart itself. As far as I am aware, the type contract isn’t broken because the tmux command I use says to make a new socket then immediately detatch, leaving the socket in /run/minecraft/servername to be attached to later.

18

Interesting thoughts on systemd service design. Interacting with the server via stdin/stdout is common (and reasonable imo) for interactive administration tasks, especially when doing so via the normal game UI is unavailable, unnecessary, or otherwise inappropriate. Perhaps there are better ways to provide this functionality besides using tmux.

19

You can still do that with journalctl -f --unit. But if you insist on having an attached console, that’s what TTYPath is for, which you could combine with conspy. You may also be interested in reptyr, which is much cleaner than abusing tmux in your service definitions - though reptyr will still break logging if you actually use it.

I’m happy to contribute something using ttypath/rcon, and make options in your module available for those, if you’re willing to take contributions like that :slight_smile: I’m definitely planning to start using this long term.

2 Likes
20

Sure, PR away! I had looked into the TTYPath stuff before, but I didn’t end up using it because I honestly just didn’t understand how to properly implement it and use it.

1 Like