Conventions for service modules?

Hi nixians! I hope you are having a nice Sunday.

I want to start by saying it would be awesome to have some kind of linting when writing modules. Some kind of clippy that tells you “you are using int instead of port here”, or “use externalPath instead of path/str for a secret”, etc. I bet there are a lot of quirks that could be recommended. Actually, an entry in the wiki with “good practices for nixos modules” would also be a great start (I would do it if I knew what to write :sweat_smile:!)

RFC 42 seems like a good initial start, but it seems to be focused in reducing the number of settings thrown at a module, and suggests using the freeform type, which I think it’s great and prevents bugs from happening.

Anyway, I find myself wondering what are good practices when writing a service module and I couldn’t find much, not in the wiki, nor manual, or the internet. I’ve been reading different modules, trying to get inspiration, but I haven’t seen a clear line, so I’m opening this discussion.

Naming conventions

When designing the interface, should one try to (1) “fit” to nix, or (2) try to follow the naming conventions of the app you are trying to nixify?
Like a lot of nix services expose an address and a port, but if the app, you are trying to wrap, has listen_address. Should you use listenAddress like the app or address and port, like used in nix? What about making it easy to search in the original application’s documentation? you might not find listenAddress in camelCase.

My intuition tells me that the config should map the underlying software, to make it as familiar as possible, and to be able to follow the original software documentation without having to guess, while trying to maintain nix conventions (like camelCase).

How much should a service module provide?

Let’s take an Identity provider, how many other systems should it configure? should it also configure all reverse proxies, even though it’s not necessary to run it? I understand if it had to configure a database, without which it wouldn’t run. I feel like here the future rfc 189 could come really in handy.

What patterns do you use or recommend when writing nixos modules?

I was quite impressed by kanidm.provision, it’s like a very nice pattern for applications which cannot be configured via configuration files, and usually have settings in a database (e.g: jellyfin, *arrstack, etc)

How to deal with applications that are mostly provisioned via environment variables?

I’ve seen applications using settings to actually configure the app via environments, while at the same time, exposing an environmentFile to fill the environment.
Which is confusing, right?

Should we use settings for setting files and environment to fill the environment? some applications accept both, and it would map to what one would expect from the underlying app, leaving nix as a thin abstraction.

E.g:

app.settings = {}; // freeform
app.environment = {};  // freeform

At the same time, how much the name of the underlying configuration should change?

For example, netbird is configured via environment variables like NB_LISTEN_ADDRESS.
Should one try to leave it like the original so it can be easily looked up? or should we build an abstraction on top to make it more “fitting” to nix? Like listenAddress, which then becomes upper snake case with prefix NB?
I bet having netbird.environment.NB_LISTEN_ADDRESS would make it straightforward to find in netbird docs, and it tells you A LOT, like you already know it’s configured via env variables, and the real name. On the other hand, when you search on nixos search, it might look a bit weird.

Should we wrap secrets under a secret freeform attribute set?

This might make little sense, but grouping the secrets together would make it easier to locate them, and signal to the user that this values must be protected, a beginner nix user might not recognize that setupKeyFile should be protected.

app.secrets = {}; // These automatically either go to settings or are configured safely as env variables

Provisioning secrets

I haven’t found much explanation on how to do this, it would be nice to have a guide on how to handle secrets, and how to feed them to systemd.

systemd explains clearly (creds) how to do it if the application accepts a path to a secret, but that’s it

Should we always use types.externalPath instead of types.str or types.path? A lot of modules seem to use str or path for secrets.

Would you say this rough example is acceptable to feed a secret file that can only be configured via environments?

{ config, lib, pkgs, ... }:
{
    options.services.myservice = {
      # ...etc...
      authSecretPath = mkOption {
        type = types.externalPath; # Correct type for sensitive external files
        description = "Path to the file containing the authentication secret.";
        example = "/run/secrets/myservice-auth";
      };
    };
    config = mkIf cfg.enable {
      systemd.services.myservice = {
        script = ''
          export AUTH_SECRET="$(< "$CREDENTIALS_DIRECTORY/auth_secret")"
          exec ${getExe cfg.package}
        '';
        serviceConfig = {
          # ...etc...

          LoadCredential = "auth_secret:${cfg.authSecretPath}";
        };
      };
    };
}

Any other tips or suggestions would be appreciated.

5 Likes

env vars

the file options there are relevant for secrets, to prevent them from leaking into the (public) nix store directory. to this end, make sure to pass file path as strings, not as actual path types.

naming

there are some common fields that maybe have nix conventions, tho where we don’t need to wrap options we do try and (as per RFC42) transparently expose upstream ones

secrets

externalPath sounds advisable yeah, tho in some toy settings people may still be passing in-store credentials as it’s easy.
for LoadCredential there should be some examples in nixpkgs - e.g. homebox’s, which does in fact look similar to your example there, if a bit further generalized (to account for multiple distinct secrets like that).

How much should a service module provide?

i think so far in practice this has ended up a function of how much effort people have wanted to put into a module / how much they wanted out of it. i agree the RFC would make things easier (and in fact, i think its initial PR could use some reviews).

linting / best practices

for linting, i think there have been several different efforts at least:

there would probably be room to consolidate some more of that.

3 Likes

Hello @woile, you’re touching to a lot of topics dear to my heart.

As author of the contracts RFC you mentioned, I’m fully biased. My take on this is there must be a layer where we provide a uniform way to configure modules. NixOS modules that only provide a full freeform .settings is hard for users to configure because they need to go read the documentation of the upstream service and write all the wiring themselves. As author of a module, there’s such a big opportunity to simplify this and do the work once.

That being said, providing a NixOS module which encapsulate all the particularities of the upstream service is hard and time consuming too. I understand not everyone wants to do it.

Provision / ensure* options

I was quite impressed by kanidm.provision, it’s like a very nice pattern for applications which cannot be configured via configuration files

True. Note there has been some valid pushbacks about these kind of options. There’s an RFC about adding those using the ensure* nomenclature. The argument against it is nix is not in a good position to deal with runtime settings, which makes sense. I’ve tried and failed myself to add such options to LLDAP. There has been some work on using OpenTofu to do runtime reconciliation which might be a good idea.

(e.g: jellyfin, *arrstack, etc)

If you want still to see how to deal with those runtime settings with nix, I’m covering Jellyfin, the *arr stack and quite a few others in my repo, all setup declaratively. I would love to be able to upstream this into nixpkgs at some point.

Settings vs. environmentFile

I’ve seen applications using settings to actually configure the app via environments, while at the same time, exposing an environmentFile to fill the environment.

Should we wrap secrets under a secret freeform attribute set?

As I understand the history of this issue, the settings option allows to craft the config file using freeform NixOS module options but does not play well with secrets. On the other side, the environmentFile let’s you deal with crafting the config yourself but allows you to include secrets.

But there’s no reason one can’t embed secrets in settings directly. Some modules do it using bespoke implementations. The main reason to me it’s not ubiquitous is we never had a function in utils that can generate a config file that includes secrets in any format. We have some for json, and other formats are handled with ad-hoc code in some modules. Usually it’s one style per maintainer.

I created a PR which I’m eager to get merged because it adds a new function which can generate a config file including secrets in any format. https://github.com/NixOS/nixpkgs/pull/503858 I realize just now you commented on it already. When that PR is added, I’ll be able to upgrade NixOS modules that lack this feature.

The snippet you shared looks good at first glance.

Secrets provisioning

I haven’t found much explanation on how to do this, it would be nice to have a guide on how to handle secrets, and how to feed them to systemd.

I agree. The PR above I linked to adds a section in the NixOS manual around this. You’re still required to use an out of tree tool like sops-nix or agenix but it’s better than no documentation.

2 Likes

that #503858 would be nice, yeah.

i basically hope we can normalize having upstreams publish JSON-Schemas that we could just reuse there.

1 Like

I don’t really think this is the driving reason behind RFC 42 and the tendency to have just a settings option whenever that makes sense. It isn’t that it’s hard to make our own NixOS option interface for whatever upstream offers; it’s that upstream’s interface likely is the way it is for a good reason, and upstream is the best source of truth about that. It’s not really simplifying anything to create options that are just 1:1 mapped to an upstream setting; that only serves to make users think there’s something special about those options in NixOS when they could have just understood the upstream documentation, and they’re better off having read upstream documentation considering it’s a better source of truth with more context about things. And if it’s not 1:1 mapping, then it calls into question: Why differ from the upstream interface? Surely if this interface was actually a better interface, it would make more sense for that to be upstream’s interface as well; so why isn’t it? Is there a good reason that NixOS is the place that this interface emerges? Really often the answer is no, and the reality is that either that interface should just be upstream or (more often, probably) the upstream interface is actually better, e.g. because maybe it has to be more complicated to be usable for a broader set of use cases.

In all those contexts, just referring users to the actual settings they’re actually using by just having a settings option is pretty much ideal. If there’s a good reason the NixOS interface should differ from upstream, then that’s a different story. But I think it’s much more common that it’s just better to expose a settings option rather than try to be smarter than upstream about upstream’s interface.

4 Likes

Seconding what @ElvishJerricco wrote:

  • Settings checking is one example of something that might seem lost with unstructured settings, but it actually need not be. It is regained by when we run the program to verify the settings file. That is done build not not eval time but whatever, that’s fine.)

  • in the cases we’re we don’t do settings because upstream has some highly bespoke or irregular format, it is also good to push upstream to adopt a more regular format.

    For example, for a long time I’ve been trying to make Nix itself have settings in a more regular format :slight_smile:

2 Likes

We can also just type-check/validate individual options under settings if needed, hence freeformType submodules.

1 Like

I do agree with your arguments. But I do also see a lot of value in providing some uniform option structure for common concerns in nixpkgs. Not having that is a missed opportunity IMO.

It’s like if packages built in nixpkgs had their output completely unstructured. But that’s not the case, we did converge to having everything live in the `.out` attribute and usually each build ecosystem (Python, Golang, etc.) end up building their own tooling that uniformize everything. I’m simplifying here and happy to be corrected, but I would bet that’s part of the reason why nixpkgs is successful.

I’m convinced there’s a place for an uniformization at the NixOS module level. Some NixOS modules do work around upstream already to make things more declarative and easier to configure. I’m thinking of Nextcloud, Home Assistant and Jellyfin for example. Seeing how much work has been poured into taming those upstream services, I cannot imagine here how sending users to read upstream documentation and just providing a minimal `.settings` option is better.

But then again, I agree with you in principle. Maybe this situation is the exception and you’re right that providing a `.settings` option is doable in most cases. And in an ideal world, we could upstream the work made in those NixOS modules and make them lighter while at the same time improving the experience to setup the upstream projects. I tried that and it’s not easy.

3 Likes

skawarePackages uses multiple outputs for a lot of things, but maybe I should do another refactor to converge onto $out (and $dev if needed).
Previous refactor I did on it:

But in general you’re right, except for $dev.

Sorry, I didn’t mean it that way. Multiple outputs is nice, there’s an endeavor going on to split packages Split outputs across nixpkgs to shrink runtime closures · Issue #515268 · NixOS/nixpkgs · GitHub . So don’t go changing already split out packages! At least not based on one of my comments. :smiley:

Just to be clear, what I meant when writing about “$out” was more on the theoretical side. nixpkgs is based on packages having inputs and outputs, and this led to some unifomization which makes, from a distance, all packages look the same. All particularities of building a given package are ironed out. You never need to read the documentation of an upstream package to know how to build it. And whatever package you build, you get a result/bin directory with the related binaries, if there are any, with some expected properties like all paths point to the nix store. There’s even a lib.getExe function that returns the main binary whatever the package. That’s the kind of uniformization I was talking about and this is one huge success for nixpkgs IMO.

1 Like

OK! Thanks for clarifying that!

1 Like

I don’t think that’s a good comparison. Because it’s not being argued that should we standardise or not, it’s should we standardise things that impede on a user’s expectations for a given software. .out is something unique to Nix derivations which won’t have such dilemmas.

I agree. IMO, in an ideal world, Nix would capture every little setting along with its documentation, etc. such that I never have to leave search.nixos.org. And I like to think not many people disagree here. But it boils down to maintenance burden, hence the pull towards a minimal module interface wherever possible (of course, the services like you mentioned not hitting the “wherever possible” criteria).

Since most modules don’t require such non-trivial work, a minimal .settings works best in both principle and practice IMO. The question is, what uniformity shall we bring about for the other instances like the OP mentioned.

2 Likes