Pre-RFC: Standardization of feature parameters

[KAction]

@7c6f434c

Given that we have camel, snake, and skewer casing happily coexisting, maybe it should be written explicitly how the naming is mangled into a uniform form.

Good point. I’ll elaborate on enableFoo in the guidelines section.

[7c6f434c]

@KAction

I don’t think we ever should make it convenient to do vendorng.

Globally: good luck keeping LibreOffice or Julia at zero vendoring for multiple releases. So what happens is we look «maybe this can be unvendored», but from time to time we are forced to go back to using the vendored version of something else. Doesn’t help that vendored versions sometimes carry project-specific patches…

The thing is, if one ever has an optional dependency in a safe-to-unvendor / too-complicated-to-unvendor-again cycle, by the wording of your proposal there will be pointless with/enable churn.

[7c6f434c]

@KAction

Yes, but as a user, I want to disable Qt5, not hunt down pythonQt5, tclQt5 and so on.

As a user you’ll need to have an idea what you have, and what has mandatory Qt dependencies. So yeah, withQt5 vs withQt6 vs withPythonPyqt is the least of your problems

[KAction]

Valid point. I’ll add clause that vendored dependencies, which technically are not external dependencies, are still with, not enable.

[KAction]

@7c6f434c

As a user you’ll need to have an idea what you have, and what has mandatory Qt dependencies. So yeah, withQt5 vs withQt6 vs withPythonPyqt is the least of your problems

Can’t argue, yet I still would rather make relatively small user’s problem even smaller.

[KAction]

Latest revision. Two more entries in changelog – about vendoring and about camel case.

As writing I realized that current wording has unfortunate implication that derivations that support multiple SSL libraries with have enableSsl flag, while derivation that only support openssl will only have withOpenssl, which is somewhat inconsistent.

I have no ideas how to fix it short of hard-coding list of blessed features, like enableSsl, enablePdf.

---

feature: feature_parameter_names
start-date: 2024-01-10
author: Dmitry Bogatov
co-authors: (find a buddy later to help out with the RFC)
shepherd-team: (names, to be nominated and accepted by RFC steering committee)
shepherd-leader: (name to be appointed by RFC steering committee)
related-issues: (will contain links to implementation PRs)

Summary

Nixpkgs usually exports compile-time options of the upstream build system
through the named parameters of derivation function, but does it inconsistent
way.

Motivation

Developer interface for building environment with specific features enabled and
disabled is more complicated and requires more knowledge of implementation
details compared to already existing systems.

For example, Gentoo Linux has exhaustive list of all known configuration
flags and has means to
programmatically query what flags are available for particular package. Nixpkgs
has neither, and this RFC strives to rectify it.

Detailed design

1. Derivation function MAY expose compile-time boolean options of the upstream
   build system to enable or disable a feature using parameters named either as
   enableX or withX, consistent with Autoconf naming convention.

   Autoconf ENABLE
   Autoconf WITH

2. If compile-time feature requires build and runtime dependency on package
   libfoo, corresponding feature parameter MUST match regular expression
   ^with[^a-z]. See guidelines below for choosing name of feature parameter.

3. If compile-time feature does not require any extra build dependencies,
   corresponding feature parameter MUST have name matching ^enable[^a-z] and
   SHOULD correspond to the upstream naming.

4. As special provision, if optional vendored dependency is exposed by feature
   parameter, that paramter name MUST have match `^with[^a-z]’ regular
   expression.

5. If upstream build features and build dependencies do not map one-to-one,
   then one with feature parameter SHOULD be added for every build dependecy
   and one enable feature SHOULD be added for every upstream build feature
   intended to be optional. Assertions to preclude incoherent feature
   configurations MAY be added.

6. These rules are to be enforced by static code analyse linter. Since no
   static code analyzis is perfect, it shall have support for inhibiting
   warnings in individual cases that do not fit into general scheme.

7. Parameter names matching ^(enable|with) regular expression MUST not be
   used for any other purpose. In particular, they always must be boolean.

8. Derivation function MAY expose compile-time string or numeric options of the
   upstream build system using feature parameters that MUST match ^conf[^a-z]
   regular expression, e.g confDefaultMaildir.

9. Due overwhelming amount of possible combinations of feature flags for some
   packages, nixpkgs maintainer is not expected to test or maintain them all,
   but SHOULD accept provided technically sound contributions related to
   configurations with non-default feature flags.

10. Due overwhelming amount of possible combinations of feature flags for some
    packages, only configurations that has name in package set (e.g emacs-nox)
    shall be built on CI.

The migration process.

Named parameters are part of the programming interface, and renaming them is a
breaking change. As such, renaming of the parameters is done in following way:

1. Following function is added into lib set:

let renamed = { oldName, newName, sunset, oldValue }: newValue:
   let warning = builtins.concatStringsSep " "
      [ "Feature flag"
      , oldName
      , "is renamed to"
      , newName
      , "; old name will no longer be available in nixpkgs="
      , sunset
      , "."
      ]
   in lib.warnIf (value != null) warning (if (value != null) then value else newValue);

Starting with following function:

{ lib
, stdenv
, nonCompliantFoo ? true
}:

# uses nonCompliantFoo
stdenv.mkDerivation { ... }

First step of migration is to replace it with the following:

{ lib
, stdenv
, nonCompliantFoo ? null
, enableFoo ? lib.renamed {
    oldName = "nonCompliantFoo";
    newName = "enableFoo";
    sunset = "25.11";
    value = nonCompliantFoo;
  } true
}:

# uses enableFoo
stdenv.mkDerivation { ... }

and after grace period of two releases, any mentions of nonCompliantFoo are
removed, and function becomes:

{ lib
, stdenv
, enableFoo ? true
}:

# uses enableFoo
stdenv.mkDerivation { ... }

Feature parameter naming guidelines

1. Feature flags that require single external dependency SHOULD be named after
   that dependency. Prefix lib SHOULD be removed. For example,

   systemd => withSystemd
   libgif  => withGif
   curl    => withCurl
   

2. When multiple feature flags require the same build dependency, for example
   derivation has optional support for FTP and HTTP protocols, any of which
   incur dependency on curl, and derivation would look like following:

   { lib, stdenv, curl, withCurl ? true, enableFTP ? true, enableHTTP ? true }:
   
   # Assertions are fully optional.
   assert enableFTP -> withCurl;
   assert enableHTTP -> withCurl;
   
   stdenv.mkDerivation {
      ...
   
      buildInputs = lib.optionals withCurl [ curl ];
   
      ...
   }
   

3. Mutually-exclusive build dependencies that provide the same feature are also
   handled with assertions. For example, if derivation has optional SSL support
   that may be provided by multiple libraries, but only one may be used and it
   must be chosen at compilation time, derivation will look like following:

   { lib, stdenv, enableSSL ? false, openssl, withOpenSSL ? false, libressl, withLibreSSL ? false }:
   
   # Assertions are fully optional.
   assert enableSSL -> withOpenSSL || withLibreSSL;
   assert !(withLibreSSL && withOpenSSL);
   
   stdenv.mkDerivation {
      ...
   
      # Asserts above make sure that at most one SSL implementation will be in
      # the list.
      buildInputs = lib.optionals withLibreSSL [ libressl ]
                  ++ lib.optionals withOpenSSL [ openssl ];
   
      ...
   }
   

4. When build dependency comes from particular package set, it MAY make sense to
   name feature parameter after it. E.g build dependency on qt6.qtbase should
   have withQt6 feature parameter.

5. If feature parameter name suggested by previous point is too generic,
   package name from the set MAY be included into the feature parameter name.
   Optional dependency on pythonPackages.pillow MAY have feature parameter
   withPythonPillow.

6. Build dependency on bindings to low-level libraries SHOULD be named after
   underlying library. For example, optional dependecy on pyqt5 Python
   bindings to Qt5 library should have withQt5 feature parameter.

7. Camel case is preferred to other styles. E.g:

pam                   => withPam
http                  => enableHttp
gemini                => enableGemini
ssl                   => enableSsl
openssl               => withOpenssl
pythonPackages.pillow => withPythonPillow

Examples and Interactions

This is how one can query list of feature parameters that incur extra runtime dependencies:

$ nix eval --json --impure --expr 'with import ./. {}; builtins.attrNames gnuplot.override.__functionArgs' | jq '.[]| select(startswith("with"))'
"withCaca"
"withLua"
"withQt"
"withTeXLive"
"withWxGTK"

I picked the gnuplot as example since it is the closest to be compliant with proposed rules.

Drawbacks

1. The migration process involves renaming feature parameters, so these changes
   will likely conflict with other outstanding changes to the derivation, and potentially
   even with automatic version bumps.

Alternatives

Group all feature parameters into separate attrset parameter

Instead of writing

{ lib, stdenv, enableFoo ? true }:

# uses enableFoo
stdenv.mkDerivation { ... }

derivation can be written as

{ lib, stdenv, features ? { enableFoo = true; }}:

# uses features.enableFoo
stdenv.mkDerivation { ... }

It is definitely looks cleaner, but unfortunately hits several limitations of
the Nix language.

1. Nix language provides way to introspect function argument names,
   but no way to learn their default values. So this approach gives consistency, but no way to query list of derivation feature parameters.

2. Overrides become much more verbose. Simple and ubiquitous

   bar.override { withFoo = true; }
   

   becomes unwieldy

   bar.override (old: old // { features = old.features // { withFoo = true }; })
   

   That can be simplified by introducing overrideFeatures function, but we
   already have way too many
   override functions. Also, this version will silently override nothing in case
   of typo.

Any approach that tries to pass attribute set to function will have these
issues. Using existing config.nix
is no different.

Do nothing

Avoids all the work and drawbacks, but there is no evidence that consistency
problem will solve itself evolutionary.
For some build dependencies, we have multiple equally popular feature parameter
names, and people keep picking random one when adding new packages.

Prior art

As mentioned in motivation part, the best in class of feature flag
configuration system is Gentoo Linux:

• [USE flag index – Gentoo Linux]

There is not much work I can find about this problem in Nixpkgs other than my
previous attempts to solve it on package-by-package basis:

• Naming convention for optional feature flags · Issue #148730 · NixOS/nixpkgs · GitHub
• dbus: rename "enableSystemd" to "withSystemd" by KAction · Pull Request #234463 · NixOS/nixpkgs · GitHub

Unresolved questions

This RFC makes it possible to introspect feature parameters of particular
derivation, but still does not provide simple and efficient way to list all
existing feature parameters.

Future work

There are other configuration scenarios not covered by this RFC:

• Optional dependencies in shell wrappers (e.g passage).
• Finding way to get list of all existing feature parameters. That can be possibly done by building and distributing the index separately,
  like nix-index does it.

Changelog

1. Changed wording to not imply that every upstream build system knob SHOULD be
   exported via feature parameters. (Thx: 7c6f434c)

2. Relaxed wording on the name of feature parameters to avoid painting ourselves
   into ugly and non-intuitive names. (Thx: 7c6f434c)

3. Fix typo in regex to be consistent that feature flag name can’t have small
   letter after with|conf|enable prefix. (Ths: don.dfh)

4. Explicitly mention that static code analysis has support for overrides based
   on human judgement call. (Thx: 7c6f434c)

5. Clarify solution scenarios when build inputs and feature flags don’t match
   one-to-one. (Thx: Atemu, 7c6f434c)

6. Refine the deprecation plan to make sure the warning includes the sunset
   timestamp. (Thx: pbsds)

7. Add rules about non-boolean feature parameters. (Thx: Atemu, pbsds)

8. Set expectations for building and maintaining multiple configurations. (Thx: pbsds)

9. Removed non-boolean parameters from “Future Work” section.

10. Relaxed requirements for assertions about conflicting flags (Thx: Atemu)

11. Add guideline so pythonPackages.pillow does not get withPython feature name. (Thx: 7c6f434c)

12. Mention that vendored dependenices are still with. (Thx: 7c6f434c)

13. Elaborate on camel case convention. (Thx: 7c6f434c)

[rnhmjoj]

It’s enough that NixOS is full of non-local effects on a fixpoint (fine with me, not going to use NixOS anytime soon anyway), but we have actual functions with actual scope in Nixpkgs and it should be kept this way.

It would have to be a somewhat restricted version of the module system anyway, otherwise there’s no chance of scaling it to ~100000 packages. Evaluating NixOS is already slow with less than 2000 modules.

[7c6f434c]

@rnhmjoj

Evaluating NixOS is already slow with less than 2000 modules.

And, separately, evaluating Nixpkgs for CI is already too slow (the current proposal looks almost neutral, though)

[Atemu]

I’d put package modularisation entirely out of scope for this RFC.

@KAction

enableSsl

I kinda don’t like camelCase’d abbreviations. I’d prefer if those were allowed to be upper case. (All the other examples also use upper case.)

@KAction

Yes, but as a user, I want to disable Qt5, not hunt down pythonQt5, tclQt5 and so on. So I’d keep this pass-through clause.

As mentioned before, I think it’s better to handle this at another level. If we want global overrides a la USE-flags, we’d simply set the precise flags dependant on the global flag.

The default value of withQT6 would simply be withQT such that, if the high-level withQT flag was set to false, the low-level withQT5, withQT6 etc. would also be set to false (unless overridden otherwise).

Speaking of dependencies, another aspect I’d like to see at least mentioned in the RFC is setting flags depending on the required with and/or enable flags. E.g. enableSSL ? withOpenSSL || withLibreSSL. I don’t think this should be required but at least recommended.

Another aspect I also think is missing is to state/limit which dependencies should be configurable because I do not believe every dependency needs to be.

[delroth]

Who’s taking on the cost of this migration? And what’s the timeline for it actually being done? Because unfortunately writing an RFC defining how things should be doesn’t make them be that way, and RFCs that don’t match reality are confusing more than anything.

If you expect package maintainers to take care of the migration, I’d like to see a discussion of how that’s supposed to go since I suspect many package maintainers don’t particularly care about exposing those configuration knobs - if anything, for the packages I maintain, they’re annoying as they increase complexity and make testing of all the configurations almost impossible.

[7c6f434c]

@delroth

I suspect many package maintainers don’t particularly care about exposing those configuration knobs

Which is why «SHOULD expose» from the first version was changed to «MAY expose».

My forecast is: if the RFC gets passed, there will be some tree-wide grepping and renaming by some extreme-uniformists (and as a 0-rebuild implementation of RFC naming, it will get merged in a couple of months), then the CI checks will be in a debugging limbo forever (not deployed as CI, but maybe used by someone to do cleanups with manual vetting).

[Atemu]

@7c6f434c

My forecast is: if the RFC gets passed, there will be some tree-wide grepping and renaming by some extreme-uniformists (and as a 0-rebuild implementation of RFC naming, it will get merged in a couple of months), then the CI checks will be in a debugging limbo forever (not deployed as CI, but maybe used by someone to do cleanups with manual vetting).

That’s also how I’d have assumed it to go but it should be added to the RFC as the implementation plan:

• In existing packages, maintainers MAY start to expose flags and MAY cease exposing them
• Package maintainers MAY change their package to be compliant with the RFC
• Interested users SHOULD change packages they’re interested in to be compliant with the RFC
• Implementers MAY involve maintainers but SHOULD NOT be required to do so because the rules MUST be clear
• New packages MUST be compliant with the RFC (either no options or compliant options)

[KAction]

Yes, that is pretty much the plan – declare the rules, accept that currently they are violated at huge scale, forbid introducing more violations, and steadily fix existing violations.

Thank you for the writing it up in such detailed manner, I’ll incorporate it, but not sure I’ll have time until next weekend to incorporate the feedback and post the new revision.

@Atemu

Another aspect I also think is missing is to state/limit which dependencies should be configurable because I do not believe every dependency needs to be.

I think it is outside of the scope of RFC. It deals with naming of flags should somebody create them, not ruling on when or whether that flag is necessary.

@Atemu

I kinda don’t like camelCase’d abbreviations. I’d prefer if those were allowed to be upper case. (All the other examples also use upper case.)

I’ll fix examples, thanks. I don’t think camelCase is fundamentally better, but it allows different flags to look at least somewhat consistent. If I allow feature name flag to follow upstream styling, I’ll get zoo like enableHTTP, enableSSL, withQt, withPAM, withMpc, enableGemini. Personally I prefer with-lisp-style, but that will looks completely alien.

@Atemu

Speaking of dependencies, another aspect I’d like to see at least mentioned in the RFC is setting flags depending on the required with and/or enable flags. E.g. enableSSL ? withOpenSSL || withLibreSSL. I don’t think this should be required but at least recommended.

Thanks. Good idea.

@Atemu

As mentioned before, I think it’s better to handle this at another level. If we want global overrides a la USE-flags, we’d simply set the precise flags dependant on the global flag.

The default value of withQT6 would simply be withQT such that, if the high-level withQT flag was set to false, the low-level withQT5, withQT6 etc. would also be set to false (unless overridden otherwise).

I want package to have withQt5 flag, but I definitely don’t want packages to have withPythonQt5 flag (unless there are multiple python Qt binding packages can be used).

[Atemu]

@KAction

I think it is outside of the scope of RFC. It deals with naming of flags should somebody create them, not ruling on when or whether that flag is necessary.

I think it’d be good to explicitly state that it is entirely at the maintainer’s discretion which flags to expose.

@KAction

I don’t think camelCase is fundamentally better, but it allows different flags to look at least somewhat consistent. If I allow feature name flag to follow upstream styling, I’ll get zoo like enableHTTP, enableSSL, withQt, withPAM, withMpc, enableGemini.

I’d personally prefer slightly inconsistent but decent looking to consistently ugly.

Take for instance enableSsh, enablePam, enableSsl, withGpl, withLlvm or withFfmpeg. Those look really bad IMHO.

@KAction

I want package to have withQt5 flag, but I definitely don’t want packages to have withPythonQt5 flag (unless there are multiple python Qt binding packages can be used).

I agree on that specific case but, IMO, this leads into “which flags should you expose” territory. Probably nobody cares about PythonQt5 but it is conceivable to want to disable Qt. Should the flag then follow the (hypothetical) configure script naming of withPythonQt5 or should it use withQt5 or even just withQt?

In a similar vein, some configureFlags may have different names across packages but practically do the same thing. I.e. X11 support where actual configureFlags may be named xorg, X11, xlib, xcb etc. or use the names of individual X libraries and/or wrappers.

I think it’d be useful to unify these. Perhaps there could be some sort of “registry” for commonly used & confused ones to declare one name to rule them all.

[KAction]

@Atemu

I think it’d be useful to unify these. Perhaps there could be some sort of “registry” for commonly used & confused ones to declare one name to rule them all.

Yes. I think it will arise naturally as part of migration process. I am currently researching on what would be the best way to automate migration.

[KAction]

---

feature: feature_parameter_names
start-date: 2024-01-10
author: Dmitry Bogatov
co-authors: (find a buddy later to help out with the RFC)
shepherd-team: (names, to be nominated and accepted by RFC steering committee)
shepherd-leader: (name to be appointed by RFC steering committee)
related-issues: (will contain links to implementation PRs)

Summary

Nixpkgs usually exports compile-time options of the upstream build system
through the named parameters of derivation function, but does it inconsistent
way.

Motivation

Developer interface for building environment with specific features enabled and
disabled is more complicated and requires more knowledge of implementation
details compared to already existing systems.

For example, Gentoo Linux has exhaustive list of all known configuration
flags and has means to
programmatically query what flags are available for particular package. Nixpkgs
has neither, and this RFC strives to rectify it.

Detailed design

When upstream build system has options to enable or disable a optional features
or configure some compile-time parameter, like path to default mailbox or
initial size of hash map, derivation function (for short, package) MAY expose
compile-time options via feature parameters that match
^(with|conf|enable)_[a-z_0-9]$ regular expression. For short, we will refer
to them as with-parameter, conf-parameter and enable-parameter correspondingly,
collectively as feature parameters.

Note that this RFC does not establish any requires whether package should or
should not have any feature parameters, this decision is left at descretion of
the package maintainer(s). This RFC only concerns itself with the naming of the
feature flags. Further text will use wording “corresponding feature parameter”
with implicit caveat “provided that package maintainer decided to export
underlying build system configuration option”.

Feature parameters MUST only be used according to the rules outlined in this
RFC. They MUST NOT be used for any other purpose.

Boolean feature parameter MUST be either with-parameter or enable-parameter.
Non-boolean (e.g. string or integer) feature parameter MUST be an conf-parameter.

Feature paramter that incurs additional runtime dependency MUST be and with-parameter.

Example

{ lib, stdenv, mpfr, with_mpfr ? true }:

stdenv.mkDerivation {
  # ... snip ...
  configureFlags = lib.optionals with_mpfr ["--with-mpfr"];
  buildInputs = lib.optionals with_mpfr [ mpfr ];
  # ... snip ...
}

As special provision, if optional vendored dependency is exposed by feature
parameter, it MUST be with-parameter.

If feature parameter does not incur extra runtime dependencies, corresponding
feature parameter MUST be enable-parameter and its name SHOULD correspond to
the upstream naming, unless is causes major inconsistency with other packages.
Consistency of feature parameters across nixpkgs is more important than
matching to the names used by particular upstream. What constitutes a major
inconsistency is left at discretion of the package maintainer(s).

Example

If upstream uses --enable-translations to denote support for NLS,
corresponding feature parameter SHOULD be enable_nls, since it is much more
common term for enabling programs to produce output in non-English languages.

Rationale

This distinction between with and enable feature parameters is based on Autoconf naming conventions.

Autoconf ENABLE
Autoconf WITH

If upstream build features and build dependencies do not map one-to-one, then
one with-parameter SHOULD be added for every build dependecy and one
enable-parameter SHOULD be added for every upstream build feature intended to
be optional. Assertions to preclude incoherent feature configurations MAY be
added.

Example

{ lib
, stdenv
, openssl
# At maintainer(s) discretion some coherent configuration is chosen.
, with_openssl ? true
, wolfssl
, with_wolfssl ? false
, enable_ssl ? with_openssl || with_wolfssl
}:

# Assertions are optional and might be infeasible when package has huge amount
# of feature flags, but in general improves user experience.
assert enable_ssl -> with_openssl || with_wolfssl;

stdenv.mkDerivation {
  # ... snip ...
  configureFlags = lib.optionals enable_ssl ["--enable-ssl"];
  buildInputs = lib.optionals with_wolfssl [ wolfssl ]
             ++ lib.optionals with_openssl [ openssl ];
  # ,,, snip ...
}

Maintainer(s) MAY choose to add both with and enable feature flags even if
they map one-to-one for consistency with other packages.

Example

{ lib
, stdenv
, openssl
, with_openssl ? true
, enable_ssl ? with_openssl
}:

assert enable_ssl -> with_openssl;

stdenv.mkDerivation {
  # ... snip ...
  configureFlags = lib.optionals enable_ssl ["--enable-ssl"];
  buildInputs = lib.optionals with_openssl [ openssl ];
  # ,,, snip ...

  passthru.features = {
    inherit with_openssl enable_ssl;
  };
}

The rules are to be enforced by static code analyse linter to be written. Since
no static code analyzis is perfect, it shall have support for inhibiting
warnings in unsual cases.

All feature parameters SHOULD be exported in features passthrough set. See example above.

Due overwhelming amount of possible combinations of feature flags for some
packages, nixpkgs maintainer is not expected to test or maintain them all, but
SHOULD accept provided technically sound contributions related to
configurations with non-default feature flags. Only configurations that has
name in package set (e.g emacs-nox) SHOULD be built on CI.

The migration process.

Orgranization rules

By the very nature of this RFC, once it passes, numerous packages in Nixpkgs
will become non-compliant. This is unavoidable, otherwise this RFC would not be
necessary. The following organizational process shall be followed to ensure
that nixpkgs will eventually become compliant with this RFC:

• In existing packages, maintainers MAY start to expose feature parameters and MAY cease exposing them.
• Further contributions MAY change their package to be compliant with the RFC
• Further contributions MUST NOT rename non-compliant feature parameters to another non-compliant name.
• Further contributions MUST NOT rename compliant feature parameters to non-compliant parameter.
• Further contributions MAY rename compliant feature parameters to another compliant name
  for the purposes of consistency with other packages.
• Further contributions that improve compliance with this RFC MAY involve
  maintainers but SHOULD NOT be required to do so because the rules MUST be
  clear.
• New packages MUST be compliant with the RFC (either no feature parameters or compliant with this RFC.)

This ensures that count of non-compliant feature parameters in nixpkgs is
non-increasing over the time.

Technical process

Named parameters are part of the programming interface, and renaming them is a
breaking change. As such, renaming of the parameters is done in following way:

1. Following function is added into lib set:

let renamed = { oldName, newName, sunset, oldValue }: newValue:
   let warning = builtins.concatStringsSep " "
      [ "Feature flag"
      , oldName
      , "is renamed to"
      , newName
      , "; old name will no longer be available in nixpkgs="
      , sunset
      , "."
      ]
   in lib.warnIf (value != null) warning (if (value != null) then value else newValue);

Starting with following package:

{ lib
, stdenv
, nonCompliantFoo ? true
}:

# uses nonCompliantFoo
stdenv.mkDerivation { ... }

the first step of migration is to replace it with the following:

{ lib
, stdenv
, nonCompliantFoo ? null
, enable_foo ? lib.renamed {
    oldName = "nonCompliantFoo";
    newName = "enable_foo";
    sunset = "25.11";
    value = nonCompliantFoo;
  } true
}:

# uses enable_foo
stdenv.mkDerivation { ... }

where sunset parameter is set two full releases after time of this change.
So, if this change is made in Jan 2024, sunset MUST be set to 25.11. After
the release 25.05 all mentions of nonCompliantFoo SHOULD be removed, and
package will look like following:

{ lib
, stdenv
, enable_foo ? true
}:

# uses enable_foo
stdenv.mkDerivation { ... }

Feature parameter naming guidelines

Rules in the first section of this RFC describe different scenarios when
with, enable and conf feature parameters must be used, but coming with
exact name that is clear, concise, describes it effect correctly and is
consistent across the nixpkgs is hard to codify with absolute rule. Instead,
this RFC provides set of guidelines and examples based on scenarios brought up
during the discussion.

Feature flags that require single external dependency SHOULD be named after
that dependency. Prefix lib SHOULD be removed. For example,

Example

systemd => with_systemd
libgif  => with_gif
curl    => with_curl
alsa    => with_alsa

# When people say "Xorg" support, they usually mean linking "X11"
# library which is one of the ways to talk X protocol to X server. "xcb"
# being another one.
#
# I am not aware of anybody implementing X protocol themself, and not
# using one of these libraries.

xorg    => with_x11
xcb     => with_xcb
Qt5     => with_qt5

When multiple feature flags require the same build dependency, for example
derivation has optional support for FTP and HTTP protocols, any of which
incur dependency on curl, the package would look like following:

Example

{ lib
, stdenv
, curl
, with_curl ? true
, enable_ftp ? true
, enable_http ? true
}:

# Assertions are fully optional.
assert enable_ftp -> with_curl;
assert enable_http -> with_curl;

stdenv.mkDerivation {
   # ... snip ...

   configureFlags = lib.optionals enable_http ["--enable-http"]
                 ++ lib.optionals enable_ftp ["--enable-ftp"];
   buildInputs = lib.optionals withCurl [ curl ];

   # ... snip ...

   passthru.features = {
      inherit with_curl enable_ftp enable_http;
   };

   # ... snip ...
}

When build dependency comes from particular package set, it MAY make sense to
name feature parameter after it. E.g build dependency on qt6.qtbase SHOULD
have with_qt6 feature parameter. If that results in feature parameter name
that is too generic, package name from the set MAY be included into the feature
parameter name. Optional dependency on pythonPackages.pillow MAY have feature
parameter with_python_pillow.

Build dependency on bindings to low-level libraries SHOULD be named after
underlying library. For example, optional dependecy on pyqt5 Python bindings
to Qt5 library should have with_qt5 feature parameter.

Examples and Interactions

This is how one can query list of feature parameters that incur extra runtime dependencies:

$ nix eval --json --impure --expr 'with import ./. {}; builtins.attrNames gnuplot.override.__functionArgs' | jq '.[]| select(startswith("with"))'
"withCaca"
"withLua"
"withQt"
"withTeXLive"
"withWxGTK"

I picked the gnuplot as example since it is the closest to be compliant with proposed rules.

Drawbacks

1. The migration process involves renaming feature parameters, so these changes
   will likely conflict with other outstanding changes to the derivation, and potentially
   even with automatic version bumps.

2. While migration is in the process (which is 1 year minimum due the grace
   period), nixpkgs source will be even less consistent than it is today.

Alternatives

Do nothing

Avoids all the work and drawbacks, but there is no evidence that consistency
problem will solve itself evolutionary.
For some build dependencies, we have multiple equally popular feature parameter
names, and people keep picking random one when adding new packages.

Other ways to pass feature parameters

Group all feature parameters into separate attrset parameter

Instead of writing

{ lib, stdenv, enable_foo ? true }:

# uses enable_foo
stdenv.mkDerivation { ... }

derivation can be written as

{ lib, stdenv, features ? { enable_foo = true; }}:

# uses features.enable_foo
stdenv.mkDerivation { ... }

It is definitely looks cleaner, but unfortunately hits several limitations of
the Nix language.

1. Nix language provides way to introspect function argument names,
   but no way to learn their default values. So this approach gives consistency, but no way to query list of derivation feature parameters.

2. Overrides become much more verbose. Simple and ubiquitous

   bar.override { with_foo = true; }
   

   becomes unwieldy

   bar.override (old: old // { features = old.features // { with_foo = true }; })
   

   That can be simplified by introducing overrideFeatures function, but we
   already have way too many
   override functions. Also, this version will silently override nothing in case
   of typo.

Any approach that tries to pass attribute set to function will have these
issues. Using existing config.nix
is no different.

This could be viable and actually more elegant solution if Nix to be extended
to be able to introspect default parameter values, but that would make nixpkgs
incompatible with older installations of the Nix.

Other ways to name parameters

Other naming conventions were considered and found less appealing.

1. Lisp-style with-foo variable names are inconvenient when they need to be
   passed to the builder environment as opposed being used exclusively at
   evaluation phase, since Bash does not support referring to environment
   variables that are not C identifier.

2. Camel case is the most popular naming convention in Nixpkgs at the time of
   writing, but adding prefix results in quite unnatural-looking identifiers
   like enableSsh or withLlvm. In addition, proposed convention visually
   distinguish feature parameters from other kinds of parameters.

3. Camel case, but allowing to follow upstream spelling, produces naturally
   looking identifies like enableSSH, withLibreSSL, enableAudio that are
   wildly inconsistent between each other.

Prior art

As mentioned in motivation part, the best in class of feature flag
configuration system is Gentoo Linux:

• [USE flag index – Gentoo Linux]

There is not much work I can find about this problem in Nixpkgs other than my
previous attempts to solve it on package-by-package basis:

• Naming convention for optional feature flags · Issue #148730 · NixOS/nixpkgs · GitHub
• dbus: rename "enableSystemd" to "withSystemd" by KAction · Pull Request #234463 · NixOS/nixpkgs · GitHub

Unresolved questions

This RFC makes it possible to introspect feature parameters of particular
derivation, and makes recomendations to keep feature parameter names consistent
across whole nixpkgs, but still does not provide simple and efficient way to
list all existing feature parameters.

Future work

There are other configuration scenarios not covered by this RFC:

• Optional dependencies in shell wrappers (e.g passage).

• Finding way to get list of all existing feature parameters, like Gentoo does.
  That can be possibly done by building and distributing the index separately, like nix-index does it,
  or requiring every feature parameter used in any package to be listed in some index file kept under version control in nixpkgs.

• Meta-feature-flags, that would allow user to set, e.g enable_gui = false on
  the top level of the overlay, and that would disable support for X11, Qt4,
  Qt6, GTK2, GTK3, Motif and all other graphical libraries and toolkits for all
  packages that support doing so.

Changelog

Folded changelog

1. Changed wording to not imply that every upstream build system knob SHOULD be
   exported via feature parameters. (Thx: 7c6f434c)

2. Relaxed wording on the name of feature parameters to avoid painting ourselves
   into ugly and non-intuitive names. (Thx: 7c6f434c)

3. Fix typo in regex to be consistent that feature flag name can’t have small
   letter after with|conf|enable prefix. (Ths: don.dfh)

4. Explicitly mention that static code analysis has support for overrides based
   on human judgement call. (Thx: 7c6f434c)

5. Clarify solution scenarios when build inputs and feature flags don’t match
   one-to-one. (Thx: Atemu, 7c6f434c)

6. Refine the deprecation plan to make sure the warning includes the sunset
   timestamp. (Thx: pbsds)

7. Add rules about non-boolean feature parameters. (Thx: Atemu, pbsds)

8. Set expectations for building and maintaining multiple configurations. (Thx: pbsds)

9. Removed non-boolean parameters from “Future Work” section.

10. Relaxed requirements for assertions about conflicting flags (Thx: Atemu)

11. Add guideline so pythonPackages.pillow does not get withPython feature name. (Thx: 7c6f434c)

12. Mention that vendored dependenices are still with. (Thx: 7c6f434c)

13. Elaborate on camel case convention. (Thx: 7c6f434c)

14. Explicitly mention that what feature parameters to introduce is left at discretion of the maintainer (Ths: Atemu)

15. Add clause about passthrough features set.

16. Switch to snake_case since it looks more consistent and less ugly than alternatives.

17. Mention meta-parameters in the “Future Work” section.

18. Elaborate on benefits if Nix were to allow introducing of default values.

[7c6f434c]

1. Conversion rules are not clear in case a package was already camel-case-named.
2. Given the motivation as written, it would be enough to say that zero-rebuild rename-with-watning of equivalent feature flags (vendoring does not break equivalence, high-level bindings are equivalent to low-level target library) towards the majority — however slim — should be accepted. If the current version demands renaming almost all the feature flags across Nixpkgs, the do-nothing should-accept-uniformisation approach is an alternative worth listing.

[KAction]

@7c6f434c

Conversion rules are not clear in case a package was already camel-case-named.

Can you provide an example, please? It sounds like violation of existing package naming convention (https://github.com/NixOS/nixpkgs/blob/f7641bb141af756507462b4f0922b26f80551797/pkgs/README.md#package-naming)

@7c6f434c

Given the motivation as written, it would be enough to say that zero-rebuild rename-with-watning of equivalent feature flags (vendoring does not break equivalence, high-level bindings are equivalent to low-level target library) towards the majority — however slim — should be accepted. If the current version demands renaming almost all the feature flags across Nixpkgs, the do-nothing should-accept-uniformisation approach is an alternative worth listing.

Good point. I’ll add idea of skip the rules, rename towards the majority on feature by feature basis. That will be less disruptive, less work but we will end up with something like supportSystemd, enableAlsa and withSDL.

[Atemu]

@KAction

Further contributions that improve compliance with this RFC MAY involve
maintainers but SHOULD NOT be required to do so because the rules MUST be
clear.

We might want an escape hatch here where we can add minor uncontroversial rules to some Nixpkgs doc later without requiring a whole RFC process such that they actually are clear in all cases.

@KAction

, nonCompliantFoo ? null
, enable_foo ? lib.renamed {
    oldName = "nonCompliantFoo";
    newName = "enable_foo";
    sunset = "25.11";
    value = nonCompliantFoo;
  } true

This is incredibly verbose and does not stand the ffmpeg test. A refactor of args for a large package would double the line numbers with mostly unimportant information.

We need to find a better way.

Perhaps for large refactors we could simply give up and say “Look, there was a large refactor and you need to redo your override entirely.”; not specifying every now-changed parameter.

@KAction

hard to codify with absolute rule. Instead,
this RFC provides set of guidelines and examples

Great approach, I like it.

@KAction

, with_curl ? true
, enable_ftp ? true
, enable_http ? true

I’d prefer

, with_curl ? true
, enable_ftp ? with_curl
, enable_http ? with_curl

such that the enable options get disabled automatically if the user does not want the required deps.

If we wanted Gentoo-style USE-flags, we need to do it this way because otherwise if you disabled a certain dep, you’d have to hunt for enable options in hundreds of packages that are still true while the actual dep the flag requires isn’t present due to the global with flag.
That should probably be part of the RFC.

---

Now for the elephant in the room: Parameter naming.

Snake_case won’t happen.

At the very least not in this RFC. That’s just too radical of a change and inconsistent with almost every other use of casing in Nixpkgs. A bare-minimum pre-requisite would be a separate RFC on standardising acceptable and unacceptable casing for Nixpkgs as a whole. I don’t see any chance for acceptance of cases other than camelCase right now.

@KAction

Camel case, but allowing to follow upstream spelling, produces naturally
looking identifies like enableSSH, withLibreSSL, enableAudio that are
wildly inconsistent between each other.

As mentioned previously, I really don’t see that as a bad thing.

Inconsistency is an issue when the same thing is represented differently at different points but here we represent different things in different manners, so consistency, while preferable, isn’t that big of an issue.

The only issue would be consistency of flag names between packages and that should definitely be enforced as we don’t want 3 packages to have 3 different ways of spelling withOpenSSL.

Even if you wanted overall consistency though, it’s still possible with this simple rule:

• Words, abbreviations and names start with an upper case character and end in lower case
• Acronyms are always in full caps

Given the same interpretation of what a word, name or acronym is, this is always consistent and does not look ugly in the vast majority of cases IME.

Trying this rule on ffmpeg’s numerous flags it works very well. Well enough that I’d consider applying it if this RFC wouldn’t likely necessitate a large refactor down the line anyways.

The only names which don’t look very well using this rule I could find were chainings of acronyms: withFDKAAC withGNUTLS withSVTAV1

Not nearly as terrible as the likes of withLlvm but they certainly don’t look amazing.

These could be fixed by enforcing a - separator between multiple acronyms: withFDK-AAC withGNU-TLS withSVT-AV1. Alternatively, withGNUTLS could also be fixed by interpreting “GNU” as a name which is also the upstream naming: withGnuTLS.
Though, honestly, I’d find these acceptable even without acronym separator.

I also believe that if we had exceptions for a couple few percent of names, I don’t think it’d be too bad despite being inconsistent. Perfection shouldn’t be the enemy of good and I think readable names have more upsides than 100% consistent names.

[KAction]

@Atemu

Snake_case won’t happen.

At the very least not in this RFC. That’s just too radical of a change and inconsistent with almost every other use of casing in Nixpkgs.

I would keep snake_case until and if it gets more wider and unanimous rejection. I know, it is inconsistent with other stuff, but as I mentioned above, it makes feature parameters to stand out from other parameters. Different things look different.

Other naming conventions are mentioned in alternatives.

@Atemu

If we wanted Gentoo-style USE-flags, we need to do it this way because otherwise if you disabled a certain dep, you’d have to hunt for enable options in hundreds of packages that are still true while the actual dep the flag requires isn’t present due to the global with flag.
That should probably be part of the RFC.

This is tricky. User may want to say I have no need for SSL, and that would ideally mean that with_openssl, with_wolfssl, with_libressl should all default on enable_ssl.

On other hand, user may say I want to get rid of curl and sacrifice whatever features it means. Then enable_http, enable_ftp, enable_pop3 should all default on with_curl.

I don’t known how to reconcile these two non-default needs, so one of them will have to do the hunting. I’d rather avoid choosing the sides here.

@Atemu

This is incredibly verbose and does not stand the ffmpeg test. A refactor of args for a large package would double the line numbers with mostly unimportant information.

I see two other options. First, is to just accept the breaking change. At my $dayjob, it usually takes at least couple days to upgrade to next nixpkgs release. Usually, because of GHC and python libraries upgrades, nix breakages are usually the least of the problems.

Second one is to sweep all this mess into callPackage. So, packages just s/nonCompliantFoo/enable_foo/g, and we have long mapping of oldName => newName in the guts of callPackage. Did not try, but I’d imagine that evaluation time will increase noticeably.