I am exploring different options to react to vulnerabilities in packages distributed by any NixOS version.
Scenario
Let’s imagine we are building a distribution based on NixOS that packages a Jellyfin and Caddy installation ready for others to self host. The promise of this distribution is that others can just use it and never need to worry about config or updates except for customizing Jellyfin.
Let’s also imagine that this distro self-updates every hour or so with what is defined at the distribution source (e.g. a Github repo). Now a vulnerability is published for Caddy. To deliver on the promise to users, that they don’t need to worry about updates, the goal is for the distro maintainers to close the vulnerability with a patched version as soon as possible.
So there are two paths: Upgrade or downgrade to a non-vulnerable version. As far as I know, upgrading involves:
- Caddy maintainers must close the vulnerability, a new version appears
- New version must be updated in nixpkgs (PRs, CI runs, …)
- Distro maintainers must update nixpkgs input
- Instances of the distro must self update
Downgrading is simpler (ignoring possible backwards incompatibilities with other deployed software):
- Downgrade the package somewhere
- Instances must self-update
Either way, the package version must be overwritten in the distro. I have explored ways to do that and have found the following options:
- Using package overrides
- Importing other nixpkgs versions (in case of downgrade)
- Deploying Docker containers to prevent issues of 1 and 2
- Maintaining a “faster” fork of nixpkgs
- Maintaining nixpkgs overlays
- Just waiting for nixpkgs
1. Using Package Overrides
Just overriding the package and fetching from source:
Example:
pkgs.htop.overrideAttrs (finalAttrs: prevAttrs:
{
src = fetchFromGitHub {
owner = "htop-dev";
repo = "htop";
tag = "3.5.0";
sha256 = "1r07ri0dl9xz0jn1hc2g8q3l0q4clvnxs05yagpzsz0zigp4flj5";
};
}
)
Pros
- Easy to automate and quick to apply
- Distro maintainers control software update time
- Easy to maintain and easy to remove once upstream nixpkgs catches up
Cons
- Build definition dependency on nixpkgs can cause trouble and more overwrite work, slowing down patching
- NixOS module code when using
services.caddy.must match. Can only be prevented if the module code is maintained by distro maintainers themselves. - The nar-hash must be computed outside of nix eval. Automating this introduces a new trust vector (hash source could be compromised).
- Builds from source and misses nixpkgs cache
2. Importing Other nixpkgs Versions
The basic idea is to just use a nixpkgs version that patched it already.
Best case scenario is replacing only the package version:
services.caddy.package = inputs.nixpkgs-2405.legacyPackages.${system}.caddy;
Otherwise, overwriting the nixpkgs module is necessary:
{ inputs, system, ... }:
{
disabledModules = [ "services/web-servers/caddy.nix" ];
imports = [
"${inputs.nixpkgs-2405}/nixos/modules/services/web-servers/caddy.nix"
];
services.caddy = {
enable = true;
package = inputs.nixpkgs-2405.legacyPackages.${system}.caddy;
};
}
Pros:
- Prevents compatibility issues of #1
- Uses nixpkgs cache
Cons:
- Not feasible if no nixpkgs version is available that contains a non-vulnerable version (especially when trying to upgrade to newer package version)
- More code to maintain
- Old nixpkgs tree may introduce other vulnerable packages
- Old imported module using newer nixpkgs can introduce subtle incompatibilities
3. Deploying Docker Containers
The idea is to simply not depend on nixpkgs, since most software maintainers publish images as part of their normal release process. I don’t like this, but listed for completeness.
Pros:
- Faster than nixpkgs (patched version means patched container image most of the time)
- Supports immutable reference with image digest pinning
Cons:
- Losing a lot of NixOS and nixpkgs benefits (e.g. the module system integration)
- Pulls in separate dependency stack that needs different handling (e.g. SBOM creation, …)
- Quality of container images depends on software maintainers directly
4. Maintaining a “Faster” Fork of nixpkgs
The idea is to fork nixpkgs, update the vulnerable package as well as the NixOS module integration and build it faster than nixpkgs does.
Pros:
- Full control
Cons:
- Needs additional build power besides own distro
- I personally have no experience with this, seems like a big undertaking
- Needs self-maintained build infra?
5. Maintaining nixpkgs Overlays
Simpler than forking all of nixpkgs, a dedicated overlay collection could offer much of the pros of #4 with less cons.
Example:
# overlays/patches.nix
final: prev: {
caddy = prev.caddy.overrideAttrs (_: {
src = prev.fetchFromGitHub {
owner = "caddyserver";
repo = "caddy";
rev = "abc123...";
hash = "sha256-...";
};
});
}
# flake.nix
{
outputs = { nixpkgs, ... }@inputs: {
nixosConfigurations.myhost = nixpkgs.lib.nixosSystem {
modules = [{
nixpkgs.overlays = [ (import ./overlays/patches.nix) ];
services.caddy.enable = true;
}];
};
};
}
Pros:
- Full control
- Simpler than nixpkgs fork
- Build definitions can be overlaid
- Requires only the overlay to be built, therefore less CI-intensive and faster
- System-wide package replacement in case it is used more than once
Cons:
- NixOS module options still bound to main nixpkgs version
- The nar-hash must be computed outside of nix eval (same as #1)
- Builds from source and misses nixpkgs cache for the overlaid package
6. Just Waiting for nixpkgs
As last option, the maintainers could just wait until nixpkgs has an updated package in the version the distro is relying on.
Pros:
- Simple
Cons:
- Patching speed depends entirely on nixpkgs release speed
- No control
Closing
These are all ideas about how to handle such a case. Of course, the fastest, most convenient and low maintenance approach is the goal. Did I miss something? Are there other ways about it ( maybe GitHub - berberman/nvfetcher: Generate nix sources expr for the latest version of packages · GitHub ?)? Should I just fetch the package from unstable?
How do you handle these cases? What do you recommend?