"Consistent channel-like behaviour" for system/HM flakes (was: Can't override/permit insecure qtwebkit (needed for OpenModelica))

ppenguin · January 10, 2023, 9:13pm

I need to install OpenModelica but the install fails with

Package ‘qtwebkit-5.212.0-alpha4’ in /nix/store/jkl0nk32sz83k6dm15lm8hqv2ack463w-source/pkgs/development/libraries/qt-5/qtModule.nix:72 is marked as insecure, refusing to evaluate.

       Known issues:
        - QtWebkit upstream is unmaintained and receives no security updates, see https://blogs.gnome.org/mcatanzaro/2022/11/04/stop-using-qtwebkit/

So I went ahead and did what I was told (configuration.nix):

nixpkgs.config.permittedInsecurePackages = [ "qtwebkit-5.212.0-alpha4" ];

=> no difference. Complicating factor:
I’m installing openmodelica.combined through home.packages, but this should not make a difference as my flake for the HM generation takes the nixpkgs input from the nixosConfigurations.<myhost>

Inspired by this qt5 override trick and this vulnerabilities override trick I added an overlay:

final: prev:

let
  ignoringVulns = x: x // { meta = (x.meta // { knownVulnerabilities = []; }); };
in
{
    qt5 = prev.qt5.overrideScope (super: self: {
        qtwebkit = super.qtwebkit.overrideAttrs ignoringVulns;
    });
}

but it still doesn’t make any difference.

Additionally, I get the following error:

warning: `overrideScope` (from `lib.makeScope`) is deprecated. Do `overrideScope' (self: super: { … })` instead of `overrideScope (super: self: { … })`

but how can I control this (the overrideScope function in qt5 is outside my control?)

Any ideas?

ppenguin · January 10, 2023, 9:21pm

Oh, wait, I actually managed to make it work by putting that permittedInsecurePackages in my general nixos-builder which has wrapper functions to generate nixosConfigurations at the location where I do import inputs.nixpkgs. While this is unsatisfactory in terms of factoring, it does the job for now.

tejing · January 10, 2023, 9:22pm

You have to make sure you’re configuring the correct instantiation of nixpkgs. If you’ve overridden pkgs directly somehow, then the nixpkgs.* options do nothing.

Does this mean you’re using home-manager.useGlobalPkgs = true; in a nixos module install? Be specific.

You may not have noticed, but there are two functions here, overrideScope and overrideScope' the ' makes it a totally different variable/attribute.

tejing · January 10, 2023, 9:23pm

This is why I always recommend against overriding pkgs directly

ppenguin · January 10, 2023, 9:28pm

Thanks, yeah (that’s what I get from staring at a screen for >10h I guess), now you’re saying it I noticed the overrideScope difference

I don’t use HM through a module, because it doesn’t provide enough separation between “sysadmins” and “users” for applying configs, but I do it like this:

  mkHome = username: hostname: (
    let
      inherit (systemConfigs."${hostname}") pkgs config;
      inherit (pkgs) unstable system;
      osConfig = config;
      homeDirectory = "${if lib.hasInfix "darwin" system then "/Users" else "/home"}/${username}";
      configHome = "${homeDirectory}/.config";
    in
    inputs.home-manager.lib.homeManagerConfiguration {
      inherit pkgs;
      modules = [
        ../home/home.nix
      ];
      extraSpecialArgs = { inherit inputs username unstable hostname homeDirectory osConfig; };
    });

and apply the config not with the HM cli but like this:

nix run "${FLAKEPATH}#homeConfigurations.${USER}@$(hostname -s).activationPackage"

tejing · January 10, 2023, 9:31pm

Ah, I see, so your home-manager pkgs truly is the same value as the nixos host’s pkgs within the same flake commit, though not necessarily the running generation of the host config.

ppenguin · January 10, 2023, 9:33pm

Yeah, that’s why indeed for a case like this (when playing with global overlays and HM packages) I have to first switch system and then home, but I didn’t forget that, so that potential cause for things not working can be discarded.

ppenguin · January 10, 2023, 9:43pm

BTW: so do you mean preferrably use overlays?
But the overlay presented above didn’t appear to have any effect, that’s why I still needed the permittedInsecurePackages thingy.

tejing · January 10, 2023, 9:44pm

No, I mean you should use the nixpkgs.* options in nixos to configure nixpkgs, rather than overriding it in your nixosSystem call.

In other words, don’t take instantiating nixpkgs out of nixos’ hands.

ppenguin · January 10, 2023, 10:02pm

Hm, seems I need to go back to basics a bit (My flake possibly got a bit out of hand…)

So just to check if I follow, I am basically doing something like

let pkgs = (import inputs.nixpkgs ({ inherit system; config..... = .... ; }) in
lib.nixosSystem { inherit system pkgs .... }

does this constitute an “override” in your meaning?
But if I don’t do this, how do I tell nixosSystem know which input to take for pkgs?
And if there is a different way, I suppose I’d still use the above way to add unstable, which I would then propagate (like now) through specialArgs?

tejing · January 10, 2023, 10:08pm

ppenguin:

So just to check if I follow, I am basically doing something like
let pkgs = (import inputs.nixpkgs ({ inherit system; config..... = .... ; }) in
lib.nixosSystem { inherit system pkgs .... }
does this constitute an “override” in your meaning?
But if I don’t do this, how do I tell nixosSystem know which input to take for pkgs ?

Yes, that’s exactly what I’m referring to. nixosSystem comes from nixpkgs itself. Take it from the nixpkgs input you want to use, and it will use the one it came from.

Yeah, passing other inputs through specialArgs for those few packages you want to get from a different version of nixpkgs is fine. There’s no reason to change that, though unless you need to configure them as well, I wouldn’t explicitly instantiate them, I’d just use their outputs.

ppenguin · January 10, 2023, 10:15pm

Thanks … seems I’ve got some refactoring to do
(The flake had grown a bit complex because I scan host definitions in a subdir and use abstractions that provide less per host boilerplate and also “host profiles” for common stuff. But in fact this particular point will just simplify matters…

… and, as you indicated, will again allow me to use nixpkgs.* on a per host basis, which is of course as it should be.

ppenguin · January 11, 2023, 2:51pm

@tejing
Just to finalise one last question (thanks for being so informative):

So if I don’t want to override pkgs in nixosSystem, but I leave the following requirements intact:

access to the unstable module argument so I can freely switch between pkgs.somepkg and unstable.somepkg
have the same config for unstable as for pkgs, since the goal is to just be ablle to use newer versions in some cases.
refer to the same pkgs and unstable instances of the system pkgs and unstable used in HM (see above, unchanged)
I want to be able to continue using impure nix-shells that use { pkgs ? import <nixpkgs> {}, unstable ? import <unstable> {}, ...}:
all of the above to achieve re-using installed nix store packages where possible and have the “sysadmin” be responsible for the “default channels”, which is in effect updated through nix flake update && nixos-rebuild switch --flake ...

I now have something like this (conceptually):

    nixosConfigurations.test = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules = [ ./configuration.nix ./overlays ];
      specialArgs = {
        inherit inputs;
      };
    };

# overlays/default.nix
{ inputs, pkgs, config, ... }:

{
    nixpkgs.overlays = [
        (final: prev: {
            unstable = (import inputs.nixpkgs-unstable { inherit (prev) system config; }) // { outPath = inputs.nixpkgs-unstable.outPath; };
            outPath = inputs.nixpkgs.outPath;
        })
    ] ++ [ /* more overlays */ ];

    nix.nixPath = [ "nixpkgs=${pkgs.outPath}" "unstable=${pkgs.unstable.outPath}" ];
    # this *adds* unstable to the available args for all modules
    _module.args.unstable = pkgs.unstable;
}

Would this be a good (i.e. idiomatic) way to achieve those requirements whithout leveraging the module system wrongly/awkwardly?

Oh, wait, this way the outPath contains the default (upstream) config instead of my own… Can I also access somewhere the outPath of the configured nixpkgs, and how to refer to it in the flake so it’s set correctly regardless of configuration.nix and HM?

tejing · January 12, 2023, 2:04am

I would do something more like:

# overlays/default.nix
{ inputs, pkgs, unstable, config, ... }:

{
    nixpkgs.overlays = [ /* more overlays */ ];

    # this *adds* unstable to the available args for all modules
    _module.args.unstable = import inputs.nixpkgs-unstable { inherit (pkgs) system config; };

    nix.nixPath = [ "nixpkgs=${pkgs.path}" "unstable=${unstable.path}" ];
}

There’s no need to shove unstable into pkgs, just pass it separately. Also, nixpkgs already provides a path attribute which does what you wanted from outPath. However, both your version and mine still get you an unconfigured version in nix.nixPath.

In my own config, I create, and register as pkgs, a flake that exposes my nixos system’s pkgs as its legacyPackages.x86_64-linux. This allows me to do nix run pkgs#hello-unfree, for example, and it works because I have allowUnfree configured in my nixos config. See my config here and the definition of mkFlake here.

I haven’t yet figured out how to get a configured version in NIX_PATH, nor whether it’s actually a good idea. Lots of things expect to be able to configure <nixpkgs> themselves.

You would probably have to make a modified copy of nixpkgs with the default.nix moved to original.default.nix, and a new default.nix in its place that calls the original with extra configuration. But probably only if the caller provided nothing in the argument; you don’t want your configuration getting in the way if something is trying to configure nixpkgs its own way. I’m really not sure it’s worth it.

ppenguin · January 12, 2023, 9:44am

Ah, very nifty! I might look into that registry stuff some more later.

The reason I did this, is that it appeared to be needed in my current mechanism where I want to take both configured pkgs and unstable from the systemConfigs.${hostname} when building my HM generation and didn’t know where unstable ended up. But now I found that I can actually just as well get it from inputs.self.nixosConfigurations.${hostname}._modules.args.unstable, so indeed I can use your (cleaner) solution with that.

More philosophically, this whole hocus pocus (also for TLDR) should, I think, result in some kind of “pre-flake channels” behaviour with the added “bonus” that the “channels” are configured so would generally lead to less duplication for packages that both are both defined in environment.systemPackages and home.packages (at the cost of having less explicit/vanilla behaviour for HM users[^1])

[^1]: this is what I want, because I like my config to have the paradigm that the “sysadmin” defines the default host environment and users are by default “subjected” to this (users are working on this host for a particular reason, so the host provides its specific defaults). This should also per default cause less duplication of installed package trees (?). This is “policy-wise” also closer the linux multi-user philosophy (e.g. free/non-free would be an admin decision rather than a user decision, just to name one).