Overlays for noobs

[dob]

Over in another post, I mentioned that I did not understand the concept of Overlays.

Thanks @Birdee, for your reply:

Overlays just cause a new pkgs object to be returned and used instead which contains whichever things the overlay adds to it.

(self: super: { jq = super.jq; })

This overlay, for example, would replace jq with itself lol

…

the two arguments for overlays are either called self: super: or final: prev: by convention and are the new and old pkgs sets before and after the overlay is called.

---

Still though, I am struggling to grasp it as I am lacking a lot for basic knowledge. I will try to break it down below. I might very well be wrong in my assumptions, so please correct me.

Overlays just cause a new pkgs object to be returned and used instead which contains whichever things the overlay adds to it.

A package object is what is found here. So, Overlays somehow alters existing package object, like parameters for a program.

Let’s take the example above:
(self: super: { jq = super.jq; })
or, using the new convention:
(final: prev: { jq = prev.jq; })

It does not help I am new to functional programming as well. So, I even find the code confusing. However, it is a function with final and prev as input parameters. All Overlay functions are defined like this. It is seemingly a part of the Nix language.

In the above example an Overlay is created for the JSON processor jq. As stated in the documentation: “prev comes from only the stage before”. So, you are replacing jq with jq. I think I got that.

But, one overlay I have seen several places is the following one, for waybar:

# fix waybar not displaying hyprland workspaces
(self: super: {
    waybar = super.waybar.overrideAttrs (oldAttrs: {
        mesonFlags = oldAttrs.mesonFlags ++ [ "-Dexperimental=true" ];
    });
})

I see that overrideAttrs is a part of Nix.

When I look at the code, as a non-functional programmer, it looks like the entire waybar package object is replaced with “super.waybar.overrideAttrs” with this code:
waybar = super.waybar.overrideAttrs …

But, I guess that is not the case. It should only be the mesonFlags that is altered, and the string -Dexperimental=true" that has been added to it, I guess. I just do not understand that the assignment above will not overwrite the entire waybar package. Or, maybe overrideAttrs returns the waybar package itself, with the changed attributes? That would make sense.

In the documentation, the following example is listed:

helloBar = pkgs.hello.overrideAttrs (finalAttrs: previousAttrs: {
  pname = previousAttrs.pname + "-bar";
});

I guess previousAttrs in the example equals oldAttrs from the waybar Overlay. But, would it work to add finalAttrs as well, like this:

# fix waybar not displaying hyprland workspaces
(self: super: {
    waybar = super.waybar.overrideAttrs (finalAttrs: oldAttrs: {
        mesonFlags = oldAttrs.mesonFlags ++ [ "-Dexperimental=true" ];
    });
})

How do one find out what attributes are accessible in a package? Like, waybar’s mesonFlags. Is the solution to check the source code of the package you want to alter?

[Birdee]

It overrides the whole pkgs set.

by pkgs set I mean, the set from import { system = “”; }

so, what you did with waybar is overrided the attribute mesonflags in the waybar derivation, and then said, in the pkgs set im going to get in my modules, its going to have a pkgs.waybar with a new mesonFlags attribute.

To find what you can override in a particular package derivation, use the nix repl

You can go to your system flake, use :lf .

then you can pkgs = import inputs.nixpkgs { system = “”; }

and then you can check out what attributes are available in pkgs.waybar

I know overlays but not overrides although I have used them some. I have only used .override I have not used .overrideAttrs

What is probably happening is it is replacing the waybar package with a function that returns the new waybar package when you do overrideAttrs

[hugosenari]

I see your confusion.

Nix != nixpkgs

Nix is the package manager,
You run Nix, it reads a nix file and if your nix file call builtins.derivation passing some attributes, like name, version, builder (the build script), etc. and others: “Every other attribute is passed as an environment variable to the builder”.. It creates a package (aka derivation).

Nix has the nice syntax of call a function passing parameter within space

# C like example
HashSet myFunc(HashSet final, HashSet prev) {
  return prev;
} 
/* to be called like */
myFunc(firstArg, secondArg);

is

# nix example
myFunc = final: prev:
  prev;
# to be called like
myFunc fistArg secondArg # Look ma! No comas or parenthesis!

But this nice syntax has the flaw that when the function call expects another function, we add parenthesis overrideAttrs (self: super: { jq = super.jq; }) is also true for lists|array like [(self: super: { jq = super.jq; })].

Other cool thing about Nix lang, it expects only one value as result, not only to the function, but to the file, being more like a JSON, where the root value of your file has to be an attrset, an array, number, null, string or even a function. This is true for functions, if else, etc.

# This is invalid in Nix
a = 1;
b = 2;
c = a + b

But there are some exception and most important one is let

# this is valid Nix
let 
  a = 1;
  b = 2;
  c = # we can nest it
    let
      d = 3; 
      e = 4; 
    in d + e;
in {
  a = a; b = b; c = c; d = d;
} # no last `;` required

Nixpkgs Is the default package repository, that when we import, it has a large attrset|object|dict|hashmap (whatever your loved lang like to call it) of nix pkgs and lib.

Overlays, is a nixpkg pattern, expecting a list of functions that return attrsets to be merged with nixpkgs.

The most common package structure is a single function (the root object), expecting a single argument of an attrset to be ‘deconstructed’ in the function definition.
ie:

# instead of this
a: b: # two args
  b

# we do this
{ a, b }: #single arg with two attrs
   b

in package:

# FUNCTION ARG
{
  # expects `pkgs` attr or import nixpkgs if missing
  pkgs  ? import <nixpkgs> {},  # call `import` with `<nixpkgs>`, 
                                # it returns a function,
                                # call this other function with an empty attrset `{}`
                                # to use its defaults

  # expects myDep or use myDep from pkgs
  myDep ? pkgs.myDep,
  ... # expects more unkown args # common in nix modules but not in pkgs
}: 
# FUNCTION BODY

# We called another function  # with 1 argument
pkgs.stdenv.mkDerivation      {
  # One attrset with N ATTRIBUTES
  name = "myPkg";
  buildInputs = [ myDep ];
  mesonFlags  = [ "-DsomeFlag=false" ];
  # ... I'm too lazy now to type the other attributes
}

When another nix file tries to import it callPackage (there is also mkOverride), it adds the override function to your package so it can be used to change ARG.

@dob

maybe overrideAttrs returns the waybar package itself, with the changed attributes?

Yes.

When your package is evaluated, the mkDerivation creates a package calling builtins.derivation and also add the function overrideAttrs that can be used to change attributes passe to mkDerivation.

This is strange, but your package is sometimes read as pkg|derivation, sometimes read as attrset, and sometimes read as path.

@dob

How do one find out what attributes are accessible in a package?

for override you can read the function ARG, for overrideAttrs you can read the call of mkDerivation of that package, the definition of mkDerivation, and bultins.derivation and the “build script” because the definition is open.

[dob]

@Birdee @hugosenari

Wonderful. Thanks a lot for taking the time to explain all this. Hopefully, there will be quite a few that finds this useful. I will have to read it a few times, and chew on it.