How to set up a GTK3 theme, thats not in the nix pkgs?

There are two ways, one simple, manual way and one using nix.

If you just want to get it working, download the theme zip and extract it to ~/.local/share/themes/<theme-name>. You’ll be able to configure gtk to use it as usual.

If you want to install it declaratively using nix, you’ll have to write a package for it. Nix makes writing packages downstream, without interacting with the upstream project at all, really easy.

Assuming you use a configuration.nix and not flakes, I’d suggest you do something like this:

  1. Create a new directory called pkgs in your /etc/nixos directory, adjacent to your configuration.nix.
  2. Write a default.nix file in this new directory, containing something like this:
# /etc/nixos/pkgs/default.nix
{pkgs, ...}: let
  callPackage = pkgs.callPackage;
in {
  nixpkgs.overlays = [(final: prev: {
    mypackages = {
      gtk-theme = callPackage ./gtk-theme.nix {};
    };
  })];
}
  1. Write a gtk-theme.nix in the same directory, adjacent to the default.nix. As you see above, we’ll be running the callPackage function on it. This is the function typically used to build nix packages using the various things available in nixpkgs.
    Without knowing what theme you want to package, I can’t help much with this file. Here’s an example from the nixpkgs repo though: https://github.com/NixOS/nixpkgs/blob/85bcb95aa83be667e562e781e9d186c57a07d757/pkgs/data/themes/juno/default.nix#L39
    The crux here is that the extracted theme is installed to $out/share/themes - NixOS automatically places the share/themes directories of all installed packages in the profile-wide share directory to make them available.
    Obviously, you can name the file whatever you want. Just make sure to change the ./gtk-theme.nix in the default.nix to whatever you call the file.
  2. Import this pkgs/default.nix file in your main configuration.nix, and use your new package:
# /etc/nixos/configuration.nix
{pkgs, lib, config, ...}: {
  imports = [
    ./pkgs
  ];
  
  environment.systemPackages = with pkgs; [
    mypackages.gtk-theme
  ];

  # This next stuff is technically not necessary if you're going to use
  # a theme chooser or set it in your user settings, but if you go
  # through all this effort might as well set it system-wide.
  #
  # Oddly, NixOS doesn't have a module for this yet.
  environment.etc."xdg/gtk-2.0/gtkrc".text = ''
    gtk-theme-name = "<gtk-theme>"
  '';

  environment.etc."xdg/gtk-3.0/settings.ini".text = ''
    [Settings]
    gtk-theme-name = <gtk-theme>
  '';
}

Substituting your theme’s name for <gtk-theme>, of course. If you like, you can change the package name to the theme’s name too, just make sure to do the same in pkgs/default.nix.