Creating an overlay for a python application

I am trying to create an overlay for a python application (octoprint) in which I would like to add my own python package to its own overridden python derivation.
For now I am trying to do that inside a shell.nix file for easier testing.

Here is my pieced together attempt to do so, but for now it fails due to some syntax errors.

let pkgs = import <nixpkgs> {
    overlays = [
      (self: super: {
        octoprint = super.octoprint.override {
          python = super.octoprint.python.override {
            packageOverrides = selfPy: superPy: {
              octoprint-ldap = selfPy.buildPythonPackage rec {
                pname = "OctoPrintPlugin-OctoPrint-LDAP";
                version = "1.1.0";
                propagatedBuildInputs = [ super.octoprint ];
                doCheck = false;

                src = pkgs.fetchGit {
                  url = "https://github.com/gillg/OctoPrint-LDAP.git";
                  ref = "master";
                };
              };
            };
          };
        };
      })
    ];
  };
in pkgs.mkShell {
  nativeBuildInputs = [ pkgs.octoprint ];
}

OctoPrint creates a custom python3 override here which itself switches out some packages for different versions and introduces some new ones (mostly octoprint plugins). Finally it is all put together using the toPythonApplication call at the bottom of the file.

What I am trying to do is to introduce another plugin to that python derivation.
Is that even possible?

EDIT: Ah, I found that my initial override of octoprint just calls the package expression, which accepts python3 as a parameter. Feels like I am getting closer to a solution.

1 Like

Hello busti,

this package looks be be not the best example - due to it’s “funny” implementation logic
a refactoring of the package could be a good idea and less effort than to adjust the override clause


are you sure about the methodology you are trying to use?

regarding the logic in your script, think about if

  • python = super.octoprint.python.override { - could it be python or python3 or python38?
  • propagatedBuildInputs = [ super.octoprint ]; ? propagatedBuildInputs = [ self.octoprint.propagatedBuildInputs ++ ];

if you would do that override it would not add anything to that package octoprint, right?

  • you “cannot” use overrideAttrs either because you don’t want to exchange anything (that is given/contained in the package code) but add a new package (to packageOverrides)
    (sure you can use those concept - but you would rewrite 90% of the package code)

Why don’t you add your plugin to plugins?

  • import ./plugins.nix
1 Like

Thanks a lot for your reply. I still haven’t gotten it working since the 3D printer and the pi are in a hackerspace at the other end of the city and I do not have a VPN connection there, so I can only work on it for some hours at a time. I could probably also test it on my laptop, but I am still working on my laptop nixos system and I am still using windows as a daily driver.
My motivation for doing it like this as opposed to a PR to nixpkgs is that I would like to test it on the pi first and cloning and maintaining a fork of nixpkgs seemed overkill.

I have since submitted a PR, but doing so was quite a hassle for something that I could do using a single command on any other distro and it also introduces a multi-day wait while the PR is getting merged, unless I want to use my own fork of nixpkgs on the pi, which is something I would like to avoid. Right now the PR went through review and it is waiting for the maintainer of the octoprint plugin to tag their release commit, which is something I do not have an influence on.
Please don’t misunderstand me though, I want to use nixos to do this. I also know that I could just install octoprint, python, pip and finally the plugin as I would on any other distro, i.e. using nix-env, but I also think that that would destroy the purpose of using nixos in the first place.

I have seen your post where you linked this thread and it resonates with me a little.
I have been using nixos for about two months now and so far the experience has been really nice.

Everything just works, until it doesn’t.
In this case I wanted to add a plugin to some application that wasn’t in the list of added plugins and something that would be a two minute job elsewhere turned into a weeklong endeavor. There are processes of manipulating nix to add that plugin, but I am too inexperienced with it to see everything through for now. Sure the package I chose to test my skills is quite complicated, but then again, it is what I want to do right now.
Especially the last line of the packages definition tripped me up.

The function toPythonApplication is a black box to me and I have no idea how it changes the contents of the expression, which leaves me clueless as to how I can manipulate it using overlays.

Also I have no tools that allow me to work on the overlay properly. Right now I am using a mixture of IntelliJ with the nix language plugin, nano on the pi and nix repl, because it has auto-completion for members.

I have made some progress though:

  • You were right, the name I need to override is python3.
  • propagatedBuildInputs seems to be used for libraries needed by the plugin during its build process, in this case super.octoprint and super.ldap are needed.

Sorry for the brain dump. I hope it helps with the other thread.

(currently it is frozen zone for the release - not sure if that makes some maintainer less available …)
I personally thing that (only) this package is no structure perfectly to use the power of nixos/nix …

nixpkgs/pkgs/top-level/python-packages.nix:88: toPythonApplication = drv: # clone the repo and grep through it
doc

In “worst case” you would not need to maintain a total copy of nixpkgs