{
lib,
python3,
fetchPypi,
python3Packages,
glib,
}:
let
pname = "workrave_break_info";
version = "0.1.1";
in
python3.pkgs.buildPythonPackage {
inherit pname version;
pyproject = true;
src = fetchPypi {
inherit pname version;
sha256 = "sha256-43b5YwTFp4GvG6usq3x7blTXK2mn/O1GtZJ+mNakezU=";
};
build-system = [ python3Packages.setuptools ];
dependencies = [ python3Packages.dasbus ];
pythonImportsCheck = [
"workrave_break_info"
];
makeWrapperArgs = [ "--set" "GI_TYPELIB_PATH" "${glib.out}/lib/girepository-1.0" ];
meta = {
homepage = "https://pypi.org/project/workrave-break-info/";
description = "Allows info from Workrave timers to appear in tools like Waybar, etc.";
license = lib.licenses.gpl3;
maintainers = [ ];
};
}
Now the derivation, near as I can tell, works, but there are a couple things I had to do to get it to work.
First, even though I see unqualified usage of buildPythonPackage in several packages in nixpkgs, such as dasbus, nix-build complained when I did the same; indeed, I got the confusing error message ‘Function called without required argument “buildPythonPackage”’ when I did so. Instead, near as I can tell, I had to pass python3 as an argument to the function generating the derivation, and then use python3.pkgs.buildPythonPackage instead of just buildPythonPackage.
Second, to get a script bundled into the package to work, I had to manually add "${glib.out}/lib/girepository-1.0" to GI_TYPELIB_PATH via makeWrapperArgs. This is the case even though GI_TYPELIB_PATH is set correctly when I run nix-shell -p python3Packages dasbus, where dasbus is a dependency of my package.
I don’t quite get why either of these things were necessary. Can anyone explain?
Regarding the first issue, the answer is most likely that those files are called with pkgs.python3Packages.callPackage rather than pkgs.callPackage. It includes anything in pkgs.python3Packages as potential sources for the given args, falling back to pkgs if not found.
I’m less certain about GI_TYPELIB_PATH, but I suspect that may be normal. What happens inside a nix-shell -p is a separate concern from what automatically goes into wrapper args. You’d need someone more familiar with this specific aspect of the ecosystem to be sure.
A little more research suggests you probably want to be using wrapGAppsHook (add it to nativeBuildInputs) to get GI_TYPELIB_PATH. Still not my area of expertise, though, so maybe someone will come along and correct me.
The standard way is to add gobject-introspection to nativeBuildInputs. As python apps/modules built with buildPythonApplication/buildPythonPackage already get wrapped, we have to do a dance to avoid double-wrapping:
(wrapGAppsHook* may not be necessary in your case, but use the rest of the snippet. If it’s not a graphical module you may want wrapGAppsNoGuiHook instead.)
The big catch I see with using wrapGApps*Hook is that the reason the script needs GI_TYPELIB_PATH is that one of its dependencies, namely dasbus, needs it. A non-hacky solution – or at least a less hacky one – would be to propagate the environment that is already associated with dasbus (about the same one that gets set up when nix-shell -p python3Packages dasbus is run) to the script.
The script from the module is not a GNOME or GTK app itself, even though it interfaces with one, so the use of wrapGApps*Hook is just as much of a hack as setting GI_TYPELIB_PATH directly.
The environment that gets set up when nix-shell -p is run is a build environment. Blindly copying what it does at runtime through the wrapper is definitely not the right solution. Perhaps some cleaner abstraction for propagating necessary wrapper args could be created, but it doesn’t exist right now.
What was explained above is the currently idiomatic approach.
It is also the environment that the Python modules listed in the -p argument should run in, as seen in these examples of nix-shell shebangs: nix-shell - Nix 2.28.5 Reference Manual
Basically, the way the script should act should be roughly equivalent to the following script:
It’s only idiomatic for GNOME and GTK apps, not for apps that depend on GI_TYPELIB_PATH because they happen to depend on Python modules that depend on it. It doesn’t cover situations where Python modules could depend on a different environment.
Your recommended approach breaks if the internals of the Python dependencies change (e.g., if a future version of dasbus no longer relies on GLib for its D-Bus implementation). That indicates that it’s not truly idiomatic for the situation at hand.
Basically, what I need is a way to specify that the script should set up whatever environment is needed for its Python dependencies to work, whatever those are, rather than relying on explicit knowledge about the implementation details of those Python dependencies. It should continue to work if those details change.
What you’re asking for is impossible; if some modules have a dependency on something outside of the python space, that has to be part of the package expression. The entire point of nix is recording such dependencies explicitly.
The Nix package for dasbus already has pyobject3 as a dependency. That’s why GI_TYPELIB_PATH already includes ${glib.out}/lib/girepository-1.0 when either nix-shell -p python3Packages.dasbus or nix-shell -p python3Packages.workrave_break_info is run. So the information about the necessary environment is already gathered by Nix.
It’s just that that environment isn’t fully included in the wrapping of the script.
So I don’t think what I’m looking for is entirely impossible.
This is pretty much an accidental byproduct of how nix-shell -p works, and one of the reasons nix shell does not work that way in the new cli. Strictly speaking, nix-shell -p is actually a build environment as well, for a hypothetical piece of software with that specific dependency.
This does not exist. It’s not clear it would be a good idea to implement, either. Wrapping executables is generally tricky, and probably not amenable to systematic treatment like you’re proposing.
First off, that misses the point, which is that Nix already gathers the information about the environment that I need, and it’s a matter of figuring out whether I can access it from within the derivation and, if so, how.
Second, where’s the documentation for that? And why would one not want the environment set so that the Python module can run? If what you say is true, it would seem to defeat the point of a Nix shebang in the first place.
Actually, it doesn’t. It collects incidentally related information, which isn’t really properly connected to the information you really want. Which is the point that you’re missing. This is why I said from the beginning that blindly copying from a build environment is not the way to do this.
Could what you want be implemented, with a major new piece of infrastructure in nixpkgs? Probably. Would anyone be happy if it actually was? That’s a whole different question. Regardless, what you’re asking for doesn’t exist, so use what does exist. (Or get started making it yourself and try convincing everyone it’s worth switching to…)
What information do you think I “really want”, and why do you think that the information collected by Nix is only “incidentally related” to that?
Really, what I want is for the DRY principle to work for my derivation, so that if the dasbus derivation changes in response to upstream changes in dasbus, my own derivation will automatically accommodate that. Neither the manual setting of GI_TYPELIB_PATH that I’ve done (which I regard as a code smell) nor introducing wrapGApps*Hook will accomplish that.
As I’ve now said several times, the information you have is about build environments. The information you want is about runtime environments. These are not the same thing, even if they often happen to significantly coincide, especially for interpreted languages. A system built on the assumption that they are the same would likely be unusably broken.
I do understand what you want and why you want it. I also understand how hard it would be to implement in a way that did more good than harm in general. I also understand, from personal experience packaging things, how minor the amount of work you’re trying to avoid is. We have a very long list of more beneficial improvements to work on in nixpkgs.
And even if everyone magically dropped everything to implement exactly what you want because you want it, it wouldn’t actually benefit you for a significant amount of time. It’s quite useless to argue further about it here. Just do the current best practice.
They don’t just coincide. Rather, they’re related by having similar technical needs – especially for interpreted languages – that the classic nix-shell shebang is recommended practice, even by Nix’s documentation.
Whatever the “current best practice” is, it hasn’t been offered in this thread. Using wrapGApps*Hook in this particular context is at least as much of a hack as a raw setting of GI_TYPELIB_HOOK, because it’s not being applied to a GNOME or GTK app, but rather to a script associated with a Python module that depends on another Python module that coincidentally shares one commonality with a GNOME or GTK app.
I linked to the nixpkgs manual, how is that not best practice? Nix is not built around DRY, it’s built around explicit dependency specification when it comes to compilation and deplyment environments, which Eelco’s thesis explains in more detail - chapter 1 suffices to get the idea here. (Dev shells are the hack here, in that regard.)
Nix does absolutely nothing to reduce boilerplate. Nixpkgs meanwhile does take care of a lot of boilerplate to avoid repeating really essential stuff (basically whatever’s in stdenv), but everything else is manually specified (very often via hooks to execute some other bash code, or by providing runtime deps explicitly). I’d be happy to see something better provided it does not sacrifice correct deployment to do so.
That manual never says to use wrapGAppsHook* for my scenario. wrapGAppsHook3 is for GTK3 apps, and wrapGAppsHook3 is for GTK4 apps. Arguably, there’s some wiggle room for wrapGAppsNoGuiHook, but judging from the intended usages of wrapGAppsHook*, it looks like it’s still more aimed at command-line applications that are part of the GTK/GNOME ecosystem. That seems especially likely given that wrapGAppsHook* affects far more environment variables than just GI_TYPELIB_PATH, and those variables are tied to particular conventions of how GTK/GNOME apps work.
At best, wrapGAppsHook* seems to do too much. Worse, the only reason I’d be using it is because I know a particular implementation detail of dasbus that really isn’t supposed to be exposed when using dasbus according to its API.
In short, there are two very good indications that wrapGAppsHook* is the wrong tool for this particular job: it’s being applied well outside its usual context (of GTK/GNOME applications), and it’s being applied due to reliance on details of a module that aren’t part of its user-facing interface and thus are subject to change.
Ideally, what I’d like to be able to do is extract the necessary information from the dasbus derivation itself (which is largely the same information that nix-shell already extracts) and use that for the wrapper. I’m not sure if that’s available in a derivation, but I’m hoping it is somewhere.
Sidenote about DRY
For what it’s worth, DRY isn’t really about boilerplate, and in spite of its name, isn’t so much about repetition as such. Rather, it’s about designing a system so that if you have to change something, especially something “simple”, you don’t have to go to a million random locations spread across the code base to make it happen, but rather can just change one general location and have the effects of that change propagate. I don’t think Nix does much that’s against that spirit.
You missed my point. I’m basically saying that using wrapGAppsHook* only works because of details about dasbus that I’m not supposed to care about. I should use dasbus under the expectation that its internal details are subject to change, and anything that relies on those details can break without warning. That goes for its packaging as well.
Because of this, using wrapGAppsHook* in this derivation is about as wise as using undocumented members of a Python class that begin with _. Sure, it will work for the moment, and if one is lucky, it may even work for a while. Nonetheless, it’s a reliance on internal details that I’m supposed to act as if I’m ignorant of.
The above, of course, also applies to setting GI_TYPELIB_PATH as well, which is why I described it as a “weirdness” and regard it as a problem. I don’t want to have a derivation that cares about that detail. That’s why I object to wrapGAppsHook* as a “solution”. It suffers from the same problem that I already have and adds complexity on top of that.
To put it another way, I want a solution that satisfies the Law of Demeter, i.e., I want to be able to have a derivation that only knows about the properties of my dependencies, not the details of the dependencies of my dependencies. I should be able to write a working derivation that allows me to not care at all what dasbus depends on. That seems to be a basic rule for good maintainability.
Packaging may break between versions, that’s a given for any software. Neither nix nor nixpkgs helps you with the arbitrary restriction that you came up with. This recommendation from the manual only applies for the current version of nixpkgs, nix, python, and the respective python modules (although nix itself generally promises some level of compatibility assuming non-experimental functionality is in play, and nixpkgs commits to compatibility within a given stable release).
Nix is not built around DRY, it’s built around explicit dependency specification when it comes to compilation and deplyment environments, which 