I have a python environment which includes camelot
:
devEnv = python.withPackages ( ps: [
ps.jupyterlab
ps.camelot
ps.ghostscript
pkgs.ghostscript
]);
during runtime, camelot requires ghostscript
camelot/backends/ghostscript_backend.py (abbreviated for clarity)
import sys
import ctypes
from ctypes.util import find_library
def installed_posix():
library = find_library("gs")
if library is not None:
return library
else:
raise OSError(
"Ghostscript is not installed. You can install it using the instructions"
" here: https://camelot-py.readthedocs.io/en/master/user/install-deps.html"
)
As you can see, including it in withPackages doesn’t allow Python to find it. I also tried overriding camelot
as follows:
camelotWithExtras = pyPkgs.camelot.overridePythonAttrs ( old: rec {
propagatedBuildInputs = old.propagatedBuildInputs ++ [pkgs.ghostscript];
buildInputs = [] ++ [pkgs.ghostscript];
}
);
That didn’t work either. I’m sure I’m just missing something simple.
Unfortunately, when I tried this, it didn’t work because I’m already using pkgs.python3Packages.ghostscript
. In my script, I set python = pkgs.python313
, and withPackages provides the package set: In the withPackages block, ps
is set to pkgs.python313Packages
.
Okay, so I found the solution. The trick here is that find_library
simply does not and will not work on Nix.
The source reveals that the tricks it uses to find a lib are just not aplicable in Nix. I thought about adding a clause that would work on Nix to the find_library declaration, but decided against it. Because Nix pulls its libs as dependent derivations, with a known path, if you can’t provide a absolute path at build-time, you’re doing something wrong.
Still, if you want to explore patching instead of replacing, here are my notes:
The ctypes.util.load_library
function used is just straight up not looking for Nix paths. It has some variables I can pass as overrides, but the guys in the discord say that won’t work right.
-
source code for load_library
-
find_library
is defined in a large if statement, that declares a version appropriate for the host OS. The linux definition has its own sub-block, which starts on util.py:101. There is a large ammount of platform specific code that doesn’t run for my machine from lines 162-280. the actual declaration for my platform is at util.py:339 The definition refers to issue #9998 and just runs _findSoname_ldconfig(name) or _get_soname(_findLib_gcc(name)) or _get_soname(_findLib_ld(name))
Short-circuiting means this just kicks back the first to return something not none.
_findSoname_ldconfig
is defined on util.py:282 in the same block
_get_soname
_findLib_gcc
is defined elsewhere util.py:114 near the top of the posix block. It literally runs gcc or cc if it can, and greps for expr = os.fsencode(r'[^\(\)\s]*lib%s\.[^\(\)\s]*' % re.escape(name))
in the output.
_findLib_ld
is defined on util.py:312 also in the same block.
Instead, I replace a find_library
with a direct call to ctypes.cdll.LoadLibrary
, which takes a direct path. I use one of the terminal utilities provided in Nix’s standard environment. Every derivation should have access to the patching utilities detailed here, and I use them as follows:
camelotWithGS = pyPkgs.camelot.overridePythonAttrs ( old: rec {
postPatch = (old.postPatch or "") +
(
if old.version == "0.11.0" then ''
substituteInPlace camelot/backends/ghostscript_backend.py \
--replace-fail 'find_library("gs")' 'ctypes.cdll.LoadLibrary("${pkgs.ghostscript}/lib/libgs.so")'
'' else ""
);
});
You can mostly check the docs for substituteInPlace
from above for how this works, but one thing they don’t mention is that the --replace
flag is now depreciated. As per issue #356002, I’ve swapped it out for --replace-fail
.
Another improvement would be to use final.version
instead of old.version
. As is, the version-handling is broken. Using final.version
should let it use what it needs, instead of ignoring any later modifications. The new override would look like: overridePythonAttrs ( final: old: rec {
I didn’t check what phases this fix works in since I don’t understand phases. But if you search github for substituteInPlace find_library
you find pretty quick that everyone seems to be using it. They probably know something I don’t.