/nix/store/gnvk1lfssjn3yahnd6cky3ay931wlj7x-postgresql-11.11-lib/lib/libpq.so: file not recognized: file format not recognized
full build log:
so I wonder what an easy way would be to cross compile this?
and I also wonder if I can create a multiarch docker image or if I would need to upload these to all under different tags? is that even desirable or will the image get huge doing that?
Since this is just a simple python program I figured it would fairly simply also run on arm.
I haven’t done much cross compilation with nix, so won’t say anything on the first question, but regarding the second:
Docker images always target a specific platform. They’re ultimately just a chroot with extra bells, so the binaries need to be compiled for the architecture they’ll run on.
Common practice is indeed to create separate tags, see the official postgres image for example: Docker Hub
This may be surprising because docker registries do have special multiarch metadata support; the registry will just serve different images with the same tag depending on the client’s requested architecture, if configured to do so upstream.
Tags don’t affect the image size if you create them per-arch, since it’s not overwriting the original image’s layer with a new set of binaries; rather you create multiple separate images with different handles. Downstream users only download the architecture tag they need (or the registry magics it away, causing misunderstandings and reproducibility issues).
None of this matters in the nix world of course; here the image will just be built for the target arch and stored in the nix store. Or should, at least.
On a side note, your main definition should probably be in a default.nix, not a shell.nix - the latter is supposed to be a development shell, providing dev dependencies, not the package.
ah that’s interesting. I’m not well versed with docker so this explained a lot.
I’ve found that this shell.nix also works perfectly for hacking on the project as well.
it is in fact the only way I know of that will properly setup console scripts that I need to run my project.
so since it both builds the project and works as a development shell it’s just easier to call it shell.nix, I could maybe add a symlink.
They’re similar, but not quite the same. Besides the shell.nix, which defines your dev environment and all the packages you want for development, you should also have a default.nix, which specifies only the package as you would deploy it.
nix-shell will fall back to default.nix, so if you’re happy with this current setup just renaming shell.nix to default.nix would turn off my alarm bells
In theory I think you should not be building the package using nix during development at all though, and just let setuptools handle it. For this project you’d use a shell.nix somewhat like this (documented in the section above the one you link, but lacks an anchor):
shell.nix:
{ pkgs ? import <nixpkgs> {} }:
let
pythonEnv = python3.withPackages (ps: with ps; [
<dependencies>
]);
in pkgs.mkShell {
buildInputs = [
setuptools
flake8
pythonEnv
];
}
That’d put python with all the dependencies installed on your path, as well as flake8 and setuptools.
Depending on how your project actually works, you could then drop setuptools and flake8 from the package definition in default.nix, and even add things like language servers or other dev-only dependencies here. The pythonEnv should probably then be in a separate nix file that you import from both default.nix and shell.nix.
But anyway, this is not going to cause any real problems, and is besides your question. The only reason I comment on this is because I’m immediately suspicious if I see import ./shell.nix {}, because that should never happen - shell.nix is a development shell, not a package. It will work, of course, but it’s kind of like putting emacs in your dependencies list.
well if I set it up the way you describe it won’t create those console scripts I need and it also won’t let me run python setup.py develop, while as the manual states
If we create a shell.nix file which calls buildPythonPackage, and if src is a local source, and if the local source has a setup.py, then development mode is activated.
my current approach does exactly what I want?
If I was using a language server or w/e I would of course separate it from my default.nix
I did not know nix-shell would fallback to default.nix though.
I don’t see why my sample shell.nix doesn’t allow python setup.py develop? It should create a python that has all the dependencies needed to do exactly that (assuming you replace <dependencies> with the long list of dependencies from your project I didn’t copy out).
I also think the CLI scripts should be there - I double checked, and if I use python3.withPackages (ps: with ps; [ flask ]) as my env I can call flask, for example. Maybe I misunderstand what CLI scripts you refer to.
You’re absolutely right, though, python is an odd case where the shell and the package often end up having the exact same contents, so sticking to this is probably fine. My comment is a nit, and unrelated to your real problem, so ignore me.
well the issue is that it will try to modify the nix store.
$ python setup.py develop
running develop
error: can't create or remove files in install directory
The following error occurred while trying to add or remove files in the
installation directory:
[Errno 13] Permission denied: '/nix/store/66fbv9mmx1j4hrn9y06kcp73c3yb196r-python3-3.8.9/lib/python3.8/site-packages/test-easy-install-556.write-test'
...
while my current approach does everything in a directory in /tmp
by cli scripts I mean that I define an entry point like this in my setup.py:
Hah, nevermind me then, I was convinced I’d used this successfully like this before. It’s probably possible to convince setuptools to work in a different directory somehow, but your current approach is simpler.
