Packaging py-spy

Help
1

I’m trying to package py-spy.

It is written in Rust, compiled to a single binary executable and distributed as
a Python wheel! Weird though this might seem at first blush, it makes sense as
the target audience are Python developers, so the author has chosen to
distribute it as something installable with pip. But, in the end, it’s just a
binary which has absolutely no dependency on Python whatsoever.

Python .whls are just Zip archives, and this one is pretty simple:

.
├── py_spy-0.3.3.data
│   └── scripts
│       └── py-spy
└── py_spy-0.3.3.dist-info
    ├── DESCRIPTION.rst
    ├── METADATA
    ├── metadata.json
    ├── RECORD
    ├── top_level.txt
    └── WHEEL

3 directories, 7 files

… the only file needed is py_spy-0.3.3.data/scripts/py-spy; it works fine as
a standalone executable.

If I try to download this .whl from PyPI using fetchurl, it complains that
it doesn’t know how to unpack it:

unpacking source archive /nix/store/zgklnbxx4qii7k8l361dy839bvr6mhhr-py_spy-0.3.3-py2.py3-none-manylinux1_x86_64.whl
do not know how to unpack source archive /nix/store/zgklnbxx4qii7k8l361dy839bvr6mhhr-py_spy-0.3.3-py2.py3-none-manylinux1_x86_64.whl

I was under the impression that fetchurl didn’t do any unpacking, but that seems to be wrong?

Can you suggest how to go about packaging this?

As a secondary issue, any given version comes in 3 varieties, Linux 32/64 +
Darwin 64 (well, 4 if you include Windows) with URLs:

  • https://files.pythonhosted.org/packages/8e/a7/ab45c9ee3c4654edda3efbd6b8e2fa4962226718a7e3e3be6e3926bf3617/py_spy-0.3.3-py2.py3-none-manylinux1_x86_64.whl
  • https://files.pythonhosted.org/packages/11/f0/20a67fd3dfe1a693bfb37b785615e4a468558d1f4607e8498fd2a959eb49/py_spy-0.3.3-py2.py3-none-manylinux1_i686.whl
  • https://files.pythonhosted.org/packages/60/fa/55656827fedfe0728ea70225115065bef1ff000d7ce0604e9c1f4c84a2f2/py_spy-0.3.3-py2.py3-none-macosx_10_7_x86_64.whl

Any suggestions on how to deal with this cleanly?

1 Like
2

Usually, with nix, it’s easier to build from source. I’d forget about python and pip, and try to build it with the buildRustPackage infra. NixOS - Nixpkgs 21.05 manual

1 Like
3

I’m sorry, it seems that my original message was not sufficiently clear on this
point: am trying to forget about Python and pip.

The author of the package distributes pre-built executables in a zip archive
with the quirk of its filename ending in .whl rather than .zip. You appear
to be suggesting that compiling the package all over again using the Nix Rust
infrastructure is preferable to extracting the executable from the archive and
copying it into $out/bin.

I’m not sure whether I wasn’t clear enough about the approach I want to take:

  1. Download zip archive
  2. Move one file from said archive to $out/bin

or whether you think that compiling from scratch is a better approach.

4

I do think that compiling from source (that is https://github.com/benfred/py-spy I think) is preferable to using the prebuilt wheel, because in my experience patching binaries to work on NixOS is a never ending source of pain. But yes, it’s slower. Pick your poison.

2 Likes
5

Compile from source using buildRustPackage and install the built wheel using buildPythonPackage or buildPythonApplication by passing the wheel as src as well as setting format = "wheel";.

1 Like