Has anyone ever tried to install Plone (and Zope) with Nix and has a recipe to use at least as starting point?
For anyone coming here, the discussion on the Plone’s community site has been more fruitful.
I’ve used your repo to get a development shell (metapensiero / plone-nix · GitLab) I had to change the commit it was point at, presumably because of a rebase. I had to change
e62703057add368d6ad7fc231955f0628171cb6c. Is that the right commit to point to?
Hi @joehealy, sorry for the late response. It has been some time since I worked on this repo, can you tell me where this commit
e5b34577bb7fc3782362d659b56ff1fea2ef7f16 is mentioned?
Thanks for the reply.
In the docs on this page: metapensiero / plone-nix · GitLab under “How to create a Plone development environment”. I’ve been using the latest commits more recently having figured out how to modify the examples to match your latest changes.
However I’m running into quite a few issues installing things (collective.celery most recently) and am thinking of moving back to a more traditional buildout install. Is the wider plone world not yet ready for a nix approach?
What issues did you stumbled into? want to share some?
Python packaging and nix do have few problems alone, mostly due to the packaging of packages with architecture specific bits. Some other problems often arise from the fact that some people still need to learn how to specify the dependencies (e.g. testing dependencies listed among the others) and that
setuptools required much effort before reaching maturity. On top of that a tool like
zc.buildout that wants to do a bit what pip does and then some product/system configuration doesn’t help. Plone with its hundred or like dependencies means that’s inevitable to have issues when you want to install it in the same environment (python/nix env) with another medium sized framework with its own dependencies that may interfere with Plone’s.
Also please bear in mind that
mach-nix work is split into two major components, there the “Nix package deriver” that runs when you use it, but this part uses the dependency tree calculated by a crawler at a specific point in time (it downloads this db from a git repo). This in turn means that if what you want to install is newer than the pin it will not be found… I pin that db here default.nix · master · metapensiero / plone-nix · GitLab
thanks for the above - I’d love to share
I’ve mostly (sort of) figured the above out. I last used plone heavily about 5 or 6 years ago, before plone 5 (4.3.2 from memory). Since then I’ve been using nix for haskell with a tiny bit of local python packages. So I’m new to mach nix.
I’ve figured out the pinning of the crawler part and updated that to a more recent revision.
I’m now wondering if there are inherent incompatibilities between the packages I’m wanting to use at the revision of the crawler I’ve picked or if somehow they don’t work nicely on nix.
my next planned step is to just get the buildout environment and try running buildout by itself, then if that still fails an ubuntu or similar vm to see if I reproduce the issue.
In terms of the direct issue I find, when using:
pypiDataRev = "1ba4b301d347c90197d99c9e476d6dc230c94dd3"; pypiDataSha256 = "1pkgjj8kdhz0vy6rklkca9hbciyxng05d8hiwpfn6naj3d2yspaq";
I can add collective.documentviewer to the “moreRequirements” and it works well, however so processing can occur in the background, I want to add collective.celery so the pdf processing occurs in the background. When I do so, the environment builds, but I get errors when trying to run
File "/nix/store/z1ivb7r03dr6gyns1vpry2srywmr4lq4-plone-5.2.3-pending-skeleton/parts/instance/etc/site.zcml", line 16.2-16.23 File "/nix/store/805sa391yav0ncm5cxd1k62vpzpf4nyg-python3-3.7.6-env/lib/python3.7/site-packages/Products/CMFPlone/configure.zcml", line 110.2-114.8 File "/nix/store/805sa391yav0ncm5cxd1k62vpzpf4nyg-python3-3.7.6-env/lib/python3.7/site-packages/collective/documentviewer/configure.zcml", line 11.2-11.34 AttributeError: can't delete attribute
At this point, I’m not to sure how to debug directly, so trying a few options as described above.
My preference is to resolve this and for this to all be controlled via nix - anything else these days feels a little shaky.
For reference, this is the shell.nix file I am using: debian Pastezone
hi Joe, I run your
shell.nix and I stumbled on your same error, which with
some more context is the following:
File "/nix/store/805sa391yav0ncm5cxd1k62vpzpf4nyg-python3-3.7.6-env/lib/python3.7/site-packages/collective/documentviewer/async_utils.py", line 40, in <module> @task() File "/nix/store/805sa391yav0ncm5cxd1k62vpzpf4nyg-python3-3.7.6-env/lib/python3.7/site-packages/collective/celery/__init__.py", line 37, in decorator return getCelery().task(base=AfterCommitTask, **task_kw)(new_func) File "/nix/store/805sa391yav0ncm5cxd1k62vpzpf4nyg-python3-3.7.6-env/lib/python3.7/site-packages/collective/celery/utils.py", line 98, in getCelery _local.celery = _getCelery() File "/nix/store/805sa391yav0ncm5cxd1k62vpzpf4nyg-python3-3.7.6-env/lib/python3.7/site-packages/collective/celery/utils.py", line 88, in _getCelery del(celery.backend) File "/nix/store/805sa391yav0ncm5cxd1k62vpzpf4nyg-python3-3.7.6-env/lib/python3.7/site-packages/celery/local.py", line 161, in __delattr__ delattr(self._get_current_object(), name) zope.configuration.xmlconfig.ZopeXMLConfigurationError: File "/nix/store/805sa391yav0ncm5cxd1k62vpzpf4nyg-python3-3.7.6-env/lib/python3.7/site-packages/collective/documentviewer/upgrades.zcml", line 6.2-12.50 File "/nix/store/s1aw12vgmv4w1005yh3yl47mzln9y1xr-plone-5.2.3-pending-skeleton/parts/instance/etc/site.zcml", line 16.2-16.23 File "/nix/store/805sa391yav0ncm5cxd1k62vpzpf4nyg-python3-3.7.6-env/lib/python3.7/site-packages/Products/CMFPlone/configure.zcml", line 110.2-114.8 File "/nix/store/805sa391yav0ncm5cxd1k62vpzpf4nyg-python3-3.7.6-env/lib/python3.7/site-packages/collective/documentviewer/configure.zcml", line 11.2-11.34 AttributeError: can't delete attribute
From this my opinion is that some celery configuration is missing. The story
here is that when plone is starting up, some celery task is scheduled, probably
after reading the
collective/documentviewer/upgrades.zcml file, between the
line 6 and 12:
<gs:upgradeStep title="Convert all valid unconverted files document viewer" description="if async installed, we queue up all processes" source="*" destination="upgrade-all" handler=".upgrades.convert_all" profile="collective.documentviewer:default" />
All runs until a
callable is decorated with
@task (from celery) to become a
collective/documentviewer/async_utils.py). Decorators do run custom
code when the code context is first evaluated (in this case the module), takes a
callable and returns another callable. During this phase the line
del(celery.backend) and this is the beginning of the end: that
attribute is missing and I suppose that this is due to a missing celery
configuration, but I’m no celery expert. What I know is that surely it needs
some configuration for the queues and the results.
Looking at the
collective.celery package repo, I see that there is some
configuration to prepare, You can find more here:
collective.celery/README.rst at 497e81ddc82694ff2388e0b44672a936f44d750e · collective/collective.celery · GitHub .
I’ve tried something like:
$ pcelery worker /nix/store/s1aw12vgmv4w1005yh3yl47mzln9y1xr-plone-5.2.3-pending-skeleton/parts/instance/etc/zope.conf Traceback (most recent call last): File "/nix/store/771bdvayqd8axcncslj3i55n8ikmx9pm-python3.7-collective.celery-1.1.4/bin/.pcelery-wrapped", line 9, in <module> sys.exit(main()) File "/nix/store/805sa391yav0ncm5cxd1k62vpzpf4nyg-python3-3.7.6-env/lib/python3.7/site-packages/collective/celery/scripts/ccelery.py", line 42, in main from Zope2.Startup.run import configure File "/nix/store/805sa391yav0ncm5cxd1k62vpzpf4nyg-python3-3.7.6-env/lib/python3.7/site-packages/zope/deferredimport/deferredmodule.py", line 70, in __getattr__ v = get.get() File "/nix/store/805sa391yav0ncm5cxd1k62vpzpf4nyg-python3-3.7.6-env/lib/python3.7/site-packages/zope/deferredimport/deferredmodule.py", line 54, in get return super(DeferredAndDeprecated, self).get() File "/nix/store/805sa391yav0ncm5cxd1k62vpzpf4nyg-python3-3.7.6-env/lib/python3.7/site-packages/zope/deferredimport/deferredmodule.py", line 37, in get v = __import__(module, *self._import_chicken) ModuleNotFoundError: No module named 'ZServer'
plone-nix environment the
zope.conf is not local but is packaged,
but as you can see I stumbled on another issue which is the missing ZServer
module, which is missing because latest Plone/Zope combo use a more generic
WSGI pipe to publish the stuff, and the ZServer isn’t used anymore.
Another issue may be that the
plone.celery dependency on the
is defined as
celery>=4 but recently celery released a new major release (the
5.x) which a friend of mine uses and he tells me that especially “startup” phase
changed much. Checking the celery version I’ve got 5.05 version:
[nix-shell:~/wip/metapensiero/joe]$ celery --version 5.0.5 (singularity)
I think you will have some work ahead if really is required for
collective.celery to work with the latest Plone releases, even in an
Wow! Thank you so much for your efforts. I really appreciate the help.
I had assumed that celery would add something in to give me a form to configure it through the management interface.
I was wondering how all this worked, being more familiar with the old z server approach (well zeo cluster and then front end workers…).
Thankyou so much for your help again - looks like I’m back on with nix and finding a new solution to viewing pdfs - probably custom views with an iframe or embedded object. Also laying out your process has given me some tips on debugging this.
I’ll keep an eye on celery and see how this evolves over time.
For reference, there appears to be a known issue with celery (as @azazel75 kindly worked out above) Doesn't work with plone 5.2.4-py3 (but works with plone 5.2.1-py3) · Issue #17 · collective/collective.celery · GitHub. I think the issue may be closed prematurely, but I’ll give applying that patch a go and report here and on the issue.