I want to retool a bit so that I can work with scientists. I’m a bit tired of buisness people. Scientists like conda. Nix has some advantages over Conda, but “favored by the scientific community” is not one of them.
So maybe it would be nice to be able to easily convert Conda packages to nix packages–then we could have the best of both worlds.
There are also tools like Galaxy, which understands how to run these tools via a planemo “tool definition file”. And of course there’s the original source code which is typically available and often has packaging gotcha’s somewhere in its documentation.
That’s a lot of context to go off of, perhaps enough to get an LLM to a point where it be useful.
I’m imagining a framework which can generate both the packages and some tests for them such that it only recommends a package for publication if the resulting build succeeds (from a return code point of view) and the nix package output behaves similarly to the conda package when harnessed by planemo.
Failures it could categorize with accompanying error info. Humans could then take it from there and build “yeah this works” type consensus around the outputs.
Have I entered the realm of science fiction, or is this doable? It’ll obviously take a lot of learning new things on my part, but that’s sort of the point. Just hoping to bounce the idea around a little bit.
that’s a problem that should be entirely solvable by “hand” written code, throwing llm’s at this will just lead to wonky inaccurate results that are hard to debug. This is of course doable but it can be quite hard and time consuming. @figsoda has already built a greate tool called nix-init which is able to do the thing you are imaging for most go, rust and python applications (conda is not supported as far as I’m aware) and using it can feel quite magical.
Running Conda environments from the Nix store?
It’s probably not too hard to use a conda-lock file as a starting point to build Conda environments that live in the store.
That would mostly be a matter of invoking the right fetchers, unpacking the environment & liberal use of autoPatchelfHook.
To get Nix expressions from Conda?
This is a pipe dream. Nix packages are much more rich than Conda packages, just for starters.
I was recently in a classroom full of students who were all terrified that they had broken their conda environments and would be unable to proceed with their assignments.
Like a fool, I had styled myself as a computer expert, here to help with the programs, here to get help with the chemistry.
Conda had failed during a switch between solvers, so even though the classic solver was enabled they were getting errors related to libmambapy. Whatever, we got it sorted.
My goal is to be able to say:
we don’t need to bother with solvers at all, here’s how to use nix!
But there is no “hicexplorer” nix package, so maybe it’s not the right direction for me to take them just yet. I guess I was just looking for short cuts because I can’t package everything myself.
This is what I needed to hear:
Nix packages are much more rich than Conda packages, just for starters.
I’ll proceed to dream different dreams, thank you.
Maybe it would be an idea to try and package hicexplorer with nix and get it into nixpkgs? Not sure how difficult this might be. But then you could provide the necessary python environment via nix entirely.
In my python projects, which are less groundbreakingly scientific I have to say, I often limit nix to providing the interpreter and use poetry to manage the dependencies. But having everything in nix is also nice.