Add libraries to sagemath jupyter sage/python kernels

Help
1

Hello,

I’m using the great sage(math) program. Unfortunately, I can’t find how to add libraries to the sage or to the python kernel included into sagemath. So for example, if I want to run a python kernel with numpy, and a sage kernel at the same time, I need to open both sage’s notebook and install + load a different python notebook by adding jupyter and numpy in the python dependencies…

Can I install directly new python libraries directly to the python or sage kernel of sage?

Thanks!

2

Hi,

unfortunately, with the current state of sage, this is difficult. Ideally sage would be just another python library that you could install on your system side by side with other python libraries. That is currently not the case for various reaons, the main one being that it depends on a bunch of environment variables at runtime that need to be set through a wrapper. There is an upstream ticket to improve this, though it doesn’t have very high priority. I have worked on hacking around this for nix, but it needs more work which I currently don’t have the time for.

The concrete problem is, that currently the sage package depends on a python environment (instead of the other way around, as it should be). It would be relatively easy to add a package to that environment: just add it to pythonRuntimeDeps in pkgs/applications/science/math/sage/default.nix. The issue it that since sage depends on that environment, it will trigger a rebuild of the long running (~1h) tests, which is annoying.

So the easiest way to achieve your goal would be to make it easily possible to add packages to that environment. To make that actually usable without long rebuilds every time something changes, we should probably disable the sage tests when any additional dependencies are specified. That is not very nice, but its would at least make it possible for now. So you’d essentially opt out of sage tests by making any changes.

What do you think about that?

3

First thanks for your great job around sage, it’s amazing!

Concerning the “sage as python library”, how would you do the annoying pre-processing of stuff like “2^3” into “Integer(2)**Integer(3)”, or “load(myfile.sage)” to make the difference between python syntax and sage syntax ?

Concerning making the python environment depend on sage, it has to be done on the nix side or there is something that make it impossible upstream? Also, if I want to upgrate the variable pythonRuntimeDeps without waiting for a simpler fix, what is the way to go ? (sorry, I’m improving everyday, but I’m not yet an expert of nix, and I guess there is a better way than maintaining a fork of the full nixpkgs repo)

Concerning the fix, it seems a nice fix, but I’ve a few remarks/questions:

  • tests are indeed a problem, so disabling them for this kind of stuff would be nice indeed (if possible an option to disable tests when somebody wants to recompile sage would be nice if it does already exist. I remember losing an afternoon to wait for the end of sage tests when the last version were not yet built by hydra)
  • does that also mean that it will rebuild the full sage at every change?
  • is the pythonRuntimeDeps also used by all kernels, namely sage’s kernel, python’s 2 kernel and python’s 3 kernels?

Thanks again for the help!

4

Thanks!

Well, the sage repl builds on top of ipython (e.g. jupyter). Just like you can now do

nix-shell -p 'python2.withPackages(ps : with ps; [ipython numpy])'

And then have a ipython repl with numpy available, you should ideally be able to do the same with sage instead of ipython. The sage binary would then just be a thin wrapper that starts the ipython based repl. More importantly, it could start that repl based on any python environment that includes sage.

Not impossible (hence the work in progress hack I linked earlier), but upstream certainly doesn’t make it easy due to all the environment variables necessary. If I get that hack to working condition, we’d be the first distro to my knowledge with a fully functional python-library sage.

  • Get a local checkout of nixpkgs (git clone https://github.com/nixos/nixpkgs.git).
  • cd into it, check out the version of nixpkgs you want to work with. If you want to stay in the release channel, that would be git checkout release-19.03. You can also stay on master to work with the bleeding edge.
  • Edit the list in the file I mentioned earlier. Maybe manually disable the tests by setting doInstallCheck = false; in pkgs/applications/science/math/sage/sage-tests.nix.
  • You can now do one of several things:
    • Just build sage without including it in you system environment: nix build -f. sage. A result symlink will be placed in your current working directory, pointing to the sage build. Launch with result/bin/sage.
    • Create a temporary environment with sage installed: nix-shell -I nixpkgs=. -p sage. You’ll be in a bash shell with the proper sage in PATH.
    • If you’re on nixos, rebuild your system based on this patched nixpkgs checkout: sudo nixos-rebuild -I nixpkgs=. switch
    • If you’re not on nixos, you can probably convince nix-env to install the right sage version. I have never really used nix-env though.

I thought about this before, but I am a bit hesitant about a user facing option to disable tests. That means it may theoretically happen that sage is broken in some non-obvious way (i.e. wrong calculation results) and you wouldn’t notice. Thats not very likely, but could happen. At least right now, sage would fail its tests in that case.

No, since the deps are only used and runtime (and therefore also test time), not build time.

You mean when using the sage jupyter notebook? I think yes, that should use sages python environment.

5

and I guess there is a better way than maintaining a fork of the full nixpkgs repo

Missed that part. Usually yes, you could either use package.override {...} or overlays. However since the sage package is pretty complicated and consists of multiple layers (unfortunately, I’d love to simplify it) its not that easy to override the right thing as long as I don’t add an explicit config option.

6

Ok great, thanks a lot for everything, and let me know if you get some progress !

7

I’m not sure what you mean. Progress on what exactly?