Anyone has a working jupyterhub + jupyter lab setup?

Help
2

Hello,

I use the thing you may see below ( it is obtained and combined from posts on this channel).
I left some comments in there if you want to play & try. And sorry for not directly answering your question (I hope this helps).

Random comments:

  • there is examples on how to add your own packages (I just marked the path with dots).
  • (one such example package definition for is even more below)
  • I think I tested with several kernels but not sure (don’t need it, this is older than three days so don’t remember any more).
  • it uses 3.7 because those tensorflow things didn’t yet work on 3.8 (haven’t checked the situation lately)
  • shows how to add extensions
  • shows how to add packages
  • shows how to add other programs / packages
  • the shellHook at the end is really bad but does it’s job
  • unfortunately, I don’t remember, how I was able to try to find out the correct commands
  • so, somebody may answer or tell me, how I did it :slight_smile: I’d prefer someone from the marketing thread: if possible, this is something that I’d like to be easily available information on how to use nix :slight_smile:
  • so, why I said this bad (I mean ugly), you see, the scripts calls jupyterlab build twice
  • the reason is that the first one fails because of missing directories but creates them
  • and then the second run succeeds by being now able to use those directories
  • as a last thing, it starts the jupyter lab
  • you may leave it out, if you want start it manually (just give that command on shell then)

I hope that this helps even though this doesn’t answer your question directly.

# default37.nix
# start with
#   nix-shell default37.nix
# or
#   nix-shell default37.nix -I nixpkgs=/home/isto/nixpkgs-unstable/
let 
  pkgs = import <nixpkgs> {};
  newpkgs = import pkgs.path { overlays = [ (pkgsself: pkgssuper: {
    python37 = let
      packageOverrides = self: super: {
            openpyxl = openpyxl2;
        # numpy = super.numpy_1_10;
      };
    in pkgssuper.python37.override {inherit packageOverrides;};
  } ) ]; };
  # pkgsConfig = { allowUnfree = true; };
  # nixpkgs = import (
  #      fetchTarball https://github.com/NixOS/nixpkgs-channels/archive/nixos-unstable.tar.gz
  #    ) { config = pkgsConfig; };
  multitasking = newpkgs.callPackage /...../......./multitasking/default.nix {
    buildPythonPackage = newpkgs.python37Packages.buildPythonPackage;
    fetchPypi = newpkgs.python37Packages.fetchPypi;
  };
  openpyxl2 = pkgs.callPackage /..../...../openpyxl/default.nix {
    buildPythonPackage = pkgs.python37Packages.buildPythonPackage;
    fetchPypi = pkgs.python37Packages.fetchPypi;
    isPy27 = pkgs.python37Packages.isPy27;
    pytest = pkgs.python37Packages.pytest;
    jdcal = pkgs.python37Packages.jdcal;
    et_xmlfile = pkgs.python37Packages.et_xmlfile;
    lxml = pkgs.python37Packages.lxml;
  };
  yfinance = newpkgs.callPackage /..../...../yfinance/default.nix {
    buildPythonPackage = newpkgs.python37Packages.buildPythonPackage;
    fetchPypi = newpkgs.python37Packages.fetchPypi;
    pandas = newpkgs.python37Packages.pandas;
    numpy = newpkgs.python37Packages.numpy;
    requests = newpkgs.python37Packages.requests;
    multitasking = multitasking;
  };
  kernels = [
    # pkgs.python37Packages.ansible-kernel
    # pythonPackages.jupyter-c-kernel
    # pkgs.gophernotes
  ];
  additionalExtensions = [
    # "@jupyterlab/toc"
    # "@jupyterlab/fasta-extension"
    # "@jupyterlab/geojson-extension"
    # "@jupyterlab/katex-extension"
    # "@jupyterlab/mathjax3-extension"
    # "@jupyterlab/plotly-extension"
    # "@jupyterlab/vega2-extension"
    # "@jupyterlab/vega3-extension"
    # "@jupyterlab/xkcd-extension"
    # "jupyterlab-drawio"
    # "@jupyterlab/hub-extension"
    #jupyter labextension install @jupyter-widgets/jupyterlab-manager
    #jupyter labextension install @bokeh/jupyter_bokeh
    "@jupyter-widgets/jupyterlab-manager"
    "@bokeh/jupyter_bokeh"
    "@pyviz/jupyterlab_pyviz"
    # "jupyterlab_bokeh"
  ];
  pythonEnv = newpkgs.python37.withPackages (ps: with ps; [
    ipykernel jupyterlab jupyterlab_server
    python-language-server pyls-isort
    matplotlib numpy pandas scikitlearn # pytorch
    tensorflow tensorflow-tensorboard # tflearn
    bokeh bkcharts networkx statsmodels seaborn
    holoviews hvplot 
    param pyviz-comms # pyviz
    Keras keras-applications keras-preprocessing
    Theano 
    nltk
    scipy plotly
    sympy
    # cufflinks
    yfinance
    openpyxl
    #openpyxl2
    sphinx nbsphinx nbconvert
    dash dash-html-components dash-core-components dash-renderer dash-table
    jinja2 # jinja2_time j2cli
    ipywidgets
  ]);
  # ] ++ [ps.jinja2])
in newpkgs.mkShell rec {
  buildInputs = [
    pythonEnv
    pkgs.nodejs
    # newpkgs.tectonic # not good
    # newpkgs.python-language-server
    # newpkgs.pyls-isort
  ] ++ kernels;
  shellHook = ''
    TEMPDIR=$(mktemp -d -p /tmp)
    mkdir -p $TEMPDIR
    cp -r ${pkgs.python37Packages.jupyterlab}/share/jupyter/lab/* $TEMPDIR
    chmod -R 755 $TEMPDIR
    echo "$TEMPDIR is the app directory"

    # kernels
    export JUPYTER_PATH="${pkgs.lib.concatMapStringsSep ":" (p: "${p}/share/jupyter/") kernels}"

# labextensions
${pkgs.stdenv.lib.concatMapStrings
     (s: "jupyter labextension install --no-build --app-dir=$TEMPDIR ${s}; ")
     (pkgs.lib.unique
       ((pkgs.lib.concatMap
           (d: pkgs.lib.attrByPath ["passthru" "jupyterlabExtensions"] [] d)
           buildInputs) ++ additionalExtensions))  }
jupyter lab build --app-dir=$TEMPDIR
chmod -R +w $TEMPDIR/staging/
jupyter lab build --app-dir=$TEMPDIR
 
# start jupyterlab
jupyter lab --app-dir=$TEMPDIR
    '';
}

Here is example of one package package definition not found from the pkgs:

# ...../yfinance/default.nix
{ stdenv, fetchPypi, buildPythonPackage
, pandas 
, numpy 
, requests 
, multitasking 
}:
buildPythonPackage rec {
  pname = "yfinance";
  version = "0.1.54";
  format = "setuptools";

  src = ./.;

  propagatedBuildInputs = [ pandas numpy requests multitasking ];

  meta = with stdenv.lib; {
    description = "Yahoo! Finance market data downloader.";
    homepage = https://github.com/ranaroussi/yfinance;
    license = licenses.asl20;
    maintainers = with maintainers; [ gspia ];
  };
}
6 Likes