Overriding a dependency of a package

Help
1

I’m trying to override a version of h5py 3.2 to 2.10 in tensorflow 2.4.1, due to an error with how h5 models are loaded. So I’m eyeing on this specific derivation of h5py in nixos-20.09. But I’m using an unstable version of nixpkgs to get tensorflow 2.4.1.

shell.nix

{ pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/3847a2a8595bba68214ac4b7e3da3fc00776989b.tar.gz") {} }:
  
with pkgs;

let
  inherit (lib) optional optionals;
  nodejs = nodejs-14_x;
  erlang = beam.interpreters.erlangR23;
  elixir = beam.packages.erlangR23.elixir;
in
mkShell {
  buildInputs = with python38Packages; [
    cacert
    nodejs
    elixir
    erlang
    tensorflow
    scikitimage
    inotify-tools
  ];

  shellHook = ''
    export LANG="en_US.UTF-8";
    export LC_TYPE="en_US.UTF-8";
  '';
}

I tried reading the nix pills on how to override packages, but the syntax is so unfamiliar to me that I can’t comprehend it. I also tried checking the source for tensorflow but proved to be too much for me to wrap my head around.

I also filed an issue here: tensorflow 2.4 depends on h5py 2.10 but is packaged with h5py 3.2 · Issue #124599 · NixOS/nixpkgs · GitHub. I appreciate any pointers!

1 Like
2

Here’s how I would approach it. It does require rebuilding tensorflow since the input changed, which may take a while.

{ pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/3847a2a8595bba68214ac4b7e3da3fc00776989b.tar.gz") {} }:
  
with pkgs;

let
  inherit (lib) optional optionals;
  nodejs = nodejs-14_x;
  erlang = beam.interpreters.erlangR23;
  elixir = beam.packages.erlangR23.elixir;
  h5py-nixpkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/76ed24ceab9ec8b520f977a2803181f0c1d86b4d.tar.gz") {};
in
mkShell {
  buildInputs = with python38Packages; [
    cacert
    nodejs
    elixir
    erlang
    (tensorflow.override {h5py = h5py-nixpkgs.python38Packages.h5py;})
    scikitimage
    inotify-tools
  ];

  shellHook = ''
    export LANG="en_US.UTF-8";
    export LC_TYPE="en_US.UTF-8";
  '';
}

this line imports the version of nixpkgs that contained the derivation for h5py 2.10

  h5py-nixpkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/76ed24ceab9ec8b520f977a2803181f0c1d86b4d.tar.gz") {};

and this tells nix to override the ‘h5py’ function argument to use that older version:

    (tensorflow.override {h5py = h5py-nixpkgs.python38Packages.h5py;})
2 Likes
3

Sweet. This is exactly what I was looking for. As you mentioned, it built it from scratch which is alright. But I found an alternative way that didn’t require me to, mach-nix. Seems to work decently!

Thank you for taking the time to explain though. This was very helpful.

4

@sekun, thanks for the info! For reference, would you be able to share how you solved the issue using mach-nix?

5

@dbdr Sure! The project is public anyway.

1 Like