How to make `withPackages` work with old Ruby 2.3 in nix-shell?

stravid · July 14, 2020, 5:10am

I’m trying to make my development environments reproducible by using / switching to nix-shell. I still have to support old Ruby versions and I’m struggling to make my shell.nix work. Specifically I struggle to install Bundler for the specific Ruby version I want.

I already figured out how to pin nixpkgs to a specific channel so I can have the ruby_2_3 package. While this works for Ruby 2.3 it installs Bundler for the most recent Ruby version in the channel.

{
  pkgs ? import (fetchGit {
    url = https://github.com/NixOS/nixpkgs-channels;
    ref = "nixos-20.03";
  }) {},
  oldpkgs ? import (fetchGit {
    url = https://github.com/NixOS/nixpkgs-channels;
    ref = "nixos-19.03";
  }) {}
}:

pkgs.mkShell {
  buildInputs = with pkgs; [
    oldpkgs.ruby_2_3
    oldpkgs.bundler
    which
    git
    postgresql_9_6
    parallel
  ];

  shellHook = ''
    mkdir -p .local-data/gems
    export GEM_HOME=$PWD/.local-data/gems
    export GEM_PATH=$GEM_HOME
    export PGHOST="$PWD/.local-data/postgresql/sockets"
    unset SSL_CERT_FILE
    unset NIX_SSL_CERT_FILE
  '';
}

Checking versions:

$ nix-shell --pure

$ which ruby
/nix/store/1vhjmpfmva4797c68q8pzc1pi8r07d08-ruby-2.3.8/bin/ruby

$ which bundle
/nix/store/75prx2g596sjxzjxsr7jnxc0vdwffggp-bundler-1.17.3/bin/bundle

$ ls -la /nix/store/75prx2g596sjxzjxsr7jnxc0vdwffggp-bundler-1.17.3/lib/ruby/gems/
total 12
dr-xr-xr-x 3 david david 4096 Jan  1  1970 .
dr-xr-xr-x 3 david david 4096 Jan  1  1970 ..
dr-xr-xr-x 9 david david 4096 Jan  1  1970 2.5.0

My attempt using withPackages results in an error I’m stuck with:


  pkgs ? import (fetchGit {
    url = https://github.com/NixOS/nixpkgs-channels;
    ref = "nixos-20.03";
  }) {},
  oldpkgs ? import (fetchGit {
    url = https://github.com/NixOS/nixpkgs-channels;
    ref = "nixos-19.03";
  }) {}
}:

pkgs.mkShell {
  buildInputs = with pkgs; [
    (oldpkgs.ruby_2_3.withPackages (rubyPackages: with rubyPackages; [
      bundler
    ]))
    which
    git
    postgresql_9_6
    parallel
  ];

  shellHook = ''
    mkdir -p .local-data/gems
    export GEM_HOME=$PWD/.local-data/gems
    export GEM_PATH=$GEM_HOME
    export PGHOST="$PWD/.local-data/postgresql/sockets"
    unset SSL_CERT_FILE
    unset NIX_SSL_CERT_FILE
  '';
}

$ nix-shell --pure --show-trace
error: while evaluating the attribute 'buildInputs' of the derivation 'nix-shell' at /nix/store/vr3wazryd2m12d940n7gsvyvf44cfws7-source/pkgs/build-support/mkshell/default.nix:28:3:
while evaluating 'getOutput' at /nix/store/vr3wazryd2m12d940n7gsvyvf44cfws7-source/lib/attrsets.nix:464:23, called from undefined position:
while evaluating anonymous function at /nix/store/vr3wazryd2m12d940n7gsvyvf44cfws7-source/pkgs/stdenv/generic/make-derivation.nix:142:17, called from undefined position:
while evaluating the attribute 'ruby_2_3.withPackages' at /nix/store/r8iifjq4qp3hwmcd5hfn9ff9cyjkbk17-source/pkgs/top-level/all-packages.nix:8235:5:
attribute 'withPackages' missing, at /home/david/Code/tiny-pale-blue-dot/shell.nix:14:6

My best guess is that somehow withPackages only works with the newest Ruby versions. I have no idea if that is true and if so, what else I can do to install Ruby 2.3 with this Rubys Bundler.

Any help and pointers are appreciated.

fzakaria · July 14, 2020, 5:21am

Looking at the buildRubyGem derivation which is what is used to create bundler; it takes in a ruby program; which is then being defaulted to the default one in nixpkgs.

You will have to override it.

github.com

NixOS/nixpkgs/blob/efdbbb02eb568167d1ba22191663fae9267a06e2/pkgs/development/ruby-modules/gem/default.nix#L33

    
      
          } @ defs:
          
          
lib.makeOverridable (
          
          
{ name ? null
          , gemName
          , version ? null
          , type ? "gem"
          , document ? [] # e.g. [ "ri" "rdoc" ]
          , platform ? "ruby"
          , ruby ? defs.ruby
          , stdenv ? ruby.stdenv
          , namePrefix ? (let
              rubyName = builtins.parseDrvName ruby.name;
            in "${rubyName.name}${rubyName.version}-")
          , buildInputs ? []
          , meta ? {}
          , patches ? []
          , gemPath ? []
          , dontStrip ? false
          # Assume we don't have to build unless strictly necessary (e.g. the source is a

If you are just using it in nix-shell and not planning to package the application;
you can also just type gem install bundler within the nix-shell

The Ruby program is patched so that it automatically installs to the user-directory or you can take a look at my JRuby example nix-environments/shell.nix at 6329cf142f59e258c16a4647d7a53864221637f1 · nix-community/nix-environments · GitHub by specifying a GEM_HOME specifically.

  shellHook = ''
    # JRuby wants to install Gem's in the nix-store which is read-only
    # set GEM_HOME to make it a writable directory
    export GEM_HOME=$(${jruby}/bin/ruby -e 'puts Gem.user_dir')
    export GEM_PATH=$GEM_HOME
    # Add the GEM binary files to the path
    export PATH=$GEM_HOME/bin:$PATH
  '';

Rather than importing a whole nix channel; I’ve had better success just making my own derivatoin at the version I want.

Example

{ pkgs ? import <nixpkgs> { } }:
with pkgs;
with stdenv;
with stdenv.lib;
let
  jruby-9_2_9_0 = jruby.overrideAttrs (oldAtrrs: rec {
    version = "9.2.9.0";
    src = fetchurl {
      url =
        "https://s3.amazonaws.com/jruby.org/downloads/${version}/jruby-bin-${version}.tar.gz";
      sha256 = "04grdf57c1dgragm17yyjk69ak8mwiwfc1vjzskzcaag3fwgplyf";
    };
  });
in mkShell {
# stuff
``

stravid · July 14, 2020, 6:57am

Thank you, I will research and try to do that.

That’s a simple and brilliant idea. I will fall back to that should I fail to make it work the other way. Currently I’m happy with only a nix-shell. (And to be honest already challenged to fully understand what I’m doing with nix. So deployments with nix and other cool stuff needs to wait until my understanding caught up.)

I’m struggling to understand the differences between what you do and what I do on a conceptual level. How do they differ (in what happens when I run nix-shell)?

My lack of understanding let’s me only come up with less used disk space?

fzakaria · July 14, 2020, 6:30pm

If you are doing a simple Ruby application that’s not tied to any particular Ruby version; honestly as much as I’m enjoying Nix it might be overkill for you.

Here are the reasons I think nix-shell & Ruby development might make sense:

Your development environment needs other tooling aside from Ruby installed such as NodeJs, Chromium, JDK etc… Simplifying the setup to a single nix-shell might be easier than a myriad of other package manager commands.
You want to run a very specific version of a tool or Ruby or apply patches. Sure you could do it with rbenv (or other alternative Ruby version managers) but it’s simpler with Nix.
Your Ruby application requires system libraries for FFI or extensions. Nix can help guarantee they are present.
You need environment variables setup etc… You can accomplish this easily with the shellHook
You might choose to package & deploy your application using Nix

I find it less clear what version of Ruby you are using by just importing a fixed version nixpkgs especially if you are using the ruby general attribute as opposed to the ones whose name includes a version ruby_2_3_0

By embedding the derivation directly also; I think overall the nix-shell is more perscriptive of what exactly it wants & lets you change / patch the Ruby version if needed ever easily.
(Rather than have to find the correct nixpkg channel version)

Those are my 2cents from someone somewhat getting started into Ruby + Nix; so please take that into account.

stravid · July 15, 2020, 2:12pm

For future reference: I made it work with following Nix expression. Thank you for your help.

{
  pkgs ? import (fetchGit {
    url = https://github.com/NixOS/nixpkgs-channels;
    ref = "nixos-20.03";
  }) {},
  oldpkgs ? import (fetchGit {
    url = https://github.com/NixOS/nixpkgs-channels;
    ref = "nixos-19.03";
  }) {},
  ruby ? oldpkgs.ruby_2_3,
  bundler ? oldpkgs.bundler.override { inherit ruby; }
}:

pkgs.mkShell {
  buildInputs = with pkgs; [
    ruby
    bundler
    which
    git
    postgresql_9_6
    parallel
  ];

  shellHook = ''
    mkdir -p .local-data/gems
    export GEM_HOME=$PWD/.local-data/gems
    export GEM_PATH=$GEM_HOME
    export PGHOST="$PWD/.local-data/postgresql/sockets"
    unset SSL_CERT_FILE
    unset NIX_SSL_CERT_FILE
  '';
}

fzakaria · July 15, 2020, 4:24pm

Nice!
Surprised you unset the SSL_CERT_FILE;
I prefer to just pull in cacerts and point it to that to make it extra reproducible

stravid · July 16, 2020, 9:57am

That’s because I had an SSL error in another project and couldn’t figure out a way to make it work. I just looked at your cacerts example and will give it a spin. Thanks!

anon31783435 · July 16, 2020, 10:13am

With the variant of not importing a whole second nix channel (oldpkgs),
a solution could be the following:

{ pkgs ? import <nixpkgs> {} }:

let
  ruby = pkgs.ruby.overrideAttrs(attrs: {
    version = "2.3.8";
    src = pkgs.fetchurl {
      url = "https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.8.tar.gz";
      sha256 = "1gwsqmrhpx1wanrfvrsj3j76rv888zh7jag2si2r14qf8ihns0dm";
    };
    patches = [];
    postPatch = "";
  });
  bundler = pkgs.bundler.overrideAttrs(attrs: {
    name = "bundler-1.17.3";
    src = pkgs.fetchurl {
      url = "https://rubygems.org/gems/bundler-1.17.3.gem";
      sha256 = "0ln3gnk7cls81gwsbxvrmlidsfd78s6b2hzlm4d4a9wbaidzfjxw";
    };
  });
in pkgs.mkShell {
  buildInputs = with pkgs; [
    ruby
    bundler
    which
  ];
}

By overriding it means you’ll need to build the derivations locally yourself.
You won’t get the benefit of just downloading the pre-build versions from the public cache.
Means your first build will take longer, depending on how long it takes to build ruby, bundler, …

PS: Ruby 2.3.8 needs do override patches & postPatch of the current builder → ruby/default.nix