How do you discover the override attributes for a derivation?

should be jetbrains = super.jetbrains // {. It is because super.jetbrains is an attrset, and I replace a jdk attribute in this attrset.

1 Like

Ah, I’d assumed that was a discourse formatting problem rather than intentional :slight_smile:.

Nevertheless, @danbst it’s still wanting to install the invalid jdk jbrsdk-11_0_2-osx-x64-b485.1.tar.gz.drv.

Any ideas why that would be?

nix-env -iA 'nixpkgs.jetbrains.idea-ultimate' --dry-run
(dry run; not doing anything)
installing 'idea-ultimate-2019.2.4'
these derivations will be built:
  /nix/store/ica1m5yq3f3y05xnw7ln1lnfvp0yjvyf-download_file?file_path=jbrsdk-11_0_4-osx-x64-b520.11.tar.gz.drv
  /nix/store/bf2hwhrvfl8g77gdiw053rayh06x0120-jetbrainsjdk-520.11.drv
  /nix/store/fazsa1a4l70s391rjk9yyi2hvrg0zbmp-download_file?file_path=jbrsdk-11_0_2-osx-x64-b485.1.tar.gz.drv
  /nix/store/fwwk976sd278zb68zy9wm5pkxss0rnhg-jetbrainsjdk-485.1.drv
  /nix/store/9kiajpmmsp3i6ysj4vdqq8dzi84mnr73-idea-ultimate-2019.2.4.drv

Is it because overrideAttrs firstly calculates the super’s attributes before overriding them?

if it helps, at all @danbst, my system info is as follows:

nix run nixpkgs.nix-info -c nix-info -m
[2 copied (0.0 MiB), 0.0 MiB DL]
 - system: `"x86_64-darwin"`
 - host os: `Darwin 19.2.0, macOS 10.15.2`
 - multi-user?: `no`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.3.1`
 - channels(ldeck): `"nixpkgs-20.03pre205792.fb1bc1b891f"`
 - nixpkgs: `/Users/ldeck/.nix-defexpr/channels/nixpkgs`

Let’s make it fully reproducible:

let
  nixpkgs = builtins.fetchTarball "https://github.com/nixos/nixpkgs/archive/4d2dd155461.tar.gz";
  pkgs = import nixpkgs {
      config.allowUnfree = true;
      overlays = [
          (self: super: {
            jetbrains = super.jetbrains // {
                jdk = super.jetbrains.jdk.overrideAttrs (oldAttrs: rec {
                    version = "520.11";
                    src = super.fetchurl {
                        url = "https://bintray.com/jetbrains/intellij-jdk/download_file?file_path=jbrsdk-11_0_4-osx-x64-b520.11.tar.gz";
                        sha256 = "0d1qwbssc8ih62rlfxxxcn8i65cjgycdfy1dc1b902j46dqjkq9z";
                    };
                });
            };
          })
      ];
  };
in pkgs.jetbrains.idea-ultimate

When I build it it doesn’t use old sources at all. But fails on fetch step (cannot download that file).

It’s there on bintray though @danbst.
See Service End for Bintray, JCenter, GoCenter, and ChartCenter | JFrog

Hmm, looks like I’ve provided the wrong sha256 though. It should be 3fe1297133440a9056602d78d7987f9215139165bd7747b3303022a6f5e23834

Perhaps that’ll help? @danbst

btw, I’ve reproduced the error. I was wrong, it indeed downloads old jdk version as well (but on linux it downloads it from binary cache). I’ll look into

2 Likes

@danbst how do you run that as a standalone reproducible thing?

save it to file.nix and run nix-build that/file.nix

1 Like

cool, thanks. Yes, here’s the complete output for me:

Okay, I found a bug in Nixpkgs. Made a PR. You can use that in a overlay like:

let
  pkgs = import <nixpkgs> {
      config.allowUnfree = true;
      overlays = [
          (self: super:
          let danbstFixedNixpkgs = builtins.fetchTarball https://github.com/danbst/nixpkgs/archive/203a8d12be0cf5609.tar.gz;
              danbstFixedPkgs = import danbstFixedNixpkgs { };
          in {
            jetbrains = super.jetbrains // {
                idea-ultimate = super.jetbrains.idea-ultimate.overrideAttrs (_: {
                    name = "idea-ultimate-2019.2.8";
                    src = super.fetchurl {
                        url = "https://download.jetbrains.com/idea/ideaIU-2018.2.8-no-jdk.tar.gz";
                        sha256 = "0lah6sb5yaq84c68ak07hsn8k3cjrq4phl4kyiydbhwi09qsc339";
                    };

                });
                jdk = danbstFixedPkgs.jetbrains.jdk.overrideAttrs (oldAttrs: rec {
                    version = "520.11";
                    src = super.fetchurl {
                        url = "https://bintray.com/jetbrains/intellij-jbr/download_file?file_path=jbrsdk-11_0_4-osx-x64-b520.11.tar.gz";
                        sha256 = "0d1qwbssc8ih62rlfxxxcn8i65cjgycdfy1dc1b902j46dqjkq9z";
                    };
                    just-to-retain-a-link-between-garbage-collection-to-patched-nixpkgs = danbstFixedNixpkgs;
                });
            };
          })
      ];
  };
in pkgs.jetbrains.idea-ultimate

PR WIP jdk: use `placeholder` for jdk.home by danbst · Pull Request #76571 · NixOS/nixpkgs · GitHub

Seems like that PR is wrong. I also found a way to override without nixpkgs patching:

let
  pkgs = import ./nixpkgs {
      config.allowUnfree = true;
      overlays = [
          (self: super: {
            jetbrains = super.jetbrains // {
                idea-ultimate = super.jetbrains.idea-ultimate.overrideAttrs (_: {
                    name = "idea-ultimate-2019.2.8";
                    src = super.fetchurl {
                        url = "https://download.jetbrains.com/idea/ideaIU-2018.2.8-no-jdk.tar.gz";
                        sha256 = "0lah6sb5yaq84c68ak07hsn8k3cjrq4phl4kyiydbhwi09qsc339";
                    };
                });
                jdk = super.jetbrains.jdk.overrideAttrs (oldAttrs: rec {
                    version = "520.11";
                    src = super.fetchurl {
                        url = "https://bintray.com/jetbrains/intellij-jbr/download_file?file_path=jbrsdk-11_0_4-linux-x64-b520.11.tar.gz";
                        sha256 = "0d1qwbssc8ih62rlfxxxcn8i65cjgycdfy1dc1b902j46dqjkq9a";
                    };
                    passthru = oldAttrs.passthru // {
                        home = self.jetbrains.jdk;
                    };
                });
            };
          })
      ];
  };
in pkgs.jetbrains.idea-ultimate

The important part was to override also passthru.home, which had a link to old jdk.

1 Like

Thanks so much @danbst!

I assume import ./nixpkgs can be replaced happily with import <nixpkgs> or fetchTarball etc as previously suggested.

Being new to nix, I’d be interested to know how you went about debugging this.

@danbst nearly there. Thanks so much for your help!

So I’ve taken what you had into an overlay ~/.config/nixpkgs/overlays/02-jetbrains.nix

self: super:
{
  jetbrains = super.jetbrains // {
    jdk = super.jetbrains.jdk.overrideAttrs (oldAttrs: rec {
      version = "520.11";
      src = super.fetchurl {
        url = "https://bintray.com/jetbrains/intellij-jbr/download_file?file_path=jbrsdk-11_0_4-osx-x64-b520.11.tar.gz";
  	sha256 = "3fe1297133440a9056602d78d7987f9215139165bd7747b3303022a6f5e23834";
      };
      passthru = oldAttrs.passthru // {
        home = self.jetbrains.jdk;
      };
    });
    idea-ultimate = super.jetbrains.idea-ultimate.overrideAttrs (_: {
      name = "idea-ultimate.2019.2.4";
      src = super.fetchurl {
        url = "https://download.jetbrains.com/idea/ideaIU-2019.2.4-no-jbr.tar.gz";
	sha256 = "09mz4dx3zbnqw0vh4iqr8sn2s8mvgr7zvn4k7kqivsiv8f79g90a";
      };
    });
  };
}

This installs happily via nix-env -iA 'nixpkgs.jetbrains.idea-ultimate’.

But when executing it complains it doesn’t know about the JDK_HOME.

ldeck@ldeck ~ % which idea-ultimate
/Users/ldeck/.nix-profile/bin/idea-ultimate

ldeck@ldeck ~ % idea-ultimate
Unable to find any JVMs matching version "(null)".
No Java runtime present, try --request to install.
ERROR: Cannot start IntelliJ IDEA
No JDK found. Please validate either IDEA_JDK, JDK_HOME or JAVA_HOME environment variable points to valid JDK installation.

It looks like the JDK_HOME is meant to be set during the install phase from applications/editors/jetbrains/common.nix.

Any thoughts on what might be missing? Thanks again!

Hmm, the executable does have a JDK_HOME…

ldeck@ldeck ~ % cat $(which idea-ultimate)    
#! /nix/store/m7g5jss6wkwx7b8m2x7qg3wdb4x0whw6-bash-4.4-p23/bin/bash -e
export PATH='/nix/store/qz2bg8ga3qnf3pnyiwrbzmpp5h8zsi6h-idea-ultimate.2019.2.4/libexec/idea-ultimate-2019.2.4:/nix/store/bsylyydil2yvxmfr96dvv8xg5h590x2k-jetbrainsjdk-520.11/jdk/Contents/Home/bin:/nix/store/bsylyydil2yvxmfr96dvv8xg5h590x2k-jetbrainsjdk-520.11/bin:/nix/store/2ck9rfwnivsj5lyzr35v3gf9854xm33a-coreutils-8.31/bin:/nix/store/s2ixkwlyqq2h3l6rbrapf2xfxkly7dsc-gnugrep-3.3/bin:/nix/store/jkh8498r42qx0kr61qb7n43lxa694y5y-which-2.21/bin:/nix/store/3fing5pkdda7l7d4h16yq07siqikdjmd-git-2.24.1/bin'${PATH:+':'}$PATH
export LD_LIBRARY_PATH='/nix/store/jvzc6v8sjwpws3kdq01m0im582kbiv47-clang-7.1.0-lib/lib:/nix/store/jd3vmjkfnf38ls52sxvrkv6flcpb3ppl-libsecret-0.19.1/lib:/nix/store/hp90sbwznq1msv327f0lb27imvcvi80h-libnotify-0.7.8/lib'${LD_LIBRARY_PATH:+':'}$LD_LIBRARY_PATH
export JDK_HOME='/nix/store/bsylyydil2yvxmfr96dvv8xg5h590x2k-jetbrainsjdk-520.11'
export IDEA_JDK='/nix/store/bsylyydil2yvxmfr96dvv8xg5h590x2k-jetbrainsjdk-520.11'
export ANDROID_JAVA_HOME='/nix/store/bsylyydil2yvxmfr96dvv8xg5h590x2k-jetbrainsjdk-520.11'
export JAVA_HOME='/nix/store/bsylyydil2yvxmfr96dvv8xg5h590x2k-jetbrainsjdk-520.11'
exec "/nix/store/qz2bg8ga3qnf3pnyiwrbzmpp5h8zsi6h-idea-ultimate.2019.2.4/idea-ultimate.2019.2.4/bin/idea.sh"  "$@"

So I wonder why it thinks it’s not set?

The path to JAVA_HOME, JDK_HOME etc is wrongly set.

It needs the suffix path /Contents/Home.

...
export JDK_HOME='/nix/store/bsylyydil2yvxmfr96dvv8xg5h590x2k-jetbrainsjdk-520.11/Contents/Home’
...

Do you require this on linux as well @danbst?

@danbst I wonder if this is the reason for home being set incorrectly? Yet when I attempt home = self.jetbrains.jdk.home I get infinite recursion.

Okay, setting the passthru.home appending /Contents/Home works. Dunno if there’s a more elegant way.

So the overlay that works for me, thanks to @danbst, is as follows:

self: super:
{
  jetbrains = super.jetbrains // {
    jdk = super.jetbrains.jdk.overrideAttrs (oldAttrs: rec {
      version = "520.11";
      src = super.fetchurl {
        url = "https://bintray.com/jetbrains/intellij-jbr/download_file?file_path=jbrsdk-11_0_4-osx-x64-b520.11.tar.gz";
  	sha256 = "3fe1297133440a9056602d78d7987f9215139165bd7747b3303022a6f5e23834";
      };
      passthru = oldAttrs.passthru // {
        home = "${self.jetbrains.jdk}/Contents/Home";
      };
    });
    idea-ultimate = super.jetbrains.idea-ultimate.overrideAttrs (_: {
      name = "idea-ultimate.2019.2.4";
      src = super.fetchurl {
        url = "https://download.jetbrains.com/idea/ideaIU-2019.2.4-no-jbr.tar.gz";
	sha256 = "09mz4dx3zbnqw0vh4iqr8sn2s8mvgr7zvn4k7kqivsiv8f79g90a";
      };
    });
  };
}
1 Like

huh, this must be some Mac specific stuff. I see that there was a change done recently to support Mac and IDEA: jetbrains.idea-community: add darwin support by uri-canva · Pull Request #64467 · NixOS/nixpkgs · GitHub

Maybe it is not complete? If you have time, you can clone nixpkgs, update IDEA sources and try install it from nixpkgs source. If it doesn’t work, then yes, some more Mac specific ifs must be added.

1 Like