Jetbrains Gateway client support on Nixos

Learn
#1

I am wondering if anyone has had any success packaging the Jetbrains Gateway client. I found this thread about running the Gateway Backend: Cannot use nixos as jetbrains gateway backend which is not quite what I want to do since I have a preexisting backend hosted on gitpod.io and I want to be able to connect to it.

I played around with modifying the files in nixpkgs/pkgs/applications/editors/jetbrains at nixos-unstable · NixOS/nixpkgs · GitHub but I think Gateway must use a different release model because update.pl seemed to get confused.

Thank you in advance if anyone has any ideas!

#2

I think packaging the gateway app itself would be quite easy–after looking at the launcher shell script in the Gateway download, I was able to get it running simply by setting the JDK to the already NixOS-packaged JetBrains variant in my nix-store:

GATEWAY_JDK=/nix/store/avjxcd46hmxfyisalg47hjqa8mrlzmzf-jetbrains-jdk-11.0.7-b64/lib/openjdk ./JetBrainsGateway-213.6777.25/bin/gateway.sh

Specifically for gitpod support, I also needed to have a desktop launcher defined with the same env var for the app, since the gitpod website needs to link back into the gateway app using the jetbrains-gateway url scheme. In ~/.local/share/applications/jetbrains-gateway.desktop:

[Desktop Entry]
Exec=/usr/bin/env GATEWAY_JDK=/nix/store/avjxcd46hmxfyisalg47hjqa8mrlzmzf-jetbrains-jdk-11.0.7-b64/lib/openjdk /path/to/JetBrainsGateway-213.6777.25/bin/gateway.sh %u
Version=1.0
Type=Application
Categories=Development
Name=JetBrains Gateway
StartupWMClass=jetbrains-ide
Terminal=false
MimeType=x-scheme-handler/jetbrains-gateway;

Unfortunately, launching the actual IDE is where it all fell apart. The Gateway downloads and manages the Jetbrains Client itself, which is not patched for NixOS, and therefore fails to launch. I haven’t yet looked further to see if the the Client’s Java location is the only problem, and/or if it is easily overriden.

In the meantime, I was able to hack a working setup by changing the desktop file use a bubble-wrapped env:

Exec=steam-run /path/to/JetBrainsGateway-213.6777.25/bin/gateway.sh %u
#3

Thanks for the help! I’ll open a ticket upstream with jetbrains about being able to use all ready downloaded ides with gateway. I’ll link it here once it’s opened.

2 Likes
#4

I started to “package” (tweak) it on my side, but I get a critical error when starting the gateway. I put the code there if someone wants to take a look or have a base to work on. :kissing_smiling_eyes:

{ lib
, stdenv
, autoPatchelfHook
, makeWrapper
, jetbrains
, zlib
, glib
, libX11
, libXi
, libXrender
, freetype
, alsa-lib
, libXtst
}:

stdenv.mkDerivation rec {
  pname = "jetbrains-gateway";
  version = "222.2270.16";

  src = builtins.fetchurl {
    url = "https://download.jetbrains.com/idea/gateway/JetBrainsGateway-222.2270.16.tar.gz";
  };

  nativeBuildInputs = [ autoPatchelfHook makeWrapper ];
  buildInputs = [ 
    jetbrains.jdk
    zlib
    glib
    libX11
    libXi
    libXrender
    freetype
    alsa-lib
    libXtst
  ];

  installPhase = ''
    mkdir -p $out/share
    cp -r . $out/share

    makeWrapper \
      $out/share/bin/gateway.sh \
      $out/bin/jetbrains-gateway \
      --prefix LD_LIBRARY_PATH : $out/lib \
      --set GATEWAY_JDK "${jetbrains.jdk}/lib/openjdk" \
      --set JAVA_HOME "${jetbrains.jdk.home}" \
      --set JDK_HOME "${jetbrains.jdk.home}"
  '';
}
#5

What’s the error out of curiosity?

#6

Sorry for the delay. I get a Java error message in window

nternal error. Please refer to https://jb.gg/ide/critical-startup-errors

com.intellij.ide.plugins.StartupAbortedException: Cannot start app
    at com.intellij.idea.StartupUtil.lambda$start$15(StartupUtil.java:265)
    at java.base/java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:986)
    at java.base/java.util.concurrent.CompletableFuture.uniExceptionallyStage(CompletableFuture.java:1004)
    at java.base/java.util.concurrent.CompletableFuture.exceptionally(CompletableFuture.java:2307)
    at com.intellij.idea.StartupUtil.start(StartupUtil.java:264)
    at com.intellij.idea.Main.bootstrap(Main.java:118)
    at com.intellij.idea.Main.main(Main.java:79)
Caused by: java.lang.NoClassDefFoundError: java/lang/constant/Constable
    at com.intellij.idea.MainImpl.<clinit>(MainImpl.java)
    at java.base/jdk.internal.misc.Unsafe.allocateInstance(Native Method)
    at java.base/java.lang.invoke.DirectMethodHandle.allocateInstance(DirectMethodHandle.java:482)
    at com.intellij.idea.StartupUtil.lambda$start$2(StartupUtil.java:131)
    at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
    at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1692)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: java.lang.ClassNotFoundException: java.lang.constant.Constable
    at com.intellij.util.lang.UrlClassLoader.findClass(UrlClassLoader.java:209)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
    ... 11 more

-----
Your JRE: 11.0.13+0-adhoc..source amd64 (Oracle Corporation)
/nix/store/29aircgjj7nd7qdjwzqqvq2h4rza8jkq-jetbrains-jdk-11_0_13-b1751.25/lib/openjdk
#7

@Nover, I think I may have led you in the wrong direction with my solution above relying on the nix jetbrains-jdk.

The error you’re seeing is because the nix jetbrains-jdk is Java 11 (as required by the other Jetbrains IDEs), but the app is trying to load a class that was only introduced in Java 12.

Gateway development is moving at a fast pace, so I would guess they changed the target JDK version between the releases we tried (213.6777.2 vs 222.2270.16).

There are two approaches to fix this:

  1. Reuse the custom JDK they ship in the Gateway archive, which would always be a compatible version. This can be found inside the archive in the ./jbr directory.
  2. Rely on the default Nix JDK, which is currently at Java 17 (for nix unstable), the same version that is found in the Gateway archive version you were trying to package.

Although #1 sounds like the most correct way to package, I think #2 would actually be fine—looking at the ./bin/gateway.sh script, it seems to think your system JVM is an acceptable option. A comment summarizes the search logic:

# Locate a JRE installation directory command -v will be used to run the IDE.
# Try (in order): $GATEWAY_JDK, .../gateway.jdk, .../jbr, $JDK_HOME, $JAVA_HOME, "java" in $PATH.
#8

Just a note, in case you’re sinking a bunch of effort into this and haven’t had a chance to try the remote dev features at all yet:

I didn’t go much further with this myself because after I got it all working I found the development experience to be sub-par. There is a new logical split between local IDE and remote server/host IDE. None of my settings, plugins, themes, or other IDE customizations synced into this new split environment properly, and even after trying to manually copy everything in, it seems many plugins just don’t work yet in this new split setup. Finally, I was hoping to use this with ephemeral remote systems (GitPod and Github Codespaces), which meant that the host side of all that manual setup would be lost the minute switched projects.

I presume that will all be fixed in time (Gateway/CodeWithMe are still in EAP), so it’s not wasted effort to figure out how to get this all setup and/or packaged now; I just wanted to let you down softly in case y’all got your hopes up like I did that it’d be seamless! :smile:

#9

Hi!

I tried using the jdk package rather than jetbrains.jdk and I can successfully get the main window. I cannot go further, I get the following warning: WARN - #c.i.r.d.CodeWithMeClientDownloader - Running client process failed after specified number of attempts
I’m definitely not skill enough to understand what is the problem for this package.

I never tried the remote dev thing, I wanted to try for the first time but if you say that it is not fully functionnal and has a lot of drawback, I may temper my expectations :stuck_out_tongue: Thanks for the heads up

#10

@jeffwilde I tried the thing on my Windows, and it is actually exciting. There are still some drawbacks that impact the experience but I’m sure in the near future, coupling this with NixOS could really be something huge.

For more information, I want to use it to develop websites with some Docker stuff. Installing Traefik on the server, making it accessible from a VPN such as Tailscale, and binding the DNS to this IP address and voilà! I’m pretty sure that everything can be flawless :smiley:

#11

@Nover, definitely the concept of remote development is killer, I use VSCode remote almost every day. Jetbrains’ brand of remote in its current state is still a bit flaky, but if you don’t need the flaky bits in your everyday use, I’m sure it’s just as killer; my setup relies on the bits that are still flaky, unfortunately.

Regarding your packaging effort:
As you discovered, because the Gateway downloads the client on-demand, it is of course not patched for nix, and fails to launch.

If you take a look where the client is actually downloaded and run from, it’s in ~/.cache/JetBrains/JetBrainsClientDist/JetBrainsClient-<version>-ide/JetBrainsClient-<version>/ (unless you have a custom XDG_CACHE_HOME).

In that directory, there is another collection of shell scripts (i.e. bin/intellij_client.sh, bin/jetbrains_client.sh), as well as another bundled JDK (in jbr/). If you look at those scripts, they by default use the bundled JDK, but they also respect an environment variable JETBRAINSCLIENT_JDK.

The bundled JDK is the jetbrains-patched JDK that’s already packaged in nix as jetbrains-jdk.

I think if you set the environment variable in your derivation makeWrapper before executing the Gateway itself, the downloaded Client will respect the variable and everything will just kinda-mostly :wink: work:

--set JETBRAINSCLIENT_JDK="${jetbrains.jdk.home}"
#12

You’re a genius! It works as expected, which means I can now do some crazy stuff. The final package definition is the following:

{ lib
, stdenv
, autoPatchelfHook
, makeWrapper
, jetbrains
, zlib
, glib
, libX11
, libXi
, libXrender
, freetype
, alsa-lib
, libXtst
, jdk
}:

stdenv.mkDerivation rec {
  pname = "jetbrains-gateway";
  version = "222.2270.16";

  src = builtins.fetchurl {
    url = "https://download.jetbrains.com/idea/gateway/JetBrainsGateway-222.2270.16.tar.gz";
  };

  nativeBuildInputs = [ autoPatchelfHook makeWrapper ];
  buildInputs = [
    jetbrains.jdk
    zlib
    glib
    libX11
    libXi
    libXrender
    freetype
    alsa-lib
    libXtst
  ];

  installPhase = ''
    mkdir -p $out/share
    cp -r . $out/share
    rm -r $out/share/jbr

    makeWrapper \
      $out/share/bin/gateway.sh \
      $out/bin/jetbrains-gateway \
      --prefix LD_LIBRARY_PATH : $out/lib \
      --set GATEWAY_JDK "${jdk}" \
      --set JETBRAINSCLIENT_JDK "${jetbrains.jdk.home}" 
  '';
}

I’m curious: what “still flaky” features do you use that are not available with the remote environment proposed by Jetbrains?

#13

Awesome, glad it is working for you now!

The flaky parts for me are just that I couldn’t get my customized settings to sync properly between environments (so there were settings I would have to redo for every new remote), and then I found many of the plugins I rely on in IntelliJ just didn’t work at all or were not available for installation in the plugin marketplace. I’m sure both of those problems will get fixed in time, but are blockers for my regular workflow right now.

If you find that your derivation is working smoothly, we should probably get it submitted to nixpkgs.

FYI, @DAlperin, if you’re not watching this thread closely, I think there is a working config you could use now!

#14

My goal is to reuse my current computer as a developement server that runs Docker and the remote dev, and use a Macbook to actually code. That will let me work on my desk and keep my environment working if I have to move somewhere. So one server only. I use PHPStorm, so I don’t use that many plugins.

#15

@jeffwilde thanks for the ping! Awesome to see this working. @Nover are you planning on putting this in nixpkgs? I was just chatting with a friend working in this space the other day so I’m excited to try it out/make it better for everyone on Nixos. If you want I can open a PR for it later this week if that’s easier (with full credit of course).

Thanks all for all your debugging work this is super neat.

Edit: oops didn’t see @jeffwilde already talked about getting it into nixpkgs.

Can’t wait to try it out later when I’m back at my machine.

#16

@Nover, @DAlperin, if we package this up, we should make sure it interoperates with the other Jetbrains products already packaged (IDEA, Rider, phcharm, etc). Most (if not all) are able to launch a remote coding session and/or host a code-with-me session.

I did a quick test with a few of the IDEs, and it seems like they all either use the shell scripts to launch the remote sessions (and therefore respect the variables), or they take the vars into account in the java code directly.

I also think using --set-default instead of --set would be preferred so that the user can still override if they like:

     --set-default GATEWAY_JDK "${jdk}" \
      --set-default JETBRAINSCLIENT_JDK "${jetbrains.jdk.home}"

I’m happy for any of us to open the PR, we just need to decide whom.

#17

I’ll be not around for weeks so if somebody wants to open the PR, be my guest :smiley:

Hosted by Flying Circus.