GPT4ALL Nix derivation

[rafaelcgs10]

I’m trying to create a derivation for chat GPT4ALL: GitHub - nomic-ai/gpt4all-chat: gpt4all-j chat

Here is what I have so far:

{ lib, stdenv, fetchFromGitHub, cmake, vulkan-headers, qt6 }:

let
  rev = "d3d8229b047939e442db797e1d1de65cdd45f462";

  src = fetchFromGitHub {
    owner = "nomic-ai";
    repo = "gpt4all-chat";
    rev = rev;
    sha256 = "sha256-7qhlFQ1IvqnoOKCtMyTz1Z4zD4RDm1259+hflZ29ZmU=";
    fetchSubmodules = true;
  };
in
stdenv.mkDerivation {
  name = "gpt4all-chat-${rev}";
  src = src;

  dontWrapQtApps = true;
  nativeBuildInputs = [ cmake qt6.qmake ];
  buildInputs = [ qt6.qtquicktimeline qt6.qtsvg ];

  buildPhase = ''
    mkdir -p build
    cd build
    cmake ..
    cmake --build .
  '';

  installPhase = ''
    mkdir -p $out/bin/
    cp ./chat $out/bin/chat
  '';
}

The build is falling with the error message: Error: could not load cache.

Once again, it is really hard to understand what exactly went wrong.

This a package that uses cmake, qt, and qmake.
Unfortunately, I don’t have any experience with that kind of build in Nix.

I would appreciate any help with this.

[Kranzes]

{ lib
, stdenv
, fetchFromGitHub
, cmake
, qmake
, qtquicktimeline
, qtsvg
}:

stdenv.mkDerivation {
  pname = "gpt4all-chat";
  version = "unstable-2023-04-30";

  src = fetchFromGitHub {
    owner = "nomic-ai";
    repo = "gpt4all-chat";
    rev = "d3d8229b047939e442db797e1d1de65cdd45f462";
    hash = "sha256-7qhlFQ1IvqnoOKCtMyTz1Z4zD4RDm1259+hflZ29ZmU=";
    fetchSubmodules = true;
  };


  postPatch = ''
    substituteInPlace CMakeLists.txt \
      --replace 'set(CMAKE_INSTALL_PREFIX ''${CMAKE_BINARY_DIR}/install)' ""
  '';

  nativeBuildInputs = [
    cmake
    qmake
  ];

  buildInputs = [
    qtquicktimeline
    qtsvg
  ];

  dontWrapQtApps = true;

  meta = with lib; {
    description = "Gpt4all-j chat";
    homepage = "https://github.com/nomic-ai/gpt4all-chat";
    license = licenses.mit;
    maintainers = with maintainers; [ ];
  };
}

This builds fine. You can build it by running:
nix build --impure --expr "(import <nixpkgs> { }).qt6Packages.callPackage ./. { }" -L

[rafaelcgs10]

Yes, this makes it build correctly.

First note:
One needs to put the ggml model in ~/.local/share/nomic.ai/GPT4All/

Now, the program still does not start because of the following error messages:

qt.core.plugin.loader: In /nix/store/g2cvi1xiawh7fn3s0zm3jjvk2h7ld17x-qtwayland-5.15.7-bin/lib/qt-5.15.7/plugins/platforms/libqwayland-egl.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /nix/store/g2cvi1xiawh7fn3s0zm3jjvk2h7ld17x-qtwayland-5.15.7-bin/lib/qt-5.15.7/plugins/platforms/libqwayland-generic.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /nix/store/g2cvi1xiawh7fn3s0zm3jjvk2h7ld17x-qtwayland-5.15.7-bin/lib/qt-5.15.7/plugins/platforms/libqwayland-xcomposite-egl.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /nix/store/g2cvi1xiawh7fn3s0zm3jjvk2h7ld17x-qtwayland-5.15.7-bin/lib/qt-5.15.7/plugins/platforms/libqwayland-xcomposite-glx.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /nix/store/i7k4plcqhpjv8ddqr66svmwq38ldxyh1-qtbase-5.15.7-bin/lib/qt-5.15.7/plugins/platforms/libqeglfs.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /nix/store/i7k4plcqhpjv8ddqr66svmwq38ldxyh1-qtbase-5.15.7-bin/lib/qt-5.15.7/plugins/platforms/libqlinuxfb.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /nix/store/i7k4plcqhpjv8ddqr66svmwq38ldxyh1-qtbase-5.15.7-bin/lib/qt-5.15.7/plugins/platforms/libqminimal.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /nix/store/i7k4plcqhpjv8ddqr66svmwq38ldxyh1-qtbase-5.15.7-bin/lib/qt-5.15.7/plugins/platforms/libqminimalegl.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /nix/store/i7k4plcqhpjv8ddqr66svmwq38ldxyh1-qtbase-5.15.7-bin/lib/qt-5.15.7/plugins/platforms/libqoffscreen.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /nix/store/i7k4plcqhpjv8ddqr66svmwq38ldxyh1-qtbase-5.15.7-bin/lib/qt-5.15.7/plugins/platforms/libqvnc.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /nix/store/i7k4plcqhpjv8ddqr66svmwq38ldxyh1-qtbase-5.15.7-bin/lib/qt-5.15.7/plugins/platforms/libqxcb.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.qpa.plugin: Could not find the Qt platform plugin "wayland" in ""
qt.core.plugin.loader: In /nix/store/i7k4plcqhpjv8ddqr66svmwq38ldxyh1-qtbase-5.15.7-bin/lib/qt-5.15.7/plugins/platformthemes/libqgtk3.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /nix/store/i7k4plcqhpjv8ddqr66svmwq38ldxyh1-qtbase-5.15.7-bin/lib/qt-5.15.7/plugins/platformthemes/libqxdgdesktopportal.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /nix/store/m9f87s8r152hia95h1lamhjrb2pd93g3-qtvirtualkeyboard-5.15.7/lib/qt-5.15.7/plugins/platforminputcontexts/libqtvirtualkeyboardplugin.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /nix/store/i7k4plcqhpjv8ddqr66svmwq38ldxyh1-qtbase-5.15.7-bin/lib/qt-5.15.7/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
qt.core.plugin.loader: In /nix/store/i7k4plcqhpjv8ddqr66svmwq38ldxyh1-qtbase-5.15.7-bin/lib/qt-5.15.7/plugins/platforminputcontexts/libibusplatforminputcontextplugin.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
gptj_model_load: loading model from '/home/rafael/.local/share/nomic.ai/GPT4All//ggml-gpt4all-j.bin' - please wait ...
gptj_model_load: n_vocab = 50400
gptj_model_load: n_ctx   = 2048
gptj_model_load: n_embd  = 4096
gptj_model_load: n_head  = 16
gptj_model_load: n_layer = 28
gptj_model_load: n_rot   = 64
gptj_model_load: f16     = 2
qt.core.plugin.loader: In /nix/store/7vcv31hypl3kbqp7fzlsspjigqcgnbs5-qtquickcontrols-5.15.7/lib/qt-5.15.7/qml/QtQuick/Controls/libqtquickcontrolsplugin.so:
  Plugin uses incompatible Qt library (5.15.0) [release]
QQmlApplicationEngine failed to load component
qrc:/gpt4all/main.qml:3:1: The plugin '/nix/store/7vcv31hypl3kbqp7fzlsspjigqcgnbs5-qtquickcontrols-5.15.7/lib/qt-5.15.7/qml/QtQuick/Controls/libqtquickcontrolsplugin.so' uses incompatible Qt library. (5.15.0) [release]

It seems to be related to the bug fixed in the PR:

nixos/qt: set QT_PLUGIN_PATH and QML2_IMPORT_PATH when enabled

But upgrading the channel didn’t solve the error here.

I’ll keep working on this problem.

[lilyinstarlight]

Oh, ugh, so that just breaks qt6 qml apps?

I guess I’ll try to come up with some more universal solution in the next few days for separating plugins by qt5 and qt6 (since right now they use the same env var)

As a workaround, if you disable your Qt platform theme (such that it doesn’t need to dynamically load a plugin), then it will hopefully work

[rafaelcgs10]

How can I disable that?

I’m in a NixOS, home-manager, KDE-plasma setup.

[NickCao]

There are some issues with the derivation, namely you should add wrapQtAppsHook to nativeBuildInputs instead of setting dontWrapQtApps, and qmake is not needed.

[polygon]

I turned this into a Nix flake, in case anyone is interested:

GitHub - polygon/gpt4all-nix: Nix flake for gpt4all

[rafaelcgs10]

I don’t have the same issue with QT with your flake. Why?

[Kranzes]

Because it uses wrapQtAppsHook.

[Isomorph70]

I tried to run it, but got an error.

$ nix --extra-experimental-features nix-command --extra-experimental-features flakes run github:polygon/gpt4all-nix
deserializing chats took: 0 ms
gptj_model_load: loading model from '/home/shs/.local/share/nomic.ai/GPT4All//ggml-gpt4all-j.bin' - please wait ...
gptj_model_load: n_vocab = 50400
gptj_model_load: n_ctx   = 2048
gptj_model_load: n_embd  = 4096
gptj_model_load: n_head  = 16
gptj_model_load: n_layer = 28
gptj_model_load: n_rot   = 64
gptj_model_load: f16     = 2
gptj_model_load: ggml ctx size = 5401,45 MB
Illegal instruction (core dumped)

[polygon]

Does your processor support AVX2? I think the normal build options require a processor that has this feature. I just added another derivation that enables the AVX compatibility mode which should work with older CPUs. Can you try running

nix --extra-experimental-features nix-command --extra-experimental-features flakes run github:polygon/gpt4all-nix#gpt4all-chat-avx

and see if this fixes things for you?

[Isomorph70]

I just checked my cpu with lscpu yesterday. Yes, my cpu is older, so only avx support.

Okey my computer is installing now.
Some probems…

[Isomorph70]

$ nix --extra-experimental-features nix-command --extra-experimental-features flakes run github:polygon/gpt4all-nix#gpt4all-chat-avx
deserializing chats took: 0 ms
gptj_model_load: loading model from '/home/shs/.local/share/nomic.ai/GPT4All//ggml-gpt4all-j.bin' - please wait ...
gptj_model_load: n_vocab = 50400
gptj_model_load: n_ctx   = 2048
gptj_model_load: n_embd  = 4096
gptj_model_load: n_head  = 16
gptj_model_load: n_layer = 28
gptj_model_load: n_rot   = 64
gptj_model_load: f16     = 2
gptj_model_load: ggml ctx size = 5401,45 MB
gptj_model_load: kv self size  =  896,00 MB
gptj_model_load: ................................... done
gptj_model_load: model size =  3609,38 MB / num tensors = 285
Couldn't find tool at "/nix/store/n0igx369akkvbgkgfi9s1npw1mjwzw7z-gpt4all-chat-nightly/bin/../maintenancetool" so cannot check for updates!
Couldn't find tool at "/nix/store/n0igx369akkvbgkgfi9s1npw1mjwzw7z-gpt4all-chat-nightly/bin/../maintenancetool" so cannot check for updates!
Couldn't find tool at "/nix/store/n0igx369akkvbgkgfi9s1npw1mjwzw7z-gpt4all-chat-nightly/bin/../maintenancetool" so cannot check for updates!

[Isomorph70]

Okey, most of it works. except the update system.

This privateGPT also looks interesting.

[3cola]

I got rid of theses messages by doing

unset QT_PLUGIN_PATH

before running gpt4all

[nomb]

Would it be possible to get a package made for this?