How to build a golang program which needs system libs

I am developing go programs and as such I need to compile them a lot. I don’t want to create nixpkg, I just want to build my go program, either for myself or for putting them into github releases.

So, with a game using ebitengine I am getting:

# github.com/hajimehoshi/ebiten/v2/internal/cglfw
In file included from ../../go/pkg/mod/github.com/hajimehoshi/ebiten/v2@v2.6.6/internal/cglfw/native_linbsd.go:14:
./glfw3native.h:103:12: fatal error: X11/Xlib.h: No such file or directory
  103 |   #include <X11/Xlib.h>
      |            ^~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:17: build] Error 1

Of course the file exists on the system (in /nix/store/vq8ndyjkli1xjmc7sf38kwqrsjmxhin4-libX11-1.8.9-dev/include/X11/Xlib.h) but this is certainly not a location where a compiler would look for it.

How do I do this?

1 Like

I tried to use nix-shell but this doesn’t work either. I found a couple of libs which are required, but not all. E.g.:

% nix-shell -p xorg.libX11 xorg.libXrandr libGL xorg.libXcursor

% make
rm -f openquell
go build -ldflags "-X 'openquell/config.VERSION=-master-933b9c40-2024.05.19.201844 '"
# github.com/hajimehoshi/ebiten/v2/internal/cglfw
In file included from internal.h:96,
                 from context.c:8:
x11_platform_linbsd.h:25:10: fatal error: X11/extensions/Xinerama.h: No such file or directory
   25 | #include <X11/extensions/Xinerama.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:17: build] Error 1

I could not find a package providing X11/extensions/Xinerama.h (at least not on https://mynixos.com/)

Now what? This would be really important to me. If I cannot build simple go programs in an easy way, I can’t switch to NixOS, which would be a pitty.

Ok, I got it to compile with nix-shell -p xorg.libX11 xorg.libXrandr libGL xorg.libXcursor xorg.libXinerama xorg.libXi xorg.libXxf86vm, but now I can’t run the game:

2024-05-19T08:26.58 CEST INFO: unable to run game: gl: failed to load libGL.so and libGLESv2.so

I’m getting this error both inside the nix-shell and outside.

I also set hardware.opengl.enable = true; but it didn’t help either, same error message.

Ok, I need to install libglvnd and set LD_LIBRARY_PATH manually. Really odd :frowning:

I don’t understand it. There are 4 copies of the library in /nix:

fd libGL.so.1 /nix -ls | grep libglvnd-1.7.0/lib/libGL.so.1
lrwxrwxrwx 1 root root   14 Jan  1  1970 /nix/store/bxrzc0s05749vjb2gixyimli4n09v44q-libglvnd-1.7.0/lib/libGL.so.1 -> libGL.so.1.7.0
-r-xr-xr-x 1 root root 594K Jan  1  1970 /nix/store/bxrzc0s05749vjb2gixyimli4n09v44q-libglvnd-1.7.0/lib/libGL.so.1.7.0
lrwxrwxrwx 1 root root   14 Jan  1  1970 /nix/store/pf13sr9c49ka84sgaml3j6qf436dpb95-libglvnd-1.7.0/lib/libGL.so.1 -> libGL.so.1.7.0
-r-xr-xr-x 1 root root 724K Jan  1  1970 /nix/store/pf13sr9c49ka84sgaml3j6qf436dpb95-libglvnd-1.7.0/lib/libGL.so.1.7.0
lrwxrwxrwx 1 root root   14 Jan  1  1970 /nix/store/v5jx7nnd9zshi5sg63hdby66qrbfmdbn-libglvnd-1.7.0/lib/libGL.so.1 -> libGL.so.1.7.0
-r-xr-xr-x 1 root root 594K Jan  1  1970 /nix/store/v5jx7nnd9zshi5sg63hdby66qrbfmdbn-libglvnd-1.7.0/lib/libGL.so.1.7.0
lrwxrwxrwx 1 root root   14 Jan  1  1970 /nix/store/wmm60qwckax108ffr8x1fm2xkzmm0vmh-libglvnd-1.7.0/lib/libGL.so.1 -> libGL.so.1.7.0
-r-xr-xr-x 1 root root 724K Jan  1  1970 /nix/store/wmm60qwckax108ffr8x1fm2xkzmm0vmh-libglvnd-1.7.0/lib/libGL.so.1.7.0

which one is the one being used currently?

I can’t help you with the main problem, but I can answer this question, I think. In NixOS there is no “currently used library” as it would be the case in Debian for example. This is because each program can use a different version of the same library. Programs can “pick” whichever version of a library from the nix store “that they want”.

2 Likes

I found aaaaxy, an ebiten game, in nixpkgs. It seems fairly documented, so I hope it helps you in your packaging.

Wow, this is one comprehensive package. Although I don’t want to build a package (I just want to build and execute locally), I’ll study it anyway. Thanks for the hint.

1 Like

No, doesn’t really help. Also, I don’t want to build a package, I just want to locally compile the game and execute it from current directory for testing purposes.

Does have anyone an idea? I am surely not the only one developing go software on NixOS, am I?

best,
Tom

I believe this is specifically an issue with ebiten as the program compiled correctly, but still refuses to run, even when I tried with a nix-ld shell:

shell-ld.nix
with import <nixpkgs> { };
mkShell {
  NIX_LD_LIBRARY_PATH = lib.makeLibraryPath [
    fuse
    desktop-file-utils
    xorg.libXcomposite
    xorg.libXtst
    xorg.libXrandr
    xorg.libXext
    xorg.libX11
    xorg.libXfixes
    libGL

    gst_all_1.gstreamer
    gst_all_1.gst-plugins-ugly
    gst_all_1.gst-plugins-base
    libdrm
    xorg.xkeyboardconfig
    xorg.libpciaccess

    glib
    glibc
    gtk2
    bzip2
    zlib
    gdk-pixbuf

    xorg.libXinerama
    xorg.libXdamage
    xorg.libXcursor
    xorg.libXrender
    xorg.libXScrnSaver
    xorg.libXxf86vm
    xorg.libXi
    xorg.libSM
    xorg.libICE
    freetype
    curlWithGnuTls
    nspr
    nss
    fontconfig
    cairo
    pango
    expat
    dbus
    cups
    libcap
    SDL2
    libusb1
    udev
    dbus-glib
    atk
    at-spi2-atk
    libudev0-shim

    xorg.libXt
    xorg.libXmu
    xorg.libxcb
    xorg.xcbutil
    xorg.xcbutilwm
    xorg.xcbutilimage
    xorg.xcbutilkeysyms
    xorg.xcbutilrenderutil
    libGLU
    libuuid
    libogg
    libvorbis
    SDL
    SDL2_image
    glew110
    openssl
    libidn
    tbb
    wayland
    mesa
    libxkbcommon
    vulkan-loader

    flac
    freeglut
    libjpeg
    libpng12
    libpulseaudio
    libsamplerate
    libmikmod
    libtheora
    libtiff
    pixman
    speex
    SDL_image
    SDL_ttf
    SDL_mixer
    SDL2_ttf
    SDL2_mixer
    libappindicator-gtk2
    libcaca
    libcanberra
    libgcrypt
    libvpx
    librsvg
    xorg.libXft
    libvdpau
    alsa-lib

    harfbuzz
    e2fsprogs
    libgpg-error
    keyutils.lib
    libjack2
    fribidi
    p11-kit

    gmp

    libtool.lib
    xorg.libxshmfence
    at-spi2-core
    gtk3
    libglvnd
    stdenv.cc.cc.lib
  ];
  NIX_LD = lib.fileContents "${stdenv.cc}/nix-support/dynamic-linker";

  nativeBuildInputs = [
    libglvnd
    libGL
    go
    xorg.libX11
    xorg.libXcursor
    xorg.libXinerama
    xorg.libXrandr
    xorg.libXi
    alsa-lib
    pkg-config
    zip
    advancecomp
    xorg_sys_opengl
  ];

  buildInputs = [
    libglvnd
    libGL
  ];

  shellHook = with pkgs; ''
    export LD_LIBRARY_PATH=${pkgs.wayland}/lib:${lib.getLib libGL}/lib/libGL.so:${lib.getLib libGL}/lib/libGLESv2.so:$LD_LIBRARY_PATH
  '';
}
$ ./aaaaxy
2024/05/24 11:06:12.184249 [FATAL] RunGame exited abnormally: gl: failed to load libGL.so and libGLESv2.so

This issue has to do with how ebiten is looking for libGL, which is being worked around by the aaaaxy package as follows:

  shellHook = with pkgs;
    ''
      export EBITENGINE_LIBGL='${lib.getLib libGL}/lib/libGL.so'
      export EBITENGINE_LIBGLESv2='${lib.getLib libGL}/lib/libGLESv2.so'
      export LD_LIBRARY_PATH=${pkgs.wayland}/lib:${lib.getLib libGL}/lib/libGL.so:${lib.getLib libGL}/lib/libGLESv2.so:$LD_LIBRARY_PATH
    '';

  postBuild = ''
    substituteInPlace 'vendor/github.com/hajimehoshi/ebiten/v2/internal/graphicsdriver/opengl/gl/procaddr_others.go' \
      --replace \
      '{"libGL.so", "libGL.so.2", "libGL.so.1", "libGL.so.0"}' \
      '{os.Getenv("EBITENGINE_LIBGL")}' \
      --replace \
      '{"libGLESv2.so", "libGLESv2.so.2", "libGLESv2.so.1", "libGLESv2.so.0"}' \
      '{os.Getenv("EBITENGINE_LIBGLESv2")}'
  '';

In essence, if you can replace the libGL libraries ebiten is looking for with the EBITENGINE env variables and set them to point to the libGL libraries in the nix store, this would work.

Or you could make an issue upstream and ask if those variables could be instroduced and be checked before ebiten tries to looks for the libs.

1 Like

I don’t know about this specific issue regarding libGL but I wanted to share some of my development practices that I’ve come up with for doing development under NixOS since it came up.

Typically, when I develop software on NixOS, I do so in the context of a dev shell or direnv.

Here, for example, is a flake I can use to compile Go software that depends on GTK:

{
  inputs.flake-utils.url = "github:numtide/flake-utils";
  outputs =
    { nixpkgs, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (
      system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
      in
      {
        devShells.default = pkgs.mkShell {
          nativeBuildInputs = with pkgs; [
            go
            gtk4
          ];
        };
      }
    );
}

If I have this flake.nix file in a folder, I can call nix develop to enter a dev shell, or add an .envrc with the directive use flake to get the environment automatically added upon cd’ing into the directory of the project.

You can combine this with other tools, too. You can make a derivation locally that compiles your project via buildGoModule but then use mkShell with inputsFrom (and possibly other inputs for convenience: e.g. you can add specific Go dev tools not needed for actually building the software). I don’t have an example off-hand for Go, but I can point to a relatively small project where I just recently set up a Nix dev environment following this pattern, hopefully it may lend some help:

This way of doing development has its ups and downs. Your flake inputs get locked which means they need to manually be updated periodically, but OTOH without updating them your project should remain buildable indefinitely.

Tailscale also has a Nix flake and it has an interesting trick to handle the vendor hash: it is a separate file imported from the flake and they have their own package for calculating it irrespective of Nix which is ran via a shell script. This allows people to work on a Go project that has an official Nix flake without breaking the Nix flake by changing dependencies, keeping the vendor hash up to date, even if they don’t have Nix installed.

There are also other patterns I use when working on other people’s projects that don’t have a Nix flake:

  • For some projects, it is enough to just use a devShell of the upstream Nixpkgs derivation, if one exists. Something like nix develop nixpkgs\#dolphinEmuMaster.
  • If I want to create a custom flake and keep it in the dev directory, I can do that. I can keep Git from trying to version it by adding some files to the local gitignore at .git/info/exclude: usually, flake.nix, flake.lock, .direnv and .envrc. Then, I can write an .envrc that does a use flake path://$PWD, which disables the Git fetcher, allowing the unversioned flake files to be seen. I use this for contributing to open source upstreams. The direnv caching makes Nix store bloat less of a problem since it caches aggressively.

There’s probably more stuff here to say. If you are using an IDE or text editor that manages its own processes, like VSCode, environments like this might be somewhat challenging; I have some solutions for this but ultimately it’s not a perfect fit.

Hope this helps.

P.S.: The approaches I outlined all mention Nix flakes, but just to be sure, none of them depend on Nix flakes particularly. For example, it is fully possible to use .envrc without Nix flakes; see the use nix directive. Tailscale’s nardump should work fine with projects that are not using flakes, too. I do find flakes useful, though.

1 Like

@eljamm I already tried it with the environment variables, it doesn’t work:

[24.Mai 12:44:15] --- [~/dev/openquell] ---
scip@c9: % fd libGL.so.1 /nix | tail -1
/nix/store/pf13sr9c49ka84sgaml3j6qf436dpb95-libglvnd-1.7.0/lib/libGL.so.1.7.0

[24.Mai 12:44:27] --- [~/dev/openquell] ---
scip@c9: % EBITENGINE_LIBGL=/nix/store/pf13sr9c49ka84sgaml3j6qf436dpb95-libglvnd-1.7.0/lib EBITENGINE_LIBGLESv2=/nix/store/pf13sr9c49ka84sgaml3j6qf436dpb95-libglvnd-1.7.0/lib  ./openquell
[..]
2024-05-24T12:44.34 CEST INFO: unable to run game: gl: failed to load libGL.so and libGLESv2.so

I also tried it with LD_LIBRARY_PATH with the same result. I think I may open up an issue.

@jchw: thanks for the elaborate response! Yes, for building I already use a dev shell which works like a charm:

scip@c9: % cd dev/openquell/

[24.Mai 12:50:58] --- [~/dev/openquell] ---
scip@c9: % make nixshell 
nix-shell -p xorg.libX11 xorg.libXrandr libGL xorg.libXcursor xorg.libXinerama xorg.libXi xorg.libXxf86vm xorg_sys_opengl

[24.Mai 12:51:01] --- [~/dev/openquell] ---
scip@record: % make
make[1]: Entering directory '/home/scip/dev/openquell'
rm -f openquell
go build -ldflags "-X 'openquell/config.VERSION=-master-0ee848ba-2024.05.24.125102 '"
ok
make[1]: Leaving directory '/home/scip/dev/openquell'

Of course, putting it into a flake file would be a good idea. But I need to be able to actually execute the binary somehow :frowning:

best,
Tom

I tried to make a flake that can run Ebiten games. Hope this helps.

{
  inputs.flake-utils.url = "github:numtide/flake-utils";
  outputs =
    { nixpkgs, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (
      system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
      in
      {
        devShells.default = pkgs.mkShell {
          nativeBuildInputs = with pkgs; [
            go
            libGL
            xorg.libX11
            xorg.libXrandr
            xorg.libXcursor
            xorg.libXinerama
            xorg.libXi
            xorg.libXxf86vm
          ];

          shellHook = ''
            export LD_LIBRARY_PATH=${pkgs.wayland}/lib:${pkgs.lib.getLib pkgs.libGL}/lib:${pkgs.lib.getLib pkgs.libGL}/lib:$LD_LIBRARY_PATH
          '';
        };
      }
    );
}

With this I can do…

# if flake.nix is inside a Git repo, but not added:
nix develop path://$PWD

# otherwise, just:
nix develop

and then

go run ./examples/2048

from the Ebitengine directory.

(I’m not sure if the Wayland lib directory needs to be added, I only did this because the shellhook for the other Ebiten game seemed to have it. Stuff only runs under X11/XWayland for me either way.)

2 Likes

Holy cow! This is working:

Many many thanks, this is awesome :slight_smile:

Tom

2 Likes

Thanks a lot for this, it’s really useful :grin:

By the way, I thought nix-ld would make the NIX_LD_LIBRARY_PATH libraries available by default, do you have an idea why this didn’t work for libGL?

Doing the following worked in this case:

  shellHook = with pkgs; ''
    export LD_LIBRARY_PATH=$NIX_LD_LIBRARY_PATH
  '';

I believe the reason why has to do with how Nix-LD works. It creates a symlink at the usual glibc ELF linker path (on AMD64, normally this is /lib64/ld-linux-x86-64.so.2) so that ELF binaries compiled for non-Nix targets will see it (to be more specific, it is set in what is called the DT_INTERP field of the ELF binary.)

However, when you’re building within Nix, the ELF linker will automatically be set to the stdenv’s Glibc linker. You can see this yourself:

# Inside of the ebiten directory
$ go build -o 2048.bin ./examples/2048

$ patchelf --print-interpreter 2048.bin 
/nix/store/k7zgvzp2r31zkg9xqgjim7mbknryv6bs-glibc-2.39-52/lib/ld-linux-x86-64.so.2

This will not trigger Nix-LD, because that doesn’t point to the Nix-LD linker. The Nix-LD linker is special. In this way, the Nix-LD linker won’t impact any of your proper Nix-compiled binaries: at worst, it might make things work when they shouldn’t because a binary is not patched, but it won’t impact e.g. dlopen calls inside of Nix-compiled binaries.

If you compile a CGo binary outside the context of Nix, say, via Docker or Distrobox or the like, then running it inside NixOS should trigger Nix-LD as you expect.

P.S.: If you want, you can also use patchelf to forcibly trigger Nix-LD, e.g. by doing patchelf --set-interpreter /lib64/ld-linux-x86-64.so.2 2048.bin. This won’t make a binary exactly like it was built outside of NixOS (e.g. there may be rpath values set into the nix store) but it should trigger Nix-LD just fine.

1 Like

That definitely clears things up, thanks a lot!

On NixOS there isn’t much difference between the two cases.
The main difference is that one gets published afterwards and the other one stays on your computer.

1 Like