When I install a software more recent than my current system (like today I wanted to install Chromium using nix run nixpkgs#chromium), I often get an error like:
/nix/store/...lib/libc.so.6: version `GLIBC_2.33' not found (required by ...)
As far as I understand, this is because the available glibc is too old. But this kind of error is extremely frustrating as it defeats to me one of the major interest of nix: being able to run an executable irrespective of the environment it is running in. And I think I never experienced this outside of the nixos ecosystem (maybe people just package their software with old enough glibc).
So is there any project to be resilient against this error? Is this a fundamental problem, or is it possible to fix it? For instance, why can’t we automatically add the appropriate LD_LIBRARIES to also include the proper GLIBC version?
For now, my manual fix is to do:
LD_LIBRARY=/nix/store/2zchy9mdx3kk166i98cx5sriq2c1bdsn-glibc-2.33-55 nix run nixpkgs#chromium
Yes, glibc has great ABI compatibility, but only one way. Running with newer version than the compile-time one. So packages from a different version that get impurely linked-in (typically from /run/opengl-driver) then can cause issues. But glibc isn’t the only problem; we can also see e.g. undefined symbol: wl_ (from libwayland, I think) and there it might be harder.
Sorry for the long time I spent to answer, I was quite busy recently ^^’ Thanks for your answers!
So I’m not sure to understand, if LD_PRELOAD can solve this problem, why don’t we use LD_PRELOAD in front of all programs to be sure all programs use a recent enough glibc?
I certainly wouldn’t do such a thing “globally”. It would be way more significant departure from purity than what we have now. Say, what if the newest glibc had some bug and now your previously working stuff would break?
Wait, I think I’m missing something (my goal was to increase purity) : when I install a brand new program, the program loads libraries in /run/opengl that use an old glibc version, but the glibc used is the one of the program (the new one) I guess, so if glibc is backward compatible, why is it even crashing?
You might want to look into why you have glibc older than 2.33-x. The oldest supported nixpkgs branch is 21.11 (since the end of 2021), and that one always had glibc 2.33-x. The master branch (and “unstable” channel) will currently get updated to 2.34-x.
Say, I commonly see people mishandling their channels. Say, they confuse root’s channels with their user’s channels (because sudo, nixos-rebuild, etc.) or they have more channels mistakenly and only update some of them, …
Hum… I am not sure why (I’m running unstable a4bf443457), but it seems that most of my system is running GLIBC 33:
$ ldd --version
ldd (GNU libc) 2.33
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
Now, I’m actually very confused: when I run nix run nixpkgs#chromium, I don’t always have the same version of chromium that is running (and to be honest, I’m not even sure which channel is supposed to be targeted by this as I don’t have any channel named nixpkgs). Last time, I was having version 95, then I managed to get the latest (99), and today… I’m back on 87!?
Also, today, I do have an error, but it does not stop Chromium from starting:
$ nix run nixpkgs#chromium
/nix/store/1yvpgm763b3hvg8q4fzpzmflr5674x4j-glibc-2.32-10/lib/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/5rkf7qnbk6a30ay27p6mmfxm81hlq4qn-gvfs-1.48.1/lib/gio/modules/libgvfsdbus.so)
Failed to load module: /nix/store/5rkf7qnbk6a30ay27p6mmfxm81hlq4qn-gvfs-1.48.1/lib/gio/modules/libgvfsdbus.so
[1735484:1735484:0405/194210.821885:ERROR:angle_platform_impl.cc(43)] Display.cpp:805 (initialize): ANGLE Display::initialize error 12289: glXQueryExtensionsString returned NULL
[1735484:1735484:0405/194210.822127:ERROR:gl_surface_egl.cc(773)] EGL Driver message (Critical) eglInitialize: glXQueryExtensionsString returned NULL
[1735484:1735484:0405/194210.822162:ERROR:gl_surface_egl.cc(1322)] eglInitialize OpenGL failed with error EGL_NOT_INITIALIZED, trying next display type
[1735484:1735484:0405/194210.822863:ERROR:angle_platform_impl.cc(43)] Display.cpp:805 (initialize): ANGLE Display::initialize error 12289: glXQueryExtensionsString returned NULL
[1735484:1735484:0405/194210.822946:ERROR:gl_surface_egl.cc(773)] EGL Driver message (Critical) eglInitialize: glXQueryExtensionsString returned NULL
[1735484:1735484:0405/194210.822978:ERROR:gl_surface_egl.cc(1322)] eglInitialize OpenGLES failed with error EGL_NOT_INITIALIZED
[1735484:1735484:0405/194210.823013:ERROR:gl_initializer_linux_x11.cc(160)] GLSurfaceEGL::InitializeOneOff failed.
[1735484:1735484:0405/194210.828185:ERROR:viz_main_impl.cc(150)] Exiting GPU process due to errors during initialization
[1735261:1735476:0405/194210.854571:ERROR:media_history_store.cc(379)] Failed to create or update the media history store.
libva error: dlopen of /run/opengl-driver/lib/dri/i965_drv_video.so failed: /nix/store/1yvpgm763b3hvg8q4fzpzmflr5674x4j-glibc-2.32-10/lib/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/ri9b63pgs5mnrk53yvckwgy0jqvbxkax-libpciaccess-0.16/lib/libpciaccess.so.0)
[1735523:1735523:0405/194210.857924:ERROR:vaapi_wrapper.cc(541)] vaInitialize failed: unknown libva error
[1735261:1735475:0405/194210.887510:ERROR:login_database.cc(654)] Password store database is too new, kCurrentVersionNumber=28, GetCompatibleVersionNumber=31
[1735261:1735475:0405/194210.887637:ERROR:password_store_default.cc(39)] Could not create/open login database.
[1735261:1735475:0405/194210.887815:ERROR:password_store_x.cc(187)] Could not start the migration into the encrypted LoginDatabase because the database failed to initialise.
[1735261:1735476:0405/194210.919088:ERROR:component_installer.cc(326)] Failed to read manifest or verify installation for Federated Learning of Cohorts (/home/leo/.config/chromium/Floc/1.0.6).: Success (0)
(chromium:1735261): Gtk-WARNING **: 19:42:11.045: /nix/store/1yvpgm763b3hvg8q4fzpzmflr5674x4j-glibc-2.32-10/lib/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/fa26m83xs9fhjm940f0j9i89hc0p1irn-ibus-1.5.24/lib/libibus-1.0.so.5)
(chromium:1735261): Gtk-WARNING **: 19:42:11.045: Loading IM context type 'ibus' failed
[1735261:1735622:0405/194220.398822:ERROR:database.cc(1696)] History sqlite error 1, errno 0: no such column: publicly_routable, sql: SELECT id,url,visit_time,from_visit,transition,segment_id,visit_duration,incremented_omnibox_typed_score,publicly_routable FROM visits WHERE url=? ORDER BY visit_time DESC, id DESC LIMIT ?
Reason: Your graphics drivers that are mounted at /run/opengl-driver/lib are likely stale, and causing an older version of glibc to be loaded first. Then when your system tries to load a newer binary which is linked against a newer glibc it will fail. This is why you are getting a lot of graphics related failures.
Oh great, I think I’m starting to understand, thanks a lot! So rebooting does not change anything, but when I run:
$ nix registry list | grep flake:nixpkgs
user flake:nixpkgs github:NixOS/nixpkgs/355df33432ac534ba2184714850c366708c84cf5
global flake:nixpkgs github:NixOS/nixpkgs/nixpkgs-unstable
I can see that I pinned an old (like from 2020) commit for nixpkgs (I guess that the global will fetch nixpkgs-unstable everytime I run the command? but only for root). The reason was that I was tired of seeing nix search nixpkgs XXX being so long because the caching was lost every time a new commit was pushed on nixpkgs. So then, I guess that chromium uses the old glib 2.32 since it was the 2020 version, and it got confused by newer libraries using glib 2.33. And now, I understand also why the “fix” I was proposing is not a fix : it’s actually what it already done by nix (i.e. fixing one glibc for each program), the problem arises when the loaded libraries are too new… i.e. when we run old programs on new OS. Then, the LD_PRELOAD solution proposed above basically forces the old program to use the glibc used by the rest of the system… So if we apply that for all programs we will add impurity. And I guess that’s the reason why other OSes don’t have this issue: they always use the “impure” version, by keeping a single version of the glibc on the system. So it’s a tradeoff between purity and usability.
So I see basically a few approaches to approach this issue:
doing nothing more: then we can’t run old softwares on recent machines. Can lead to confusing errors for non-experts like I just experienced. And even if we have purity, we don’t have perfect reproducibility (due to the impurity coming from the drivers), and this choice actually decreases reproducibility: instead of having non-reproducible stuff only when the errors comes from different version of GLIBC (which is quite rare, especially when the used GLIBC is more recent than expected), we always crash when there is a version mismatch
if we always force the program to use the system GLIBC, then we lose purity, and we may find strange bugs when a recent program runs on an old system, as it will use an outdated GLIBC
best of both world: if we check right before running the program if the system’s GLIBC is more recent that the program’s GLIBC and use a LD_PRELOAD-like solution only for that case, then we get, I think, best of both world: if the program is more recent, then it will run the expected GLIBC in a pure way, and if the program is older than the system, instead of crashing it will run with a more recent GLIBC (and hopefully, recent GLIBC are compatible with older ones, so we expect few bugs due to this impurity… at least way less harmful than a big crash).
Do you think that this last strategy could make sense?
