Using nix run to run a rust program fails, but entering the devShell and using cargo run works. What's going on?

I’m encountering a strange phenomenon, and I was wondering if someone could explain what’s going on. I don’t necessarily know if it’s a problem, yet.

I have the following configuration:

{
	inputs = {
		nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
		flake-parts = { url = "github:hercules-ci/flake-parts"; inputs.nixpkgs-lib.follows = "nixpkgs"; };

		crane.url = "github:ipetkov/crane";
		fenix = { url = "github:nix-community/fenix"; inputs.nixpkgs.follows = "nixpkgs"; };
	};
	outputs = inputs@{
		self,
		nixpkgs,
		flake-parts,
		crane,
		fenix,
		...
	}: flake-parts.lib.mkFlake { inherit inputs; }
	{
		systems = [
			"x86_64-linux"
			"x86_64-darwin"
			"aarch64-linux"
			"aarch64-darwin"
			"i686-linux"
		];

		imports = [
			inputs.flake-parts.flakeModules.easyOverlay
		];

		perSystem = {self', pkgs, lib, system, ... }:
		let
			craneLib = (crane.mkLib pkgs).overrideToolchain
				fenix.packages.${system}.stable.toolchain;
			
			cargoTOML = lib.importTOML ./Cargo.toml;

			pname = "bevy-play";
			version = cargoTOML.package.version;

			src = craneLib.cleanCargoSource (lib.fileset.toSource {
				root = ./.;
				fileset = lib.fileset.unions [
					./Cargo.toml
					./Cargo.lock
					# ./rustfmt.toml
					# ./crates
					# ./docs
					# ./tests
					./src
				];
			});

			nativeBuildInputs = with pkgs; [
				pkg-config
			];

			buildInputs = with pkgs; [
				# Add additional build inputs here

				udev
				alsa-lib-with-plugins
				vulkan-loader
				
				# To use the x11 feature
				xorg.libX11
				xorg.libXcursor
				xorg.libXi
				xorg.libXrandr

				# To use the wayland feature
				libxkbcommon
				wayland

			] ++ lib.optionals stdenv.isDarwin [
				# Additional darwin specific inputs can be set here
				libiconv
			];
		in
		rec {
			# checks = {
			# 	;
			# };
			
			packages = rec {
				default = bevy-play;
				bevy-play = craneLib.buildPackage {
					inherit src pname version;
					inherit nativeBuildInputs buildInputs;
					meta.mainProgram = pname;
				};
			};

			apps.default = {
				type = "app";
				program = lib.getExe packages.bevy-play;
			};

			devShells.default = craneLib.devShell {
				# checks = self'.checks;
				inputsFrom = [ packages.bevy-play ];

				packages = with pkgs; [
					# python3
				];

				shellHook = ''
					export CARGO_MANIFEST_DIR=$(pwd)
					export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:${pkgs.lib.makeLibraryPath (nativeBuildInputs++buildInputs)}"
				'';
			};
		};
	};
}

If I run nix run, I get the following error:

thread 'main' panicked at /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-vendor-cargo-deps/c19b7c6f923b580ac259164a89f2577984ad5ab09ee9d583b888f934adbbe8d0/xkbcommon-dl-0.4.2/src/x11.rs:59:28:
Library libxkbcommon-x11.so could not be loaded.

However, if I enter the devShell, and I run cargo run, I get no error, and the program runs successfully.

From what I understand, libxkbcommon is part of LD_LIBRARY_PATH in the devShell, which allows cargo run to work in the devShell.

Why all the crane and fenix antics just to use stable rust at the end of the day? Much ado has been made about fenix’s inadequacy here.

You can use rustPlatform.buildRustPackage to build, and it should work.

You need to wrap it.

1 Like

Basically the previous answer is on point; you need a wrapper.
I’ll go in a bit more detail here to give context.

There are three things that are notable here:

  • the crate name is “xkbcommon-dl
  • this is a Rust runtime panic of something
  • the error message is not the usual “No such file or directory”

Why are those important? Because this all points to the library using Dynamic Loading for the library instead of linking to it at compile-time. If the latter were used then the resulting binary would have the path to the library embedded in its ELF metadata (you can check that using lld). Since it does not have that it needs to be told where to look for the library at runtime, which is your LD_LIBRARY_PATH export in the shellHook.

To expand on the link to wrapProgram; as explained you need to set the LD_LIBRARY_PATH. Ideally you would not overwrite it entirely since some paths may be set by the environment itself (for instance on my local machine the pipewire-jack path is provided that way), so setting those in your code will just complicate your code and potentially cause interoperability issues. However those libraries you do want to provide should be preferred over user choices usually, so that the X11 libraries match what you actually compiled against (and pull in the dependency properly).

The nixpkgs documentation has an example for PATH which you can adapt relatively easily by expanding the program = lib.getExe packages.bevy-play part of your Nix code. Note that this is just a hacky way to test this, since the wrapper should be part of the package, not the app. If you want to go about it properly you could adapt the approach that e.g. the firefox (but many others too) uses and have two packages, the -unwrapped version and the regularly named one which wraps the other in the same way shown below if you do not want to mess around with crane (which I don’t know how to do, and I also agree with the earlier comment about rustPlatform.buildRustPackage; I don’t think you gain much from using crane here).

{
  program = let bevy-play = packages.bevy-play; in lib.getExe (pkgs.runCommandLocal
    "${bevy-play.name}-wrapper"
    {
      nativeBuildInputs = [ pkgs.makeWrapper ];
      meta = { inherit (bevy-play.meta) mainProgram; };
    }
    ''
      makeWrapper ${lib.getExe bevy-play} $out/bin/${bevy-play.meta.mainProgram} \
        --prefix LD_LIBRARY_PATH : ${lib.makeLibraryPath [ buildInputs ]}
    '');
}

Note: I’ve tested only part of this, you may need to tinker with the code above to get it to work, and as implied by my earlier comment it’s likely you would be better off integrating with the underlying build system. Also trimming down the LD_LIBRARY_PATH to the libraries which are actually loaded dynamically instead of setting all of them might be a good idea too.

1 Like

autoPatchelfHook would be better for programs that use dlopen assuming they actually declare the dependency correctly, if not a manual patchelf would be needed. LD_LIBRARY_PATH should be heavily discouraged and therefore a wrapper should be unnecessary.

2 Likes

Ah, autoPatchelfHook will just add the missing libraries to the RPATH? Good to know.
And since it is a hook, one should be able to specify it as a nativeBuildInputs entry and have it automatically do its thing even with crane doing its thing then I guess.

That would of course obsolete the entire wrapper and make things a lot easier in that respect, while also causing an implicit nix dependency due to the RPATH of the ELF referencing the library.

So the following should be enough, right?

diff --git a/flake.nix b/flake.nix
index 2044086..2a66f7e 100644
--- a/flake.nix
+++ b/flake.nix
@@ -52,6 +52,7 @@

                        nativeBuildInputs = with pkgs; [
                                pkg-config
+                               autoPatchelfHook
                        ];

                        buildInputs = with pkgs; [

Thanks! This helped. Here’s the change that worked:

diff --git a/flake.nix b/flake.nix
index 5a35a68..d35a6b4 100644
--- a/flake.nix
+++ b/flake.nix
@@ -90,6 +90,13 @@
                                cargoArtifacts = craneLib.buildDepsOnly commonCraneArgs;
                                bevy-play = craneLib.buildPackage (commonCraneArgs // {
                                        inherit cargoArtifacts;
+                                       postFixup = ''
+                                               patchelf \
+                                               --set-rpath ${lib.makeLibraryPath (
+                                                       buildInputs
+                                               )} \
+                                               $out/bin/${pname}
+                                       '';
                                        meta.mainProgram = pname;
                                });
                        };

This was helpful, thanks!