Using Metal shader compiler inside derivation

Help
1

I’m currently packaging https://zed.dev, a Rust-based editor for macOS / darwin, for nixpkgs and am encountering an issue I feel like is not solvable: zed.dev uses their own GPUI library for the GUI, which makes heavy use of Metal. That in itself is not an issue as I can provide Metal.frameowork just fine, but in the build.rs of GPUI their are precompiling their shaders from .metal => .air => .metallib, meaning the proprietary CLI tools metal and metallib are required. These are closed source and only bundled with Xcode, not even with the Command Line Tools.

So over all the options are:

  1. Getting access to a real Xcode installation and use the original tools => not possible in nixpkgs
  2. Packaging the closed source, proprietary tools => not possible because of copyright / licensing
  3. Precompile the shader binary blob myself and bundle it in nixpkgs => feels bad and I don’t think that will be accepted
  4. Add patches to the package that do the following: disable the build.rs shader compilation phase, replace the Shader creation from metallib with Shader creation from source code (not a problem in Metal, just might mean slightly increased startup time for Zed)

Are there any other packages that have encountered the issue, or is there any approach I am missing to precompile the shaders inside Nix?

2

I’m also trying to work with this new codebase with nix. I specifically just want to try GPUI in a flake devshell, as I’m used to with other projects. I posted this issue, which was actually caused by my nix shell, which I still haven’t fixed yet. Have you run into this error message? How did you solve it?

Here’s my current flake for reference:

{
  inputs.flakelight-rust.url = "github:accelbread/flakelight-rust";
  outputs = { flakelight-rust, ... }:
    flakelight-rust ./. {
      systems = ["aarch64-darwin"];
      # devShell = {
      #   packages = pkgs: with pkgs; [
      #       curl
      #       libiconv
      #       zlib

      #       llvmPackages_17.llvm
      #       llvmPackages_17.clang
      #   ];
      #   inputsFrom = pkgs: with pkgs.darwin.apple_sdk.frameworks; [
      #     CoreMedia
      #     Security
      #     CoreFoundation
      #     CoreServices
      #   ];

      #   env = pkgs: {
      #     LIBCLANG_PATH = "${pkgs.llvmPackages_17.clang-unwrapped.lib}/lib";
      #   };
      # };

      devShells.testing = {mkShell, pkgs}: (mkShell.override {
        stdenv = pkgs.clangStdenv;
      }) {
        nativeBuildInputs = with pkgs; [
            rustc cargo
            curl
        ];
        buildInputs = with pkgs.darwin.apple_sdk.frameworks; [
          CoreMedia
        ] ++ [
          pkgs.libiconv
          pkgs.clang
        ];
        LIBCLANG_PATH = "${pkgs.libclang.lib}/lib";
      };
    };
}

I started with flakelight, but wanted more control to try things like LIBCLANG_PATH (necessary for bindgen to use our libclang, not the system Xcode’s…

Run the shell with nix develop .#testing and try cargo build to see this error:

error: failed to run custom build command for `media v0.1.0 (/Users/evanrichter/projects/gpui-test/zed/crates/media)`

Caused by:
  process didn't exit successfully: `/Users/evanrichter/projects/gpui-test/target/debug/build/media-3603fc04948a557e/build-script-build` (exit status: 101)
  --- stdout
  libclang path: "/nix/store/f3j1g5k8adp12mrygw72mvqdsmbsm743-clang-16.0.6-lib/lib/libclang.dylib"
  Ok("clang")
  ClangVersion { parsed: Some((16, 0)), full: "clang version 16.0.6" }
  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk
  cargo:rerun-if-changed=src/bindings.h
  cargo:rerun-if-env-changed=LIBCLANG_PATH
  cargo:rerun-if-env-changed=TARGET
  cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS_aarch64-apple-darwin
  cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS_aarch64_apple_darwin
  cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS

  --- stderr
  src/bindings.h:1:9: fatal error: 'CoreMedia/CMFormatDescription.h' file not found
  thread 'main' panicked at zed/crates/media/build.rs:38:10:
  unable to generate bindings: ClangDiagnostic("src/bindings.h:1:9: fatal error: 'CoreMedia/CMFormatDescription.h' file not found\n")
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I added some extra prints to the build.rs script, including “cargo:rerun-if-env-changed=LIBCLANG_PATH” which I intend to upstream to bindgen

Any idea why CoreMedia headers aren’t found even though I’m including the framework in buildInputs?

3

I’ve opened a related discussion on Zed: Building GPUI without Xcode's `metal` and `metallib` · zed-industries/zed · Discussion #7016 · GitHub
I think precompiling the shaders is not necessary, and probably doesn’t even give much benefit as wgpu for example only supports compiling Metal shaders at runtime.

1 Like