Need help from Darwin users: Syntax errors in `.../Library/Frameworks/Foundation.framework/Headers`?

doronbehar · July 15, 2023, 1:58pm

I’m trying to make a package build on Darwin at musescore: 4.0.2 -> 4.1.0 by doronbehar · Pull Request #243298 · NixOS/nixpkgs · GitHub and I encounter these weird syntax issues in ofborg:

In file included from /nix/store/yjj6klsa9sk1yq6dzmpd4snrr3dmbcfn-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/CoreFoundation.framework/Headers/CFStream.h:16:
/nix/store/yjj6klsa9sk1yq6dzmpd4snrr3dmbcfn-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:1189:83: error: expected ','
    kCFURLBookmarkCreationWithSecurityScope API_AVAILABLE(macos(10.7), macCatalyst(13.0))  API_UNAVAILABLE(ios, watchos, tvos) = ( 1UL << 11 ), // Mac OS X 10.7.3 and later, include information in the bookmark data which allows the same sandboxed process to access the resource after being relaunched
                                                                                  ^
/nix/store/yjj6klsa9sk1yq6dzmpd4snrr3dmbcfn-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:1190:98: error: expected ','
    kCFURLBookmarkCreationSecurityScopeAllowOnlyReadAccess API_AVAILABLE(macos(10.7), macCatalyst(13.0)) API_UNAVAILABLE(ios, watchos, tvos) = ( 1UL << 12 ), // Mac OS X 10.7.3 and later, if used with kCFURLBookmarkCreationWithSecurityScope, at resolution time only read access to the resource will be granted
                                                                                                 ^
/nix/store/yjj6klsa9sk1yq6dzmpd4snrr3dmbcfn-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:1200:85: error: expected ','
    kCFURLBookmarkResolutionWithSecurityScope API_AVAILABLE(macos(10.7), macCatalyst(13.0)) API_UNAVAILABLE(ios, watchos, tvos) = ( 1UL << 10 ), // Mac OS X 10.7.3 and later, use the secure information included at creation time to provide the ability to access the resource in a sandboxed process.
                                                                                    ^
In file included from /tmp/nix-build-musescore-4.1.0.drv-0/source/src/framework/global/internal/platform/macos/macosinteractivehelper.mm:26:
In file included from /nix/store/z1h7s3bvqg65y9m0nwhx02gjw92ggmz0-apple-framework-Cocoa-11.0.0/Library/Frameworks/Cocoa.framework/Headers/Cocoa.h:12:
In file included from /nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/Foundation.h:10:
In file included from /nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSArray.h:9:
In file included from /nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSOrderedCollectionDifference.h:7:
/nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSOrderedCollectionChange.h:51:11: error: expected ';' after method prototype
- (id)init API_UNAVAILABLE(macos, ios, watchos, tvos);
          ^
In file included from /tmp/nix-build-musescore-4.1.0.drv-0/source/src/framework/global/internal/platform/macos/macosinteractivehelper.mm:26:
In file included from /nix/store/z1h7s3bvqg65y9m0nwhx02gjw92ggmz0-apple-framework-Cocoa-11.0.0/Library/Frameworks/Cocoa.framework/Headers/Cocoa.h:12:
In file included from /nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/Foundation.h:12:
/nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSBundle.h:156:21: error: expected ';' after method prototype
- (instancetype)init API_UNAVAILABLE(macos, ios, watchos, tvos);
                    ^
In file included from /tmp/nix-build-musescore-4.1.0.drv-0/source/src/framework/global/internal/platform/macos/macosinteractivehelper.mm:26:
In file included from /nix/store/z1h7s3bvqg65y9m0nwhx02gjw92ggmz0-apple-framework-Cocoa-11.0.0/Library/Frameworks/Cocoa.framework/Headers/Cocoa.h:12:
In file included from /nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/Foundation.h:14:
/nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSCalendar.h:115:21: error: expected ';' after method prototype
- (instancetype)init API_UNAVAILABLE(macos, ios, watchos, tvos);
                    ^
In file included from /tmp/nix-build-musescore-4.1.0.drv-0/source/src/framework/global/internal/platform/macos/macosinteractivehelper.mm:26:
In file included from /nix/store/z1h7s3bvqg65y9m0nwhx02gjw92ggmz0-apple-framework-Cocoa-11.0.0/Library/Frameworks/Cocoa.framework/Headers/Cocoa.h:12:
In file included from /nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/Foundation.h:26:
In file included from /nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSMeasurement.h:8:
/nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSUnit.h:96:21: error: expected ';' after method prototype
- (instancetype)init API_UNAVAILABLE(macos, ios, watchos, tvos);
                    ^
/nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSUnit.h:97:20: error: expected ';' after method prototype
+ (instancetype)new API_UNAVAILABLE(macos, ios, watchos, tvos);
                   ^
In file included from /tmp/nix-build-musescore-4.1.0.drv-0/source/src/framework/global/internal/platform/macos/macosinteractivehelper.mm:26:
In file included from /nix/store/z1h7s3bvqg65y9m0nwhx02gjw92ggmz0-apple-framework-Cocoa-11.0.0/Library/Frameworks/Cocoa.framework/Headers/Cocoa.h:12:
In file included from /nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/Foundation.h:26:
/nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSMeasurement.h:22:21: error: expected ';' after method prototype
- (instancetype)init API_UNAVAILABLE(macos, ios, watchos, tvos);
                    ^
In file included from /tmp/nix-build-musescore-4.1.0.drv-0/source/src/framework/global/internal/platform/macos/macosinteractivehelper.mm:26:
In file included from /nix/store/z1h7s3bvqg65y9m0nwhx02gjw92ggmz0-apple-framework-Cocoa-11.0.0/Library/Frameworks/Cocoa.framework/Headers/Cocoa.h:12:
In file included from /nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/Foundation.h:27:
In file included from /nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSMeasurementFormatter.h:11:
/nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSLocale.h:89:21: error: expected ';' after method prototype
- (instancetype)init API_UNAVAILABLE(macos, ios, watchos, tvos);     /* do not invoke; not a valid initializer for this class */
                    ^
In file included from /tmp/nix-build-musescore-4.1.0.drv-0/source/src/framework/global/internal/platform/macos/macosinteractivehelper.mm:26:
In file included from /nix/store/z1h7s3bvqg65y9m0nwhx02gjw92ggmz0-apple-framework-Cocoa-11.0.0/Library/Frameworks/Cocoa.framework/Headers/Cocoa.h:12:
In file included from /nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/Foundation.h:39:
In file included from /nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSFileManager.h:11:
/nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSURL.h:395:82: error: expected ','
    NSURLBookmarkCreationWithSecurityScope API_AVAILABLE(macos(10.7), macCatalyst(13.0)) API_UNAVAILABLE(ios, watchos, tvos) = ( 1 << 11 ), /* include information in the bookmark data which allows the same sandboxed process to access the resource after being relaunched */
                                                                                 ^
/nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSURL.h:396:97: error: expected ','
    NSURLBookmarkCreationSecurityScopeAllowOnlyReadAccess API_AVAILABLE(macos(10.7), macCatalyst(13.0)) API_UNAVAILABLE(ios, watchos, tvos) = ( 1 << 12 ), /* if used with kCFURLBookmarkCreationWithSecurityScope, at resolution time only read access to the resource will be granted */
                                                                                                ^
/nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSURL.h:402:84: error: expected ','
    NSURLBookmarkResolutionWithSecurityScope API_AVAILABLE(macos(10.7), macCatalyst(13.0)) API_UNAVAILABLE(ios, watchos, tvos) = ( 1 << 10 ) /* use the secure information included at creation time to provide the ability to access the resource in a sandboxed process */
                                                                                   ^
In file included from /tmp/nix-build-musescore-4.1.0.drv-0/source/src/framework/global/internal/platform/macos/macosinteractivehelper.mm:26:
In file included from /nix/store/z1h7s3bvqg65y9m0nwhx02gjw92ggmz0-apple-framework-Cocoa-11.0.0/Library/Frameworks/Cocoa.framework/Headers/Cocoa.h:12:
In file included from /nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/Foundation.h:69:
/nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSProxy.h:28:28: error: expected ';' after method prototype
- (BOOL)allowsWeakReference API_UNAVAILABLE(macos, ios, watchos, tvos);
                           ^
/nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSProxy.h:29:28: error: expected ';' after method prototype
- (BOOL)retainWeakReference API_UNAVAILABLE(macos, ios, watchos, tvos);
                           ^
In file included from /tmp/nix-build-musescore-4.1.0.drv-0/source/src/framework/global/internal/platform/macos/macosinteractivehelper.mm:26:
In file included from /nix/store/z1h7s3bvqg65y9m0nwhx02gjw92ggmz0-apple-framework-Cocoa-11.0.0/Library/Frameworks/Cocoa.framework/Headers/Cocoa.h:12:
In file included from /nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/Foundation.h:87:
In file included from /nix/store/g9q6d6p98w0gd9ymv60kawm15kg24fym-apple-framework-Foundation-11.0.0/Library/Frameworks/Foundation.framework/Headers/NSURLCredential.h:9:
In file included from /nix/store/5c3545spggcjv0wv6sjnbjznymqp0b5y-apple-framework-Security-11.0.0/Library/Frameworks/Security.framework/Headers/Security.h:27:
/nix/store/5c3545spggcjv0wv6sjnbjznymqp0b5y-apple-framework-Security-11.0.0/Library/Frameworks/Security.framework/Headers/SecBase.h:142:73: error: expected ';' after top level declarator
typedef struct CF_BRIDGED_TYPE(id) __SecKeychainItem *SecKeychainItemRef API_UNAVAILABLE(ios, watchos, tvos, macCatalyst);
                                                                        ^
/nix/store/5c3545spggcjv0wv6sjnbjznymqp0b5y-apple-framework-Security-11.0.0/Library/Frameworks/Security.framework/Headers/SecBase.h:148:77: error: expected ';' after top level declarator
typedef struct CF_BRIDGED_TYPE(id) __SecKeychainSearch *SecKeychainSearchRef API_UNAVAILABLE(ios, watchos, tvos, macCatalyst);
                                                                            ^
/nix/store/5c3545spggcjv0wv6sjnbjznymqp0b5y-apple-framework-Security-11.0.0/Library/Frameworks/Security.framework/Headers/SecBase.h:154:35: error: expected ';' after top level declarator
typedef OSType SecKeychainAttrType API_UNAVAILABLE(ios, watchos, tvos, macCatalyst);
                                  ^
/nix/store/5c3545spggcjv0wv6sjnbjznymqp0b5y-apple-framework-Security-11.0.0/Library/Frameworks/Security.framework/Headers/SecBase.h:163:27: error: expected ')'
struct API_UNAVAILABLE(ios, watchos, tvos, macCatalyst) SecKeychainAttribute
                          ^
/nix/store/5c3545spggcjv0wv6sjnbjznymqp0b5y-apple-framework-Security-11.0.0/Library/Frameworks/Security.framework/Headers/SecBase.h:163:8: note: to match this '('
struct API_UNAVAILABLE(ios, watchos, tvos, macCatalyst) SecKeychainAttribute
       ^
/nix/store/vw0zbvb4n6c1mwfj5x4ggngqlkfgb070-Libsystem-1238.60.2/include/os/availability.h:9:132: note: expanded from macro 'API_UNAVAILABLE'
  #define API_UNAVAILABLE(...) __API_UNAVAILABLE_GET_MACRO(__VA_ARGS__, __API_UNAVAILABLE3, __API_UNAVAILABLE2, __API_UNAVAILABLE1)(__VA_ARGS__)
                                                                                                                                   ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.

Do all Darwin users experience these errors right now? This is very weird to me.

reckenrode · July 15, 2023, 3:09pm

Those errors happen when headers are mixed between different SDK versions. It looks like one of your dependencies (portaudio) is propagating Dariwn frameworks from the 10.12 SDK even though Qt is having you build against the 11.0 one.

On Darwin (at least x86_64-darwin), try rebuilding portaudio against the 11.0 SDK. Something like:

let
  # portaudio propagates Darwin frameworks. Rebuild it using the 11.0 stdenv from Qt and the 11.0 SDK frameworks.
  portaudio' = if !stdenv.isDarwin
    then portaudio
    else portaudio.override {
      inherit stdenv;
      inherit (darwin.apple_sdk_11_0.frameworks) AudioUnit AudioToolbox CoreAudio CoreServices Carbon;
    };
in
# ... (use `portaudio'` in your `buildInputs`

jost-s · May 23, 2024, 7:27pm

I have a similar problem when nix building a package with crane on x86_64-darwin, that depends on Carbon. The error message I get is

The following warnings were emitted during compilation:
hc-launch-deps-workspace> warning: tao@0.16.8: In file included from src/platform_impl/macos/carbon_hotkey/carbon_hotkey_binding.c:7:
hc-launch-deps-workspace> warning: tao@0.16.8: In file included from /nix/store/a10l2jpkmqn2ysrxd27vzvgqd0zz59w0-apple-framework-Carbon-11.0.0/Library/Frameworks/Carbon.framework/Headers/Carbon.h:20:
hc-launch-deps-workspace> warning: tao@0.16.8: In file included from /nix/store/n1mayiw8rhx0i5z5mpc27ccsvdnqpfwz-apple-framework-CoreServices-11.0.0/Library/Frameworks/CoreServices.framework/Headers/CoreServices.h:19:
hc-launch-deps-workspace> warning: tao@0.16.8: In file included from /nix/store/pzfpgshpdailjfqvh48sh12ayix10l4s-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:60:
hc-launch-deps-workspace> warning: tao@0.16.8: In file included from /nix/store/pzfpgshpdailjfqvh48sh12ayix10l4s-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/CoreFoundation.framework/Headers/CFPropertyList.h:17:
hc-launch-deps-workspace> warning: tao@0.16.8: In file included from /nix/store/pzfpgshpdailjfqvh48sh12ayix10l4s-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/CoreFoundation.framework/Headers/CFStream.h:16:
hc-launch-deps-workspace> warning: tao@0.16.8: /nix/store/pzfpgshpdailjfqvh48sh12ayix10l4s-apple-framework-CoreFoundation-11.0.0/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:1189:83: error: expected ','
hc-launch-deps-workspace> warning: tao@0.16.8:     kCFURLBookmarkCreationWithSecurityScope API_AVAILABLE(macos(10.7), macCatalyst(13.0))  API_UNAVAILABLE(ios, watchos, tvos) = ( 1UL << 11 ), // Mac OS X 10.7.3 and later, include information in the bookmark data which allows the same sandboxed process to access the resource after being relaunched

There are several other ones of that kind (see Github workflow nix: please · holochain/holonix@7937faa · GitHub).

That happens with these build inputs:

pkgs.darwin.apple_sdk_11_0.frameworks.Carbon
pkgs.darwin.apple_sdk_11_0.frameworks.WebKit

When I change the build inputs to just

pkgs.darwin.apple_sdk_10_12.frameworks.Carbon

I get a different error:

The following warnings were emitted during compilation:
hc-launch-deps-workspace> warning: tao@0.16.8: In file included from src/platform_impl/macos/carbon_hotkey/carbon_hotkey_binding.c:7:
hc-launch-deps-workspace> warning: tao@0.16.8: In file included from /nix/store/zz0b1r8gmclavz92my2ifv6a8hfr1vjw-apple-framework-Carbon/Library/Frameworks/Carbon.framework/Headers/Carbon.h:24:
hc-launch-deps-workspace> warning: tao@0.16.8: In file included from /nix/store/g4zyacnlqm5mmhzbysd73lvc89i643id-apple-framework-ApplicationServices/Library/Frameworks/ApplicationServices.framework/Headers/ApplicationServices.h:31:
hc-launch-deps-workspace> warning: tao@0.16.8: In file included from /nix/store/ck3awwmd2pca7cs50jbghcxfh32k1jc2-apple-framework-ColorSync-11.0.0/Library/Frameworks/ColorSync.framework/Headers/ColorSync.h:11:
hc-launch-deps-workspace> warning: tao@0.16.8: /nix/store/ck3awwmd2pca7cs50jbghcxfh32k1jc2-apple-framework-ColorSync-11.0.0/Library/Frameworks/ColorSync.framework/Headers/ColorSyncProfile.h:26:51: error: expected ';' after top level declarator
hc-launch-deps-workspace> warning: tao@0.16.8: CSEXTERN CFStringRef kColorSyncGenericGrayProfile CS_AVAILABLE_PUBLIC_STARTING(10.4, CS_UNAVAILABLE_PUBLIC_EMBEDDED);          /* com.apple.ColorSync.GenericGray  */

Github workflow

I see it’s still pulling in the apple-framework-11.0.0 packages. How can I figure out what’s causing this? Or is there another fix?

reckenrode · May 23, 2024, 9:53pm

Are you using overrideSDK stdenv "11.0" or apple_sdk_11_0.callPackage? The default x86_64-darwin SDK is 10.12. Mixing frameworks from the 11.0 SDK with the 10.12 SDK leads to errors like you’re seeing. (You can sometimes get away with using the frameworks with the default stdenv, but using overrideSDK will normally be more reliable.)

Using a non-default SDK is messy right now. It’s something I’d like to improve for 24.11 as part of the SDK refactor.

jost-s · May 23, 2024, 10:15pm

No, I’m using neither of those. The corresponding portion of the flake is this:

craneLib.buildPackage {
  pname = "hc-launch";
  version = "workspace";
  cargoExtraArgs = "--bin hc-launch";
  src = pkgs.lib.cleanSourceWith {
    src = launcher-src;
  };
  buildInputs = [
    pkgs.go
    pkgs.perl
  ]
  ++ (pkgs.lib.optionals pkgs.stdenv.isDarwin [
    pkgs.darwin.apple_sdk_10_12.frameworks.Carbon
  ]);
};

Should I override tao?

reckenrode · May 23, 2024, 11:08pm

According to the documentation, crateLib.buildPackage accepts a stdenv attribute. If you use the following, it should (hopefully) do the right thing. For frameworks, use the unversioned ones (pkgs.darwin.apple_sdk.frameworks.Carbon). overrideSDK will take care of switching them to the right version.

craneLib.buildPackage {
  pname = "hc-launch";
  version = "workspace";
  stdenv = 
    if pkgs.stdenv.isDarwin then
      pkgs.overrideSDK pkgs.stdenv "11.0"
    else
      pkgs.stdenv;
  cargoExtraArgs = "--bin hc-launch";
  src = pkgs.lib.cleanSourceWith {
    src = launcher-src;
  };
  buildInputs = [
    pkgs.go
    pkgs.perl
  ]
  ++ (pkgs.lib.optionals pkgs.stdenv.isDarwin [
    pkgs.darwin.apple_sdk.frameworks.Carbon
  ]);
};

jost-s · May 24, 2024, 1:48am

Mmh, I’ve tried it and it doesn’t work on the x86_64-darwin platform. Even with nix flake check, I get the following error:

… while calling the 'derivationStrict' builtin
         at /builtin/derivation.nix:9:12: (source not available)
       … while evaluating derivation 'hc-launch-workspace'
         whose name attribute is located at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/generic/make-derivation.nix:331:7
       … while evaluating attribute '__impureHostDeps' of derivation 'hc-launch-workspace'
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/generic/make-derivation.nix:446:7:
          445|       __propagatedSandboxProfile = unique (computedPropagatedSandboxProfile ++ [ propagatedSandboxProfile ]);
          446|       __impureHostDeps = computedImpureHostDeps ++ computedPropagatedImpureHostDeps ++ __propagatedImpureHostDeps ++ __impureHostDeps ++ stdenv.__extraImpureHostDeps ++ [
             |       ^
          447|         "/dev/zero"
       … while calling anonymous lambda
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/generic/make-derivation.nix:430:30:
          429|         computedImpureHostDeps =
          430|           unique (concatMap (input: input.__propagatedImpureHostDeps or [])
             |                              ^
          431|             (stdenv.extraNativeBuildInputs
       … while calling anonymous lambda
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/generic/make-derivation.nix:302:13:
          301|       (map (drv: getDev drv.__spliced.buildBuild or drv) (checkDependencyList "depsBuildBuild" depsBuildBuild))
          302|       (map (drv: getDev drv.__spliced.buildHost or drv) (checkDependencyList "nativeBuildInputs" nativeBuildInputs'))
             |             ^
          303|       (map (drv: getDev drv.__spliced.buildTarget or drv) (checkDependencyList "depsBuildTarget" depsBuildTarget))
       … from call site
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/generic/make-derivation.nix:302:18:
          301|       (map (drv: getDev drv.__spliced.buildBuild or drv) (checkDependencyList "depsBuildBuild" depsBuildBuild))
          302|       (map (drv: getDev drv.__spliced.buildHost or drv) (checkDependencyList "nativeBuildInputs" nativeBuildInputs'))
             |                  ^
          303|       (map (drv: getDev drv.__spliced.buildTarget or drv) (checkDependencyList "depsBuildTarget" depsBuildTarget))
       … while calling 'getOutput'
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/lib/attrsets.nix:1796:23:
         1795|   */
         1796|   getOutput = output: pkg:
             |                       ^
         1797|     if ! pkg ? outputSpecified || ! pkg.outputSpecified
       … while calling anonymous lambda
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/lib/lists.nix:334:29:
          333|   */
          334|   imap1 = f: list: genList (n: f (n + 1) (elemAt list n)) (length list);
             |                             ^
          335|
       … from call site
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/lib/lists.nix:334:32:
          333|   */
          334|   imap1 = f: list: genList (n: f (n + 1) (elemAt list n)) (length list);
             |                                ^
          335|
       … while calling anonymous lambda
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/generic/make-derivation.nix:276:15:
          275|     imap1
          276|       (index: dep:
             |               ^
          277|         if isDerivation dep || dep == null || builtins.isString dep || builtins.isPath dep then dep
       … from call site
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/generic/make-derivation.nix:277:12:
          276|       (index: dep:
          277|         if isDerivation dep || dep == null || builtins.isString dep || builtins.isPath dep then dep
             |            ^
          278|         else if isList dep then checkDependencyList' ([index] ++ positions) name dep
       … while calling 'isDerivation'
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/lib/attrsets.nix:1283:5:
         1282|   isDerivation =
         1283|     value: value.type or null == "derivation";
             |     ^
         1284|
       … while calling 'getReplacement'
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/darwin/override-sdk.nix:260:18:
          259|   getReplacement =
          260|     newPackages: pkg:
             |                  ^
          261|     let
       … from call site
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/darwin/override-sdk.nix:397:29:
          396|               input = args.${inputName};
          397|               newPackages = getPackageMapping baseSdkMapping input;
             |                             ^
          398|             in
       … while calling 'getPackageMapping'
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/darwin/override-sdk.nix:328:18:
          327|   getPackageMapping =
          328|     baseMapping: input:
             |                  ^
          329|     let
       … from call site
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/darwin/override-sdk.nix:332:5:
          331|     in
          332|     lib.foldr (
             |     ^
          333|       pkg: newPackages:
       … while calling 'foldr'
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/lib/lists.nix:121:20:
          120|   */
          121|   foldr = op: nul: list:
             |                    ^
          122|     let
       … from call site
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/lib/lists.nix:128:8:
          127|         else op (elemAt list n) (fold' (n + 1));
          128|     in fold' 0;
             |        ^
          129|
       … while calling 'fold''
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/lib/lists.nix:124:15:
          123|       len = length list;
          124|       fold' = n:
             |               ^
          125|         if n == len
       … from call site
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/darwin/override-sdk.nix:330:22:
          329|     let
          330|       dependencies = topologicallyOrderedPropagatedDependencies input;
             |                      ^
          331|     in
       … while calling 'topologicallyOrderedPropagatedDependencies'
         at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/darwin/override-sdk.nix:310:5:
          309|   topologicallyOrderedPropagatedDependencies =
          310|     pkgs:
             |     ^
          311|     let
       error: cannot coerce a list to a string

This is on a Github runner. In a repl on that runner I can run :b pkgs.overrideSDK pkgs.stdenv "11.0" and it builds it without errors. I’ve tried pkgs.overrideSDK pkgs.stdenv "10.12" to check the expression and nix flake check succeeds.

When I try your suggestion locally on my aarch64-darwin machine, it works fine too.

Do you have a clue where the list to a string comes from?

reckenrode · May 24, 2024, 2:57am

Something’s build inputs are of an unexpected type. overrideSDK does some complicated stuff to rewrite the dependencies propagated to a package.

I cloned your PR and ran it with --debugger. It appears rustc has the following, but I don’t know where it’s getting it because it’s not in the nixpkgs rustc derivation.

nix-repl> pkg.name
"rust-default-1.77.2"

nix-repl> pkg.depsTargetTargetPropagated
[ [ ... ] ]

Wait a moment …. That’s not the nixpkgs rustc.

github.com

oxalica/rust-overlay/blob/66a43411079d0d1b1b776c0a6ced20a5df896edb/mk-aggregated.nix#L29-L30


      
          depsTargetTargetPropagated =
            optional (targetPlatform.isDarwin) [ pkgsTargetTarget.libiconv ];

There it is. It should be optionals with a list, but it’s using optional.

I don’t know whether that’s valid, but it works without overrideSDK, so it should work when using overrideSDK. I’m testing a patch now to confirm it works.

reckenrode · May 24, 2024, 3:19am

It builds. I had to add darwin.apple_sdk.libs.xpc as a buildInput to the package. PR is up at overrideSDK: handle propagated lists of inputs by reckenrode · Pull Request #314167 · NixOS/nixpkgs · GitHub.

$ nix build .#packages.x86_64-darwin.launcher
$ ./result/bin/hc-launch --version
holochain_cli_launch 0.300.0-dev.2 (holochain 0.3.1-rc.0)

jost-s · May 24, 2024, 12:19pm

Amazing, a huge thank you!

Would it ideally be fixed upstream in the rust-overlay repo as well? What surprises me is that it works under aarch64-darwin. That would mean that overrideSDK receives different values on the two platforms.

reckenrode · May 24, 2024, 12:59pm

Probably, but if Nix accepts the nested lists, then overrideSDK also needs to be able to handle them.

aarch64-darwin uses the 11.0 SDK by default, so overriding doesn’t do anything currently on that platform because it’s also the only SDK choice. That will change after the refactor, but exactly how is still to be determined.

jost-s · May 24, 2024, 1:59pm

aarch64-darwin uses the 11.0 SDK by default, so overriding doesn’t do anything currently on that platform because it’s also the only SDK choice. That will change after the refactor, but exactly how is still to be determined.

Got it.

Probably, but if Nix accepts the nested lists, then overrideSDK also needs to be able to handle them.

Right! A bit funny the optional function and its documentation. That means it takes any type. Alright, I’ll make a PR for the rust-overlay anyway.

jost-s · May 24, 2024, 4:46pm

I’m using your patched branch as nixpkgs input in the flake and added apple_sdk.libs.xpc. The flake compiles correctly now, but the package still does not build with the same error as before nix: use patched nixpkgs · holochain/holonix@6c46f6f · GitHub

You must have worked some trick there on your machine

jost-s · May 27, 2024, 12:25pm

@reckenrode Please post the complete flake that let you build successfully. The debugger keeps me in an endless loop with error: QuickTime framework not available.

reckenrode · May 27, 2024, 3:53pm

Sorry about not getting back to you. I was building against my ld64/cross/clang18 branch with the overrideSDK fix applied. I had started a build using your PR but got distracted by other things. This is the flake I’m using that built for me.

gist.github.com

https://gist.github.com/reckenrode/606bdd17ede67f3f5debf02e2beb09e1

flake.nix

{
  # flake format https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html#flake-format
  description = "Holonix - Holochain Nix flake";

  # specify all input dependencies needed to create the outputs of the flake
  inputs = {
    # nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
    nixpkgs.url = "/Users/reckenrode/Developer/nixpkgs";

    # utility to iterate over multiple target platforms

This file has been truncated. show original

jost-s · May 27, 2024, 4:11pm

Sorry about not getting back to you.

Not at all! I’m happy about the help you’ve given me already.

I was building against my ld64/cross/clang18 branch with the overrideSDK fix applied. I had started a build using your PR but got distracted by other things. This is the flake I’m using that built for me.

Thank you.
Mmh, I’m using the same flake but get the initial error still. Does that point to the compiler being the problem?

reckenrode · May 27, 2024, 6:46pm

It could be that or (more likely) the updates required for the ld64 update. I had to find another source for the availability headers, which could make mixing headers from different SDK versions a little less problematic.

jost-s · May 28, 2024, 11:34am

I see. I’ve tried to use your ld64 branch as nixpkgs input, but [the runner times out after 6 hrs of building](test: try with only ld64 · holochain/holonix@09ee24a · GitHub =\ It’s probably building every single of the nearly 2000 packages.

The upstream fix in the Rust overlay has been merged.

If you can offer any more help, I’d be grateful. Instructions for me to debug are welcome, or should I ask in the matrix chat or on Discord as well?

reckenrode · May 28, 2024, 12:32pm

The ld64 branch involves changes that affect the bootstrap, so it takes a while to build. A full bootstrap of just the stdenv on aarch64-darwin is ~2 hours on my M3 Max.

Once the 24.05 release is out, which should be soon, I’ll be cherry-picking changes from my branch to separate PRs into staging and master. Unfortunately, the timing meant it was too close to release to do (especially since there are breaking changes in the ld64 upgrade).

Until those land, you may have to target only aarch64-darwin and mark x86_64-darwin as broken. The problem you’re encountering is that something is retaining the 10.12 libSystem, which has incomplete availability headers that don’t work with newer SDKs. It may be the replacement ones are why it builds on my system, but I don’t know.

Ideally, overrideSDK would work with Rust, but rustPlatform embeds references to various compilers in its hooks’ scripts, which are impossible to detect generally. rustPlatform would need modifying to be friendly to overriding the SDK, and I don’t know what the frameworks outside of nixpkgs would need.

I have an idea for how SDKs can be changed to make using a specific one less painful (at the cost of radically changing how SDKs are used in nixpkgs), but that work hasn’t started yet since my focus currently is on the ld64 upgrade, then Darwin cross-compilation, then doing the clang 18 update. I’m still hopeful it can be done this summer, but that stuff comes first.

jost-s · May 28, 2024, 10:34pm

Thank you for the explanation. I’m usually following nixos-unstable anyway, so IIUC as soon as your cherry-picked fixes are on master, I would benefit from them.

So other than overrideSDK, there’s no generalized way of making sure a particular version of the SDK is used. Well if rustPlatform is inconsistent in that area, that would be the place to fix, right?!

For the time being I’ll exclude this package for the x86_64-darwin platform and look forward to your updates. Is there an issue I can subscribe to or something like that?