No, I’m using neither of those. The corresponding portion of the flake is this:
craneLib.buildPackage {
pname = "hc-launch";
version = "workspace";
cargoExtraArgs = "--bin hc-launch";
src = pkgs.lib.cleanSourceWith {
src = launcher-src;
};
buildInputs = [
pkgs.go
pkgs.perl
]
++ (pkgs.lib.optionals pkgs.stdenv.isDarwin [
pkgs.darwin.apple_sdk_10_12.frameworks.Carbon
]);
};
Should I override tao?
According to the documentation, crateLib.buildPackage accepts a stdenv attribute. If you use the following, it should (hopefully) do the right thing. For frameworks, use the unversioned ones (pkgs.darwin.apple_sdk.frameworks.Carbon). overrideSDK will take care of switching them to the right version.
craneLib.buildPackage {
pname = "hc-launch";
version = "workspace";
stdenv =
if pkgs.stdenv.isDarwin then
pkgs.overrideSDK pkgs.stdenv "11.0"
else
pkgs.stdenv;
cargoExtraArgs = "--bin hc-launch";
src = pkgs.lib.cleanSourceWith {
src = launcher-src;
};
buildInputs = [
pkgs.go
pkgs.perl
]
++ (pkgs.lib.optionals pkgs.stdenv.isDarwin [
pkgs.darwin.apple_sdk.frameworks.Carbon
]);
};
Mmh, I’ve tried it and it doesn’t work on the x86_64-darwin platform. Even with nix flake check, I get the following error:
… while calling the 'derivationStrict' builtin
at /builtin/derivation.nix:9:12: (source not available)
… while evaluating derivation 'hc-launch-workspace'
whose name attribute is located at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/generic/make-derivation.nix:331:7
… while evaluating attribute '__impureHostDeps' of derivation 'hc-launch-workspace'
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/generic/make-derivation.nix:446:7:
445| __propagatedSandboxProfile = unique (computedPropagatedSandboxProfile ++ [ propagatedSandboxProfile ]);
446| __impureHostDeps = computedImpureHostDeps ++ computedPropagatedImpureHostDeps ++ __propagatedImpureHostDeps ++ __impureHostDeps ++ stdenv.__extraImpureHostDeps ++ [
| ^
447| "/dev/zero"
… while calling anonymous lambda
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/generic/make-derivation.nix:430:30:
429| computedImpureHostDeps =
430| unique (concatMap (input: input.__propagatedImpureHostDeps or [])
| ^
431| (stdenv.extraNativeBuildInputs
… while calling anonymous lambda
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/generic/make-derivation.nix:302:13:
301| (map (drv: getDev drv.__spliced.buildBuild or drv) (checkDependencyList "depsBuildBuild" depsBuildBuild))
302| (map (drv: getDev drv.__spliced.buildHost or drv) (checkDependencyList "nativeBuildInputs" nativeBuildInputs'))
| ^
303| (map (drv: getDev drv.__spliced.buildTarget or drv) (checkDependencyList "depsBuildTarget" depsBuildTarget))
… from call site
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/generic/make-derivation.nix:302:18:
301| (map (drv: getDev drv.__spliced.buildBuild or drv) (checkDependencyList "depsBuildBuild" depsBuildBuild))
302| (map (drv: getDev drv.__spliced.buildHost or drv) (checkDependencyList "nativeBuildInputs" nativeBuildInputs'))
| ^
303| (map (drv: getDev drv.__spliced.buildTarget or drv) (checkDependencyList "depsBuildTarget" depsBuildTarget))
… while calling 'getOutput'
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/lib/attrsets.nix:1796:23:
1795| */
1796| getOutput = output: pkg:
| ^
1797| if ! pkg ? outputSpecified || ! pkg.outputSpecified
… while calling anonymous lambda
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/lib/lists.nix:334:29:
333| */
334| imap1 = f: list: genList (n: f (n + 1) (elemAt list n)) (length list);
| ^
335|
… from call site
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/lib/lists.nix:334:32:
333| */
334| imap1 = f: list: genList (n: f (n + 1) (elemAt list n)) (length list);
| ^
335|
… while calling anonymous lambda
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/generic/make-derivation.nix:276:15:
275| imap1
276| (index: dep:
| ^
277| if isDerivation dep || dep == null || builtins.isString dep || builtins.isPath dep then dep
… from call site
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/generic/make-derivation.nix:277:12:
276| (index: dep:
277| if isDerivation dep || dep == null || builtins.isString dep || builtins.isPath dep then dep
| ^
278| else if isList dep then checkDependencyList' ([index] ++ positions) name dep
… while calling 'isDerivation'
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/lib/attrsets.nix:1283:5:
1282| isDerivation =
1283| value: value.type or null == "derivation";
| ^
1284|
… while calling 'getReplacement'
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/darwin/override-sdk.nix:260:18:
259| getReplacement =
260| newPackages: pkg:
| ^
261| let
… from call site
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/darwin/override-sdk.nix:397:29:
396| input = args.${inputName};
397| newPackages = getPackageMapping baseSdkMapping input;
| ^
398| in
… while calling 'getPackageMapping'
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/darwin/override-sdk.nix:328:18:
327| getPackageMapping =
328| baseMapping: input:
| ^
329| let
… from call site
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/darwin/override-sdk.nix:332:5:
331| in
332| lib.foldr (
| ^
333| pkg: newPackages:
… while calling 'foldr'
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/lib/lists.nix:121:20:
120| */
121| foldr = op: nul: list:
| ^
122| let
… from call site
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/lib/lists.nix:128:8:
127| else op (elemAt list n) (fold' (n + 1));
128| in fold' 0;
| ^
129|
… while calling 'fold''
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/lib/lists.nix:124:15:
123| len = length list;
124| fold' = n:
| ^
125| if n == len
… from call site
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/darwin/override-sdk.nix:330:22:
329| let
330| dependencies = topologicallyOrderedPropagatedDependencies input;
| ^
331| in
… while calling 'topologicallyOrderedPropagatedDependencies'
at /nix/store/57rsy3z79kvxah2l4j13bjwpsgxkbhby-source/pkgs/stdenv/darwin/override-sdk.nix:310:5:
309| topologicallyOrderedPropagatedDependencies =
310| pkgs:
| ^
311| let
error: cannot coerce a list to a string
This is on a Github runner. In a repl on that runner I can run :b pkgs.overrideSDK pkgs.stdenv "11.0" and it builds it without errors. I’ve tried pkgs.overrideSDK pkgs.stdenv "10.12" to check the expression and nix flake check succeeds.
When I try your suggestion locally on my aarch64-darwin machine, it works fine too.
Do you have a clue where the list to a string comes from?
Something’s build inputs are of an unexpected type. overrideSDK does some complicated stuff to rewrite the dependencies propagated to a package.
I cloned your PR and ran it with --debugger. It appears rustc has the following, but I don’t know where it’s getting it because it’s not in the nixpkgs rustc derivation.
nix-repl> pkg.name
"rust-default-1.77.2"
nix-repl> pkg.depsTargetTargetPropagated
[ [ ... ] ]
Wait a moment …. That’s not the nixpkgs rustc.
There it is. It should be optionals with a list, but it’s using optional.
I don’t know whether that’s valid, but it works without overrideSDK, so it should work when using overrideSDK. I’m testing a patch now to confirm it works.
It builds. I had to add darwin.apple_sdk.libs.xpc as a buildInput to the package. PR is up at overrideSDK: handle propagated lists of inputs by reckenrode · Pull Request #314167 · NixOS/nixpkgs · GitHub.
$ nix build .#packages.x86_64-darwin.launcher
$ ./result/bin/hc-launch --version
holochain_cli_launch 0.300.0-dev.2 (holochain 0.3.1-rc.0)
Amazing, a huge thank you!
Would it ideally be fixed upstream in the rust-overlay repo as well? What surprises me is that it works under aarch64-darwin. That would mean that overrideSDK receives different values on the two platforms.
Probably, but if Nix accepts the nested lists, then overrideSDK also needs to be able to handle them.
aarch64-darwin uses the 11.0 SDK by default, so overriding doesn’t do anything currently on that platform because it’s also the only SDK choice. That will change after the refactor, but exactly how is still to be determined.
aarch64-darwin uses the 11.0 SDK by default, so overriding doesn’t do anything currently on that platform because it’s also the only SDK choice. That will change after the refactor, but exactly how is still to be determined.
Got it.
Probably, but if Nix accepts the nested lists, then
overrideSDKalso needs to be able to handle them.
Right! A bit funny the optional function and its documentation. That means it takes any type. Alright, I’ll make a PR for the rust-overlay anyway.
I’m using your patched branch as nixpkgs input in the flake and added apple_sdk.libs.xpc. The flake compiles correctly now, but the package still does not build with the same error as before nix: use patched nixpkgs · holochain/holonix@6c46f6f · GitHub
You must have worked some trick there on your machine 
@reckenrode Please post the complete flake that let you build successfully. The debugger keeps me in an endless loop with error: QuickTime framework not available.
Sorry about not getting back to you. I was building against my ld64/cross/clang18 branch with the overrideSDK fix applied. I had started a build using your PR but got distracted by other things. This is the flake I’m using that built for me.
Sorry about not getting back to you.
Not at all! I’m happy about the help you’ve given me already.
I was building against my ld64/cross/clang18 branch with the
overrideSDKfix applied. I had started a build using your PR but got distracted by other things. This is the flake I’m using that built for me.
Thank you.
Mmh, I’m using the same flake but get the initial error still. Does that point to the compiler being the problem?
It could be that or (more likely) the updates required for the ld64 update. I had to find another source for the availability headers, which could make mixing headers from different SDK versions a little less problematic.
I see. I’ve tried to use your ld64 branch as nixpkgs input, but [the runner times out after 6 hrs of building](test: try with only ld64 · holochain/holonix@09ee24a · GitHub =\ It’s probably building every single of the nearly 2000 packages.
The upstream fix in the Rust overlay has been merged.
If you can offer any more help, I’d be grateful. Instructions for me to debug are welcome, or should I ask in the matrix chat or on Discord as well?
The ld64 branch involves changes that affect the bootstrap, so it takes a while to build. A full bootstrap of just the stdenv on aarch64-darwin is ~2 hours on my M3 Max.
Once the 24.05 release is out, which should be soon, I’ll be cherry-picking changes from my branch to separate PRs into staging and master. Unfortunately, the timing meant it was too close to release to do (especially since there are breaking changes in the ld64 upgrade).
Until those land, you may have to target only aarch64-darwin and mark x86_64-darwin as broken. The problem you’re encountering is that something is retaining the 10.12 libSystem, which has incomplete availability headers that don’t work with newer SDKs. It may be the replacement ones are why it builds on my system, but I don’t know.
Ideally, overrideSDK would work with Rust, but rustPlatform embeds references to various compilers in its hooks’ scripts, which are impossible to detect generally. rustPlatform would need modifying to be friendly to overriding the SDK, and I don’t know what the frameworks outside of nixpkgs would need.
I have an idea for how SDKs can be changed to make using a specific one less painful (at the cost of radically changing how SDKs are used in nixpkgs), but that work hasn’t started yet since my focus currently is on the ld64 upgrade, then Darwin cross-compilation, then doing the clang 18 update. I’m still hopeful it can be done this summer, but that stuff comes first.
Thank you for the explanation. I’m usually following nixos-unstable anyway, so IIUC as soon as your cherry-picked fixes are on master, I would benefit from them.
So other than overrideSDK, there’s no generalized way of making sure a particular version of the SDK is used. Well if rustPlatform is inconsistent in that area, that would be the place to fix, right?!
For the time being I’ll exclude this package for the x86_64-darwin platform and look forward to your updates. Is there an issue I can subscribe to or something like that?
One other possibility is using darwin.apple_sdk_11_0.callPackage, but it doesn’t do framework substitution or handle propagated inputs.
That’s the backup fix if the refactor doesn’t or can’t address the problem generally.
I could hardly believe it, but I got a working build eventually with stock nixpkgs. This comes from taking a legacy flake which is super complicated as a starting point. As background, I’m rewriting the flake with a flat and simple structure.
I’m still in the process of figuring out why exactly this is working. The main change I made is adding the SDK override and updating nixpkgs to a include a higher version of go to address a _SecTrustEvaluateWithError I got when using SDK 10_12 throughout.
There’s another bit which may have an influence holochain/nix/modules/darwin.nix at 00cf54944968c3bc1b5d9ce0ae9eb0c28529df2e · jost-s/holochain · GitHub.
Once I’ve come to understand it, I’ll report back.
This is as far as my understanding goes. I have a minimum working version of the legacy flake that successfully builds on aarch64-darwin, x86_64-darwin and x86_64-linux.
I think the go override is what did the trick. Can you identify anything that needs fixing upstream?
With the new flake I’ve been getting an error unrelated to symbols for aarch64-darwin. It builds correctly on Linux, but on aarch64-darwin in the installPhase it fails with an error xargs: signIfRequired: No such file or directory.
I have no clue where it comes from and even if I copy the working flake above to the new flake’s repo, I get the same error on aarch64-darwin. I couldn’t find anything relevant about this error. Do you know why that is happening?
[edit]
It’s to do with the flake.lock. Some input is causing a problem. I copied the working lock file over and it builds on aarch64-darwin. I’m replacing the inputs one by one to hone in on the faulty one.
[edit2]
Turns out that it was a problem with the revision of crane. I updated the flake input which fixed the build.
Go is already using the 11.0 SDK. It’s surprising that wrapping it would resolve the issue.