This thread is a place to share updates to Darwin separate from Nix š¤ macOS Monthly to avoid creating confusion about which work is sponsored and which is not.
10 Likes
This is a repost from Nix š¤ macOS Monthly - #50 by reckenrode .
NixOS:staging
ā reckenrode:system_cmds-970.0.4
opened 02:38AM - 16 Mar 24 UTC
## Description of changes
Update system_cmds to 970.0.4. I replaced the Xcodeā¦ build system with Meson for easier maintenance.
Tested building both aarch64-darwin and x86_64-darwin early in the bootstrap. Tested full bootstrap with aarch64-darwin.
## Things done
- Built on platform(s)
- [ ] x86_64-linux
- [ ] aarch64-linux
- [x] x86_64-darwin
- [x] aarch64-darwin
- For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] `sandbox = relaxed`
- [ ] `sandbox = true`
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://nixos.org/manual/nixpkgs/unstable/#sec-package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [x] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [24.05 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) (or backporting [23.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2305.section.md) and [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) Release notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://nixos.org/manual/nixpkgs/unstable/#chap-reviewing-contributions
-->
---
Add a :+1: [reaction] to [pull requests you find important].
[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/
[pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc
This PR updates system_cmds to 970.0.4 on both aarch64-darwin and x86_64-darwin. This version corresponds to the system_cmds in macOS 14, but they are built with the default SDKs and should work on older systems. Five binaries were lost in the update:
makekey - removed upstream
login - requires entitlements and non-public headers
sadc - removed upstream
sar - removed upstream
zprint - requires IOKitUser framework that isnāt packaged
Several more were added:
arch (aarch64-darwin only)
chpass
cpuctl
dynamic_pager
iosim
kpgo
ltop
memory_pressure
mslutil (aarch64-darwin only)
pagesize
passwd
proc_uuid_policy
purge
stackshot (aarch64-darwin only)
taskpolicy (aarch64-darwin only)
vm_purgeable_stat (aarch64-darwin only)
wait4path
wordexp-helper
zlog (aarch64-darwin only)
The aarch64-darwin only ones are due to the default SDK. If you override the SDK, they should build and work on x86_64-darwin. The following will also be available with the 12.0 SDK (once available):
lskq
lsmp
reboot - if/when IOKitUser framework is added
shutdown - if/when IOKitUser framework is added
2 Likes
Once the current staging-next cycle completes, Darwin will no longer require curl in the stdenv bootstrap build. It was only needed previously for fetchers, which can use the one bundled in the bootstrap tools tarball, and for hatch-vcs tests.
NixOS:staging
ā reckenrode:darwin-nocurl
opened 08:50PM - 17 Mar 24 UTC
The Darwin bootstrap currently requires curl, but it is not strictly required. Tā¦ he bootstrap requires it for two things:
* Fetchers; and
* As a transitive dependency of llvm-manpages (via Sphinx).
For the fetchers, the bootstrap curl can be used. For hatch-vcs, the dependency, its tests can be disabled. Doing this allows curl to be dropped from the Darwin stdenv bootstrap.
Closes #276762.
Tested by building the stdenvs against staging. I then changed the name of `curl` and did a new build, confirming that it only triggered a rebuild of curl and not the whole stdenv.
## Description of changes
<!--
For package updates please link to a changelog or describe changes, this helps your fellow maintainers discover breaking updates.
For new packages please briefly describe the package or provide a link to its homepage.
-->
## Things done
- Built on platform(s)
- [ ] x86_64-linux
- [ ] aarch64-linux
- [x] x86_64-darwin
- [x] aarch64-darwin
- For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] `sandbox = relaxed`
- [ ] `sandbox = true`
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://nixos.org/manual/nixpkgs/unstable/#sec-package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [x] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [24.05 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) (or backporting [23.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2305.section.md) and [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) Release notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://nixos.org/manual/nixpkgs/unstable/#chap-reviewing-contributions
-->
---
Add a :+1: [reaction] to [pull requests you find important].
[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/
[pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc
2 Likes
@a-n-n-a-l-e-e fixed the issue with linking libc++abi on Darwin. All of the hacks that were put in place during the LLVM 16 update and final staging-next cycle before the release of 23.11 have been removed.
NixOS:staging
ā a-n-n-a-l-e-e:libcxx-libcxxabi-merge
opened 09:47AM - 28 Feb 24 UTC
## Description of changes
- merge libcxxabi into libcxx for LLVM 12, 13, 14, ā¦ 15, 16, 17, and git.
- remove the link time workaround `-lc++ -lc++abi` from 58 packages as it is no longer required.
- fixes https://github.com/NixOS/nixpkgs/issues/166205
- provides alternative fixes for. https://github.com/NixOS/nixpkgs/issues/269548 https://github.com/NixOS/nix/issues/9640
- pkgsCross.x86_64-freebsd builds work again
- merge should be squashed
This change can be represented in 3 stages
1. merge libcxxabi into libcxx -- files: pkgs/development/compilers/llvm/[12, git]/{libcxx, libcxxabi}
2. update stdenv to account for merge -- files: stdenv.{adapters, cc.wrapper, darwin}
3. remove all references to libcxxabi outside of llvm (about 58 packages modified)
### merging libcxxabi into libcxx
- take the union of the libcxxabi and libcxx cmake flags
- eliminate the libcxx-headers-only package - it was only needed to break libcxx <-> libcxxabi circular dependency
- libcxx.cxxabi is removed. external cxxabi (freebsd) will symlink headers / libs into libcxx.
- darwin will re-export the libcxxabi symbols into libcxx so linking `-lc++` is sufficient.
- linux/freebsd `libc++.so` is a linker script `LINK(libc++.so.1, -lc++abi)` making `-lc++` sufficient.
- libcxx/default.nix [12, 17] are identical except for patches and `LIBCXX_ADDITIONAL_LIBRARIES` (only used in 16+)
- git/libcxx/defaul.nix does not link with -nostdlib when useLLVM is true so flag is removed. this is not much different than before as libcxxabi used -nostdlib where libcxx did not, so libc was linked in anyway.
### stdenv changes
- darwin bootstrap, remove references to libcxxabi and cxxabi
- cc-wrapper: remove c++ link workaround when libcxx.cxxabi doesn't exist (still exists for LLVM pre 12)
- adapter: update overrideLibcxx to account for a pkgs.stdenv that only has libcxx
### 58 package updates
- remove `NIX_LDFLAGS = "-l${stdenv.cc.libcxx.cxxabi.libName}` as no longer needed
- swift, nodejs_v8 remove libcxxabi references in the clang override
Supersedes https://github.com/NixOS/nixpkgs/pull/282624
tests bellow all pass:
<details>
<summary>linux x64 builds</summary>
```nix
{pkgs ? import ./. {}}:
pkgs.mkShell {
packages = let
inherit (pkgs) lib;
versions = map toString (lib.range 12 17) ++ [ "git" ];
in with pkgs; (
(map (ver: pkgsCross.wasi32.${"llvmPackages_${ver}"}.libcxx) versions)
++ (map (ver: pkgsLLVM.${"llvmPackages_${ver}"}.libcxx) versions)
++ (map (ver: pkgs.${"llvmPackages_${ver}"}.libcxx) versions)
++ (map (ver: pkgsMusl.${"llvmPackages_${ver}"}.libcxx) versions)
++ (map (ver: pkgsCross.x86_64-freebsd.${"llvmPackages_${ver}"}.libcxx) versions)
);
```
</details>
<details>
<summary>linux / darwin x64 builds</summary>
```nix
{pkgs ? import ./. {}}:
let
inherit (pkgs) lib stdenv;
libcxxabi-removed = (with pkgs; ([
ast-grep
bazel_7
beancount-language-server
bfc
blackmagic-desktop-video
boolector
bpf-linker
cbfmt
cdrtools
citrix_workspace
crystal
difftastic
dolphin-emu
done
dump_syms
espanso
firefox
flow
frawk
gcs
geos
harmonia
hashcat
haskellPackages.hercules-ci-cli
jemalloc
kanata
kotatogram-desktop
ladybird
lean4
lib2geom
libserdes
lua-language-server
mold
nickel
nix-du
nodejs_18
nova-filters
opam
open-watcom-v2
osl
ouch
p4
pjsip
postgresql15Packages.postgis
proj
prusa-slicer
python3Packages.uamqp
qdrant
rust-code-analysis
snapdragon-profiler
sonic-server
swift
symbolicator
teamspeak_client
tectonic
topiary
unar
usql
]));
libcxxabi-removed-env = pkgs.buildEnv {
name = "libcxxabi-removed-env";
paths = lib.filter (p: lib.meta.availableOn stdenv p) libcxxabi-removed;
};
channel-blockers-env = pkgs.buildEnv {
name = "channel-blockers-env";
paths = lib.filter (p: lib.meta.availableOn stdenv p) (with pkgs; [
cabal2nix cachix emacs ghc gimp git go inkscape mariadb nix-info-tested
nix-info nix nixpkgs-review openssh openssl pandoc postgresql python3
qt5.qtmultimedia ruby rustc stdenv # self.checks.${system}.tarball
transmission-gtk vim
]);
};
in
pkgs.mkShell {
packages = [ libcxxabi-removed-env channel-blockers-env ]
++ lib.filter (p: lib.meta.availableOn stdenv p) (
with pkgs; [ crystal_1_2 crystal_1_7 crystal_1_8 crystal_1_9 ]
) ;
}
```
</details>
<details>
<summary>dlopen and link tests</summary>
```nix
# shell.nix
{pkgs ? import ./. {}}:
let
dltest = pkgs.callPackage (
{ lib
, runCommandCC
, writeText
, libcxx
}:
let
name = "dltest";
code = writeText "${name}.c" ''
#include <dlfcn.h>
#include <stdio.h>
int
main(int argc, const char **argv)
{
int numFailed = 0;
for (int i = 1; i < argc; i++) {
int failed = dlopen(argv[i], RTLD_LOCAL | RTLD_LAZY) == 0;
numFailed += failed;
fprintf(stderr, "dlopen %s: %s\n", argv[i], failed ? "failed" : "succeeded");
}
return numFailed;
}
'';
in
runCommandCC name {src = code;} ''
mkdir -p "$out"/bin
cc -Wall -Werror -O2 -o "$out/bin/${name}" "$src" -ldl -L${lib.getLib libcxx}/lib -lc++
''
) {
libcxx = pkgs.llvmPackages_git.libcxx;
};
linkTest = pkgs.callPackage (
{ runCommandWith
, writeText
, name
, stdenv
}:
let
runCommandCC = name: env: runCommandWith {
stdenv = stdenv;
runLocal = false;
inherit name;
derivationArgs = env;
};
code = writeText "${name}.cxx" ''
#include <iostream>
int
main(void)
{
std::cout << "${name} succeeded" << std::endl;
return 0;
}
'';
in
runCommandCC name {src = code;} ''
mkdir -p "$out"/bin
cd "$(mktemp -d)"
$CXX $src -c -o main.o
$CC main.o -o "$out/bin/${name}" -lc++
''
);
buildTest = pkgs.callPackage (
{ runCommandWith
, writeText
, name
, stdenv
}:
let
runCommandCC = name: env: runCommandWith {
stdenv = stdenv;
runLocal = false;
inherit name;
derivationArgs = env;
};
code = writeText "${name}.cxx" ''
#include <iostream>
int
main(void)
{
std::cout << "${name} succeeded" << std::endl;
return 0;
}
'';
in
runCommandCC name {src = code;} ''
mkdir -p "$out"/bin
$CXX $src -o "$out/bin/${name}"
''
);
shell =
let
inherit (pkgs) lib stdenv mkShell writeShellScriptBin;
ext = stdenv.hostPlatform.extensions.sharedLibrary;
#versions = [ "12" "13" "14" "15" "16" "17" "git" ];
versions = map toString (lib.range 12 17) ++ [ "git" ];
getLibcxx = libcxx: "${lib.getLib libcxx}/lib/libc++${ext}" + lib.optionalString stdenv.isLinux ".1";
getStdenv = pfx: if stdenv.isLinux then pfx.libcxxStdenv else pfx.stdenv;
getLinkTest = ver:
linkTest {
name = "main${ver}";
stdenv = getStdenv pkgs.${"llvmPackages_${ver}"};
};
getBuildTest = ver:
buildTest {
name = "mainxx${ver}";
stdenv = getStdenv pkgs.${"llvmPackages_${ver}"};
};
in
mkShell {
packages = (map getLinkTest versions) ++
(map getBuildTest versions) ++ [
dltest
(writeShellScriptBin "runtests" ''
set -e
${lib.getBin dltest}/bin/dltest \
${toString (map (ver: getLibcxx pkgs.${"llvmPackages_${ver}"}.libcxx) versions)}
for ver in ${toString versions}; do
mainxx$ver
main$ver
done
'')
];
shellHook = "runtests";
};
in
shell
```
</details>
<!--
### fold libcxxabi into libcxx package
- stand alone libcxxabi is unsupported https://github.com/llvm/llvm-project/issues/78860
- stand alone libcxxabi resulted in bugs https://github.com/NixOS/nixpkgs/issues/166205 https://github.com/NixOS/nixpkgs/issues/269548 https://github.com/NixOS/nix/issues/9640
- no packages use libcxxabi independent of libcxx
- building libcxx and libcxxabi together is supported and recommended
### building together
- for darwin. cmakeFlags are simply `-DLLVM_ENABLE_RUNTIMES="libcxxabi;libcxx" -DLIBCXX_CXX_ABI=libcxxabi`
- flags for other build types are folded into the libcxx package
- remove libcxx header package -- only needed to break libcxx/libcxxabi circular dependency
### pkgs/stdenv/darwin/default.nix
- remove all references to libcxxabi
- set cxxabi = null
### pkgs/stdenv/adapters.nix overrideLibcxx
-
building libcxxabi and libcxx together solves the bugs without requiring any cmakefile changes. For darw
the bulk of the changes are in:
- pkgs/compilers/llvm/[15-git] (merged cmakeFlags from libcxxabi into libcxx). the changes between versions are nearly identical.
- pkgs/stdenv/adapters.nix overrideLibcxx
- pkgs/stdenv/darwin/default.nix delete all references to libcxxabi, make cxxabi null.
- pkgs/build-support/cc-wrapper/default.nix remove c++ hack for libc++abi when cxxabi is null.
- rest of files are removing workarounds needed to link libc++abi when using clang w/ libcxx
--!>
<!--
For package updates please link to a changelog or describe changes, this helps your fellow maintainers discover breaking updates.
For new packages please briefly describe the package or provide a link to its homepage.
-->
## Things done
- Built on platform(s)
- [ ] x86_64-linux
- [ ] aarch64-linux
- [ ] x86_64-darwin
- [ ] aarch64-darwin
- For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] `sandbox = relaxed`
- [ ] `sandbox = true`
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://nixos.org/manual/nixpkgs/unstable/#sec-package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [ ] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [24.05 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) (or backporting [23.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2305.section.md) and [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) Release notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [ ] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://nixos.org/manual/nixpkgs/unstable/#chap-reviewing-contributions
-->
---
Add a :+1: [reaction] to [pull requests you find important].
[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/
[pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc
5 Likes
The darwin
attrset now supports aliases, which will allow packages to be moved and deprecated.
NixOS:master
ā reckenrode:darwin-aliases
opened 01:39AM - 27 Mar 24 UTC
## Description of changes
Darwin needs a way to deprecate and move packages iā¦ n the `darwin` scope. This is needed to support transitioning to upstream packages for libiconv and icu. Itās also useful for moving things out of `darwin` to `by-name`.
This PR also moves two packages to darwin-aliases.nix:
* builder (moved from an ad hoc alias)
* ~~moltenvk (moved to `by-name`)~~
## Things done
- Built on platform(s)
- [ ] x86_64-linux
- [ ] aarch64-linux
- [x] x86_64-darwin
- [x] aarch64-darwin
- For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] `sandbox = relaxed`
- [ ] `sandbox = true`
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://nixos.org/manual/nixpkgs/unstable/#sec-package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [ ] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [24.05 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) (or backporting [23.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2305.section.md) and [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) Release notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#reviewing-contributions
-->
---
Add a :+1: [reaction] to [pull requests you find important].
[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/
[pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc
2 Likes
Darwin is now using the upstream libiconv. It is built with a compatible ABI, so it can be used with install_name_tool
to replace references to the system libiconv with one from nixpkgs.
Update: This PR had to be reverted. It caused libgit2ās tests to fail due to a dependency on a Mac-specific UTF-8 encoding for handling precomposed characters. I have a replacement in the works, but it needs to be reviewed and merged first before Iāll post it here.
NixOS:staging
ā reckenrode:libiconv-switch
opened 12:11AM - 28 Mar 24 UTC
## Description of changes
With https://github.com/NixOS/nixpkgs/pull/238993 aā¦ nd https://github.com/NixOS/nixpkgs/pull/299360, it is now possible to deprecate darwin.libiconv and switch Darwin to using upstream libiconv with a compatible ABI. This PR does that and updates packages that were using `darwin.libiconv` to use just `libiconv`.
Since this affects the stdenv, Iām targeting staging. I tested by building the stdenv on both platforms and GHC on aarch64-darwin. GHC uses `install_name_tool` to swap libiconv from the system one to the one in nixpkgs, so that it built successfully is a good test of the ABI compatibility implemented. I also confirmed eval locally using ofborgās `outpaths.nix`.
## Things done
- Built on platform(s)
- [ ] x86_64-linux
- [ ] aarch64-linux
- [x] x86_64-darwin
- [x] aarch64-darwin
- For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] `sandbox = relaxed`
- [ ] `sandbox = true`
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://nixos.org/manual/nixpkgs/unstable/#sec-package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [x] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [24.05 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) (or backporting [23.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2305.section.md) and [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) Release notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#reviewing-contributions
-->
---
Add a :+1: [reaction] to [pull requests you find important].
[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/
[pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc
2 Likes
A rewrite of overrideSDK
was merged into staging today. This version improves the robustness of overrideSDK
to handle all inputs and (hopefully) play nicely with cross-compilation. It also sets the SDK root, which results in a binary being linked with the correct SDK version. On master, binaries will be linked with the SDK set to the deployment target due to how clang infers the SDK version.
Setting the SDK version correctly is important for two reasons:
macOS changes behavior or enables functionality depending on the linked SDK. For example, dark mode support requires a newer SDK even with an older back deployment target; and
It affects how the OS version is reported. I am working on adding patches for D3DMetal support in the nixpkgs version of Wine. D3DMetal requires macOS 14. The version check fails when linked with the current stdenv because SDK versions older than 11 return a compatibility version it doesnāt recognize (10.20 instead of 14.x).
It is planned to update rustPlatform
to handle overriding better than it does currently. This is needed for applications like wezterm, neovide, and lapce to build currently with the newer SDK. After that, Iāll be updating the documentation to reflect a preference for overrideSDK
over darwin.apple_sdk_11_0.callPackage
.
Note: overrideSDK
is a stdenv adapter. Use it as follows. The attrset can be replaced with the desired SDK version for brevity. overrideSDK
only supports Darwin, so you will need to make it conditional on stdenv.isDarwin
if you need to support other platforms.
The version number will become a constraint in the future. It will match the newest version up to the next major version. "11.0"
is the same as "^11.0"
(though the latter syntax is not currently supported). When the 11.0 SDK is updated to the 11.3 SDK, "11.0"
(or "^11.0"
will match 11.3. Currently, you should use "11.0"
. The deployment target is passed through as-is.
stdenv = overrideSDK stdenv {
darwinMinVersion = <deployment target>;
darwinSdkVersion = <SDK version>;
};
# or (using the platformās default deployment target)
stdenv = overrideSDK stdenv "11.0";
# or (to set just the deployment target)
stdenv = overrideSDK stdenv { darwinMinVersion = "10.7"; }
NixOS:staging
ā reckenrode:darwin-stdenv-improvements
opened 11:30PM - 09 Feb 24 UTC
## Description of changes
This PR makes several improvements to the Darwin stā¦ denv situation.
### Adds SDK root
It adds the SDK version to the stdenv as a new `darwin.apple_sdk.sdkRoot` package. This is done via a hook that adds `-isysroot` with a path to a stub SDK that just defines version information (based on the `xcbuild.sdk`). This is needed because clang passes `-platform_version <deployment target> <sdk version>` unconditionally to the linker, and the SDK version is inferred to be the same as the deployment target if `-isysroot` or `SDKROOT` do not point to SDK from which it can get the version.
macOS implements different behaviors depending on the SDK version. There are two notable examples: dark mode and version checks.
macOS requires the linked SDK to be at least 10.14 for dark mode to work. Currently, x86_64-darwin binaries in nixpkgs do not work with dark mode. With the SDK root specified, clang will pass the correct SDK version, ensuring that binaries linked against the 11.0 SDK work as expected.
For version checks, macOS returns a compatible version when the linked SDK is older than 11.0. I ran into that issue while working on https://github.com/NixOS/nixpkgs/issues/236414. MetalD3D checks the macOS version, but because the load commands in Wineās binary indicate an older SDK, macOS returns a ācompatibleā version: 10.20. Since the check is for 14, the check fails, and MetalD3D does not work as expected.
Closes #265139.
### Fleshes out `overrideSDK`
This PR includes a rewrite of `overrideSDK`, addressing most of the issues identified in https://github.com/NixOS/nixpkgs/issues/242666#issuecomment-1803866452. This was necessary to support building Rust applications such as lapce, neovide, and wezterm on x86_64-darwin using the 11.0 SDK. It is also necessary to fix building applications that use `wrapGAppsHook` and the 11.0 SDK.
According to ofborgās evalulation performance check, the rewritten `overrideSDK` performs about the same as the previous version even though it is more capable: it supports replacing inputs of all types, supports additional SDK-based overrides (libobjc, libs, private frameworks), and should be easier to maintain. It should also work with cross-compilation, but this is untested due to needing https://github.com/NixOS/nixpkgs/pull/256590 to fix cross-compilation support.
I plan to address overriding the SDK for Rust in a separate PR. There are some changes to `rustPlatform` needed to make it work robustly with overriding the SDK. Documentation will also follow in a separate PR.
<!--
For package updates please link to a changelog or describe changes, this helps your fellow maintainers discover breaking updates.
For new packages please briefly describe the package or provide a link to its homepage.
-->
## Things done
- Built on platform(s)
- [ ] x86_64-linux
- [ ] aarch64-linux
- [x] x86_64-darwin
- [x] aarch64-darwin
- For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] `sandbox = relaxed`
- [ ] `sandbox = true`
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://nixos.org/manual/nixpkgs/unstable/#sec-package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [ ] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [24.05 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) (or backporting [23.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2305.section.md) and [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) Release notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://nixos.org/manual/nixpkgs/unstable/#chap-reviewing-contributions
-->
---
Add a :+1: [reaction] to [pull requests you find important].
[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/
[pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc
6 Likes
The default version of GCC on Darwin will be GCC 13 after the next staging-next cycle.
NixOS:staging
ā wegank:gcc-13-everywhere
opened 04:15AM - 27 Jan 24 UTC
## Description of changes
I imagine this would rebuild `liblapack`, since `gfā¦ ortran` is now at 13. An easy test would then be to see if `python3Packages.numpy` still works. (EDIT: OK)
Also Wine according to #283642. (EDIT: OK)
## Things done
- Built on platform(s)
- [ ] x86_64-linux
- [ ] aarch64-linux
- [ ] x86_64-darwin
- [x] aarch64-darwin
- For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] `sandbox = relaxed`
- [ ] `sandbox = true`
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://nixos.org/manual/nixpkgs/unstable/#sec-package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [ ] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [24.05 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) (or backporting [23.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2305.section.md) and [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) Release notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://nixos.org/manual/nixpkgs/unstable/#chap-reviewing-contributions
-->
---
Add a :+1: [reaction] to [pull requests you find important].
[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/
[pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc
2 Likes
This one is more of an announcement than news. 24.11 will be updating the SDK on x86_64-darwin. This is anticipated to be done after the SDK refactor lands post-24.05. The specific version has not yet been determined. It will likely be either 10.13 or 10.14. Which version will be based on the needs of software in the bootstrap (e.g., LLVM doesnāt really support 10.12 anymore, and Darwin will be updating to LLVM 18 for 24.11).
7 Likes
I am working on updates to cctools, ld64, and libtapi. While itās possible they may land for 24.05, itās very likely they wonāt make the freeze in time. I wanted to post something because this is a potentially bit update, and I havenāt posted news in a while.
Packaging Changes
cctools, ld64, and libtapi are now built from Appleās OSS distributions repo instead of the cctools-port and apple-libtapi repos.
These versions (cctools 1010.6, ld64 951.9, libtapi 1500.0.12.3) correspond to the versions shipped with Xcode 15. cctools and ld64 are current as of Xcode 15.3. Packages should no longer have to patch around missing flags or feature support.
Restoring Linux support is a WIP. I have made some changes to use non-private and non-portable APIs, but I am currently focused on getting Darwin updates first.
The packages have been moved to the by-name
hierarchy. Aliases are provided to the old names, but there are some breaking changes (see below).
cctools is now only cctools. This is technically a breaking change (see below). ld64 has been moved to its own, separate package.
darwin.binutils is now the package that defines the mix of cctools and LLVM tools. It has been update with additional tools. See below for the breakdown of what it contains.
The build system has been replaced with Meson. I did this because dealing with xcbuild stinks, and these packages eventually need to support Linux for cross-compilation.
corecrypto and CommonCrypto hashing APIs are reimplemented using OpenSSL on both Darwin (and eventually) Linux. cctools-port only does this on non-Darwin platforms.
libtapi has been split into separate out
and lib
outputs. Most packages only need libtapi.dylib
, which is ~500 KiB compared to ~30 MiB for the tapi
binary.
libtapi is linked against zlib. I donāt know what it uses zlib to do, but the upstream libtapi is also linked against zlib, and I want to match what upstream is doing.
Note: ld-prime is will not be packaged because it is not currently included in the source releases. While I think itās unlikely, ld-prime will be added separately as a new package should the source be released.
New Features
LTO support. With this update, LTO works. I had to patch ld64 to look for libLTO.dylib
in the store and clang to pass the correct path (see llvmPackages_{9,12,13,14,15,16,17,18,git}.clang: fix libLTO.dylib path by reckenrode Ā· Pull Request #302481 Ā· NixOS/nixpkgs Ā· GitHub ). I bootstrap ld64 and link it with itself with LTO enabled to make sure that LTO actually works. Fixes Clang link time optimization not working on OSX Ā· Issue #19098 Ā· NixOS/nixpkgs Ā· GitHub .
Code directory support. cctools is built with libcodedirectory support, which allows strip
and install_name_tool
to identify linker-created signatures and update them automatically. This removes the need for postLinkSignHook
and actually makes it harmful to continue using it by default (because it prevents automatic signature updates by replacing them with a non-linker-created signature).
Breaking Changes
ld64 and cctools use GCD (libdispatch) for parallelism. This should improve performance, but it technically ups the required macOS version to 10.14. The APIs it uses are present under different names in 10.12, so I have attempted to patch around them. It builds with the 10.12 SDK, but it will need testing to confirm the tools actually work. This was the driver of updating the default SDK on x86_64-darwin for 24.11.
Apple removed support for nested static archives from ld64. This happened with the release of Xcode 14, so it has been out in the ecosystem for a few years. Hopefully packages have adapted by now. For example, Kyua adds libutils.a
to libcli.a
using libcli_a_LIBADD
in automake. This will fail to link with ld64 951.9.
I have chosen not to build GNU as in cctools. It is deprecated upstream. The default in nixpkgs is to use the clang integrated assembler (matching upstream). I am aware that GNAT is documented as requiring GNU as for its bootstrap. I plan to investigate and make changes accordingly to make sure GNAT still builds.
Update: This has changed. GNU as is built, but it is now found in the gas
output on cctools. darwin.binutilsDualAs has been redefined as a buildEnv
of darwin.binutils and cctools.gas, so packages that need both should continue to build. Those that need gas alone can add cctools.gas as an input.
cctools only provides binaries from cctools. ld64 is a separate package now.
darwin.cctools-apple, darwin.cctools-port, and darwin.cctools-llvm are replaced by just cctools. cctools-llvm will be an alias to darwin.bintools.
The old names canāt be used in nixpkgs because aliases are disabled when ofborg eval is run. The PR updating the tools will include changes for all packages using the old names. (Updates for LTO will be done in a separate PR or PRs.)
Use of postLinkSignHook
has been removed from bintools-wrapper. Because it removes linker-created signatures, it breaks the automatic updating of signatures by strip
and install_name_tool
. Unless youāre manually invoking codesign
, you shouldnāt be affected. install_name_tool
will do the right thing by default.
strip
and install_name_tool
are no longer wrapped. If you are modifying linker-signed binaries, which should be the typical case, they will update the signatures automatically. If you are manually invoking codesign
, you will need to update the signatures manually after running strip
or install_name_tool
.
stdenv updates
Stage 2 has been collapsed from two parts into one stage. The source-based CF is not coming back. It adds a lot of complexity to the bootstrap.
As noted in breaking changes, the Darwin stdenv no longer provides bintools using postLinkSignHook
. Once the bootstrap tools are updated with these releases, it may be possible to drop sigtool completely from the bootstrap. Until that happens, early stages manually implement postLinkSignHook
because the bootstrap tools ld64 does not appear capable of signing binaries on its own.
As part of this update, I have to update the stdenv to use the new paths.
The overall stdenv size is down by ~20 MiB due to dropping tapi
from the closure. It also drops signtool and signingUtils, but those binaries are small.
curl and GNU binutils throw when used during the bootstrap. curl should never be used during the bootstrap (to allow it to be updated without triggering a whole rebuild of Darwin). GNU binutils generally shouldnāt be used on Darwin (e.g., Rust 1.77 will clash harder with GNU binutils on Darwin Nix Ā· Issue #299606 Ā· NixOS/nixpkgs Ā· GitHub ), so the bootstrap shouldnāt need to use it.
darwin.bintools changes
Variants of darwin.bintools will likely be removed. There is only darwin.bintools and darwin.binutils (for wrapped versions similar to top-level binutils). If GNAT still requires it, something will be done (tbd) to ensure it still has access to GNU as.
Most packages are from LLVM. c++filter is no longer linked from GNU binutils. It is linked from LLVM. A handful of packages are linked from cctools because LLVM does not provided drop-in replacements for them. The lists of tools are exported as attributes on darwin.bintools (llvm_cmds
and cctools_cmds
respectively). All binaries are symlinked to their traditional names (e.g., llvm-strip
is symlinked to strip
).
LLVM tools: addr2line, ar, c++filt, dsymutil, lipo, install-name-tool, nm, objcopy, objdump, otool, size, strings, strip
cctools tools: codesign_allocate, gprof, ranlib (and ld from ld64)
Darwin is still isCCTools
and not isLLVM
because it does not use lld by default. Maybe that could change with lld 18, but Iām doubtful.
Iāll announce here once I have a branch available for testing. Iāll also see if I can get a Hydra job for it because I want to check for any build failures.
18 Likes
As a follow up to Darwin Updates News - #6 by reckenrode , libiconv has been updated for Darwin to libiconv-99, which matches the version of libiconv shipped in macOS 14.4. Updating Appleās libiconv was necessary because of the need for the special, Mac-specific UTF-8 encoding. This libiconv implementation is based on the one from FreeBSD.
In addition to the libiconv update, three packages have been introduced that may be of interest to FreeBSD users: ATF , Kyua , and Lutok . The versions of these packages in nixpkgs should correspond to the current versions in the FreeBSD ports tree.
NixOS:staging
ā reckenrode:libiconv-switch-mk2
opened 09:10PM - 03 Apr 24 UTC
## Description of changes
This is another attempt at https://github.com/NixOSā¦ /nixpkgs/pull/299613. The reasoning behind https://github.com/NixOS/nixpkgs/pull/238993 turned out to be mistaken. In the time between that PR and today, Apple has made several source releases. As it turns out, they switched their libiconv implementation from GNU libiconv to the implementation in FreeBSDās libc. Updating libiconv proved to be non-trivial for a few reasons.
* Upstream bugs. Appleās implementation had bugs that caused libarchive test failures (leading to my adding a test case as I was working on diagnosing before a source release on Monday included a fix);
* Build system issues. xcbuild cannot build libiconv correctly. The install names are a mess. It doesnāt handle tests are installation. Fixing would require relinking or hacking up the Xcode build project; and
* Packaging ATF, Kyua, and Lutok. These are requirements for libiconvās tests. They also mutually depend upon each other and have had releases in years. I opted to pin them to the same commits that FreeBSDās ports does.
Fortunately, those issues were solvable and should not be an issue for updates going forward. libiconv now has a test suite, and the work to make the other dependencies build should be a one time cost for packaging them. I built for both aarch64-darwin and x86_64-darwin, and I confirmed that libarchive and libgit2 build and pass their tests.
The Linux builds were done on ATF, Kyua, and Lutok. I tried to build libiconv-darwin on Linux, but it failed. It may be possible with effort to make it build, but it seems rather pointless since glibc includes a libiconv implementation.
## Things done
- Built on platform(s)
- [x] x86_64-linux
- [ ] aarch64-linux
- [x] x86_64-darwin
- [x] aarch64-darwin
- For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] `sandbox = relaxed`
- [ ] `sandbox = true`
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://nixos.org/manual/nixpkgs/unstable/#sec-package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [x] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [24.05 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) (or backporting [23.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2305.section.md) and [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) Release notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#reviewing-contributions
-->
---
Add a :+1: [reaction] to [pull requests you find important].
[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/
[pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc
1 Like
I have opened the draft PR for the cctools and ld64 updates. I will be rebasing it weekly on staging and pushing once the Darwin blockers channel and my configs build, which takes about a day and a half on an M3 Max.
I plan to take a break from this and look at rebasing the Darwin cross PR on this branch then get back to cherry-picking some of the commits into separate PRs. The libunwind change will probably require an upstream PR to add the missing pkg-config file.
NixOS:staging
ā reckenrode:ld64
opened 02:09AM - 30 Apr 24 UTC
## Description of changes
This PR updates cctools and ld64. It also fixes relā¦ ated breakage. It is currently set to draft until 24.05 is released. I will be rebasing weekly against staging, resolving any merge conflicts then. Before updating the ld64 branch, I build my configs and the Darwin channel blockers to confirm no regressions.
The following notes are copied from https://discourse.nixos.org/t/darwin-updates-news/42249/10.
### Packaging Changes
* cctools, ld64, and libtapi are now built from Appleās [OSS distributions](https://github.com/apple-oss-distributions) repo instead of the [cctools-port](https://github.com/tpoechtrager/cctools-port) and [apple-libtapi](https://github.com/tpoechtrager/apple-libtapi) repos.
* These versions (cctools 1010.6, ld64 951.9, libtapi 1500.0.12.3) correspond to the versions shipped with Xcode 15. cctools and ld64 are current as of Xcode 15.3. Packages should no longer have to patch around missing flags or feature support.
* Restoring Linux support is a WIP. I have made some changes to use non-private and non-portable APIs, but I am currently focused on getting Darwin updates first.
* The packages have been moved to the by-name hierarchy. Aliases are provided to the old names, but there are some breaking changes (see below).
* cctools is now only cctools. This is technically a breaking change (see below). ld64 has been moved to its own, separate package.
* darwin.binutils is now the package that defines the mix of cctools and LLVM tools. It has been update with additional tools. See below for the breakdown of what it contains.
* The build system has been replaced with Meson. I did this because dealing with xcbuild stinks, and these packages eventually need to support Linux for cross-compilation.
* corecrypto and CommonCrypto hashing APIs are reimplemented using OpenSSL on both Darwin (and eventually) Linux. cctools-port only does this on non-Darwin platforms.
* libtapi has been split into separate out and lib outputs. Most packages only need libtapi.dylib, which is ~500 KiB compared to ~30 MiB for the tapi binary.
* libtapi is linked against zlib. I donāt know what it uses zlib to do, but the upstream libtapi is also linked against zlib, and I want to match what upstream is doing.
_Note:_ ld-prime is will not be packaged because it is not currently included in the source releases. While I think itās unlikely, ld-prime will be added separately as a new package should the source be released.
### New Features
* LTO support. With this update, LTO works. I had to patch ld64 to look for libLTO.dylib in the store and clang to pass the correct path (see https://github.com/NixOS/nixpkgs/pull/304350). I bootstrap ld64 and link it with itself with LTO enabled to make sure that LTO actually works.
* Code directory support. cctools is built with libcodedirectory support, which allows strip and install_name_tool to identify linker-created signatures and update them automatically. This removes the need for postLinkSignHook and actually makes it harmful to continue using it by default (because it prevents automatic signature updates by replacing them with a non-linker-created signature).
###
Breaking Changes
* ld64 and cctools use GCD (libdispatch) for parallelism. This should improve performance, but it technically ups the required macOS version to 10.14. The APIs it uses are present under different names in 10.12, so I have attempted to patch around them. It builds with the 10.12 SDK, but it will need testing to confirm the tools actually work. This was the driver of updating the default SDK on x86_64-darwin for 24.11.
* Apple removed support for nested static archives from ld64. This happened with the release of Xcode 14, so it has been out in the ecosystem for a few years. Hopefully packages have adapted by now. For example, Kyua adds libutils.a to libcli.a using libcli_a_LIBADD in automake. This will fail to link with ld64 951.9.
* ld64 is more picking about command-line arguments. Passing `-rpath` when merging Mach-O object files fails.
* GNU as has been moved to the cctools^gas output. `-q` and `-Q` flags are supported to control whether it calls GNU as or the clang assembler (matching the upstream behavior of `as`).
* cctools only provides binaries from cctools. ld64 is a separate package now.
* darwin.cctools-apple, darwin.cctools-port, and darwin.cctools-llvm are replaced by just cctools. cctools-llvm is an alias to darwin.bintools.
* The old names canāt be used in nixpkgs because aliases are disabled when ofborg eval is run. The PR updating the tools will include changes for all packages using the old names. (Updates for LTO will be done in a separate PR or PRs.)
* Use of postLinkSignHook has been removed from bintools-wrapper. Because it removes linker-created signatures, it breaks the automatic updating of signatures by strip and install_name_tool. Unless youāre manually invoking codesign, you shouldnāt be affected. install_name_tool will do the right thing by default.
* `strip` and `install_name_tool` are no longer wrapped with `sigtool`. If you are modifying linker-signed binaries, which should be the typical case, they will update the signatures automatically. If you are manually invoking `codesign`, you will need to update the signatures manually after running `strip` or `install_name_tool`.
### stdenv updates
* Stage 2 has been collapsed from two parts into one stage. The source-based CF is not coming back. It adds a lot of complexity to the bootstrap.
* As noted in breaking changes, the Darwin stdenv no longer provides bintools using postLinkSignHook. Once the bootstrap tools are updated with these releases, it may be possible to drop sigtool completely from the bootstrap. Until that happens, early stages manually implement postLinkSignHook because the bootstrap tools ld64 does not appear capable of signing binaries on its own.
* As part of this update, I have to update the stdenv to use the new paths.
* The overall stdenv size is down by ~20 MiB due to dropping tapi from the closure. It also drops signtool and signingUtils, but those binaries are small.
* curl and GNU binutils throw when used during the bootstrap. curl should never be used during the bootstrap (to allow it to be updated without triggering a whole rebuild of Darwin). GNU binutils generally shouldnāt be used on Darwin (e.g., https://github.com/NixOS/nixpkgs/issues/299606), so the bootstrap shouldnāt need to use it.
### darwin.bintools changes
darwin.bintools changes
* c++filter is no longer linked from GNU binutils. It is linked from LLVM. A handful of packages are linked from cctools because LLVM does not provided drop-in replacements for them. The lists of tools are exported as attributes on darwin.bintools (`llvm_cmds` and `cctools_cmds` respectively). All binaries are symlinked to their traditional names (e.g., llvm-strip is symlinked to strip).
* LLVM tools: addr2line, ar, c++filt, dsymutil, lipo, install-name-tool, nm, objcopy, objdump, otool, size, strings, strip
* cctools tools: codesign_allocate, gprof, ranlib (and ld from ld64)
* Darwin is still `isCCTools` and not `isLLVM` because it does not use lld by default. Maybe that could change with lld 18, but Iām doubtful.
### Resolved Issues
* LTO works. Closes https://github.com/NixOS/nixpkgs/issues/19098.
* ld64 no longer crashes linking Haskell applications. Closes https://github.com/NixOS/nixpkgs/issues/149692.
## Things done
- Built on platform(s)
- [ ] x86_64-linux
- [ ] aarch64-linux
- [x] x86_64-darwin
- [x] aarch64-darwin
- For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] `sandbox = relaxed`
- [ ] `sandbox = true`
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [x] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [24.05 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) (or backporting [23.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2305.section.md) and [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) Release notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#reviewing-contributions
-->
---
Add a :+1: [reaction] to [pull requests you find important].
[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/
[pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc
7 Likes
Darwin will be updating to clang 18 for 24.11, so I have opened the issue to track updates. Depending on the timing, some of the updates may make it into 24.05 since they donāt cause breaking changes. The plan is to open the PR to update the version once I have all of the Darwin blockers building and 24.05 has released.
opened 08:21PM - 05 May 24 UTC
This is the tracking issue for this yearās LLVM update for Darwin. Linux is alreā¦ ady on LLVM 17, but both will be going to LLVM 18. My goal with this issue is to document common problems as well as their fixes. Iāve encountered a few breaking changes so far that have common solutions. Unlike last yearās update, the Darwin stdenv is in pretty good shape. Itās other packages that need fixed.
The target is to do the update early after the release of 24.05, so there is time to fix problems.
Previous tracking issue: https://github.com/NixOS/nixpkgs/issues/234710
### error: non-private labels cannot appear between .cfi_startproc / .cfi_endproc pairs
This is due to https://github.com/llvm/llvm-project/commit/d506aa4edfa66074db3dc1fa84da9d9c80d71500. It affects GCC and compiler-rt versions from LLVM 17 and earlier.
- [ ] https://github.com/NixOS/nixpkgs/pull/309405
### error: '\_\_abi_tag\_\_' attribute only applies to structs, variables, functions, and namespaces
This is a GCC bug when bootstrapping against newer versions of libc++. It is fixed upstream, but the patches are not yet available on a released version of GCC 6 through GCC 13.
### undefined symbol: `__availability_version_check`
LLVM 17 weakly links `_availability_verison_check`, but it is not available in the SDK until 10.15. This error shouldnāt happen because it has been fixed, but if it does, the fix is to revert back to finding the symbol at runtime using `dlsym`.
- [ ] https://github.com/NixOS/nixpkgs/pull/309385
### C23 attributes expose bugs in attribute macros
Clang 18 adds support for C23 attributes, which gnulib will use preferentially. Unfortunately, older versions of gnulib either donāt try or donāt detect correctly whether the compiler supports `inline` in headers. This results in trying to use `static [[__maybe_unused__]]`, which is not valid C23.
Fix this issue by cherry-picking the following commits from gnulib. Depending on how recent your packageās vendored gnulib is, you may only have to cherry-pick some of them.
* tbd
Another example is darwin.system_cmds, which attempts to use `static [[noreturn]]`. Even though `static _Noreturn` is valid in C11, `_Noreturn` is deprecated in C23, and `[[noreturn]]` is intended to replace `_Noreturn`, `static [[noreturn]]` is not valid C23. The fix is to use `[[noreturn]] static` instead.
3 Likes
The 12.3 SDK has been added to master.
It is usable via overrideSDK
. The apple_sdk.callPackage
pattern is not supported.
The overrideSDK
adapter is only for Darwin stdenvs. If a package supports Linux, the override should be done conditionally.
SDK frameworks should used from the unversioned apple_sdk.frameworks
. Versioned frameworks (e.g., apple_sdk_11_0.frameworks.CoreAudio
) are not overridden by overrideSDK
.
For an example of a package using the 12.3 SDK, see MoltenVK, which has been updated to use it.
Requesting an SDK version:
stdenv = overrideSDK stdenv "12.3";
Requesting an SDK version with a deployment target.
stdenv = overrideSDK stdenv {
darwinMinVersion = "10.15";
darwinSdkVersion = "12.3";
};
Note: This is an interim addition while the SDK refactor is being done to allow packages requiring a newer SDK to be built.
NixOS:master
ā reckenrode:darwin-sdk-12.3
opened 12:08AM - 12 Jul 24 UTC
## Description of changes
This PR extracts the 12.3 SDK from https://github.cā¦ om/NixOS/nixpkgs/pull/229210 by @ConnorBaker.
Note that it does not replace the refactor work I plan to do nor does it integrate the Apple OSS releases to provide headers, but I am adding the 12.3 SDK for packages that need it before the rework can land. To use the 12.3 SDK, use `overrideSDK`. See the MoltenVK derivation for an example. The `apple_sdk.callPackage` pattern is not supported.
Testing was done by updating MoltenVK to build with the 12.3 SDK.
```shell
$ nix build .#darwin.moltenvk^out
$ nix shell .#darwin.binutils --command otool -l ./result/lib/libMoltenVK.dylib | rg LC_BUILD_VERSION -A7
cmd LC_BUILD_VERSION
cmdsize 32
platform 1
sdk 12.3
minos 11.0
ntools 1
tool 3
version 951.9
```
```shell
$ nix build .#darwin.moltenvk^out --system x86_64-darwin
$ nix shell .#darwin.binutils --command otool -l ./result/lib/libMoltenVK.dylib | rg LC_BUILD_VERSION -A7
cmd LC_BUILD_VERSION
cmdsize 32
platform 1
sdk 12.3
minos 10.15
ntools 1
tool 3
version 951.9
```
```shell
$ diff -u 11.0.log 12.3.log # to show that it is using the version of MSL corresponding to the newer SDK.
--- 11.0.log 2024-07-11 20:05:24.096319472 -0400
+++ 12.3.log 2024-07-11 20:01:22.596312512 -0400
@@ -119,7 +119,7 @@
GPU memory available: 36864 MB
GPU memory used: 0 MB
supports the following Metal Versions, GPU's and Feature Sets:
- Metal Shading Language 2.3
+ Metal Shading Language 2.4
GPU Family Apple 7
GPU Family Apple 6
GPU Family Apple 5
```
## Things done
- Built on platform(s)
- [ ] x86_64-linux
- [ ] aarch64-linux
- [x] x86_64-darwin
- [x] aarch64-darwin
- For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] `sandbox = relaxed`
- [ ] `sandbox = true`
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [x] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [ ] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [24.11 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2411.section.md) (or backporting [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) and [24.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) Release notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#reviewing-contributions
-->
---
Add a :+1: [reaction] to [pull requests you find important].
[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/
[pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc
10 Likes
The cctools and ld64 update has been merged into staging. I will be following up with PRs to enable LTO on packages where it is disabled and to implement the treewide changes needed to move cctools to darwin-aliases.nix
.
NixOS:staging
ā reckenrode:ld64
opened 02:09AM - 30 Apr 24 UTC
## Description of changes
This PR updates cctools and ld64. It also fixes relā¦ ated breakage. It is currently set to draft until 24.05 is released. I will be rebasing weekly against staging, resolving any merge conflicts then. Before updating the ld64 branch, I build my configs and the Darwin channel blockers to confirm no regressions.
The following notes are copied from https://discourse.nixos.org/t/darwin-updates-news/42249/10.
### Packaging Changes
* cctools, ld64, and libtapi are now built from Appleās [OSS distributions](https://github.com/apple-oss-distributions) repo instead of the [cctools-port](https://github.com/tpoechtrager/cctools-port) and [apple-libtapi](https://github.com/tpoechtrager/apple-libtapi) repos.
* These versions (cctools 1010.6, ld64 951.9, libtapi 1500.0.12.3) correspond to the versions shipped with Xcode 15. cctools and ld64 are current as of Xcode 15.3. Packages should no longer have to patch around missing flags or feature support.
* Restoring Linux support is a WIP. I have made some changes to use non-private and non-portable APIs, but I am currently focused on getting Darwin updates first.
* The packages have been moved to the by-name hierarchy. Aliases are provided to the old names, but there are some breaking changes (see below).
* cctools is now only cctools. This is technically a breaking change (see below). ld64 has been moved to its own, separate package.
* darwin.binutils is now the package that defines the mix of cctools and LLVM tools. It has been update with additional tools. See below for the breakdown of what it contains.
* The build system has been replaced with Meson. I did this because dealing with xcbuild stinks, and these packages eventually need to support Linux for cross-compilation.
* corecrypto and CommonCrypto hashing APIs are reimplemented using OpenSSL on both Darwin (and eventually) Linux. cctools-port only does this on non-Darwin platforms.
* libtapi has been split into separate out and lib outputs. Most packages only need libtapi.dylib, which is ~500 KiB compared to ~30 MiB for the tapi binary.
* libtapi is linked against zlib. I donāt know what it uses zlib to do, but the upstream libtapi is also linked against zlib, and I want to match what upstream is doing.
_Note:_ ld-prime is will not be packaged because it is not currently included in the source releases. While I think itās unlikely, ld-prime will be added separately as a new package should the source be released.
### New Features
* LTO support. With this update, LTO works. I had to patch ld64 to look for libLTO.dylib in the store and clang to pass the correct path (see https://github.com/NixOS/nixpkgs/pull/304350). I bootstrap ld64 and link it with itself with LTO enabled to make sure that LTO actually works.
* Code directory support. cctools is built with libcodedirectory support, which allows strip and install_name_tool to identify linker-created signatures and update them automatically. This removes the need for postLinkSignHook and actually makes it harmful to continue using it by default (because it prevents automatic signature updates by replacing them with a non-linker-created signature).
###
Breaking Changes
* ld64 and cctools use GCD (libdispatch) for parallelism. This should improve performance, but it technically ups the required macOS version to 10.14. The APIs it uses are present under different names in 10.12, so I have attempted to patch around them. It builds with the 10.12 SDK, but it will need testing to confirm the tools actually work. This was the driver of updating the default SDK on x86_64-darwin for 24.11.
* Apple removed support for nested static archives from ld64. This happened with the release of Xcode 14, so it has been out in the ecosystem for a few years. Hopefully packages have adapted by now. For example, Kyua adds libutils.a to libcli.a using libcli_a_LIBADD in automake. This will fail to link with ld64 951.9.
* ld64 is more picking about command-line arguments. Passing `-rpath` when merging Mach-O object files fails.
* GNU as has been moved to the cctools^gas output. `-q` and `-Q` flags are supported to control whether it calls GNU as or the clang assembler (matching the upstream behavior of `as`).
* cctools only provides binaries from cctools. ld64 is a separate package now.
* darwin.cctools-apple, darwin.cctools-port, and darwin.cctools-llvm are replaced by just cctools. cctools-llvm is an alias to darwin.bintools.
* The old names canāt be used in nixpkgs because aliases are disabled when ofborg eval is run. The PR updating the tools will include changes for all packages using the old names. (Updates for LTO will be done in a separate PR or PRs.)
* Use of postLinkSignHook has been removed from bintools-wrapper. Because it removes linker-created signatures, it breaks the automatic updating of signatures by strip and install_name_tool. Unless youāre manually invoking codesign, you shouldnāt be affected. install_name_tool will do the right thing by default.
* `strip` and `install_name_tool` are no longer wrapped with `sigtool`. If you are modifying linker-signed binaries, which should be the typical case, they will update the signatures automatically. If you are manually invoking `codesign`, you will need to update the signatures manually after running `strip` or `install_name_tool`.
### stdenv updates
* Stage 2 has been collapsed from two parts into one stage. The source-based CF is not coming back. It adds a lot of complexity to the bootstrap.
* As noted in breaking changes, the Darwin stdenv no longer provides bintools using postLinkSignHook. Once the bootstrap tools are updated with these releases, it may be possible to drop sigtool completely from the bootstrap. Until that happens, early stages manually implement postLinkSignHook because the bootstrap tools ld64 does not appear capable of signing binaries on its own.
* As part of this update, I have to update the stdenv to use the new paths.
* The overall stdenv size is down by ~20 MiB due to dropping tapi from the closure. It also drops signtool and signingUtils, but those binaries are small.
* curl and GNU binutils throw when used during the bootstrap. curl should never be used during the bootstrap (to allow it to be updated without triggering a whole rebuild of Darwin). GNU binutils generally shouldnāt be used on Darwin (e.g., https://github.com/NixOS/nixpkgs/issues/299606), so the bootstrap shouldnāt need to use it.
### darwin.bintools changes
darwin.bintools changes
* c++filter is no longer linked from GNU binutils. It is linked from LLVM. A handful of packages are linked from cctools because LLVM does not provided drop-in replacements for them. The lists of tools are exported as attributes on darwin.bintools (`llvm_cmds` and `cctools_cmds` respectively). All binaries are symlinked to their traditional names (e.g., llvm-strip is symlinked to strip).
* LLVM tools: addr2line, ar, c++filt, dwarfdump, dsymutil, lipo, install-name-tool, nm, objcopy, objdump, otool, size, strings, strip
* cctools tools: codesign_allocate, gprof, ranlib (and ld from ld64)
* Darwin is still `isCCTools` and not `isLLVM` because it does not use lld by default. Maybe that could change with lld 18, but Iām doubtful.
### Resolved Issues
* LTO works. Closes https://github.com/NixOS/nixpkgs/issues/19098.
* ld64 no longer crashes linking Haskell applications. Closes https://github.com/NixOS/nixpkgs/issues/149692.
## Things done
- Built on platform(s)
- [ ] x86_64-linux
- [ ] aarch64-linux
- [x] x86_64-darwin
- [x] aarch64-darwin
- For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] `sandbox = relaxed`
- [ ] `sandbox = true`
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [x] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [24.05 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) (or backporting [23.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2305.section.md) and [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) Release notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#reviewing-contributions
-->
---
Add a :+1: [reaction] to [pull requests you find important].
[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/
[pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc
5 Likes
Two follow-ups to the cctools/ld64 update have been merged into staging.
The second PR also enables LTO for Python, which results in a ~10% performance improvement on py-bench.
NixOS:staging
ā reckenrode:cctools-darwin-aliases
opened 02:58AM - 18 Jul 24 UTC
## Description of changes
This PR is the first follow-up to https://github.coā¦ m/NixOS/nixpkgs/pull/307880. It finishes moving darwin.cctools to the top-level by adding darwin.cctools and darwin.cctools-port to darwin-aliases.nix.
<!--
For package updates please link to a changelog or describe changes, this helps your fellow maintainers discover breaking updates.
For new packages please briefly describe the package or provide a link to its homepage.
-->
## Things done
- Built on platform(s)
- [ ] x86_64-linux
- [ ] aarch64-linux
- [x] x86_64-darwin
- [x] aarch64-darwin
- For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] `sandbox = relaxed`
- [ ] `sandbox = true`
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [x] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [ ] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [24.11 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2411.section.md) (or backporting [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) and [24.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) Release notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#reviewing-contributions
-->
---
Add a :+1: [reaction] to [pull requests you find important].
[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/
[pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc
NixOS:staging
ā reckenrode:darwin-lto
opened 05:15AM - 20 Jul 24 UTC
## Description of changes
This PR is the second follow-up to https://github.cā¦ om/NixOS/nixpkgs/pull/307880. It enables LTO in packages flagged by https://github.com/NixOS/nixpkgs/issues/19098 as well as others I could identify as having it disabled on Darwin.
I tested building Python on both aarch64- and x86_64-darwin. I also ran a `nixpkgs-review` on aarch64-darwin with the Python LTO disabled (to keep the number of changed packages manageable). Most packages built. Those that failed did not appear to be LTO-related. I have a number of PRs I will be submitting for some of those broken packages.
## Things done
- Built on platform(s)
- [ ] x86_64-linux
- [ ] aarch64-linux
- [x] x86_64-darwin
- [x] aarch64-darwin
- For non-Linux: Is sandboxing enabled in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- [ ] `sandbox = relaxed`
- [ ] `sandbox = true`
- [ ] Tested, as applicable:
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [x] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [x] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [24.11 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2411.section.md) (or backporting [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) and [24.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) Release notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module
- [x] Fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!--
To help with the large amounts of pull requests, we would appreciate your
reviews of other pull requests, especially simple package updates. Just leave a
comment describing what you have tested in the relevant package/service.
Reviewing helps to reduce the average time-to-merge for everyone.
Thanks a lot if you do!
List of open PRs: https://github.com/NixOS/nixpkgs/pulls
Reviewing guidelines: https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#reviewing-contributions
-->
---
Add a :+1: [reaction] to [pull requests you find important].
[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/
[pull requests you find important]: https://github.com/NixOS/nixpkgs/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc
8 Likes