The apply script won’t be a problem for a nixos-rebuild reimplementation. More than anything it is a bottom-up improvement to give a better interface to the toplevel, such that the operations on toplevel are self-contained. The benefits aren’t limited to systemd-run; also the management of the profile link is taken care of, and it improves the architecture / separation of concerns.
Specifically it simplifies the interface between the deployable operating system (toplevel) and deployment tools such as nixos-rebuild, especially when support for the “legacy” non-apply code path can be dropped from such tools.
All improvements to the “local” part of the switch operation can be decided by NixOS and picked up by any deployment tool. This includes swapping out systemd-run for something better, if it turns out we were mistaken.
Speaking of mistakes, the mistake I made in nixos-rebuild in NixOS `apply` script by roberth · Pull Request #344407 · NixOS/nixpkgs · GitHub was a simple logic inversion (-n vs -z) that I didn’t spot, and wasn’t sufficiently tested, so your effort to better test the rewrite is much appreciated.
I expect that we can merge a fixed version after branch-off without much trouble and we might even end up backporting the apply script; maybe even the nixos-rebuild improvement as well.
nixos-rebuild-ng can implement this change at its own pace, because the addition of apply is not a breaking change.
Regarding systemd-run, one of the goals is to persist into journald all logs that are produced as part of the switching operations.
The fact that some activations may succeed despite a broken SSH connection doesn’t mean that all of them will. For instance, if they log more, they may receive a SIGHUP.
It solves a real problem, using an appropriate tool (a transient systemd unit), so no “shenanigans”.
I’m happy to discuss alternative ways to make it more robust.