Nixos-rebuild issues after latest unstable upgrade

Help
1

Original:
nixos-rebuild --show-trace --upgrade --flake .\#machine switch --target-host machine --use-remote-sudo --use-substitutes

recent change → nixos-rebuild-ng: enable by default by thiagokokada · Pull Request #415640 · NixOS/nixpkgs · GitHub

Updated:
nixos-rebuild-ng --show-trace --upgrade --flake .\#machine switch --target-host machine --ask-sudo-password --sudo --use-substitutes

Now gives me the following error:

sudo: sorry, you must have a tty to run sudo
Command '['ssh', '-o', 'ControlMaster=auto', '-o', 'ControlPath=/tmp/nix-shell-517870-0/nixos-rebuild.cyhqgl0e/ssh-%n', '-o', 'ControlPersist=60', 'machine', '--', 'sudo', '--prompt=', '--stdin', 'nix-env', '-p', '/nix/var/nix/profiles/system', '--set', '/nix/store/vlml0513air9kv7prv5pc0xv03ipp331-nixos-system-machine-25.11.20250607.3e3afe5']' returned non-zero exit status 1.

I’ve checked the unstable manual NixOS Manual but cannot seem to find anything on the changes.

Any help would be appreciated :), thanks.

2

Doesn’t look like a default configuration for sudo.

3

Since you mentioned it I checked, and this machine does have some hardening enabled;

  security.sudo = {
    execWheelOnly = true;
    extraConfig = lib.mkBefore ''
      Defaults requiretty,use_pty,env_reset

      Defaults:root !requiretty
    '';
  };

I am trying again with this disabled, thanks for highlighting.

Was the linked update tested with the hardened profile or is that not supported.

Edit: Removing the extraConfig doesn’t appear to change the error.

Edit: for clarity the local machine doing the building has no hardening, while the remote machine did have hardening which I have removed, but I guess I cannot override due to nix-rebuild failing…will have to revert to an older profile so I can push the unhardened update.

4

Is only a default config supported?

Does forcing false revert to the old nixos rebuild for unstable users?

system.switch.enableNg

Thanks

5

I doubt 90% of the stuff in nixpkgs is tested with the hardened profile, which is part of why the profile is controversial.

1 Like
6

I’d tested without any hardening and the same error, so seems unrelated.

Hard to see how hardening is controversial, but I’ll take your word on that :slight_smile:

I’d just be happy to just revert and continue.

7

Did you re-apply the config first locally and then tried to do remotely? Because the fact that your current sudo config have this option mean the nixos-rebuild switch --target-host will not work until this is removed and applied. After applying the configuration it --target-host should work.

No, of course not, but adding random things to your configuration will of course break random things. Keep in mind that running nixos-rebuild switch locally instead of remotely should always work.

If anything, adding Defaults requiretty is doing exactly what you want in this case: it is requiring a TTY for remote commands. nixos-rebuild-ng explicitly doesn’t allocate a pseudo-TTY for a whole other issues that is described in the post from Stack Overflow that I posted. nixos-rebuild does allocate a pseudo-TTY but this is mostly regarded as a mistake and causes multiple issues.

Yes, but keep in mind that this will be removed once 25.11 is released.

1 Like
8

It’s controversial because the hardening is half-assed, but yeah that’s fair if it’s still happening without that profile, then something else is involved.

And yes as you pointed out you can use the old nixos-rebuild for now by disabling that ng option. However, ng is a fully rewritten script in python, and it doesn’t appear that there are plans to maintain the old bash script, so while that may help you for now, it’s not going to work longterm.

With that said, if you can’t get it working with nixos-rebuild-ng, there are over 40 other deployment tools for NixOS systems, (I’m personally partial to nh, though I don’t believe it supports remote deployment yet) and many others may suit your needs.

1 Like
9

Yes just 5 mins ago I realized it wouldn’t apply the unhardened, so will try that another day.

Random? you mean Hardening?

Sure, but none of which I have come across.

Sure, but as we’ve just had a release, I have plenty of time to look at this over the next number of months, while today being able to remotely update my email build.

The local machine doing the building does not have any hardening, only the remote one does.

Don’t get me wrong, thanks for your work on a rust rewrite, I guess I am just not a fan of force pushing to unstable users months away from a release cycle.

10

I thought it was in rust, but anyway, change is a good thing, as I said above, I guess I am just not a fan of force pushing to unstable users months away from a release cycle.

I’ll disable it for now, do my updates and look another day when I have more time.

Thanks again.

11

IMHO:

I honestly don’t see any other option that fits better, but directly after a release cycle.
That give the most time for unstable users to try out their configs (esp the customized ones) and receive fixes for it untill it becomes a thing for stable users.

If it’s shortly before a release cycle that means that there is not much time for testing things in the wild, resulting in a risky for breaking stable users, which is more sad than breaking unstable (tbh, FYI I am using also unstable on most of my machines)

As the fallback option to revert to non-ng is still there I don’t even see the issue tbh. Yeah it’s sad to break users, but if that’s not tolerated, I don’t understand what unstable should be used for…

3 Likes
12

For an option that was enabled and eventually disabled in a distro that generally focus in security (Fedora), because the security benefits are controversial at best and it breaks valid use cases? Yes, this is for me a random option.

https://bugzilla.redhat.com/show_bug.cgi?id=1020147.

The reason we pushed this so early after a release is exactly to get those kind of issues earlier and fix it before the next release. If we waited until the end of the cycle to do this we wouldn’t have enough time to fix the issues and it would be worse since this would push those issues to the stable users.

1 Like
13

Don’t disagree, I guess for such a material change to a core script a notice might be helpful.

Clearly there isn’t, but it took a post and queries to find out.

Of course the catch-all ‘unstable’, I guess I am used to small group beta testing for material changes, but as unstable works for most, I am cool with that.

14

There is, release notes: Appendix B. Release Notes.

It is in the release notes.

The small group beta testing was the pre-25.05 release where people could opt-in to nixos-rebuild-ng. We fixed tons of issues during that cycle, this is why it is enabled by default from this release.

1 Like
15 
  # NOTE: disabled for now while testing new nixos-rebuild
  system.switch.enableNg = lib.mkForce false;

Works in the short term.

16

maybe an additional hint for unstable users.

if you are using unstable you may want to keep regularly an eye on the Announcements - NixOS Discourse category.

For nixos-rebuild-rebuild-ng the relevant announcement (including the option flip) was Nixos-rebuild-ng: a nixos-rebuild rewrite

And there is also Breaking changes announcement for unstable which is a thread to keep an eye on. Sadly not all breaking changes are posted there.

Also nixpkgs/doc/release-notes at master · NixOS/nixpkgs · GitHub contains the “running” release notes for nixpkgs things and nixpkgs/nixos/doc/manual/release-notes/rl-2511.section.md at 74e468ca541b70961cd629915f8ea6429942088c · NixOS/nixpkgs · GitHub for nixos things, containing the discussed change here as first item.

I would recomment every unstable user to keep at least a look at the last link (not that as its linking a commit, but the respective nixos changelogs that are edited with the breaking changes) if something breaks unexpected before digging to deep into code or forums.

2 Likes
17

That seems to sum things up nicely - the information is everywhere…

18

For anyone in the same situation: you don’t need to disable system.rebuild.enableNg in the local host to fix the issue. You can simply run nix run nixpkgs#nixos-rebuild switch --target-host foo@bar or nix-shell -p nixos-rebuild to start a shell with the old nixos-rebuild and run the commands inside the shell to apply the configuration without the problematic extraConfig, and afterwards you can apply future updates using nixos-rebuild-ng.

1 Like