As far as I understand this, both the nixos-unstable and nixos-18.09 channels are currently blocked by a problem with systemd. @andir is currently working on this, but perhaps he could need some help. See discussion on IRC in #nixos-security and rebased onto upstreams stable v239 by andir · Pull Request #24 · NixOS/systemd · GitHub
Now even nixos-18.09-small has been stuck for a week. If there isn’t any low-hanging fruit left I think the way NixOS is developed must drastically change to be viable.
I think we can definitely do a lot to make some of these tests less flaky. Especially, the installer tests have had issues that are usually random. But, we actually do want things to get stuck when things are legitimately broken.
This particular hang wasn’t about tests themselves but about the build farm “maintenance”. TL;DR: you can’t well automatize everything – some humans still have to watch things and fix them when needed.
1 Like
For those of you who were brought here by Google looking for help updating NixOS:
(I know this thread is only tangentially related, but this is where google keeps corralling that search query)
Here is my update/upgrade process…
Initially, using whatever channel you want:
sudo nix-channel --add https://nixos.org/channels/nixos-unstable nixos
Then to update:
sudo nix-channel --update
nix-channel --update
sudo nix-env --upgrade
nix-env --upgrade
sudo nixos-rebuild switch --upgrade
This was suggested to me on the IRC when I started asking questions about “why doesn’t setting xyz in my configuration.nix cause my system to upgrade?”
I was very surprised to learn that most of those settings didn’t do what I had imagined based on their names. This process might not be the “proper way” but it works well for me.
1 Like
There are ux problems related to channels. The replacement is work in progress: nix-flakes.md · GitHub
1 Like
To revive this thread: Why hasn’t trunk-combined be evaluated over the last 5 days? Here is the last eval: Hydra - Build 86749887 of job nixos:trunk-combined:tested
All evals: Hydra - nixos:trunk-combined:tested
There seems to have been an introduced an error in the kerberos test. @pbogdan have supplied a fix which should hopefully do the trick:
Thanks! Is there some way to find out about failed hydra evals?
As in be notified when it happens? Not that I know of. Perhaps if you have an hydra account, but that seems pretty restrictive at the moment.
Is there a manual way to check?
There’s the Last checked timestamp in the Evaluation tab, so when that doesn’t result in a new eval there’s probably something wrong. The errors for the last check can be found in the Evaluation errors tab.
For some reason the last check didn’t go through either, but the error blocking for the last days is gone at least.
Thanks, that’s good to know for the next time.
The nixos-20.09 channel still is set to a 7 day old commit. There have be many successful builds since them, including one 6 minutes ago. What is stopping the channel update. Is there any place I can get information on what is going on. I have read the general info on this, but not on where there is any specific discussion about when an update is being postponed.
Great question. I don’t personally know.
I think for the 20.09 initial cut, we couldn’t find anyone with intimate hydra knowledge. And looking at the history of jobs that were canceled, I’m assuming there’s some infrastructure that misconfigured or missing some links in a chain.
1 Like
But successful builds are happening. This thread is over 2 years old, and I still have no idea of what happens.
For example, what is the process (or code) between https://hydra.nixos.org/job/nixos/release-20.09/tested
seeing a build success on all the required jobs and the channel being updated? Is there code somewhere checked into github?
1 Like
according to nix-channels, it’s now caught up. Not sure why it was lagging behind almost a week for a while
seems like everything is stuck again this week, with nixos-20.03-small being 4 days old with multiple passed tested jobs.
does anyone actually know when channels are updated after successful hydra builds?
1 Like
I think the actual channel update happens, effectively, via this script: $title.
And I think you want to look here as well: https://github.com/NixOS/nixos-org-configurations/blob/c70e0578d70d5a228a0a50eb3d7fece2a0282ace/modules/hydra-mirror.nix#L19…
It looks like the hydra box runs systemd services that run the mirror-nixos-branch.pl script. I didn’t try very hard to read the Perl but I’m guessing it syncs to Amazon and then also does the actual channel update/pointer switch. (Skimmed again, I’m fairly sure this is true, I even see the git push where it pushes the new tag/branch).
OnUnitInactiveSec = 600;
So it should try updating every 10 minutes? Seems to me something is wrong then 
Thanks!
1 Like