Why do some packages (LibreOffice, Thunderbird) compile locally?

I have upgraded to 20.09 on Friday and just noticed that libreoffice-fresh and thunderbird are now compiling locally, which seems to take forever and causes my CPU to be fully utilised for a long time.

I thought that these packages were downloaded from a cache server before. How come that some packages compile locally and others are available for download from a cache server?

(I solved the issue for Thunderbird by switching to thunderbird-bin, but a similar package doesn’t seem to exist for LibreOffice, or doe sit?)

3 Likes

Normally they all come from cache, but looks like some packages from last built was aborted so the package didn’t land on the cache if I understand correctly.

2 Likes

There was a configuration error where some build jobs were getting mis-distributed, and causing the builds to be aborted.

Someone with hydra restart privileges will have to restart those builds, or change the derivation so that it attempts the newer build.

Sad to see Thunderbird still wants to be locally built after 8 days (or I am doing a modification on my system which I wasn’t figure out yet).

It just finished four hours ago, https://hydra.nixos.org/eval/1625285#tabs-now-succeed

if you update your channel, it should pull down.

(however, the binary cache is populated through a script. And might not have ran yet)

1 Like

The issue seems to persist since October. Is this still due to aborted builds?

Not sure honestly. Even though it’s worse, I switched to thunderbird-bin long time ago.

Also this general issue caused me to not upgrade my system frequently. Once in a while, one package starts building and I hear the laptop fan, then I cancel the upgrade and try later.

Maybe a system needs to be in place, like a separate branch etc. so it could wait for :heavy_check_mark: from Hydra, before moving big packages to the release branch.

There is, it’s called channel blockers, which are now referred to as constituents .

https://github.com/NixOS/nixpkgs/blob/a7b60e6bdfb578404be012b0f2296fb49c3b93a6/pkgs/top-level/release.nix#L91

The problem with adding many packages to this is that the entire channel will block until all of those packages pass successfully. So there’s a potential delay in all other packages, if a given constituent fails to build.

3 Likes

Sorry for necro-bumping this again, since I’ve seen thunderbird was building locally on the weekend, I had another idea. I wanted to ask here first.

Not sure how high/low level feature it’d be but: Would it be possible to have a system option which we can list packages locally to not allow local building of them? Something like local constituents to block upgrade for disallowed builds.

1 Like

I like this idea. It would even be nice to have a standard list of packages that take an eternity ti compile. I’m currently in hour 5 of building libreoffice.

The reason it’s building locally is because the pre-built packages are not available from any binary cache. This can happen for 3 reasons:

  1. the package simply isn’t being built (ie hydraPlatforms = [];)
  2. the package isn’t build yet
  3. you have changed a dependency of the package in question locally (hence this combination isn’t being built by any hydras)
  4. the package fails to build

If you are tracking master, that would explain why item 2 happens. The solution: don’t track master or put up with the compile times.

For a common package, that normally should be built and you are tracking one of the release channels, the only reason you’re getting local builds is either 3 or 4. In case of 3, it will complete (eventually), but if you don’t want to wait, don’t override anything.

In case of 4, it obviously will not finish.

The only solution then is to either fix the package or wait for someone else to do it and then wait for the fix to hit a release channel.

Knowing which packages take a long time to compile really don’t get you anything.

Is this only happening on master? If I remember correctly that happened on release branch when backports happened which caused a re-build, but I now forgot which package it was.
Hence, not a bad idea if we could tell nix which packages to not try building. As far as I understand there is only a global switch currently.

I mean, it’ll give me the opportunity to pin the current version of nixpkgs that I probably won’t upgrade if I know it’s going to take long. Or at least find a version that’s been built by the hydras.

But I agree with your point that I shouldn’t be tracking master. It was the only option for my hardware for a while, but that’s not the case anymore.

I do wish each package could be upgraded separately. I know pinning (especially with flakes) gives me that, but there should be an easier way.

Hosted by Flying Circus.