I’m using flakes with stable 24.11 as input. It works great and I tend to update weekly in case security fixes and the such have been added. This is super fast 99% of the time, but every so often I find that my system has to build a few packages from scratch. Why is this happening?
Don’t take me wrong, I don’t mind building locally. It’s much slower but I understand that packages need to be built somewhere; but it’s such as rare event that I had come to assume that stable is only update when it passes some sort of CI and Hydra builds (and chaches?) them successfully. At least that’s what I deduced from reading this other thread.
Might my inputs or config be flawed, or is this expected?
Thanks!
I’m a bit new so I might have it wrong. But no overlays, and although I have unfree enabled, the packets that compile locally are open source tools like KiCAD (this one had to recompile today but I had to halt it because it was taking forever) or programs that depend on a given open source library (sorry, can’t give a concrete example).
Might it be because of config.allowUnfree = true;?
But it they fail to compile, wouldn’t it mean that they are broken? Wouldn’t that be catched by the CI and prevent the merge? Or are they compiled and cached after the merge?
Now that I think about it, I don’t know if I’ve ever let something compile to the end locally. It usually happens on my low-endish laptop. I just wait a few days, update the inputs again, and then everything gets downloaded from the cache
That will just happen if the package (or a test that uses the package) is defined as channel blocker.
If hydra would block on any failing package, it would never update the channel at all (as there are always some fails).
Did you check what’s the state on hydra for the packages you need to compile locally?
To do more remote analysis it might be useful to know a bit more about your configuration (the actual code could be useful). And if you are using channels, the channel list. If you use some channel pinning tool that state is useful.
The most common error I have encountered for such a question was until now that people followed the source branch and not the ci checked.
So for table you channel/flake input should be nixos-24.11
If you follow a release-* that is prior ci, and would explain local building.
I still have no public repo with my config (still figuring secrets out with agenix, WIP) but I have no issue sharing my flake here (is there a way to make it collapsible?).
Note that I add unstable as nixpkgs-unstable in my config, but only use it for Firefox. Everything else is installed from nixos-24.11.
{
description = "NixOS configuration";
# ------------------------------------------------------------------------------------------------
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-24.11";
nixpkgs-unstable.url = "github:nixos/nixpkgs/nixos-unstable";
home-manager = {
url = "github:nix-community/home-manager/release-24.11";
inputs.nixpkgs.follows = "nixpkgs"; # This line makes sure that "nixpkgs.url" and "home-manager.url" stay in sync and can work together.
};
agenix = {
url = "github:ryantm/agenix";
inputs.nixpkgs.follows = "nixpkgs";
};
nurpkgs.url = "github:nix-community/NUR"; # Nix User Repository.
nixos-hardware.url = "github:NixOS/nixos-hardware/master"; # HW specific tweaks.
};
# ------------------------------------------------------------------------------------------------
outputs =
{
self,
nixpkgs,
nixpkgs-unstable,
home-manager,
agenix,
nur,
...
}@inputs:
let
# Modules that are common to all host instances.
commonModules = [
home-manager.nixosModules.home-manager # Will search for a `home-manager` configuration struct in the host config and deplopy it.
nur.nixosModules.nur # Adds a `nur` configuration option. e.g., `environment.systemPackages = [ config.nur.repos.mic92.hello-nur ];`.
];
# SpecialArgs that are common to all host instances.
commonSpecialArgs = system: {
inherit inputs; # Inputs from the flake, includes nixpkgs and home-manager.
# Add "non-standard" input to make `pkgs-unstable` available in the system configuration.
# <"https://nixos-and-flakes.thiscute.world/nixos-with-flakes/downgrade-or-upgrade-packages">
# Requires `system` to be passed as an argument.
pkgs-unstable = import nixpkgs-unstable {
inherit system;
config.allowUnfree = true;
};
};
# Helper function to call `nixpkgs.lib.nixosSystem.
#
# Since all host instances are created in a very similar maner, with almost identical modules
# and specialArgs, this function will wrap around the creation process of a host instance.
#
# Inputs:
# - name: The name of the system, will be used to find the host .nix configuration file.
# - system: The system to build, default = "x86_64-linux".
# - extraModules: Extra modules to include in the system configuration. default = [ ].
makeConfig =
{
name,
system ? "x86_64-linux",
extraModules ? [ ],
}:
nixpkgs.lib.nixosSystem {
inherit system; # Equivalent to system = system;
modules =
[
./nixos/hosts/${name}.nix
agenix.nixosModules.default
]
++ commonModules
++ extraModules;
specialArgs = (commonSpecialArgs system);
};
in
{
# Host instances.
nixosConfigurations = {
lenovo = makeConfig {
name = "lenovo";
extraModules = [ inputs.nixos-hardware.nixosModules.lenovo-thinkpad ];
};
};
};
}
Thx so the obvious error does not apply here.
Aside that the config is not evaluatable (due to missing files as you left them out as they contained credentials, but that results in us not being able to evaluate them).
For kicad it exceeds the output limit Hydra - Build 290858899 of job nixos:trunk-combined:nixpkgs.kicad.x86_64-linux
That is the reason it’s not in cache but build able.
You might want to report that on nixpkgs as build issue, then maintainers (and infra) could possibly figure out what to do.
Possibly the closure references something that is absurdly large (idk kicad).
Edit: inst ad of nixpkgs-unstable you might want to reference nixos-unstable, (esp. If you are planning to use more from unstable), Firefox a lone might be okay from there.
Nixpkgs-unstable is missing nixos specific tests (which might be the reason something is in cache at the end)
As a temporary workaround, you could use kicad-small which is green on Hydra. It does not have the 3D stuff, but if you don’t urgently need it, it might be a stopgap until the main package is fixed.
That sounds for me like that.
Tho after some git history research one might find that the build was disabled between 2020 and somewhere in 2023.
There was a PR trying to fix that it’s build able by hydra again, but as it’s pointing out a successful build it sounds for me that it wasn’t building since 2 years.
Maybe people just use kicad-small, but that’s just blind guessing.
Thanks for looking into it! I can’t access the Hydra server though, I get a 403 Edit: it seems to be an issue of my browser, I can access it from Chrome.
Edit: inst ad of nixpkgs-unstable you might want to reference nixos-unstable, (esp. If you are planning to use more from unstable), Firefox a lone might be okay from there.
Nixpkgs-unstable is missing nixos specific tests (which might be the reason something is in cache at the end)
I’m not sure I follow. Is there a separate URL for the unstable branch? I’ve been googling it but I couldn’t find anything on that topic.
@polygon I’m actually using kicad-small and it was the package that started to recompile. I’ll check later today if it’s still an issue or if it’s already in the cache. I have never tried to install the full package, so I can’t confirm if it has ever installed from cache for me - maybe it’s been broken since 2020 as @truh mentions
For the record, it’s the source code that’s failing to upload because it’s too large. I don’t know if the actual kicad build result would be small enough to be uploaded to the cache, but I checked the source tarball. The source is only ~900M, which is plenty small enough to be uploaded, but it’s using one of the fetchers that automatically decompresses and extracts the archive file it downloaded. So it explodes to ~5G, which is significantly higher than the cache’s limit. It seems to me like this is unnecessary though, and the kicad src could instead just be the compressed, unextracted tarball instead, which would be fine. This is actually a pretty major frustration I have with the fetchers like fetchFromGitHub.
It however is currently building on nixos-unstable. I just ran nix run github:NixOS/nixpkgs/nixos-unstable#kicad-small which correctly got everything from the cache, no manual builds required.
Just changed kicad-small to unstable and it worked. Thanks! My system (around 28GB) was successfully deployed without stalling during local compilations.
I’ve learned a lot from all you comment and explanations. From now on I’ll check the status of Hydra whenever I run into this issue. I’m slowly wrapping my head around Nix
Edit: it seems to be an issue of my browser, I can access it from Chrome.