No. Someone needs to build each package at least once, of course, but usually this is done upstream by hydra.
Yes, this is the idea.
However, if the binary cache does not have a specific package cached, nix will automatically build it on your computer. This can happen if you make modifications to a package, or somehow depend on a branch that hydra has not built yet or something. Changes in dependency packages will also cause cascading rebuilds all the way to leaf packages you actually use (because there is no way to guarantee ABI/API compatibility), so a small change can cause a lot of churn.
It’s not particularly common to see accidental changes like that, but your flake is quite complex with many different inputs (some of which do not have first-party support and will therefore not be built by hydra), so something might have snuck in.
I think aarch64 also generally builds a bit less than x86_64, so maybe this is indeed a case where aarch64 just doesn’t have this, but usually that’s packages marked as not working on the platform, so you would need to opt-in to attempting to build such packages.