I definitely recommend that you try cross-compiling from an architecture with fast cheap hardware (i.e. x86).
My laptop is arm64 and I compile the whole system this way (using much faster x86 and powerpc64 machines), from the bootloader to the web browser. You can also cross-compile the NixOS installer, although I’ve only done that for mips64:
Nixpkgs’ support for cross compiling is really good at this point; basically everything works except Haskell, Emacs, and QT. The Haskell situation is likely to improve soon.