Turns out this was the last missing piece of the puzzle. Thanks for your help!
As a summary:
I had to add boot.extraModulePackages = [config.boot.kernelPackages.nvidiaPackages.legacy_470]; to my config as well as change services.xserver.videoDrivers to ["nvidia"] The blacklist for nouveau wasn’t necessary.