You can enable the CUDA Cache for Nix Community so a pre-compiled package is downloaded, if it exists. I don’t know if the cache can be enabled per development shell, but I think it would be better if you do it system-wide as it’s quite useful in general:
# configuration.nix
nix.settings = {
substituters = [ "https://nix-community.cachix.org" ];
# Compare to the key published at https://nix-community.org/cache
trusted-public-keys = [ "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=" ];
};
Note that you’ll have to switch to your configuration with the cache enabled first, before it can take any effects.