I understand that nixvim will create a 24.05 branch sometime soon (see here).
As suggested in that issue I have been using the nixvim main branch successfully, but will use the 24.05 branch when it’s available.
nixvim = {
url = "github:nix-community/nixvim";
inputs.nixpkgs.follows = "nixpkgs";
};