This is an interesting topic that i would like to explore in the context of comin, a pull based deployment tool presented at the last Nixcon.
how to centrally manage the configuration on multiple machines?
There are a lot of deployment tools available, most are push based while some are pull based. Basically, you could have a Git repository containing your configurations and either a CICD for pushing configuration changes to machines or an agent running on all machines which pulls the repository (this is how comin is working).
how to do this based on role? various roles will need different setups, eg “devops” vs “ML dev” vs “mobile dev”
I think the easiest way would be to have a NixOS configuration per employee machine. They could then be built by composing NixOS modules, which could describe roles. Also, this would allow employees to submit configuration changes to their own machine. Note I don’t know how this could scale if you have hundreds of machines to deploy.
how to do this based on single user? centrally managing secrets for users such as vpn dial-in, ssh keyring for infrastructure access, etc.
For secret management, a lot of of people are using sops-nix. Providing yubikeys to employees could also be a way of managing their secrets (since a yubikey holds private keys).
enrolling new machines into the configuration and how to keep track of which machine should have which config
The deployment agent could report a status or expose prometheus metrics. There are also tooling such as osquery (recently added to NixOS) that can be used to build a global overview of your fleet.