This most likely means that you’re trying to use a private repository. For that to work, Nix will need a netrc file. Take a look at Enterprise - NixOS Wiki - it only mentions fetchurl, but the same mechanism is used for flakes as well.
The gitlab schema does not correctly work with subgroups. The subgroups are parsed as branch…
Using gitlab:user/group%2Fsubgroup%2Fpackage might work, though it might as well happen that the percentsigns are escaped then… You need to try it out…
I also was under the impression that there has been a bug report about that, but I can not find it right now, though if Leverage the "uniform" in URLs for flake refs more · Issue #6304 · NixOS/nix · GitHub would get implemented, gitlab subgroups would be fixed as a drive by, due to the strict question for making ref a query parameter and not allow it in the path anymore.