Good question! Traditionally, those ports are reserved for root. The idea is that, in theory, when the system boots a non-privileged user may be able to start a service a bit more quickly than root, thereby reserving the port for their application before root has a chance to.
This would result in them being able to launch an ssh server or a web server, and take over the respective services. You can imagine how much of a security nightmare this could be if a malicious user does so. Hence, ports < 1024 are reserved for important services like httpd or sshd.
A lot of system security today relies on encapsulating services under different namespaces, which can do exactly the same thing as an unprivileged user, and are only prevented from doing so by this same security feature.
If you want to start caddy on low ports, I’d suggest giving it the capability (CAP_NET_BIND_SERVICE, probably what the NixOS module does too) to do so instead. This also protects caddy from being taken over by a different unprivileged user, which is probably better than running it on high ports.
That said, on a dev machine this sysctl may be totally valid. I’m not your mom, here’s the scissors, don’t tell me I didn’t warn you if you end up running with them: NixOS Search