In my case, the instance above was a little jump box, and there was an additional ssh jump step in my config:
Host nixhop
HostName 127.0.0.1
Port 2022
User ec2-user
Host nixdev
HostName <private ip>
ProxyJump nixhop
Compression yes
Both hosts and all the various dependencies are built via a cloudformation template, run as spot instances, and the template has outputs for the instance id and ip, above. I did briefly have it launch from direnv when I changed into the directory, but that was a little too annoying and slow.
The tunnel worked fine for vscode remote as well as a build server.