I upgraded to unstable version and sshd crashed randomly.
After that I roll back to stable 9.9p2, seems problem resolved since then.
After some research, I think it maybe caused by the new --with-linux-memlock-onfault
:
Bug report in Debian
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1103418
Issue tracker in openssh
https://bugzilla.mindrot.org/show_bug.cgi?id=3822
Debian disable this flag again