Setting PYTHONHOME to Python environment created by python310.withPackages helps.
For example, following works now:
[nix-shell:~]$ source venv/bin/activate
[nix-shell:~]$ export PYTHONHOME=/nix/store/xv1vlbkpn26bg09yx5sm1ls9nkn3s8v4-python3-3.10.9-env
[nix-shell:~]$ python -c "import numpy; print(numpy.version.full_version)"
1.23.3