nix develop assumes that you’re trying to create an environment to build a package. Listing those packages in nativeBuildInputs will make the bin output also available. cowsay works, because it only has an out output, so it will default to that which also contains the executable underl bin/.
In debian terms, nix assumed you wanted caffe-dev because buildInputs usually contains headers and libraries (usually this is in a *-lib output in practice) usually needed for C compilation. But you wanted caffe-bin instead, and in nix, we declare hostPlatform compatible executables in nativeBuildInputs.
Another option would have been to do lib.getBin <pkg>, but then libraries likely wouldn’t be available.