- should B be in the buildInputs of A?
Unless the build procedure detects the path from
PATH during the build and hardcodes the path into the program, this will not do anything.
- should the absolute path of the executable from B be used in A, instead of relying on finding it in the PATH environment variable?
Yes, for hard dependencies, hardcoding the absolute path is necessary. Some people prefer directly patching the path in code, others just wrap the program and set
PATH in the wrapper. The later will not work for libraries but for programs it is really just a matter of taste. For optional dependencies, you can let users install them to profile manually, or hardcode them just like hard dependencies (this might be fine for end-user programs but controversial for system components). As a compromise, you can also hardcode the dependencies optionally.
- when adding A to systemPackages, B should also be explicitly added?
If you decide you want users to manage the optional dependencies then this is fine. You should clarify the requirements somewhere, for example, in a comment at the top of the expression or in
meta.description. Though, most commonly we encounter this with dconf, which is required by every GTK application, see https://github.com/NixOS/nixpkgs/issues/52777#issuecomment-449866799. Unfortunately, it would be really inconvenient to mention it everywhere so I would really like for packages to be able to declare external dependencies and Nix to be able to inform user when they are missing.