This is indeed clumsy and has bugged me as well since the very beginning. In short: What you’re trying isn’t possible with the module system because of how module imports work.
It so happens that @infinisil and me recently discussed related ideas at the 23.11 ZHF hackathon.
Silvan, you’ll vividly remember how our brainstorming got me excited. Today I’m proud to show the one-weekend prototype as promised: