Just a shot in the dark, but in the past, when I had similar issues, it turned out that the Linux kernel’s USB autosuspend feature was involved and the devices apparently could not properly handle the autosuspend.
You can search directories under /sys/bus/usb/devices for your affected USB devices. Subsequently, you could check the contents of a file named power/control stored in the respective directory. The file may contain the string auto, which instructs the kernel to use USB autosuspend when the devices are not in use. You can then manually disable auto-suspend by writing on to these files instead, as in: echo on | sudo tee /sys/bus/usb/devices/…/power/control.
When your problem no longer occurs after disabling USB autosuspend, you could configure things so that in the future, USB autosuspend will be automatically disabled for these devices. E.g., see Power management - ArchWiki or compare the NixOS documentation on setting up udev rules.