Does anybody here know how to use ATK to interact with application windows on Wayland (Gnome)?
Ideally I’d be able to script Chromium.
I am not sure AT-SPI is intended for this use case. libei is probably more suitable but that will not have access to the accessibility info. Though, dogtail is a thing, and it uses the a11y API. Except on Wayland, that will likely require something like gnome-ponytail-daemon and patched GNOME.
For web browser, something using the WebDriver Protocol might be the easiest.
1 Like
Thanks for these pointers!
Maybe ATK is not intended for desktop automation, but it’s enough, no? I asked on the Chromium issue tracker, there’s nothing else (besides webdriver which requires opening a TCP port), and as I understand it, their internal a11y representation is also used for macOS scripting.
Sniff from python39Packages.dogtail works a bit, except that many labels don’t have text and Chromium doesn’t show up but e.g. Konsole does.
Ponytail looks somewhat like what macOS Scripting has to offer. Would it be possible to build Gnome on NixOS with the introspection API by default? I take it there’s a reason it’s not enabled?
Webdriver requires the remote debugging TCP port to be enabled, which is just awkward to manage and possibly a security issue, and then any scripting for automation is completely different from other solutions :-/
There’s many things I dislike about macOS nowadays, including the scripting language itself, but it does allow you to automate just about everything on the desktop.