Attempt to fix problems with flatpacks, e.g. Steam.
This commit is contained in:
1 parent
1c4e0a99bc
commit
53d14d561b
20 files changed
+510
-20
No files matched your search
@@ -7,6 +7,7 @@ Cthulhu is a Linux screen reader written in Go.
|
||||
- [LibreOffice support](LIBREOFFICE_SUPPORT.md): integration boundaries, initial Calc behavior, remaining work, and verification.
|
||||
- [Progress reporting](PROGRESS_BARS.md): web progress bars, Steam shader processing, and verification.
|
||||
- [Steam keyboard focus](STEAM_FOCUS.md): Xorg workspace focus recovery and verification.
|
||||
- [Flatpak foreground accessibility](docs/testing/flatpak-foreground.md): process identity and proxy matching, tests, and live acceptance.
|
||||
- [Wayland keyboard access](WAYLAND_SUPPORT.md): shared protocol support, limitations, and verification.
|
||||
|
||||
AI-assisted contributions are welcome. Contributors remain responsible for human review and testing before submission; see the [full policy](CONTRIBUTING.md#ai-assisted-contributions).
|
||||
|
||||
@@ -22,6 +22,7 @@ pkgbase = cthulhu-git
|
||||
depends = brltty
|
||||
depends = liblouis
|
||||
depends = libx11
|
||||
depends = libxres
|
||||
depends = libxtst
|
||||
depends = libxfixes
|
||||
depends = libxkbcommon
|
||||
|
||||
@@ -31,6 +31,7 @@ depends=(
|
||||
|
||||
# Xorg input, window capture, and clipboard operations
|
||||
libx11
|
||||
libxres
|
||||
libxtst
|
||||
libxfixes
|
||||
libxkbcommon
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
# Flatpak foreground accessibility
|
||||
|
||||
The Python `master` reference's `script_utilities.py:canBeActiveWindow`
|
||||
requires active, showing, non-iconified state. It does not require X11 and
|
||||
AT-SPI process IDs or window rectangles to agree. Go's additional correlation
|
||||
checks exposed two Flatpak cases:
|
||||
|
||||
- `_NET_WM_PID` can be relative to the application's PID namespace. The Xorg
|
||||
foreground reader now prefers XRes 1.2's local connection PID. It retains
|
||||
the property as a fallback when server-side lookup is unavailable.
|
||||
- AT-SPI's process query returns the D-Bus connection owner, which can be a
|
||||
proxy. The adapter accepts different processes only when their
|
||||
`/proc/PID/root/.flatpak-info` files identify the same application and running
|
||||
instance. Active state, geometry and existing window ownership still apply.
|
||||
|
||||
This is shared foreground handling, with no Steam name or window-title match.
|
||||
It covers direct events, foreground recovery, menu correlation and OCR's
|
||||
process check. It does not establish compatibility with every Flatpak, native
|
||||
Wayland focus, or application-specific scripts behind a proxy. Missing or
|
||||
unreadable instance metadata does not relax matching. The reported Steam 2x2
|
||||
auxiliary frames remain subject to the ordinary geometry checks.
|
||||
|
||||
XRes adds the `libxres` Arch dependency (`xres` pkg-config module). Debug logs
|
||||
record both the resolved X11 PID and original property PID.
|
||||
|
||||
The Xorg owner caches one client's lookup until the window or property changes.
|
||||
The AT-SPI GLib owner bounds provider comparisons to 128 entries and clears them
|
||||
when the foreground window, process or known state changes. Metadata reads are
|
||||
limited to regular files of at most 64 KiB. No process scan or service manager
|
||||
is used.
|
||||
|
||||
References: [AT-SPI process query](https://gnome.pages.gitlab.gnome.org/gtk/atspi2/method.Accessible.get_process_id.html),
|
||||
[proxy credentials](https://github.com/flatpak/xdg-dbus-proxy/blob/main/xdg-dbus-proxy.xml),
|
||||
[Flatpak instance metadata](https://github.com/flatpak/flatpak/blob/main/doc/flatpak-metadata.xml),
|
||||
[libwnck's XRes namespace correction](https://lists.gnome.org/archives/commits-list/2022-March/msg10404.html).
|
||||
|
||||
## Automated checks
|
||||
|
||||
Run the private fixtures sequentially. The GTK mismatch test fails before the
|
||||
XRes change. The proxy fixture uses real bubblewrap mount namespaces and
|
||||
xdg-dbus-proxy, with synthetic instance metadata; it needs neither an installed
|
||||
Flatpak nor a running desktop reader.
|
||||
|
||||
```sh
|
||||
CTHULHU_FOREGROUND_PID=1 python3 tests/native_atspi.py
|
||||
CTHULHU_FOREGROUND_PID=1 CTHULHU_FOREGROUND_PROXY=1 python3 tests/native_atspi.py
|
||||
CTHULHU_FOREGROUND_PID=1 CTHULHU_FOREGROUND_PROXY=1 CTHULHU_FOREGROUND_PROXY_OTHER=1 python3 tests/native_atspi.py
|
||||
xvfb-run -a env CTHULHU_NATIVE_WINDOW_TITLES=1 go test ./internal/platform/xorg -run '^TestNativeWindowTitleDiscovery$' -count=1
|
||||
xvfb-run -a -s '-screen 0 1280x1024x24 -extension X-Resource' env CTHULHU_NATIVE_WINDOW_TITLES=1 CTHULHU_XRES_DISABLED=1 go test ./internal/platform/xorg -run '^TestNativeWindowTitleDiscovery$' -count=1
|
||||
```
|
||||
|
||||
The final proxy case must reject the control from a different instance of the
|
||||
same application. Xorg tests cover cached reads, property changes, empty
|
||||
workspaces, destroyed windows and fallback without XRes. Also run the affected
|
||||
Go packages and `tests/native_ownership.py` with `CTHULHU_BINARY` pointing to a
|
||||
fresh isolated build; that checks GTK, Qt and Brave foreground boundaries.
|
||||
|
||||
## Live acceptance
|
||||
|
||||
Actual Flatpak Steam, audible speech and physical Braille remain unverified.
|
||||
On the maintainer's i3 setup:
|
||||
|
||||
1. Quit Cthulhu with Cthulhu+Q. Press Ctrl+Escape, then Shift+G as separate key
|
||||
presses. Shift+G runs `~/devel/cthulhu-go.sh` and rebuilds both binaries.
|
||||
2. Launch `flatpak --user run com.valvesoftware.Steam -tenfoot`. Press Tab and
|
||||
Shift+Tab through controls. Each focus move should announce the control
|
||||
label and role; a working Braille display should follow that same control.
|
||||
3. Open and close a Steam menu with Escape, navigate with arrows and Enter,
|
||||
then type in search. Keys should reach Steam and speech should track the
|
||||
selected control or edited text.
|
||||
4. Switch to another application's window and back. Steam's ordinary control
|
||||
events should not interrupt the other application; returning should restore
|
||||
announcements. Repeat with another available Flatpak and a native browser.
|
||||
5. Report the exact spoken text, Braille text if connected, focused control and
|
||||
key that differed from expectation. Preserve `~/devel/cthulhu-go.log` after
|
||||
an unexpected result. A remote tester should use their own equivalent rebuild
|
||||
and debug-log launcher; the i3 bindings above are local customizations.
|
||||
|
||||
The original report also contains independent Braille geometry errors. This
|
||||
foreground change does not repair the Braille helper's connection failure.
|
||||
@@ -132,7 +132,7 @@ func (handler *accessibilityEventHandler) handle(event atspi.Event) {
|
||||
"bus", event.Window.BusName, "window_path", event.Window.Path,
|
||||
"x_window", client.Window, "epoch", event.Foreground.Epoch,
|
||||
"current_epoch", state.foregroundState.Epoch, "reason", check.Reason,
|
||||
"x_pid", client.PID, "accessible_pid", check.AccessiblePID,
|
||||
"x_pid", client.PID, "x_property_pid", client.PropertyPID, "accessible_pid", check.AccessiblePID,
|
||||
"active", check.Active, "showing", check.Showing, "iconified", check.Iconified,
|
||||
"accessible_bounds", check.Bounds,
|
||||
"x_bounds", [4]int{client.X, client.Y, client.Width, client.Height},
|
||||
|
||||
@@ -195,7 +195,8 @@ func (state *eventLoopState) syncForeground(foreground foregroundMonitor, logger
|
||||
state.owner.clear()
|
||||
clearOwnedFocus()
|
||||
logger.Debug("X11 foreground ownership changed", "x_window", latest.Client.Window,
|
||||
"pid", latest.Client.PID, "known", latest.Client.Known, "epoch", latest.Epoch)
|
||||
"pid", latest.Client.PID, "property_pid", latest.Client.PropertyPID,
|
||||
"known", latest.Client.Known, "epoch", latest.Epoch)
|
||||
}
|
||||
state.foregroundState = latest
|
||||
}
|
||||
@@ -114,6 +114,7 @@ type Adapter struct {
|
||||
applicationProcesses map[string]string // GLib owner; bounded provider identity cache
|
||||
webNotificationRoots map[ObjectID]bool // copied identities only; no native references
|
||||
foregroundReader *xorg.ForegroundReader // GLib thread only
|
||||
foregroundProcesses map[int]bool // GLib thread; provider PID matches for current client
|
||||
foreground atomic.Pointer[ForegroundState]
|
||||
echoFocus atomic.Pointer[echoFocus]
|
||||
eventListener *C.AtspiEventListener // GLib owner; released with subscriptions.
|
||||
|
||||
@@ -29,6 +29,9 @@ func (adapter *Adapter) refreshForeground() {
|
||||
return
|
||||
}
|
||||
state := ForegroundState{Client: value, Epoch: previous.Epoch}
|
||||
if value.Window != previous.Client.Window || value.PID != previous.Client.PID || value.Known != previous.Client.Known {
|
||||
clear(adapter.foregroundProcesses)
|
||||
}
|
||||
if value.Known != previous.Client.Known || value.Window != previous.Client.Window {
|
||||
state.Epoch++
|
||||
adapter.foregroundWindow = ObjectID{}
|
||||
@@ -78,6 +81,7 @@ func (adapter *Adapter) matchForeground(window *C.AtspiAccessible, id ObjectID,
|
||||
if check != nil {
|
||||
check.AccessiblePID = pid
|
||||
}
|
||||
pid = adapter.foregroundProcessID(pid)
|
||||
if adapter.foregroundWindow.Valid() {
|
||||
if adapter.foregroundWindow == id {
|
||||
return true
|
||||
@@ -265,7 +269,7 @@ func (adapter *Adapter) recoverForeground() {
|
||||
continue
|
||||
}
|
||||
pid := int(C.atspi_accessible_get_process_id(application, nil))
|
||||
if pid != state.Client.PID {
|
||||
if adapter.foregroundProcessID(pid) != state.Client.PID {
|
||||
C.g_object_unref(C.gpointer(application))
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -97,7 +97,7 @@ func (adapter *Adapter) nativeOCRGeometry(ctx context.Context, object ObjectID,
|
||||
return OCRGeometry{}, ErrOCRGeometry
|
||||
}
|
||||
defer C.g_object_unref(C.gpointer(window))
|
||||
if int(C.atspi_accessible_get_process_id(window, nil)) != before.Client.PID || (adapter.foregroundWindow.Valid() && adapter.foregroundWindow != nativeObjectID(window)) {
|
||||
if adapter.foregroundProcessID(int(C.atspi_accessible_get_process_id(window, nil))) != before.Client.PID || (adapter.foregroundWindow.Valid() && adapter.foregroundWindow != nativeObjectID(window)) {
|
||||
return OCRGeometry{}, ErrOCRGeometry
|
||||
}
|
||||
states := C.atspi_accessible_get_state_set(window)
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
// Copyright (c) 2026 Storm Dragon
|
||||
// Authors: Storm Dragon, Angus Kola
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package atspi
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"strings"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
type flatpakInstance struct {
|
||||
application, instance string
|
||||
}
|
||||
|
||||
// Flatpak gives its D-Bus proxy and application the same /.flatpak-info.
|
||||
// Compare a running instance, never just an application ID: two instances of
|
||||
// the same application must not acquire one another's foreground window.
|
||||
func readFlatpakInstance(procRoot string, pid int) flatpakInstance {
|
||||
if pid <= 0 {
|
||||
return flatpakInstance{}
|
||||
}
|
||||
file, err := os.OpenFile(fmt.Sprintf("%s/%d/root/.flatpak-info", procRoot, pid), os.O_RDONLY|syscall.O_NONBLOCK, 0)
|
||||
if err != nil {
|
||||
return flatpakInstance{}
|
||||
}
|
||||
defer file.Close()
|
||||
const limit = 64 * 1024
|
||||
stat, err := file.Stat()
|
||||
if err != nil || !stat.Mode().IsRegular() || stat.Size() > limit {
|
||||
return flatpakInstance{}
|
||||
}
|
||||
data, err := io.ReadAll(io.LimitReader(file, limit+1))
|
||||
if err != nil || len(data) > limit {
|
||||
return flatpakInstance{}
|
||||
}
|
||||
var result flatpakInstance
|
||||
var section string
|
||||
for _, line := range strings.Split(string(data), "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" || strings.HasPrefix(line, "#") || strings.HasPrefix(line, ";") {
|
||||
continue
|
||||
}
|
||||
if strings.HasPrefix(line, "[") && strings.HasSuffix(line, "]") {
|
||||
section = line
|
||||
continue
|
||||
}
|
||||
key, value, ok := strings.Cut(line, "=")
|
||||
if !ok {
|
||||
return flatpakInstance{}
|
||||
}
|
||||
key, value = strings.TrimSpace(key), strings.TrimSpace(value)
|
||||
if section == "[Application]" && key == "name" {
|
||||
if result.application != "" {
|
||||
return flatpakInstance{}
|
||||
}
|
||||
result.application = value
|
||||
}
|
||||
if section == "[Instance]" && key == "instance-id" {
|
||||
if result.instance != "" {
|
||||
return flatpakInstance{}
|
||||
}
|
||||
result.instance = value
|
||||
}
|
||||
}
|
||||
if result.application == "" || result.instance == "" {
|
||||
return flatpakInstance{}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func sameFlatpakInstance(procRoot string, first, second int) bool {
|
||||
identity := readFlatpakInstance(procRoot, first)
|
||||
return identity != (flatpakInstance{}) && identity == readFlatpakInstance(procRoot, second)
|
||||
}
|
||||
|
||||
// Owned by the GLib thread and cleared with the foreground client. This only
|
||||
// resolves process identity; fresh window state and geometry still decide
|
||||
// whether an accessible can own focus.
|
||||
func (adapter *Adapter) foregroundProcessID(pid int) int {
|
||||
client := adapter.Foreground().Client
|
||||
if client.Window == 0 || client.PID <= 0 || pid <= 0 || pid == client.PID {
|
||||
return pid
|
||||
}
|
||||
matched, known := adapter.foregroundProcesses[pid]
|
||||
if !known {
|
||||
matched = sameFlatpakInstance("/proc", client.PID, pid)
|
||||
if adapter.foregroundProcesses == nil || len(adapter.foregroundProcesses) >= 128 {
|
||||
adapter.foregroundProcesses = make(map[int]bool)
|
||||
}
|
||||
adapter.foregroundProcesses[pid] = matched
|
||||
}
|
||||
if matched {
|
||||
return client.PID
|
||||
}
|
||||
return pid
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
// Copyright (c) 2026 Storm Dragon
|
||||
// Authors: Storm Dragon, Angus Kola
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package atspi
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestFlatpakProcessInstanceBoundaries(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
write := func(pid int, data string) {
|
||||
t.Helper()
|
||||
dir := filepath.Join(root, fmt.Sprint(pid), "root")
|
||||
if err := os.MkdirAll(dir, 0700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, ".flatpak-info"), []byte(data), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
metadata := "[Application]\nname=org.example.App\n[Instance]\ninstance-id=123\n"
|
||||
write(100, metadata)
|
||||
for _, test := range []struct {
|
||||
name, data string
|
||||
want bool
|
||||
}{
|
||||
{"same instance", metadata, true},
|
||||
{"other instance of same app", strings.ReplaceAll(metadata, "123", "456"), false},
|
||||
{"other app", strings.ReplaceAll(metadata, "org.example.App", "org.example.Other"), false},
|
||||
{"app ID alone", "[Application]\nname=org.example.App\n", false},
|
||||
{"instance ID alone", "[Instance]\ninstance-id=123\n", false},
|
||||
{"wrong section", "[Application]\nname=org.example.App\ninstance-id=123\n", false},
|
||||
{"empty metadata", "", false},
|
||||
{"malformed metadata", metadata + "broken", false},
|
||||
{"duplicate identity", metadata + "instance-id=456\n", false},
|
||||
{"oversized metadata", metadata + strings.Repeat("#", 65536), false},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
write(200, test.data)
|
||||
if got := sameFlatpakInstance(root, 100, 200); got != test.want {
|
||||
t.Fatalf("same instance = %v; want %v", got, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
if sameFlatpakInstance(root, 100, 300) || sameFlatpakInstance(root, 300, 400) || sameFlatpakInstance(root, 0, 0) {
|
||||
t.Fatal("missing process or unknown identity matched")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
// Copyright (c) 2026 Storm Dragon
|
||||
// Authors: Storm Dragon, Angus Kola
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package atspi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strconv"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The GTK provider publishes its host PID over AT-SPI, but its X11 property
|
||||
// deliberately contains a different PID, as with a Flatpak PID namespace.
|
||||
func TestNativeForegroundProcessIdentity(t *testing.T) {
|
||||
if os.Getenv("CTHULHU_FOREGROUND_PID") != "1" {
|
||||
t.Skip("requires isolated GTK/X11 process identity fixture")
|
||||
}
|
||||
pid, err := strconv.Atoi(os.Getenv("CTHULHU_TREE_PID"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
adapter := New()
|
||||
if err := adapter.Start(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer adapter.Stop(context.Background())
|
||||
if _, err := adapter.CaptureDesktop(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if foreground := adapter.Foreground(); foreground.Client.PID != pid {
|
||||
t.Fatalf("foreground PID = %d; want host provider PID %d (window property is 252)", foreground.Client.PID, pid)
|
||||
}
|
||||
if err := exec.CommandContext(ctx, "xdotool", "key", "F6").Run(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for {
|
||||
select {
|
||||
case event := <-adapter.Events():
|
||||
if event.Type != FocusedStateEvent || event.Detail1 == 0 || event.Source.Name != "Caret review probe" {
|
||||
continue
|
||||
}
|
||||
wantMismatch := os.Getenv("CTHULHU_FOREGROUND_PROXY_OTHER") == "1"
|
||||
if event.ForegroundMismatch != wantMismatch || event.Foreground.Client.PID != pid || !event.Window.Valid() {
|
||||
t.Fatalf("foreground mismatch = %v; want %v: %+v", event.ForegroundMismatch, wantMismatch, event)
|
||||
}
|
||||
if os.Getenv("CTHULHU_FOREGROUND_PROXY") == "1" && event.Source.ApplicationProcess != "xdg-dbus-proxy" {
|
||||
t.Fatalf("fixture did not use a separate D-Bus proxy: %q", event.Source.ApplicationProcess)
|
||||
}
|
||||
return
|
||||
case <-ctx.Done():
|
||||
t.Fatal("foreground control focus event missing")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
package xorg
|
||||
|
||||
/*
|
||||
#cgo pkg-config: xres
|
||||
#include "commands_native.h"
|
||||
#include <X11/XKBlib.h>
|
||||
*/
|
||||
@@ -13,30 +14,52 @@ func ActiveWindowIsXTerm() bool {
|
||||
// Foreground is an immutable X11 client identity and its screen rectangle.
|
||||
// Known with Window == 0 means no foreground client, not missing metadata.
|
||||
type Foreground struct {
|
||||
Known bool
|
||||
Window uint64
|
||||
PID int
|
||||
Known bool
|
||||
Window uint64
|
||||
PID int
|
||||
// PropertyPID retains _NET_WM_PID for diagnostics. PID uses the X server's
|
||||
// local client identity when available, so Flatpak namespaces agree with AT-SPI.
|
||||
PropertyPID int
|
||||
X, Y, Width, Height int
|
||||
OuterX, OuterY, OuterWidth, OuterHeight int
|
||||
}
|
||||
|
||||
// ForegroundReader has no goroutine. Its native event-loop owner opens, reads,
|
||||
// and closes it on that thread; no consumer performs X11 I/O.
|
||||
type ForegroundReader struct{ display *C.Display }
|
||||
type ForegroundReader struct {
|
||||
display *C.Display
|
||||
clientPIDs bool
|
||||
pidWindow uint64
|
||||
propertyPID, clientPID int
|
||||
}
|
||||
|
||||
func OpenForegroundReader() *ForegroundReader {
|
||||
display := C.XOpenDisplay(nil)
|
||||
if display == nil {
|
||||
return nil
|
||||
}
|
||||
return &ForegroundReader{display: display}
|
||||
return &ForegroundReader{display: display, clientPIDs: C.cthulhu_xres_client_pids_supported(display) != 0}
|
||||
}
|
||||
func (reader *ForegroundReader) Read() Foreground {
|
||||
if reader == nil {
|
||||
return Foreground{}
|
||||
}
|
||||
value := C.read_foreground(reader.display)
|
||||
return Foreground{Known: value.known != 0, Window: uint64(value.window), PID: int(value.pid),
|
||||
window, propertyPID := uint64(value.window), int(value.pid)
|
||||
// The connection's owner also owns this one-client cache. Window departure
|
||||
// (including no foreground) or a changed property discards the old identity.
|
||||
// Resizing and ordinary input never trigger an extra server query.
|
||||
if window != reader.pidWindow || propertyPID != reader.propertyPID {
|
||||
reader.pidWindow, reader.propertyPID, reader.clientPID = window, propertyPID, 0
|
||||
if window != 0 && reader.clientPIDs {
|
||||
reader.clientPID = int(C.cthulhu_xres_client_pid(reader.display, C.Window(window)))
|
||||
}
|
||||
}
|
||||
pid := propertyPID
|
||||
if reader.clientPID > 0 {
|
||||
pid = reader.clientPID
|
||||
}
|
||||
return Foreground{Known: value.known != 0, Window: window, PID: pid, PropertyPID: propertyPID,
|
||||
X: int(value.x), Y: int(value.y), Width: int(value.width), Height: int(value.height), OuterX: int(value.outerX), OuterY: int(value.outerY), OuterWidth: int(value.outerWidth), OuterHeight: int(value.outerHeight)}
|
||||
}
|
||||
func (reader *ForegroundReader) Close() {
|
||||
|
||||
@@ -36,6 +36,8 @@ typedef struct { int xterm; SteamFocusSample steam; } CommandFocusSample;
|
||||
int cthulhu_active_window_is_xterm(void);
|
||||
int cthulhu_display_active_window_xterm_match(Display *display);
|
||||
Foreground read_foreground(Display *display);
|
||||
int cthulhu_xres_client_pids_supported(Display *display);
|
||||
int cthulhu_xres_client_pid(Display *display, Window window);
|
||||
PointerPosition cthulhu_read_pointer(Display *display);
|
||||
// Caller holds the X11 read guard; optional Steam metadata shares the same
|
||||
// active-window and WM_CLASS reads as the XTerm match.
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
// Copyright (c) 2026 Storm Dragon
|
||||
// Authors: Storm Dragon, Angus Kola
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
#include "commands_native.h"
|
||||
#include <sys/types.h>
|
||||
#include <X11/extensions/XRes.h>
|
||||
|
||||
int cthulhu_xres_client_pids_supported(Display *display) {
|
||||
int event, error, major = 0, minor = 0;
|
||||
cthulhu_x11_begin_read(display);
|
||||
int supported = XResQueryExtension(display, &event, &error) &&
|
||||
XResQueryVersion(display, &major, &minor) &&
|
||||
(major > 1 || (major == 1 && minor >= 2));
|
||||
if (cthulhu_x11_end_read(display)) return 0;
|
||||
return supported;
|
||||
}
|
||||
|
||||
// _NET_WM_PID is written by the application in its own PID namespace. XRes
|
||||
// identifies the local connection from the server side instead. Unsupported
|
||||
// servers, remote clients and disappeared clients retain the property fallback.
|
||||
int cthulhu_xres_client_pid(Display *display, Window window) {
|
||||
XResClientIdSpec spec = {.client = window, .mask = XRES_CLIENT_ID_PID_MASK};
|
||||
XResClientIdValue *ids = NULL;
|
||||
long count = 0;
|
||||
int pid = 0;
|
||||
cthulhu_x11_begin_read(display);
|
||||
if (XResQueryClientIds(display, 1, &spec, &count, &ids) == Success) {
|
||||
for (long i = 0; i < count; i++) {
|
||||
pid_t candidate = XResGetClientPid(&ids[i]);
|
||||
if (candidate > 0) {
|
||||
pid = candidate;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (ids) XResClientIdsDestroy(count, ids);
|
||||
if (cthulhu_x11_end_read(display)) return 0;
|
||||
return pid;
|
||||
}
|
||||
@@ -60,6 +60,35 @@ func TestNativeWindowTitleDiscovery(t *testing.T) {
|
||||
t.Fatal("cannot open X11")
|
||||
}
|
||||
defer reader.Close()
|
||||
// Flatpak's _NET_WM_PID is in a different namespace from the AT-SPI
|
||||
// provider. XRes must identify the actual local connection owner instead.
|
||||
xres := os.Getenv("CTHULHU_XRES_DISABLED") != "1"
|
||||
if reader.clientPIDs != xres {
|
||||
t.Fatalf("XRes client PID support = %v; want %v", reader.clientPIDs, xres)
|
||||
}
|
||||
checkPID := func(property int) {
|
||||
t.Helper()
|
||||
want := property
|
||||
if xres {
|
||||
want = fixture.Process.Pid
|
||||
}
|
||||
foreground := reader.Read()
|
||||
if foreground.PID != want || foreground.PropertyPID != property {
|
||||
t.Fatalf("foreground = %+v; want PID %d and property %d", foreground, want, property)
|
||||
}
|
||||
}
|
||||
checkPID(0)
|
||||
command("pid", "normal", "252")
|
||||
checkPID(252)
|
||||
checkPID(252) // cached reads must retain the host identity
|
||||
command("pid", "normal", "253")
|
||||
checkPID(253)
|
||||
command("empty", "normal", "")
|
||||
if foreground := reader.Read(); foreground.Window != 0 || foreground.PID != 0 || foreground.PropertyPID != 0 {
|
||||
t.Fatalf("empty workspace retained a client: %+v", foreground)
|
||||
}
|
||||
command("activate", "normal", "")
|
||||
checkPID(253)
|
||||
applications := reader.ApplicationWindows()
|
||||
var foundExample, foundGame bool
|
||||
for _, application := range applications {
|
||||
@@ -89,6 +118,7 @@ func TestNativeWindowTitleDiscovery(t *testing.T) {
|
||||
command("legacy", "normal", "Legacy title")
|
||||
expect("normal", "normal", "Legacy title", "example")
|
||||
command("activate", "desktop", "")
|
||||
checkPID(0)
|
||||
command("focus", "control", "")
|
||||
expect("desktop", "game", "Wine game", "game.exe")
|
||||
command("title", "game", "Wine game loading")
|
||||
@@ -100,9 +130,13 @@ func TestNativeWindowTitleDiscovery(t *testing.T) {
|
||||
command("hide", "game", "")
|
||||
expect("desktop", "second", "Wine dialog", "game.exe")
|
||||
command("destroy", "desktop", "")
|
||||
if foreground := reader.Read(); foreground.Window != 0 || foreground.PID != 0 {
|
||||
t.Fatalf("destroyed window retained a client: %+v", foreground)
|
||||
}
|
||||
if value := reader.ReadTitle(windows["desktop"]); value != (WindowTitle{}) {
|
||||
t.Fatalf("destroyed desktop = %+v", value)
|
||||
}
|
||||
command("activate", "normal", "")
|
||||
checkPID(253)
|
||||
expect("normal", "normal", "Legacy title", "example")
|
||||
}
|
||||
Vendored
+1
@@ -61,3 +61,4 @@ web-browse-recovery-activation,Orca src/orca/scripts/web/script_utilities.py; ma
|
||||
web-structural-lists,Orca src/orca/structural_navigator.py; src/orca/structural_navigator_command_definitions.py,web,List browser elements and move to a selected item,,internal/input internal/review internal/interaction internal/app,in-progress,"go test -race ./... -count=1; CTHULHU_NATIVE_BROWSER_TESTS=lists,keys,document,modes,editing python3 tests/native_browser.py; python3 tests/native_internal_ui.py",Private Brave/Xvfb list workflow covers 302 links hidden content current row selection Escape clickables heading level and empty list.,,Live listening pending.,Shared internal UI frames covered; physical Braille acceptance pending.,25 browse shortcuts use Alt+Shift. All Orca list types are registered including unbound annotations internal frames and math. Exact targets and page ownership are validated before selection. See docs/testing/browser-lists.md for manual review and regression checks.
|
||||
mozilla-leaf-text-reading,Orca src/orca/scripts/web/script_utilities.py,web,Read native heading and link text leaves in Firefox and Thunderbird,,internal/atspi tests/native_mozilla.py,in-progress,python3 tests/native_firefox.py; CTHULHU_BINARY=/tmp/cthulhu-mozilla-review /usr/bin/python3 tests/native_mozilla.py; go test -race ./internal/atspi ./internal/review ./internal/presentation ./internal/app -count=1 -timeout=120s,Private Firefox 158.0 and Thunderbird 158.0 sessions each cover 17 navigation commands and Read All with zero native caret synchronization errors.,,Recorded SSIP includes both headings and the link in forward/reverse reading and Read All; audible listening pending.,Shared Chromium selection and routing checks passed; physical Mozilla Braille acceptance pending.,Shared document text metrics now include heading and link roles. Native Firefox metadata assertion failed before the fix. See docs/testing/mozilla-reading.md. Thunderbird folders message lists and composition remain outside this reading baseline.
|
||||
thunderbird-local-mail-composition,Orca src/orca/scripts/apps/Thunderbird/script.py;src/orca/script_utilities.py,application,Navigate local folders and message tabs; select messages; compose replies and forwards; save and add attachments,,internal/atspi internal/app tests/native_thunderbird.py,in-progress,go test -race ./internal/atspi ./internal/app ./internal/review ./internal/presentation -count=1 -timeout=120s; CTHULHU_BINARY=/tmp/cthulhu-mozilla-review /usr/bin/python3 tests/native_thunderbird.py,Private Thunderbird 158.0 HTML and plain-text local mailbox checks cover message-tab return; Subject and body editing; reply and reply-all recipients with Cc; inline forwarding; exact selected rows after Shift and Ctrl navigation; received attachment saving with content verification; and adding a local file to an unsent draft.,,SSIP verifies focus handoff and row roles; selection toggling; Subject and body echo; reply and forward editing; attachment menus and picker return; audible acceptance pending.,Event-loop regression checks paragraph cursor and routing metadata; physical display acceptance pending.,Foreground replacement requires a registered application and fresh complete window membership. Editable paragraph events belong only to their focused editable document. Title echoes require owned editable focus and recent editing plus unchanged title prefix/suffix; unrelated status titles still announce. See docs/testing/thunderbird-workflows.md; no sending remote mailbox or page-load-option acceptance claimed.
|
||||
flatpak-foreground-identity,src/cthulhu/script_utilities.py:canBeActiveWindow,focus,Correlate Flatpak accessibility providers with the X11 foreground across PID namespaces and D-Bus proxies,,internal/platform/xorg internal/atspi,in-progress,go test -race ./internal/platform/xorg ./internal/atspi ./internal/app -count=1; CTHULHU_FOREGROUND_PID=1 python3 tests/native_atspi.py,Private XRes lookup and fallback; real proxy with matching and different synthetic instance metadata; GTK Qt and Brave ownership checks,,Actual Flatpak Steam speech pending,Physical Braille pending; original report also has helper geometry errors,docs/testing/flatpak-foreground.md; preserves active state geometry and sibling ownership checks; no application name exception
|
||||
|
+73
-3
@@ -1,6 +1,8 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Isolated native AT-SPI tree capture and invalidation proof."""
|
||||
import os
|
||||
import ast
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
@@ -19,6 +21,12 @@ def wait_for(check, message, timeout=15):
|
||||
|
||||
|
||||
def run_inside():
|
||||
if os.environ.get("CTHULHU_FOREGROUND_PID") == "1":
|
||||
if os.environ.get("CTHULHU_FOREGROUND_PROXY") == "1":
|
||||
run_proxy_probe()
|
||||
else:
|
||||
run_probe("list")
|
||||
return
|
||||
if os.environ.get("CTHULHU_MOUSE_REVIEW") == "1":
|
||||
run_probe("list")
|
||||
return
|
||||
@@ -114,14 +122,71 @@ def run_table_probe(multi: bool = False, screen_lock: bool = False, list_role: b
|
||||
fixture.wait(timeout=3)
|
||||
|
||||
|
||||
def run_probe(container, flat=False, terminal=False, terminal_mode="", echo_focus=False):
|
||||
def run_proxy_probe():
|
||||
# Real D-Bus proxy and separate mount namespaces, with the shared instance
|
||||
# metadata Flatpak gives its application and proxy. No installed Flatpak
|
||||
# application or running desktop reader is used.
|
||||
address = ast.literal_eval(subprocess.check_output([
|
||||
"gdbus", "call", "--session", "--dest", "org.a11y.Bus",
|
||||
"--object-path", "/org/a11y/bus", "--method", "org.a11y.Bus.GetAddress",
|
||||
], text=True))[0]
|
||||
with tempfile.TemporaryDirectory(prefix="cthulhu-flatpak-") as directory:
|
||||
root = Path(directory)
|
||||
info = root / "info"
|
||||
info.write_text("[Application]\nname=org.example.AccessibilityProbe\n"
|
||||
"[Instance]\ninstance-id=12345\n")
|
||||
# A fresh tmpfs root lets us add the metadata without writing to /.
|
||||
prefix = ["bwrap", "--die-with-parent"]
|
||||
for path in ("/usr", "/etc", "/var", "/run", "/tmp", "/home", "/proc",
|
||||
"/bin", "/sbin", "/lib", "/lib64"):
|
||||
if Path(path).is_symlink():
|
||||
prefix += ["--symlink", os.readlink(path), path]
|
||||
elif Path(path).exists():
|
||||
prefix += ["--ro-bind", path, path]
|
||||
prefix += ["--dev", "/dev", "--bind", directory, directory,
|
||||
"--ro-bind", str(info), "/.flatpak-info"]
|
||||
read_fd, write_fd = os.pipe()
|
||||
socket = str(root / "bus")
|
||||
proxy = subprocess.Popen(prefix + ["xdg-dbus-proxy", address, socket,
|
||||
"--fd=" + str(write_fd)], pass_fds=(write_fd,))
|
||||
os.close(write_fd)
|
||||
try:
|
||||
with os.fdopen(read_fd, "rb") as ready:
|
||||
if ready.read(1) != b"x":
|
||||
raise RuntimeError("Accessibility proxy failed to start")
|
||||
fixture_prefix = prefix
|
||||
if os.environ.get("CTHULHU_FOREGROUND_PROXY_OTHER") == "1":
|
||||
other = root / "other-info"
|
||||
other.write_text(info.read_text().replace("12345", "67890"))
|
||||
fixture_prefix = prefix + ["--ro-bind", str(other), "/.flatpak-info"]
|
||||
run_probe("list", fixture_prefix=fixture_prefix,
|
||||
fixture_overrides={"AT_SPI_BUS_ADDRESS": "unix:path=" + socket,
|
||||
"GSETTINGS_BACKEND": "memory"})
|
||||
finally:
|
||||
proxy.terminate()
|
||||
proxy.wait(timeout=3)
|
||||
|
||||
|
||||
def run_probe(container, flat=False, terminal=False, terminal_mode="", echo_focus=False,
|
||||
fixture_prefix=(), fixture_overrides=None):
|
||||
environment = dict(os.environ, CTHULHU_TREE_CONTAINER=container)
|
||||
environment["CTHULHU_VTE_MODE"] = terminal_mode
|
||||
fixture = subprocess.Popen([
|
||||
prefix = list(fixture_prefix)
|
||||
pass_fds = ()
|
||||
if prefix:
|
||||
read_fd, write_fd = os.pipe()
|
||||
prefix += ["--info-fd", str(write_fd)]
|
||||
pass_fds = (write_fd,)
|
||||
fixture = subprocess.Popen(prefix + [
|
||||
sys.executable,
|
||||
str(Path(__file__).with_name("vte_terminal.py" if terminal else "gtk_tree.py")),
|
||||
], env=environment)
|
||||
], env=dict(environment, **(fixture_overrides or {})), pass_fds=pass_fds)
|
||||
try:
|
||||
fixture_pid = fixture.pid
|
||||
if prefix:
|
||||
os.close(write_fd)
|
||||
with os.fdopen(read_fd) as information:
|
||||
fixture_pid = json.load(information)["child-pid"]
|
||||
def find_window():
|
||||
result = subprocess.run(
|
||||
["xdotool", "search", "--onlyvisible", "--name", "^Cthulhu Tree Probe$"],
|
||||
@@ -132,6 +197,10 @@ def run_probe(container, flat=False, terminal=False, terminal_mode="", echo_focu
|
||||
|
||||
window = wait_for(find_window, "GTK tree probe window missing")
|
||||
subprocess.run(["xdotool", "windowfocus", window], check=True)
|
||||
if os.environ.get("CTHULHU_FOREGROUND_PID") == "1":
|
||||
environment["CTHULHU_TREE_PID"] = str(fixture_pid)
|
||||
subprocess.run(["xprop", "-id", window, "-f", "_NET_WM_PID", "32c",
|
||||
"-set", "_NET_WM_PID", "252"], check=True)
|
||||
environment.update(CTHULHU_NATIVE_ATSPI="1", CTHULHU_TREE_WINDOW=window)
|
||||
if os.environ.get("CTHULHU_MOUSE_REVIEW") == "1":
|
||||
environment["CTHULHU_NATIVE_MOUSE_REVIEW"] = "1"
|
||||
@@ -146,6 +215,7 @@ def run_probe(container, flat=False, terminal=False, terminal_mode="", echo_focu
|
||||
"test",
|
||||
"./internal/review" if flat else "./internal/atspi",
|
||||
"-run",
|
||||
"^TestNativeForegroundProcessIdentity$" if os.environ.get("CTHULHU_FOREGROUND_PID") == "1" else
|
||||
"^TestNativeMouseReviewPointerAndHitTest$" if os.environ.get("CTHULHU_MOUSE_REVIEW") == "1" else
|
||||
"^TestNativeEchoEditFieldFocus$" if echo_focus else
|
||||
"^TestNativeSpeechMetadata$" if os.environ.get("CTHULHU_SPEECH_METADATA") == "1" else
|
||||
|
||||
@@ -1325,19 +1325,31 @@ def inside():
|
||||
focus(named("Use custom echo voice")); key("space")
|
||||
echo_rate = named("Key echo rate")
|
||||
wait_for(lambda: echo_rate.getState().contains(pyatspi.STATE_SENSITIVE), "custom voice controls disabled")
|
||||
focus(echo_rate); echo_rate.queryValue().currentValue = 83
|
||||
focus(named("Key echo pitch")); named("Key echo pitch").queryValue().currentValue = 7.5
|
||||
focus(named("Key echo volume")); named("Key echo volume").queryValue().currentValue = 10
|
||||
# Only one preview may be outstanding. Read and acknowledge each
|
||||
# changed value before moving focus, which discards stale previews.
|
||||
for control, field, value in (("rate", "Rate", 83),
|
||||
("pitch", "AveragePitch", 7.5),
|
||||
("volume", "Gain", 10)):
|
||||
slider = named("Key echo " + control)
|
||||
focus(slider)
|
||||
slider.queryValue().currentValue = value
|
||||
preview = request()
|
||||
assert preview["Preview"] and preview.get("EchoPreview") == control, preview
|
||||
assert preview["Values"]["EchoVoice"][field] == value, preview
|
||||
send({"Preview": True, "Accepted": True})
|
||||
family = named("Key echo person")
|
||||
family.querySelection().selectChild(1)
|
||||
# Echo selection previews require keyboard focus in the selector.
|
||||
focus(named("Use custom echo voice")); key("Tab", "Tab", "Down")
|
||||
preview = request()
|
||||
assert preview["Preview"] and preview.get("EchoPreview") == "person", preview
|
||||
assert preview["Values"]["EchoVoice"]["Family"]["Name"] == "Test voice", preview
|
||||
assert family.querySelection().getSelectedChild(0).name == "Test voice (en)"
|
||||
send({"Preview": True, "Accepted": True})
|
||||
word_voice = named("Words")
|
||||
assert not word_voice.getState().contains(pyatspi.STATE_CHECKED)
|
||||
focus(word_voice); key("space")
|
||||
wait_for(lambda: word_voice.getState().contains(pyatspi.STATE_CHECKED),
|
||||
"custom word voice did not become checked")
|
||||
preview = request()
|
||||
assert preview["Preview"], preview
|
||||
send({"Preview": True, "Accepted": True, "Message": "Speech settings previewed."})
|
||||
focus(named("OK")); key("space")
|
||||
echo_saved = request()
|
||||
ev = echo_saved["Values"]
|
||||
|
||||
@@ -57,6 +57,11 @@ def main():
|
||||
window.set_wm_name(command["title"])
|
||||
elif action == "class":
|
||||
window.set_wm_class(command["instance"], command["class"])
|
||||
elif action == "pid":
|
||||
window.change_property(connection.intern_atom("_NET_WM_PID"),
|
||||
Xatom.CARDINAL, 32, [int(command["title"])])
|
||||
elif action == "empty":
|
||||
root.change_property(activeAtom, Xatom.WINDOW, 32, [0])
|
||||
elif action == "hide":
|
||||
window.unmap()
|
||||
elif action == "destroy":
|
||||
|
||||
Reference in new issue
Block a user