Attempt to fix problems with flatpacks, e.g. Steam.

This commit is contained in:
ak47 committed 2026-10-17 20:58:35 -04:00
1 parent 1c4e0a99bc
commit 53d14d561b
20 files changed
+510 -20

No files matched your search

+1
View File
@@ -7,6 +7,7 @@ Cthulhu is a Linux screen reader written in Go.
- [LibreOffice support](LIBREOFFICE_SUPPORT.md): integration boundaries, initial Calc behavior, remaining work, and verification.
- [Progress reporting](PROGRESS_BARS.md): web progress bars, Steam shader processing, and verification.
- [Steam keyboard focus](STEAM_FOCUS.md): Xorg workspace focus recovery and verification.
- [Flatpak foreground accessibility](docs/testing/flatpak-foreground.md): process identity and proxy matching, tests, and live acceptance.
- [Wayland keyboard access](WAYLAND_SUPPORT.md): shared protocol support, limitations, and verification.
AI-assisted contributions are welcome. Contributors remain responsible for human review and testing before submission; see the [full policy](CONTRIBUTING.md#ai-assisted-contributions).
@@ -22,6 +22,7 @@ pkgbase = cthulhu-git
depends = brltty
depends = liblouis
depends = libx11
depends = libxres
depends = libxtst
depends = libxfixes
depends = libxkbcommon
@@ -31,6 +31,7 @@ depends=(
# Xorg input, window capture, and clipboard operations
libx11
libxres
libxtst
libxfixes
libxkbcommon
+80
View File
@@ -0,0 +1,80 @@
# Flatpak foreground accessibility
The Python `master` reference's `script_utilities.py:canBeActiveWindow`
requires active, showing, non-iconified state. It does not require X11 and
AT-SPI process IDs or window rectangles to agree. Go's additional correlation
checks exposed two Flatpak cases:
- `_NET_WM_PID` can be relative to the application's PID namespace. The Xorg
foreground reader now prefers XRes 1.2's local connection PID. It retains
the property as a fallback when server-side lookup is unavailable.
- AT-SPI's process query returns the D-Bus connection owner, which can be a
proxy. The adapter accepts different processes only when their
`/proc/PID/root/.flatpak-info` files identify the same application and running
instance. Active state, geometry and existing window ownership still apply.
This is shared foreground handling, with no Steam name or window-title match.
It covers direct events, foreground recovery, menu correlation and OCR's
process check. It does not establish compatibility with every Flatpak, native
Wayland focus, or application-specific scripts behind a proxy. Missing or
unreadable instance metadata does not relax matching. The reported Steam 2x2
auxiliary frames remain subject to the ordinary geometry checks.
XRes adds the `libxres` Arch dependency (`xres` pkg-config module). Debug logs
record both the resolved X11 PID and original property PID.
The Xorg owner caches one client's lookup until the window or property changes.
The AT-SPI GLib owner bounds provider comparisons to 128 entries and clears them
when the foreground window, process or known state changes. Metadata reads are
limited to regular files of at most 64 KiB. No process scan or service manager
is used.
References: [AT-SPI process query](https://gnome.pages.gitlab.gnome.org/gtk/atspi2/method.Accessible.get_process_id.html),
[proxy credentials](https://github.com/flatpak/xdg-dbus-proxy/blob/main/xdg-dbus-proxy.xml),
[Flatpak instance metadata](https://github.com/flatpak/flatpak/blob/main/doc/flatpak-metadata.xml),
[libwnck's XRes namespace correction](https://lists.gnome.org/archives/commits-list/2022-March/msg10404.html).
## Automated checks
Run the private fixtures sequentially. The GTK mismatch test fails before the
XRes change. The proxy fixture uses real bubblewrap mount namespaces and
xdg-dbus-proxy, with synthetic instance metadata; it needs neither an installed
Flatpak nor a running desktop reader.
```sh
CTHULHU_FOREGROUND_PID=1 python3 tests/native_atspi.py
CTHULHU_FOREGROUND_PID=1 CTHULHU_FOREGROUND_PROXY=1 python3 tests/native_atspi.py
CTHULHU_FOREGROUND_PID=1 CTHULHU_FOREGROUND_PROXY=1 CTHULHU_FOREGROUND_PROXY_OTHER=1 python3 tests/native_atspi.py
xvfb-run -a env CTHULHU_NATIVE_WINDOW_TITLES=1 go test ./internal/platform/xorg -run '^TestNativeWindowTitleDiscovery$' -count=1
xvfb-run -a -s '-screen 0 1280x1024x24 -extension X-Resource' env CTHULHU_NATIVE_WINDOW_TITLES=1 CTHULHU_XRES_DISABLED=1 go test ./internal/platform/xorg -run '^TestNativeWindowTitleDiscovery$' -count=1
```
The final proxy case must reject the control from a different instance of the
same application. Xorg tests cover cached reads, property changes, empty
workspaces, destroyed windows and fallback without XRes. Also run the affected
Go packages and `tests/native_ownership.py` with `CTHULHU_BINARY` pointing to a
fresh isolated build; that checks GTK, Qt and Brave foreground boundaries.
## Live acceptance
Actual Flatpak Steam, audible speech and physical Braille remain unverified.
On the maintainer's i3 setup:
1. Quit Cthulhu with Cthulhu+Q. Press Ctrl+Escape, then Shift+G as separate key
presses. Shift+G runs `~/devel/cthulhu-go.sh` and rebuilds both binaries.
2. Launch `flatpak --user run com.valvesoftware.Steam -tenfoot`. Press Tab and
Shift+Tab through controls. Each focus move should announce the control
label and role; a working Braille display should follow that same control.
3. Open and close a Steam menu with Escape, navigate with arrows and Enter,
then type in search. Keys should reach Steam and speech should track the
selected control or edited text.
4. Switch to another application's window and back. Steam's ordinary control
events should not interrupt the other application; returning should restore
announcements. Repeat with another available Flatpak and a native browser.
5. Report the exact spoken text, Braille text if connected, focused control and
key that differed from expectation. Preserve `~/devel/cthulhu-go.log` after
an unexpected result. A remote tester should use their own equivalent rebuild
and debug-log launcher; the i3 bindings above are local customizations.
The original report also contains independent Braille geometry errors. This
foreground change does not repair the Braille helper's connection failure.
+1 -1
View File
@@ -132,7 +132,7 @@ func (handler *accessibilityEventHandler) handle(event atspi.Event) {
"bus", event.Window.BusName, "window_path", event.Window.Path,
"x_window", client.Window, "epoch", event.Foreground.Epoch,
"current_epoch", state.foregroundState.Epoch, "reason", check.Reason,
"x_pid", client.PID, "accessible_pid", check.AccessiblePID,
"x_pid", client.PID, "x_property_pid", client.PropertyPID, "accessible_pid", check.AccessiblePID,
"active", check.Active, "showing", check.Showing, "iconified", check.Iconified,
"accessible_bounds", check.Bounds,
"x_bounds", [4]int{client.X, client.Y, client.Width, client.Height},
+2 -1
View File
@@ -195,7 +195,8 @@ func (state *eventLoopState) syncForeground(foreground foregroundMonitor, logger
state.owner.clear()
clearOwnedFocus()
logger.Debug("X11 foreground ownership changed", "x_window", latest.Client.Window,
"pid", latest.Client.PID, "known", latest.Client.Known, "epoch", latest.Epoch)
"pid", latest.Client.PID, "property_pid", latest.Client.PropertyPID,
"known", latest.Client.Known, "epoch", latest.Epoch)
}
state.foregroundState = latest
}
+1
View File
@@ -114,6 +114,7 @@ type Adapter struct {
applicationProcesses map[string]string // GLib owner; bounded provider identity cache
webNotificationRoots map[ObjectID]bool // copied identities only; no native references
foregroundReader *xorg.ForegroundReader // GLib thread only
foregroundProcesses map[int]bool // GLib thread; provider PID matches for current client
foreground atomic.Pointer[ForegroundState]
echoFocus atomic.Pointer[echoFocus]
eventListener *C.AtspiEventListener // GLib owner; released with subscriptions.
+5 -1
View File
@@ -29,6 +29,9 @@ func (adapter *Adapter) refreshForeground() {
return
}
state := ForegroundState{Client: value, Epoch: previous.Epoch}
if value.Window != previous.Client.Window || value.PID != previous.Client.PID || value.Known != previous.Client.Known {
clear(adapter.foregroundProcesses)
}
if value.Known != previous.Client.Known || value.Window != previous.Client.Window {
state.Epoch++
adapter.foregroundWindow = ObjectID{}
@@ -78,6 +81,7 @@ func (adapter *Adapter) matchForeground(window *C.AtspiAccessible, id ObjectID,
if check != nil {
check.AccessiblePID = pid
}
pid = adapter.foregroundProcessID(pid)
if adapter.foregroundWindow.Valid() {
if adapter.foregroundWindow == id {
return true
@@ -265,7 +269,7 @@ func (adapter *Adapter) recoverForeground() {
continue
}
pid := int(C.atspi_accessible_get_process_id(application, nil))
if pid != state.Client.PID {
if adapter.foregroundProcessID(pid) != state.Client.PID {
C.g_object_unref(C.gpointer(application))
continue
}
+1 -1
View File
@@ -97,7 +97,7 @@ func (adapter *Adapter) nativeOCRGeometry(ctx context.Context, object ObjectID,
return OCRGeometry{}, ErrOCRGeometry
}
defer C.g_object_unref(C.gpointer(window))
if int(C.atspi_accessible_get_process_id(window, nil)) != before.Client.PID || (adapter.foregroundWindow.Valid() && adapter.foregroundWindow != nativeObjectID(window)) {
if adapter.foregroundProcessID(int(C.atspi_accessible_get_process_id(window, nil))) != before.Client.PID || (adapter.foregroundWindow.Valid() && adapter.foregroundWindow != nativeObjectID(window)) {
return OCRGeometry{}, ErrOCRGeometry
}
states := C.atspi_accessible_get_state_set(window)
+100
View File
@@ -0,0 +1,100 @@
// Copyright (c) 2026 Storm Dragon
// Authors: Storm Dragon, Angus Kola
// SPDX-License-Identifier: GPL-3.0-or-later
package atspi
import (
"fmt"
"io"
"os"
"strings"
"syscall"
)
type flatpakInstance struct {
application, instance string
}
// Flatpak gives its D-Bus proxy and application the same /.flatpak-info.
// Compare a running instance, never just an application ID: two instances of
// the same application must not acquire one another's foreground window.
func readFlatpakInstance(procRoot string, pid int) flatpakInstance {
if pid <= 0 {
return flatpakInstance{}
}
file, err := os.OpenFile(fmt.Sprintf("%s/%d/root/.flatpak-info", procRoot, pid), os.O_RDONLY|syscall.O_NONBLOCK, 0)
if err != nil {
return flatpakInstance{}
}
defer file.Close()
const limit = 64 * 1024
stat, err := file.Stat()
if err != nil || !stat.Mode().IsRegular() || stat.Size() > limit {
return flatpakInstance{}
}
data, err := io.ReadAll(io.LimitReader(file, limit+1))
if err != nil || len(data) > limit {
return flatpakInstance{}
}
var result flatpakInstance
var section string
for _, line := range strings.Split(string(data), "\n") {
line = strings.TrimSpace(line)
if line == "" || strings.HasPrefix(line, "#") || strings.HasPrefix(line, ";") {
continue
}
if strings.HasPrefix(line, "[") && strings.HasSuffix(line, "]") {
section = line
continue
}
key, value, ok := strings.Cut(line, "=")
if !ok {
return flatpakInstance{}
}
key, value = strings.TrimSpace(key), strings.TrimSpace(value)
if section == "[Application]" && key == "name" {
if result.application != "" {
return flatpakInstance{}
}
result.application = value
}
if section == "[Instance]" && key == "instance-id" {
if result.instance != "" {
return flatpakInstance{}
}
result.instance = value
}
}
if result.application == "" || result.instance == "" {
return flatpakInstance{}
}
return result
}
func sameFlatpakInstance(procRoot string, first, second int) bool {
identity := readFlatpakInstance(procRoot, first)
return identity != (flatpakInstance{}) && identity == readFlatpakInstance(procRoot, second)
}
// Owned by the GLib thread and cleared with the foreground client. This only
// resolves process identity; fresh window state and geometry still decide
// whether an accessible can own focus.
func (adapter *Adapter) foregroundProcessID(pid int) int {
client := adapter.Foreground().Client
if client.Window == 0 || client.PID <= 0 || pid <= 0 || pid == client.PID {
return pid
}
matched, known := adapter.foregroundProcesses[pid]
if !known {
matched = sameFlatpakInstance("/proc", client.PID, pid)
if adapter.foregroundProcesses == nil || len(adapter.foregroundProcesses) >= 128 {
adapter.foregroundProcesses = make(map[int]bool)
}
adapter.foregroundProcesses[pid] = matched
}
if matched {
return client.PID
}
return pid
}
+54
View File
@@ -0,0 +1,54 @@
// Copyright (c) 2026 Storm Dragon
// Authors: Storm Dragon, Angus Kola
// SPDX-License-Identifier: GPL-3.0-or-later
package atspi
import (
"fmt"
"os"
"path/filepath"
"strings"
"testing"
)
func TestFlatpakProcessInstanceBoundaries(t *testing.T) {
root := t.TempDir()
write := func(pid int, data string) {
t.Helper()
dir := filepath.Join(root, fmt.Sprint(pid), "root")
if err := os.MkdirAll(dir, 0700); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, ".flatpak-info"), []byte(data), 0600); err != nil {
t.Fatal(err)
}
}
metadata := "[Application]\nname=org.example.App\n[Instance]\ninstance-id=123\n"
write(100, metadata)
for _, test := range []struct {
name, data string
want bool
}{
{"same instance", metadata, true},
{"other instance of same app", strings.ReplaceAll(metadata, "123", "456"), false},
{"other app", strings.ReplaceAll(metadata, "org.example.App", "org.example.Other"), false},
{"app ID alone", "[Application]\nname=org.example.App\n", false},
{"instance ID alone", "[Instance]\ninstance-id=123\n", false},
{"wrong section", "[Application]\nname=org.example.App\ninstance-id=123\n", false},
{"empty metadata", "", false},
{"malformed metadata", metadata + "broken", false},
{"duplicate identity", metadata + "instance-id=456\n", false},
{"oversized metadata", metadata + strings.Repeat("#", 65536), false},
} {
t.Run(test.name, func(t *testing.T) {
write(200, test.data)
if got := sameFlatpakInstance(root, 100, 200); got != test.want {
t.Fatalf("same instance = %v; want %v", got, test.want)
}
})
}
if sameFlatpakInstance(root, 100, 300) || sameFlatpakInstance(root, 300, 400) || sameFlatpakInstance(root, 0, 0) {
t.Fatal("missing process or unknown identity matched")
}
}
+60
View File
@@ -0,0 +1,60 @@
// Copyright (c) 2026 Storm Dragon
// Authors: Storm Dragon, Angus Kola
// SPDX-License-Identifier: GPL-3.0-or-later
package atspi
import (
"context"
"os"
"os/exec"
"strconv"
"testing"
"time"
)
// The GTK provider publishes its host PID over AT-SPI, but its X11 property
// deliberately contains a different PID, as with a Flatpak PID namespace.
func TestNativeForegroundProcessIdentity(t *testing.T) {
if os.Getenv("CTHULHU_FOREGROUND_PID") != "1" {
t.Skip("requires isolated GTK/X11 process identity fixture")
}
pid, err := strconv.Atoi(os.Getenv("CTHULHU_TREE_PID"))
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
adapter := New()
if err := adapter.Start(ctx); err != nil {
t.Fatal(err)
}
defer adapter.Stop(context.Background())
if _, err := adapter.CaptureDesktop(ctx); err != nil {
t.Fatal(err)
}
if foreground := adapter.Foreground(); foreground.Client.PID != pid {
t.Fatalf("foreground PID = %d; want host provider PID %d (window property is 252)", foreground.Client.PID, pid)
}
if err := exec.CommandContext(ctx, "xdotool", "key", "F6").Run(); err != nil {
t.Fatal(err)
}
for {
select {
case event := <-adapter.Events():
if event.Type != FocusedStateEvent || event.Detail1 == 0 || event.Source.Name != "Caret review probe" {
continue
}
wantMismatch := os.Getenv("CTHULHU_FOREGROUND_PROXY_OTHER") == "1"
if event.ForegroundMismatch != wantMismatch || event.Foreground.Client.PID != pid || !event.Window.Valid() {
t.Fatalf("foreground mismatch = %v; want %v: %+v", event.ForegroundMismatch, wantMismatch, event)
}
if os.Getenv("CTHULHU_FOREGROUND_PROXY") == "1" && event.Source.ApplicationProcess != "xdg-dbus-proxy" {
t.Fatalf("fixture did not use a separate D-Bus proxy: %q", event.Source.ApplicationProcess)
}
return
case <-ctx.Done():
t.Fatal("foreground control focus event missing")
}
}
}
+29 -6
View File
@@ -1,6 +1,7 @@
package xorg
/*
#cgo pkg-config: xres
#include "commands_native.h"
#include <X11/XKBlib.h>
*/
@@ -13,30 +14,52 @@ func ActiveWindowIsXTerm() bool {
// Foreground is an immutable X11 client identity and its screen rectangle.
// Known with Window == 0 means no foreground client, not missing metadata.
type Foreground struct {
Known bool
Window uint64
PID int
Known bool
Window uint64
PID int
// PropertyPID retains _NET_WM_PID for diagnostics. PID uses the X server's
// local client identity when available, so Flatpak namespaces agree with AT-SPI.
PropertyPID int
X, Y, Width, Height int
OuterX, OuterY, OuterWidth, OuterHeight int
}
// ForegroundReader has no goroutine. Its native event-loop owner opens, reads,
// and closes it on that thread; no consumer performs X11 I/O.
type ForegroundReader struct{ display *C.Display }
type ForegroundReader struct {
display *C.Display
clientPIDs bool
pidWindow uint64
propertyPID, clientPID int
}
func OpenForegroundReader() *ForegroundReader {
display := C.XOpenDisplay(nil)
if display == nil {
return nil
}
return &ForegroundReader{display: display}
return &ForegroundReader{display: display, clientPIDs: C.cthulhu_xres_client_pids_supported(display) != 0}
}
func (reader *ForegroundReader) Read() Foreground {
if reader == nil {
return Foreground{}
}
value := C.read_foreground(reader.display)
return Foreground{Known: value.known != 0, Window: uint64(value.window), PID: int(value.pid),
window, propertyPID := uint64(value.window), int(value.pid)
// The connection's owner also owns this one-client cache. Window departure
// (including no foreground) or a changed property discards the old identity.
// Resizing and ordinary input never trigger an extra server query.
if window != reader.pidWindow || propertyPID != reader.propertyPID {
reader.pidWindow, reader.propertyPID, reader.clientPID = window, propertyPID, 0
if window != 0 && reader.clientPIDs {
reader.clientPID = int(C.cthulhu_xres_client_pid(reader.display, C.Window(window)))
}
}
pid := propertyPID
if reader.clientPID > 0 {
pid = reader.clientPID
}
return Foreground{Known: value.known != 0, Window: window, PID: pid, PropertyPID: propertyPID,
X: int(value.x), Y: int(value.y), Width: int(value.width), Height: int(value.height), OuterX: int(value.outerX), OuterY: int(value.outerY), OuterWidth: int(value.outerWidth), OuterHeight: int(value.outerHeight)}
}
func (reader *ForegroundReader) Close() {
+2
View File
@@ -36,6 +36,8 @@ typedef struct { int xterm; SteamFocusSample steam; } CommandFocusSample;
int cthulhu_active_window_is_xterm(void);
int cthulhu_display_active_window_xterm_match(Display *display);
Foreground read_foreground(Display *display);
int cthulhu_xres_client_pids_supported(Display *display);
int cthulhu_xres_client_pid(Display *display, Window window);
PointerPosition cthulhu_read_pointer(Display *display);
// Caller holds the X11 read guard; optional Steam metadata shares the same
// active-window and WM_CLASS reads as the XTerm match.
+40
View File
@@ -0,0 +1,40 @@
// Copyright (c) 2026 Storm Dragon
// Authors: Storm Dragon, Angus Kola
// SPDX-License-Identifier: GPL-3.0-or-later
#include "commands_native.h"
#include <sys/types.h>
#include <X11/extensions/XRes.h>
int cthulhu_xres_client_pids_supported(Display *display) {
int event, error, major = 0, minor = 0;
cthulhu_x11_begin_read(display);
int supported = XResQueryExtension(display, &event, &error) &&
XResQueryVersion(display, &major, &minor) &&
(major > 1 || (major == 1 && minor >= 2));
if (cthulhu_x11_end_read(display)) return 0;
return supported;
}
// _NET_WM_PID is written by the application in its own PID namespace. XRes
// identifies the local connection from the server side instead. Unsupported
// servers, remote clients and disappeared clients retain the property fallback.
int cthulhu_xres_client_pid(Display *display, Window window) {
XResClientIdSpec spec = {.client = window, .mask = XRES_CLIENT_ID_PID_MASK};
XResClientIdValue *ids = NULL;
long count = 0;
int pid = 0;
cthulhu_x11_begin_read(display);
if (XResQueryClientIds(display, 1, &spec, &count, &ids) == Success) {
for (long i = 0; i < count; i++) {
pid_t candidate = XResGetClientPid(&ids[i]);
if (candidate > 0) {
pid = candidate;
break;
}
}
}
if (ids) XResClientIdsDestroy(count, ids);
if (cthulhu_x11_end_read(display)) return 0;
return pid;
}
@@ -60,6 +60,35 @@ func TestNativeWindowTitleDiscovery(t *testing.T) {
t.Fatal("cannot open X11")
}
defer reader.Close()
// Flatpak's _NET_WM_PID is in a different namespace from the AT-SPI
// provider. XRes must identify the actual local connection owner instead.
xres := os.Getenv("CTHULHU_XRES_DISABLED") != "1"
if reader.clientPIDs != xres {
t.Fatalf("XRes client PID support = %v; want %v", reader.clientPIDs, xres)
}
checkPID := func(property int) {
t.Helper()
want := property
if xres {
want = fixture.Process.Pid
}
foreground := reader.Read()
if foreground.PID != want || foreground.PropertyPID != property {
t.Fatalf("foreground = %+v; want PID %d and property %d", foreground, want, property)
}
}
checkPID(0)
command("pid", "normal", "252")
checkPID(252)
checkPID(252) // cached reads must retain the host identity
command("pid", "normal", "253")
checkPID(253)
command("empty", "normal", "")
if foreground := reader.Read(); foreground.Window != 0 || foreground.PID != 0 || foreground.PropertyPID != 0 {
t.Fatalf("empty workspace retained a client: %+v", foreground)
}
command("activate", "normal", "")
checkPID(253)
applications := reader.ApplicationWindows()
var foundExample, foundGame bool
for _, application := range applications {
@@ -89,6 +118,7 @@ func TestNativeWindowTitleDiscovery(t *testing.T) {
command("legacy", "normal", "Legacy title")
expect("normal", "normal", "Legacy title", "example")
command("activate", "desktop", "")
checkPID(0)
command("focus", "control", "")
expect("desktop", "game", "Wine game", "game.exe")
command("title", "game", "Wine game loading")
@@ -100,9 +130,13 @@ func TestNativeWindowTitleDiscovery(t *testing.T) {
command("hide", "game", "")
expect("desktop", "second", "Wine dialog", "game.exe")
command("destroy", "desktop", "")
if foreground := reader.Read(); foreground.Window != 0 || foreground.PID != 0 {
t.Fatalf("destroyed window retained a client: %+v", foreground)
}
if value := reader.ReadTitle(windows["desktop"]); value != (WindowTitle{}) {
t.Fatalf("destroyed desktop = %+v", value)
}
command("activate", "normal", "")
checkPID(253)
expect("normal", "normal", "Legacy title", "example")
}
+1
View File
@@ -61,3 +61,4 @@ web-browse-recovery-activation,Orca src/orca/scripts/web/script_utilities.py; ma
web-structural-lists,Orca src/orca/structural_navigator.py; src/orca/structural_navigator_command_definitions.py,web,List browser elements and move to a selected item,,internal/input internal/review internal/interaction internal/app,in-progress,"go test -race ./... -count=1; CTHULHU_NATIVE_BROWSER_TESTS=lists,keys,document,modes,editing python3 tests/native_browser.py; python3 tests/native_internal_ui.py",Private Brave/Xvfb list workflow covers 302 links hidden content current row selection Escape clickables heading level and empty list.,,Live listening pending.,Shared internal UI frames covered; physical Braille acceptance pending.,25 browse shortcuts use Alt+Shift. All Orca list types are registered including unbound annotations internal frames and math. Exact targets and page ownership are validated before selection. See docs/testing/browser-lists.md for manual review and regression checks.
mozilla-leaf-text-reading,Orca src/orca/scripts/web/script_utilities.py,web,Read native heading and link text leaves in Firefox and Thunderbird,,internal/atspi tests/native_mozilla.py,in-progress,python3 tests/native_firefox.py; CTHULHU_BINARY=/tmp/cthulhu-mozilla-review /usr/bin/python3 tests/native_mozilla.py; go test -race ./internal/atspi ./internal/review ./internal/presentation ./internal/app -count=1 -timeout=120s,Private Firefox 158.0 and Thunderbird 158.0 sessions each cover 17 navigation commands and Read All with zero native caret synchronization errors.,,Recorded SSIP includes both headings and the link in forward/reverse reading and Read All; audible listening pending.,Shared Chromium selection and routing checks passed; physical Mozilla Braille acceptance pending.,Shared document text metrics now include heading and link roles. Native Firefox metadata assertion failed before the fix. See docs/testing/mozilla-reading.md. Thunderbird folders message lists and composition remain outside this reading baseline.
thunderbird-local-mail-composition,Orca src/orca/scripts/apps/Thunderbird/script.py;src/orca/script_utilities.py,application,Navigate local folders and message tabs; select messages; compose replies and forwards; save and add attachments,,internal/atspi internal/app tests/native_thunderbird.py,in-progress,go test -race ./internal/atspi ./internal/app ./internal/review ./internal/presentation -count=1 -timeout=120s; CTHULHU_BINARY=/tmp/cthulhu-mozilla-review /usr/bin/python3 tests/native_thunderbird.py,Private Thunderbird 158.0 HTML and plain-text local mailbox checks cover message-tab return; Subject and body editing; reply and reply-all recipients with Cc; inline forwarding; exact selected rows after Shift and Ctrl navigation; received attachment saving with content verification; and adding a local file to an unsent draft.,,SSIP verifies focus handoff and row roles; selection toggling; Subject and body echo; reply and forward editing; attachment menus and picker return; audible acceptance pending.,Event-loop regression checks paragraph cursor and routing metadata; physical display acceptance pending.,Foreground replacement requires a registered application and fresh complete window membership. Editable paragraph events belong only to their focused editable document. Title echoes require owned editable focus and recent editing plus unchanged title prefix/suffix; unrelated status titles still announce. See docs/testing/thunderbird-workflows.md; no sending remote mailbox or page-load-option acceptance claimed.
flatpak-foreground-identity,src/cthulhu/script_utilities.py:canBeActiveWindow,focus,Correlate Flatpak accessibility providers with the X11 foreground across PID namespaces and D-Bus proxies,,internal/platform/xorg internal/atspi,in-progress,go test -race ./internal/platform/xorg ./internal/atspi ./internal/app -count=1; CTHULHU_FOREGROUND_PID=1 python3 tests/native_atspi.py,Private XRes lookup and fallback; real proxy with matching and different synthetic instance metadata; GTK Qt and Brave ownership checks,,Actual Flatpak Steam speech pending,Physical Braille pending; original report also has helper geometry errors,docs/testing/flatpak-foreground.md; preserves active state geometry and sibling ownership checks; no application name exception
1 id python_source subsystem behavior owner go_destination status automated_evidence x11_evidence wayland_evidence speech_evidence braille_evidence notes
61 web-structural-lists Orca src/orca/structural_navigator.py; src/orca/structural_navigator_command_definitions.py web List browser elements and move to a selected item internal/input internal/review internal/interaction internal/app in-progress go test -race ./... -count=1; CTHULHU_NATIVE_BROWSER_TESTS=lists,keys,document,modes,editing python3 tests/native_browser.py; python3 tests/native_internal_ui.py Private Brave/Xvfb list workflow covers 302 links hidden content current row selection Escape clickables heading level and empty list. Live listening pending. Shared internal UI frames covered; physical Braille acceptance pending. 25 browse shortcuts use Alt+Shift. All Orca list types are registered including unbound annotations internal frames and math. Exact targets and page ownership are validated before selection. See docs/testing/browser-lists.md for manual review and regression checks.
62 mozilla-leaf-text-reading Orca src/orca/scripts/web/script_utilities.py web Read native heading and link text leaves in Firefox and Thunderbird internal/atspi tests/native_mozilla.py in-progress python3 tests/native_firefox.py; CTHULHU_BINARY=/tmp/cthulhu-mozilla-review /usr/bin/python3 tests/native_mozilla.py; go test -race ./internal/atspi ./internal/review ./internal/presentation ./internal/app -count=1 -timeout=120s Private Firefox 158.0 and Thunderbird 158.0 sessions each cover 17 navigation commands and Read All with zero native caret synchronization errors. Recorded SSIP includes both headings and the link in forward/reverse reading and Read All; audible listening pending. Shared Chromium selection and routing checks passed; physical Mozilla Braille acceptance pending. Shared document text metrics now include heading and link roles. Native Firefox metadata assertion failed before the fix. See docs/testing/mozilla-reading.md. Thunderbird folders message lists and composition remain outside this reading baseline.
63 thunderbird-local-mail-composition Orca src/orca/scripts/apps/Thunderbird/script.py;src/orca/script_utilities.py application Navigate local folders and message tabs; select messages; compose replies and forwards; save and add attachments internal/atspi internal/app tests/native_thunderbird.py in-progress go test -race ./internal/atspi ./internal/app ./internal/review ./internal/presentation -count=1 -timeout=120s; CTHULHU_BINARY=/tmp/cthulhu-mozilla-review /usr/bin/python3 tests/native_thunderbird.py Private Thunderbird 158.0 HTML and plain-text local mailbox checks cover message-tab return; Subject and body editing; reply and reply-all recipients with Cc; inline forwarding; exact selected rows after Shift and Ctrl navigation; received attachment saving with content verification; and adding a local file to an unsent draft. SSIP verifies focus handoff and row roles; selection toggling; Subject and body echo; reply and forward editing; attachment menus and picker return; audible acceptance pending. Event-loop regression checks paragraph cursor and routing metadata; physical display acceptance pending. Foreground replacement requires a registered application and fresh complete window membership. Editable paragraph events belong only to their focused editable document. Title echoes require owned editable focus and recent editing plus unchanged title prefix/suffix; unrelated status titles still announce. See docs/testing/thunderbird-workflows.md; no sending remote mailbox or page-load-option acceptance claimed.
64 flatpak-foreground-identity src/cthulhu/script_utilities.py:canBeActiveWindow focus Correlate Flatpak accessibility providers with the X11 foreground across PID namespaces and D-Bus proxies internal/platform/xorg internal/atspi in-progress go test -race ./internal/platform/xorg ./internal/atspi ./internal/app -count=1; CTHULHU_FOREGROUND_PID=1 python3 tests/native_atspi.py Private XRes lookup and fallback; real proxy with matching and different synthetic instance metadata; GTK Qt and Brave ownership checks Actual Flatpak Steam speech pending Physical Braille pending; original report also has helper geometry errors docs/testing/flatpak-foreground.md; preserves active state geometry and sibling ownership checks; no application name exception
+73 -3
View File
@@ -1,6 +1,8 @@
#!/usr/bin/env python3
"""Isolated native AT-SPI tree capture and invalidation proof."""
import os
import ast
import json
from pathlib import Path
import subprocess
import sys
@@ -19,6 +21,12 @@ def wait_for(check, message, timeout=15):
def run_inside():
if os.environ.get("CTHULHU_FOREGROUND_PID") == "1":
if os.environ.get("CTHULHU_FOREGROUND_PROXY") == "1":
run_proxy_probe()
else:
run_probe("list")
return
if os.environ.get("CTHULHU_MOUSE_REVIEW") == "1":
run_probe("list")
return
@@ -114,14 +122,71 @@ def run_table_probe(multi: bool = False, screen_lock: bool = False, list_role: b
fixture.wait(timeout=3)
def run_probe(container, flat=False, terminal=False, terminal_mode="", echo_focus=False):
def run_proxy_probe():
# Real D-Bus proxy and separate mount namespaces, with the shared instance
# metadata Flatpak gives its application and proxy. No installed Flatpak
# application or running desktop reader is used.
address = ast.literal_eval(subprocess.check_output([
"gdbus", "call", "--session", "--dest", "org.a11y.Bus",
"--object-path", "/org/a11y/bus", "--method", "org.a11y.Bus.GetAddress",
], text=True))[0]
with tempfile.TemporaryDirectory(prefix="cthulhu-flatpak-") as directory:
root = Path(directory)
info = root / "info"
info.write_text("[Application]\nname=org.example.AccessibilityProbe\n"
"[Instance]\ninstance-id=12345\n")
# A fresh tmpfs root lets us add the metadata without writing to /.
prefix = ["bwrap", "--die-with-parent"]
for path in ("/usr", "/etc", "/var", "/run", "/tmp", "/home", "/proc",
"/bin", "/sbin", "/lib", "/lib64"):
if Path(path).is_symlink():
prefix += ["--symlink", os.readlink(path), path]
elif Path(path).exists():
prefix += ["--ro-bind", path, path]
prefix += ["--dev", "/dev", "--bind", directory, directory,
"--ro-bind", str(info), "/.flatpak-info"]
read_fd, write_fd = os.pipe()
socket = str(root / "bus")
proxy = subprocess.Popen(prefix + ["xdg-dbus-proxy", address, socket,
"--fd=" + str(write_fd)], pass_fds=(write_fd,))
os.close(write_fd)
try:
with os.fdopen(read_fd, "rb") as ready:
if ready.read(1) != b"x":
raise RuntimeError("Accessibility proxy failed to start")
fixture_prefix = prefix
if os.environ.get("CTHULHU_FOREGROUND_PROXY_OTHER") == "1":
other = root / "other-info"
other.write_text(info.read_text().replace("12345", "67890"))
fixture_prefix = prefix + ["--ro-bind", str(other), "/.flatpak-info"]
run_probe("list", fixture_prefix=fixture_prefix,
fixture_overrides={"AT_SPI_BUS_ADDRESS": "unix:path=" + socket,
"GSETTINGS_BACKEND": "memory"})
finally:
proxy.terminate()
proxy.wait(timeout=3)
def run_probe(container, flat=False, terminal=False, terminal_mode="", echo_focus=False,
fixture_prefix=(), fixture_overrides=None):
environment = dict(os.environ, CTHULHU_TREE_CONTAINER=container)
environment["CTHULHU_VTE_MODE"] = terminal_mode
fixture = subprocess.Popen([
prefix = list(fixture_prefix)
pass_fds = ()
if prefix:
read_fd, write_fd = os.pipe()
prefix += ["--info-fd", str(write_fd)]
pass_fds = (write_fd,)
fixture = subprocess.Popen(prefix + [
sys.executable,
str(Path(__file__).with_name("vte_terminal.py" if terminal else "gtk_tree.py")),
], env=environment)
], env=dict(environment, **(fixture_overrides or {})), pass_fds=pass_fds)
try:
fixture_pid = fixture.pid
if prefix:
os.close(write_fd)
with os.fdopen(read_fd) as information:
fixture_pid = json.load(information)["child-pid"]
def find_window():
result = subprocess.run(
["xdotool", "search", "--onlyvisible", "--name", "^Cthulhu Tree Probe$"],
@@ -132,6 +197,10 @@ def run_probe(container, flat=False, terminal=False, terminal_mode="", echo_focu
window = wait_for(find_window, "GTK tree probe window missing")
subprocess.run(["xdotool", "windowfocus", window], check=True)
if os.environ.get("CTHULHU_FOREGROUND_PID") == "1":
environment["CTHULHU_TREE_PID"] = str(fixture_pid)
subprocess.run(["xprop", "-id", window, "-f", "_NET_WM_PID", "32c",
"-set", "_NET_WM_PID", "252"], check=True)
environment.update(CTHULHU_NATIVE_ATSPI="1", CTHULHU_TREE_WINDOW=window)
if os.environ.get("CTHULHU_MOUSE_REVIEW") == "1":
environment["CTHULHU_NATIVE_MOUSE_REVIEW"] = "1"
@@ -146,6 +215,7 @@ def run_probe(container, flat=False, terminal=False, terminal_mode="", echo_focu
"test",
"./internal/review" if flat else "./internal/atspi",
"-run",
"^TestNativeForegroundProcessIdentity$" if os.environ.get("CTHULHU_FOREGROUND_PID") == "1" else
"^TestNativeMouseReviewPointerAndHitTest$" if os.environ.get("CTHULHU_MOUSE_REVIEW") == "1" else
"^TestNativeEchoEditFieldFocus$" if echo_focus else
"^TestNativeSpeechMetadata$" if os.environ.get("CTHULHU_SPEECH_METADATA") == "1" else
+19 -7
View File
@@ -1325,19 +1325,31 @@ def inside():
focus(named("Use custom echo voice")); key("space")
echo_rate = named("Key echo rate")
wait_for(lambda: echo_rate.getState().contains(pyatspi.STATE_SENSITIVE), "custom voice controls disabled")
focus(echo_rate); echo_rate.queryValue().currentValue = 83
focus(named("Key echo pitch")); named("Key echo pitch").queryValue().currentValue = 7.5
focus(named("Key echo volume")); named("Key echo volume").queryValue().currentValue = 10
# Only one preview may be outstanding. Read and acknowledge each
# changed value before moving focus, which discards stale previews.
for control, field, value in (("rate", "Rate", 83),
("pitch", "AveragePitch", 7.5),
("volume", "Gain", 10)):
slider = named("Key echo " + control)
focus(slider)
slider.queryValue().currentValue = value
preview = request()
assert preview["Preview"] and preview.get("EchoPreview") == control, preview
assert preview["Values"]["EchoVoice"][field] == value, preview
send({"Preview": True, "Accepted": True})
family = named("Key echo person")
family.querySelection().selectChild(1)
# Echo selection previews require keyboard focus in the selector.
focus(named("Use custom echo voice")); key("Tab", "Tab", "Down")
preview = request()
assert preview["Preview"] and preview.get("EchoPreview") == "person", preview
assert preview["Values"]["EchoVoice"]["Family"]["Name"] == "Test voice", preview
assert family.querySelection().getSelectedChild(0).name == "Test voice (en)"
send({"Preview": True, "Accepted": True})
word_voice = named("Words")
assert not word_voice.getState().contains(pyatspi.STATE_CHECKED)
focus(word_voice); key("space")
wait_for(lambda: word_voice.getState().contains(pyatspi.STATE_CHECKED),
"custom word voice did not become checked")
preview = request()
assert preview["Preview"], preview
send({"Preview": True, "Accepted": True, "Message": "Speech settings previewed."})
focus(named("OK")); key("space")
echo_saved = request()
ev = echo_saved["Values"]
+5
View File
@@ -57,6 +57,11 @@ def main():
window.set_wm_name(command["title"])
elif action == "class":
window.set_wm_class(command["instance"], command["class"])
elif action == "pid":
window.change_property(connection.intern_atom("_NET_WM_PID"),
Xatom.CARDINAL, 32, [int(command["title"])])
elif action == "empty":
root.change_property(activeAtom, Xatom.WINDOW, 32, [0])
elif action == "hide":
window.unmap()
elif action == "destroy":