More browser work this time focusing on Firefox and Thunderbird.

This commit is contained in:
Storm Dragon committed 2026-10-17 13:15:51 -04:00
1 parent a763714fb4
commit 824a4d2369
22 files changed
+1272 -8

No files matched your search

+86
View File
@@ -0,0 +1,86 @@
# Firefox and Thunderbird reading baseline
The first Mozilla release-readiness slice covers reading an ordinary Firefox
page and a saved HTML message in Thunderbird. Both applications can expose a
heading or link as a text-bearing leaf without a static text child. Document
capture previously recorded zero characters for these roles. Structural heading
navigation still worked, but arrow reading and Read All skipped their content.
Capturing their native text counts fixes this at the shared AT-SPI boundary.
It does not add browser- or application-specific navigation rules.
## Automated verification
Build a separate reader and run:
```sh
go build -o /tmp/cthulhu-mozilla-review ./cmd/cthulhu
python3 tests/native_firefox.py
CTHULHU_BINARY=/tmp/cthulhu-mozilla-review /usr/bin/python3 tests/native_mozilla.py
```
The last command requires Firefox, Thunderbird, i3, Xvfb, xdotool, D-Bus and
Python with pyatspi. Pass `firefox` or `thunderbird` to run just one application.
It creates a separate X display, accessibility/session buses, application
profile, reader configuration and speech protocol recorder for each run.
Thunderbird uses a synthetic local identity and a saved message; no mail account
credentials or SMTP server are configured. Existing profiles and the active
desktop reader are not used.
The evidence directory printed for each application retains the profile,
fixture, reader/application/session logs, `speech.json` and `report.json`.
The test checks:
- Ctrl+Home reads the initial heading.
- Down and Up traverse all six content items in order, including headings and
the link.
- Heading jumps followed by Down reach the following paragraph.
- Link navigation reaches the expected link.
- Read All emits every content item in order, with heading/link roles.
The original Firefox metadata test now asserts heading and link character counts
in the captured document. This assertion fails on the original implementation.
It also retains the existing address-bar/page handoff and browser-tab checks.
The reader-level checks passed with Firefox 158.0 and Thunderbird 158.0. Each
run completed 17 navigation commands and Read All; the recorded navigation
results contained no native caret synchronization errors.
Shared capture changes also need Chromium checks:
```sh
go test -race ./internal/atspi ./internal/review ./internal/presentation ./internal/app -count=1 -timeout=120s
CTHULHU_NATIVE_BROWSER_TESTS=fragments,linkedheadings,document,editing,selection,modes python3 tests/native_browser.py
```
These cover plausible regressions in child-text traversal, linked headings,
structural movement, native editing, selection, Braille routing and focus modes.
## Human review and live acceptance
Review the diff and test through the normal rebuilt reader before submission,
as required by the contribution policy. Protocol recordings prove requested
speech, not audible output, physical keyboard behavior or physical Braille.
The Mozilla reader-level harness disables Braille.
1. Open the retained `reading.html` in Firefox and `message.eml` in Thunderbird.
Move focus into their content.
2. Press Ctrl+Home. Expect "Reading start", heading level 1. Down must read
"First readable paragraph.", "Reading section", "Second readable paragraph.",
"Local destination link" and "Final readable paragraph." in that order.
Up must reverse that order. Check spoken roles and matching Braille.
3. From the first heading, press H and then Down. Expect "Reading section"
followed by "Second readable paragraph.". Check ordinary character and word
movement within a heading and link as well.
4. Return to the start and invoke Read All. Expect every content item, including
both headings and the link. Interrupt it and verify the next navigation
command reads from the expected position.
5. In Firefox, use Ctrl+L and browser-tab switching, then return to the page.
In Thunderbird, move between message headers and body. Verify native controls
retain their keyboard behavior and reading resumes in the correct document.
6. Repeat with representative plain-text and HTML email and ordinary websites.
Check existing Chromium reading, linked headings, editing and selection too.
This is a reading baseline, not complete Thunderbird acceptance. Folder and
message-list navigation and unsent composition now have their own
[local-mail workflow checks](thunderbird-workflows.md). Settings and real mailbox
workflows still need dedicated acceptance checks.
Page summaries, automatic Read All on load and math support remain separate work.
+168
View File
@@ -0,0 +1,168 @@
# Thunderbird local mail and composition
This slice covers a synthetic local mailbox, folder/message-list navigation,
opening and returning from a message tab, unsent HTML and plain-text
composition, replies/reply-all, inline forwarding, attachment saving/adding,
and multiple message selection. The separate
[Mozilla reading baseline](mozilla-reading.md) covers Firefox pages and saved
Thunderbird messages. Page-load summaries and automatic Read All remain
separate feature work.
## Changes and boundaries
Thunderbird can register a GTK accessibility application before replacing it
with its Gecko application while keeping the same X11 client. Previously the
reader could retain the GTK frame and reject real folder/message focus events.
Foreground ownership now accepts a replacement only after fresh active/showing
state, process and geometry checks, and bounded window enumeration. The
candidate's parent application must still be registered on the accessibility
desktop; its current window list must include the candidate and exclude the old
owner. Existing same-process siblings cannot take ownership. Retained focus and
document references belonging to the retired frame are released on the native
owner thread. Unknown or incomplete queries reject replacement.
The composer keeps native focus on its editable web document while paragraphs
emit text, caret and selection events. Previously those insertions were rejected
as outside the focused editor; subsequent caret events spoke the growing line,
and selection feedback was missing. Native capture now copies the nearest
editable-document identity. Paragraph events belong to that document only while
it owns focus. They use ordinary typing, deletion, selection, clipboard and
Braille presentation. Focus stays on the editor, while the current paragraph
provides text and cursor/routing metadata. Moving to Subject or another editor
revokes the relationship immediately. A nested read-only document is a boundary.
These are shared AT-SPI and event ownership changes. No Thunderbird-specific
application-name, window-title or widget-label matching selects production
behavior. Orca's Thunderbird editable-message handling and shared web/focus
utilities were used as behavioral references.
Automatic title changes no longer repeat the focused editor's text during
typing, deletion or clipboard edits when the rest of the title stays the same.
This uses owned editable focus and recent input, with matching title prefix and
suffix. Both AT-SPI and X11 title sources use the same check. Window introductions,
unrelated status changes, empty values and ambiguous matches keep their existing
behavior. There is no application-name or Subject-label exception. The new
recorded-speech assertion reproduced the title echo on the pre-fix reader.
## Automated checks
```sh
go build -o /tmp/cthulhu-mozilla-review ./cmd/cthulhu
CTHULHU_BINARY=/tmp/cthulhu-mozilla-review /usr/bin/python3 tests/native_thunderbird.py
```
The harness requires Thunderbird, i3, Xvfb, xdotool, D-Bus and Python with
pyatspi. It starts a private reader before Thunderbird, with its own display,
session/accessibility buses, SSIP recorder, profile and configuration. It seeds
Inbox and Archive mbox files, a synthetic local identity, a Cc recipient and a
plain-text MIME attachment. No mail credentials or SMTP server are configured
and no send command is used. Pass `--format html`
or `--format plain` to select one run.
Each run checks native focus and recorded speech for:
- Folder Home/End and Down navigation, including returning to Inbox.
- F6/Shift+F6 between the folder tree and message list, retaining the row.
- Up/Down through messages, with native selection and spoken row roles.
- Shift+Down/Shift+Up to extend/shrink message selection; Ctrl+Down to move
without extending it; Ctrl+Space to add/remove the focused message. Native
selected rows must match at each step, and state changes must speak.
- Enter to open the selected message in a tab, Ctrl+Home to read its body,
Ctrl+W to return to the selected row, and Down to the following message.
- Tab from the message body to its attachment bar, Enter to expand the list,
and Shift+F10/Down through Open and Save As. Save the received attachment
into the evidence directory and verify its contents, then return to the
selected message row.
- Ctrl+R/Ctrl+Shift+R/Ctrl+L for reply, reply-all and inline forward, with
expected Subject and quoted body. Replies retain the sender address chip;
reply-all also retains Cc. Forwarding includes recipient entry and Tab to
the body. Type into each body, close it, focus Discard changes and confirm
return to the selected message. These messages are never sent.
- Ctrl+M, recipient typing and committing its address chip, then Subject.
- Subject character echo and Backspace without repeated composer titles.
- Tab from Subject to the editable body; H reaches the editor as typed text.
- Character echo and Backspace, two-line body input, Ctrl+Home/Down reading,
selected-word speech and native text selection, and deletion of that word.
- Shift+Tab/Tab between the body and Subject.
- Ctrl+Shift+A to open the file picker, attach a generated local file, check
the selected attachment row and its speech, Escape to clear selection, and
a second Escape back to the body.
Initial message-list focus need not select the first row: Thunderbird can expose
a focused but unselected row until a navigation key selects it. The harness
requires selection after Up/Down, rather than treating initial focus as selection.
The parent document can expose a selection as embedded-object placeholders; the
native selection assertion checks the selected paragraph text instead.
The harness ignores hidden duplicate controls in retained tabs and inspects
both Thunderbird accessibility applications and the GTK desktop portal's file
dialogs. Attachment row names can separate the filename and extension with a
space; their descriptions retain the complete filename. File-picker paths and
saved content are checked separately from recorded speech.
Printed evidence directories retain the profile, mailbox fixtures, settings,
reader/application/session logs, `tree.json`, `speech.json` and `report.json`.
Successful HTML and plain-text runs were verified with Thunderbird 158.0,
with 60 workflow checks in each format.
The tests disable Braille; the event-loop tests separately check paragraph
Braille cursor and routing metadata.
Run the shared regression checks as well:
```sh
go test -race ./internal/atspi ./internal/app ./internal/review ./internal/presentation -count=1 -timeout=120s
go vet ./internal/atspi ./internal/app ./internal/review ./internal/presentation
python3 tests/native_firefox.py
python3 tests/check_speech.py
CTHULHU_BINARY=/tmp/cthulhu-mozilla-review /usr/bin/python3 tests/native_mozilla.py
CTHULHU_BINARY=/tmp/cthulhu-mozilla-review /usr/bin/python3 tests/native_ownership.py
CTHULHU_NATIVE_BROWSER_TESTS=editing,selection,modes python3 tests/native_browser.py
```
These cover Firefox tabs/address focus and native metadata, Firefox/Thunderbird
reading and Read All, GTK/Qt controls and dialogs, same-process overlapping window
ownership, workspace return, and Chromium editing/selection/focus modes. Passing
the new tests alone does not establish that these existing behaviors still work.
## Human review and live acceptance
Review the diff and test the rebuilt reader before submission, under the
[contribution policy](../../CONTRIBUTING.md#ai-assisted-contributions). Recorded
SSIP and synthetic X11 input establish requested behavior, not audible speech,
physical input, a physical Braille display or native Wayland acceptance.
1. Start Thunderbird with the rebuilt reader already running. In a test mailbox,
use arrows, Home/End, expansion/collapse, F6 and Shift+F6 across folders and
message rows. Check sender, subject, unread/read state and selected rows.
2. Open a message, read its headers and body, return to the list and switch
folders. Check that focus and reading position belong to the current message.
3. Create an unsent HTML message. Type and commit a recipient, Tab to Subject,
and type slowly, pause, then use Backspace and paste text. Confirm ordinary
echo without repeated composer titles. Clear Subject completely; the empty
subject title can still announce. Tab to the body and confirm line/word
movement, selection, Backspace/Delete, cut/paste, and header/body return with
Tab/Shift+Tab. Repeat in plain text. Check speech and Braille text, selection,
cursor and routing.
4. Select several messages with Shift+arrows, move without changing selection
with Ctrl+arrows, and toggle rows with Ctrl+Space. Check announced selection
state and matching Braille, then return to single selection and open a
message. Try a larger folder and nonadjacent selections in a test mailbox.
5. Reply, reply-all and forward a test message without sending it. Verify
recipients, Cc, Subject, quoted content and ordinary body editing. Close the
composer, inspect the save/discard/cancel prompt, cancel once to return to
editing, then discard and verify the original selected message.
6. Tab to a received attachment, expand its list, inspect the context menu,
and save a harmless test file to a chosen directory. Verify the saved file.
Add a local file to an unsent message with Ctrl+Shift+A, check its filename
and selection, then Escape to clear selection and Escape again to return
to editing. Also check removal, multiple
attachments and other file types; those exceed the automated fixture.
7. Move from the body to Subject, another window or another message. Old
body activity must not interrupt the newly focused control. Switch between
multiple Thunderbird windows and ordinary Firefox and Chromium windows.
8. Recheck Firefox tab creation/switch/close and address editing; Chromium
editing and selection; GTK/Qt menus and dialogs; and empty-workspace return.
Remote accounts, sending, forwarding as an attached message, external attachment
opening, attachment removal, larger/nonadjacent selections, real mailbox
performance and physical speech/Braille remain outside this automated local
baseline. Page-load summaries and automatic Read All remain separate features.
+80
View File
@@ -0,0 +1,80 @@
// Copyright (c) 2026 Storm Dragon
// Authors: Storm Dragon, Angus Kola
// SPDX-License-Identifier: GPL-3.0-or-later
package app
import (
"context"
"log/slog"
"strings"
"testing"
"time"
"git.stormux.org/storm/cthulhu/internal/atspi"
"git.stormux.org/storm/cthulhu/internal/braille"
)
func TestEditableDocumentParagraphEvents(t *testing.T) {
queue := &fakeSpeechQueue{spoken: make(chan string, 16)}
adapter := &fakeAdapter{events: make(chan atspi.Event, 16)}
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
go func() {
defer close(done)
processEventsWithInput(ctx, slog.New(slog.DiscardHandler), queue, adapter, nil, nil, 0, nil, nil, nil, nil)
}()
defer func() { cancel(); <-done }()
editor := atspi.Snapshot{BusName: ":1.7", Path: "/editor", Name: "Message body", Role: "document web", Editable: true}
paragraph := atspi.Snapshot{BusName: editor.BusName, Path: "/paragraph", Role: "paragraph", Editable: true,
EditableRoot: editor.ID(), Text: "a", TextCaretKnown: true, BrailleLine: "a", BrailleCaret: 1}
adapter.events <- atspi.NewEvent(atspi.FocusEvent, editor)
adapter.events <- atspi.NewEventWithPayload(atspi.TextInsertedEvent, paragraph, "a")
// This caret must not reread the growing line after accepted typing.
adapter.events <- atspi.NewEvent(atspi.CaretEvent, paragraph)
paragraph.SelectionStart, paragraph.SelectionEnd, paragraph.SelectionText = 0, 1, "a"
adapter.events <- atspi.NewEvent(atspi.TextSelectionChangedEvent, paragraph)
adapter.events <- atspi.NewEventWithPayload(atspi.TextDeletedEvent, paragraph, "a")
subject := atspi.Snapshot{BusName: editor.BusName, Path: "/subject", Name: "Subject", Role: "entry", Editable: true}
adapter.events <- atspi.NewEvent(atspi.FocusEvent, subject)
// A delayed mutation from the old body cannot interrupt a header field.
adapter.events <- atspi.NewEventWithPayload(atspi.TextInsertedEvent, paragraph, "unfocused text")
other := paragraph
other.Path, other.EditableRoot = "/other_paragraph", atspi.NewObjectID(editor.BusName, "/other_editor")
adapter.events <- atspi.NewEventWithPayload(atspi.TextInsertedEvent, other, "other editor")
adapter.events <- atspi.NewEvent(atspi.FocusEvent, atspi.Snapshot{BusName: editor.BusName, Path: "/end", Role: "button", Name: "End"})
for _, want := range []string{"Message body, document web", "a", "a selected", "a", "Subject, entry", "End, button"} {
select {
case got := <-queue.spoken:
if got != want {
t.Fatalf("speech = %q, want %q", got, want)
}
case <-time.After(time.Second):
t.Fatalf("missing %q", want)
}
}
}
func TestEditableDocumentCaretAndSelectionBraille(t *testing.T) {
root := atspi.NewObjectID(":1.7", "/editor")
frames := make(chan braille.Frame, 2)
state := &eventLoopState{}
state.focusedID = root
handler := accessibilityEventHandler{state: state, logger: slog.New(slog.DiscardHandler), brailleOutput: frameRecorder{frames: frames}}
source := atspi.Snapshot{BusName: root.BusName, Path: "/paragraph", Role: "paragraph", Editable: true,
EditableRoot: root, Text: "body", TextCaretKnown: true, BrailleLine: "body", BrailleCaret: 2}
for _, eventType := range []string{atspi.CaretEvent, atspi.TextSelectionChangedEvent} {
handler.acceptFocusedEvent(atspi.NewEvent(eventType, source), recentInput(nil))
if state.focusedID != root || state.current.ID() != source.ID() || state.brailleSnapshot.ID() != source.ID() {
t.Fatal("paragraph presentation lost native editor focus or Braille ownership")
}
select {
case frame := <-frames:
if !strings.HasPrefix(frame.Text, "body") || frame.Cursor != 2 || len(frame.Routes) == 0 || frame.Routes[0].Path != source.Path {
t.Fatalf("paragraph Braille frame = %+v", frame)
}
default:
t.Fatal("missing paragraph Braille frame")
}
}
}
+7 -1
View File
@@ -96,6 +96,7 @@ func (handler *accessibilityEventHandler) handle(event atspi.Event) {
"prompt_message_present", event.Source.Text != "" && (event.Source.Role == "frame" || event.Source.Role == "alert") || event.WindowMessage != "", "prompt_message_present", event.Source.Text != "" && (event.Source.Role == "frame" || event.Source.Role == "alert") || event.WindowMessage != "",
"snapshot_duration", event.SnapshotDuration, "snapshot_duration", event.SnapshotDuration,
"editable", event.Source.Editable, "editable", event.Source.Editable,
"editable_root", event.Source.EditableRoot.Path,
"autocomplete", event.Source.Autocomplete, "autocomplete", event.Source.Autocomplete,
"autocomplete_popup", event.Source.AutocompletePopup, "autocomplete_popup", event.Source.AutocompletePopup,
"event_detail1", event.Detail1, "event_detail1", event.Detail1,
@@ -185,9 +186,14 @@ func (handler *accessibilityEventHandler) handle(event atspi.Event) {
activity = browser.RecentInput() activity = browser.RecentInput()
} }
intent := activity.Intent intent := activity.Intent
if event.Type == atspi.CaretEvent && event.Source.EditableRoot.Valid() && state.focusedID.Valid() &&
!belongsToFocusedEditor(event.Source, state.focusedID) {
logger.Debug("skipped caret outside focused editor", "role", event.Source.Role)
return
}
if isTextChangeEvent(event.Type) && event.Source.Editable && if isTextChangeEvent(event.Type) && event.Source.Editable &&
event.Source.Role != "terminal" && !event.Source.LiveRegion && state.focusedID.Path != "" && event.Source.Role != "terminal" && !event.Source.LiveRegion && state.focusedID.Path != "" &&
event.Source.ID() != state.focusedID && event.Source.ID() != state.managedDescendantID { !belongsToFocusedEditor(event.Source, state.focusedID) && event.Source.ID() != state.managedDescendantID {
// Settings pages initialize editable number fields while focus // Settings pages initialize editable number fields while focus
// remains in their navigation tree. These are not user edits. // remains in their navigation tree. These are not user edits.
logger.Debug("skipped text change outside focused editor", "type", event.Type, "role", event.Source.Role) logger.Debug("skipped text change outside focused editor", "type", event.Type, "role", event.Source.Role)
+6
View File
@@ -68,6 +68,12 @@ func speaksTextChanges(snapshot atspi.Snapshot) bool {
(snapshot.Editable && !snapshot.ReadOnly && snapshot.Role != "password text") (snapshot.Editable && !snapshot.ReadOnly && snapshot.Role != "password text")
} }
func belongsToFocusedEditor(source atspi.Snapshot, focus atspi.ObjectID) bool {
return focus.Path != "" && (source.ID() == focus || focus.Valid() && source.Editable && !source.ReadOnly &&
source.Role != "password text" && source.EditableRoot.Valid() && source.EditableRoot == focus &&
source.BusName == focus.BusName)
}
func isTextChangeEvent(eventType string) bool { func isTextChangeEvent(eventType string) bool {
return eventType == atspi.TextInsertedEvent || eventType == atspi.TextDeletedEvent return eventType == atspi.TextInsertedEvent || eventType == atspi.TextDeletedEvent
} }
+7
View File
@@ -95,6 +95,13 @@ func (handler *accessibilityEventHandler) acceptFocusedEvent(event atspi.Event,
logger.Debug("skipped selected state already presented by focus", "role", event.Source.Role) logger.Debug("skipped selected state already presented by focus", "role", event.Source.Role)
return true return true
} }
// Keep native focus on the editor, but present the paragraph which owns
// its current caret/selection. Leaving that editor immediately revokes
// this relationship; no extra mutable ownership state is retained.
if belongsToFocusedEditor(event.Source, state.focusedID) &&
(isTextChangeEvent(event.Type) || event.Type == atspi.CaretEvent || event.Type == atspi.TextSelectionChangedEvent) {
state.current = event.Source
}
if isFocusEvent(event.Type) || isManagedDescendantEvent(event.Type) || if isFocusEvent(event.Type) || isManagedDescendantEvent(event.Type) ||
event.Type == atspi.AccessibleNameEvent && presentsReusedRow(event.Source) && event.Type == atspi.AccessibleNameEvent && presentsReusedRow(event.Source) &&
event.Source.ID() == state.managedDescendantID || event.Source.ID() == state.managedDescendantID ||
+8 -2
View File
@@ -29,7 +29,7 @@ func (handler *accessibilityEventHandler) handleSelectionAndClipboard(event atsp
return true return true
} }
if event.Type == atspi.TextSelectionChangedEvent { if event.Type == atspi.TextSelectionChangedEvent {
if event.Source.ID() != state.focusedID || !event.Source.Editable || event.Source.ReadOnly || if !belongsToFocusedEditor(event.Source, state.focusedID) || !event.Source.Editable || event.Source.ReadOnly ||
event.Source.Role == "password text" || event.Source.Autocomplete { event.Source.Role == "password text" || event.Source.Autocomplete {
return true return true
} }
@@ -37,6 +37,12 @@ func (handler *accessibilityEventHandler) handleSelectionAndClipboard(event atsp
if activity.Selecting && intent == input.LineIntent { if activity.Selecting && intent == input.LineIntent {
fragmentLimit = lineSelectionFragmentLimit fragmentLimit = lineSelectionFragmentLimit
} }
if event.Source.EditableRoot == state.focusedID && event.Source.ID() != state.focusedID &&
state.selection.id != event.Source.ID() {
// Native focus stays on the document. The first selection in a
// newly visited paragraph still needs its selected text spoken.
state.selection.reset(atspi.Snapshot{BusName: event.Source.BusName, Path: event.Source.Path})
}
message := state.selection.changedWithLimit(event.Source, fragmentLimit) message := state.selection.changedWithLimit(event.Source, fragmentLimit)
if message != "" { if message != "" {
state.pending.caret.clear() state.pending.caret.clear()
@@ -50,7 +56,7 @@ func (handler *accessibilityEventHandler) handleSelectionAndClipboard(event atsp
return true return true
} }
if activity.Sequence != 0 && activity.Sequence != state.lastClipboardSequence && if activity.Sequence != 0 && activity.Sequence != state.lastClipboardSequence &&
event.Source.ID() == state.focusedID && event.Source.Editable && !event.Source.ReadOnly && belongsToFocusedEditor(event.Source, state.focusedID) && event.Source.Editable && !event.Source.ReadOnly &&
!event.Source.Autocomplete && event.Source.Role != "password text" { !event.Source.Autocomplete && event.Source.Role != "password text" {
message := "" message := ""
if event.Type == atspi.TextDeletedEvent && activity.Clipboard == input.CutAction { if event.Type == atspi.TextDeletedEvent && activity.Clipboard == input.CutAction {
+41 -2
View File
@@ -10,6 +10,7 @@ import (
"time" "time"
"git.stormux.org/storm/cthulhu/internal/atspi" "git.stormux.org/storm/cthulhu/internal/atspi"
"git.stormux.org/storm/cthulhu/internal/input"
"git.stormux.org/storm/cthulhu/internal/platform/xorg" "git.stormux.org/storm/cthulhu/internal/platform/xorg"
"git.stormux.org/storm/cthulhu/internal/wine" "git.stormux.org/storm/cthulhu/internal/wine"
) )
@@ -21,6 +22,7 @@ type windowTitleSource interface {
type pendingWindowTitle struct { type pendingWindowTitle struct {
text string text string
previousText string
window atspi.ObjectID window atspi.ObjectID
epoch uint64 epoch uint64
native xorg.WindowTitle native xorg.WindowTitle
@@ -103,6 +105,9 @@ func (titles *windowTitleController) queue(pending pendingWindowTitle) {
if titles.pending != nil { if titles.pending != nil {
// Coalescing must not revive a burst interrupted since its first sample. // Coalescing must not revive a burst interrupted since its first sample.
pending.inputSequence = titles.pending.inputSequence pending.inputSequence = titles.pending.inputSequence
// Compare the whole burst so an intervening status change cannot
// disappear behind a later title which mirrors an edit.
pending.previousText = titles.pending.previousText
} }
if previous := titles.pending; previous != nil && !previous.changed && previous.epoch == pending.epoch && if previous := titles.pending; previous != nil && !previous.changed && previous.epoch == pending.epoch &&
previous.window == pending.window && previous.native.Window == pending.native.Window && previous.window == pending.window && previous.native.Window == pending.native.Window &&
@@ -137,11 +142,13 @@ func (titles *windowTitleController) accessible(event atspi.Event) bool {
titles.lastTitle = "" titles.lastTitle = ""
return true return true
} }
previousTitle := titles.state.owner.title
if !titles.state.owner.changeTitle(event) { if !titles.state.owner.changeTitle(event) {
return true return true
} }
titles.queue(pendingWindowTitle{text: strings.TrimSpace(event.Source.Name), titles.queue(pendingWindowTitle{text: strings.TrimSpace(event.Source.Name),
window: titles.state.owner.window, epoch: titles.state.foregroundState.Epoch, previousText: previousTitle,
window: titles.state.owner.window, epoch: titles.state.foregroundState.Epoch,
changed: true, application: titles.application}) changed: true, application: titles.application})
return true return true
} }
@@ -185,7 +192,11 @@ func (titles *windowTitleController) nativeChanged() {
latest.Title.TitleWindow == latest.Title.Window && !strings.EqualFold(latest.Title.Application, "xterm") { latest.Title.TitleWindow == latest.Title.Window && !strings.EqualFold(latest.Title.Application, "xterm") {
changed = true changed = true
} }
titles.queue(pendingWindowTitle{text: latest.Title.Title, epoch: latest.Epoch, previousTitle := previous.Title.Title
if switched {
previousTitle = ""
}
titles.queue(pendingWindowTitle{text: latest.Title.Title, previousText: previousTitle, epoch: latest.Epoch,
native: latest.Title, changed: changed}) native: latest.Title, changed: changed})
} }
@@ -203,6 +214,31 @@ func (titles *windowTitleController) enabled() bool {
return titles.live == nil || titles.live.Load().Reading.AutomaticallyReadTitleChanges return titles.live == nil || titles.live.Load().Reading.AutomaticallyReadTitleChanges
} }
// repeats_focused_edit recognizes titles whose only changed portion mirrors
// the owned entry. Use copied focus and input evidence, never application names
// or field labels. Window introductions and unrelated status changes still speak.
func (titles *windowTitleController) repeats_focused_edit(previousTitle, title string) bool {
current := titles.state.current
if previousTitle == "" || !current.Editable || current.ReadOnly || !current.Focused ||
!titles.state.focusedID.Valid() || current.ID() != titles.state.focusedID ||
titles.state.owner.focus != current.ID() || current.BusName != titles.state.owner.window.BusName {
return false
}
activity := recentInput(titles.inputSource)
if activity.Intent != input.TypingIntent && activity.Intent != input.DeleteIntent &&
activity.Clipboard != input.CutAction && activity.Clipboard != input.PasteAction {
return false
}
text := strings.TrimSpace(current.Text)
if text == "" || strings.Count(title, text) != 1 {
return false
}
start := strings.Index(title, text)
prefix, suffix := title[:start], title[start+len(text):]
return len(previousTitle) >= len(prefix)+len(suffix) &&
strings.HasPrefix(previousTitle, prefix) && strings.HasSuffix(previousTitle, suffix)
}
func (titles *windowTitleController) flush() { func (titles *windowTitleController) flush() {
pending := titles.pending pending := titles.pending
titles.clearPending() titles.clearPending()
@@ -253,6 +289,9 @@ func (titles *windowTitleController) flush() {
return return
} }
titles.lastTitle = pending.text titles.lastTitle = pending.text
if pending.changed && titles.repeats_focused_edit(pending.previousText, pending.text) {
return
}
if pending.inputSequence != recentInput(titles.inputSource).TitleInterruptionSequence { if pending.inputSequence != recentInput(titles.inputSource).TitleInterruptionSequence {
return return
} }
+83
View File
@@ -108,6 +108,89 @@ func TestScheduledTitleInterruptedBeforeSpeechQueue(t *testing.T) {
} }
} }
func TestAutomaticTitleDoesNotRepeatFocusedEdit(t *testing.T) {
for _, native := range []bool{false, true} {
t.Run(fmt.Sprintf("native=%v", native), func(t *testing.T) {
titles, queue, changed := titleControllerFixture(t)
tracker := &input.IntentTracker{}
titles.inputSource = tracker
entry := atspi.Snapshot{BusName: changed.Source.BusName, Path: "/entry", Role: "entry", Editable: true, Focused: true}
titles.state.current = entry
titles.state.focusedID, titles.state.owner.focus = entry.ID(), entry.ID()
titles.state.owner.title, titles.lastTitle = "Compose: (empty) - Mail", "Compose: (empty) - Mail"
source := &titleProbeSource{title: atspi.ForegroundTitle{Epoch: 1, Title: xorg.WindowTitle{Window: 20, TitleWindow: 20, Title: titles.lastTitle}}}
if native {
titles.source, titles.foreground, titles.lastNative = source, source, source.title
}
change := func(text string) {
if native {
source.title.Title.Title = text
titles.nativeChanged()
} else {
changed.Source.Name = text
titles.accessible(changed)
}
titles.flush()
}
for _, step := range []struct{ key, text string }{{"D", "D"}, {"r", "Dr"}, {"BackSpace", "D"}} {
tracker.Observe(input.NewKeyEvent(0, true, step.key))
titles.state.current.Text = step.text
change("Compose: " + step.text + " - Mail")
}
if len(queue.requests) != 0 {
t.Fatalf("titles repeated the edited text: %+v", queue.requests)
}
// The entry still owns focus and input is still recent. A changed
// status outside the edited value must nevertheless be announced.
change("Compose: D - Mail (saved)")
if len(queue.requests) != 1 || queue.requests[0].Text != "Compose: D - Mail (saved)" {
t.Fatalf("unrelated status title = %+v", queue.requests)
}
})
}
}
func TestAutomaticTitleEditFilterRequiresOwnedEdit(t *testing.T) {
for _, name := range []string{"no-input", "navigation", "read-only", "unfocused", "other-focus", "other-window", "empty", "ambiguous", "introduction"} {
t.Run(name, func(t *testing.T) {
titles, queue, changed := titleControllerFixture(t)
tracker := &input.IntentTracker{}
titles.inputSource = tracker
tracker.Observe(input.NewKeyEvent(0, true, "r"))
titles.state.current = atspi.Snapshot{BusName: changed.Source.BusName, Path: "/entry", Role: "entry", Text: "Draft", Editable: true, Focused: true}
titles.state.focusedID, titles.state.owner.focus = titles.state.current.ID(), titles.state.current.ID()
titles.state.owner.title, titles.lastTitle = "Compose: Draf - Mail", "Compose: Draf - Mail"
changed.Source.Name = "Compose: Draft - Mail"
switch name {
case "no-input":
titles.inputSource = nil
case "navigation":
tracker.Observe(input.NewKeyEvent(0, true, "Left"))
case "read-only":
titles.state.current.ReadOnly = true
case "unfocused":
titles.state.current.Focused = false
case "other-focus":
titles.state.owner.focus.Path = "/other"
case "other-window":
titles.state.current.BusName = ":1.99"
titles.state.focusedID, titles.state.owner.focus = titles.state.current.ID(), titles.state.current.ID()
case "empty":
titles.state.current.Text = ""
case "ambiguous":
changed.Source.Name = "Compose: Draft - Draft"
case "introduction":
titles.state.owner.title, titles.lastTitle = "", ""
}
titles.accessible(changed)
titles.flush()
if len(queue.requests) != 1 || queue.requests[0].Text != changed.Source.Name {
t.Fatalf("title without matching edit = %+v", queue.requests)
}
})
}
}
func TestScheduledNativeIntroductionPreservesSwitchAndRepeatInput(t *testing.T) { func TestScheduledNativeIntroductionPreservesSwitchAndRepeatInput(t *testing.T) {
titles, queue, _ := titleControllerFixture(t) titles, queue, _ := titleControllerFixture(t)
titles.state.owner.clear() titles.state.owner.clear()
+3
View File
@@ -852,7 +852,10 @@ gchar *cthulhu_capture_caret_unit(AtspiAccessible *source, AtspiTextGranularity
static void cthulhu_capture_document_metrics(AtspiAccessible *source, cthulhu_snapshot *snapshot) { static void cthulhu_capture_document_metrics(AtspiAccessible *source, cthulhu_snapshot *snapshot) {
const gchar *role = snapshot->role; const gchar *role = snapshot->role;
/* Gecko can expose heading/link text directly without static children. */
if (g_strcmp0(role, "paragraph") != 0 && if (g_strcmp0(role, "paragraph") != 0 &&
g_strcmp0(role, "heading") != 0 &&
g_strcmp0(role, "link") != 0 &&
g_strcmp0(role, "list item") != 0 && g_strcmp0(role, "list item") != 0 &&
g_strcmp0(role, "math") != 0 && g_strcmp0(role, "math") != 0 &&
g_strcmp0(role, "static") != 0 && g_strcmp0(role, "static") != 0 &&
@@ -110,6 +110,13 @@ func (adapter *Adapter) captureNativeEventSnapshot(event *C.AtspiEvent, target *
captureCaret = role != C.ATSPI_ROLE_DOCUMENT_WEB && role != C.ATSPI_ROLE_DOCUMENT_FRAME && role != C.ATSPI_ROLE_EMBEDDED captureCaret = role != C.ATSPI_ROLE_DOCUMENT_WEB && role != C.ATSPI_ROLE_DOCUMENT_FRAME && role != C.ATSPI_ROLE_EMBEDDED
} }
snapshot := snapshotFromNativeWithCaret(target, captureCaret, eventType == CaretEvent, false, adapter.captureSound.Load(), adapter.captureTextAttributes.Load()) snapshot := snapshotFromNativeWithCaret(target, captureCaret, eventType == CaretEvent, false, adapter.captureSound.Load(), adapter.captureTextAttributes.Load())
if snapshot.Editable && (eventType == CaretEvent || eventType == TextInsertedEvent ||
eventType == TextDeletedEvent || eventType == TextSelectionChangedEvent) {
if editor := C.cthulhu_editable_document(target); editor != nil {
snapshot.EditableRoot = nativeObjectID(editor)
C.g_object_unref(C.gpointer(editor))
}
}
snapshot.ApplicationProcess = adapter.applicationProcess(target) snapshot.ApplicationProcess = adapter.applicationProcess(target)
if snapshot.Role == "slider" || if snapshot.Role == "slider" ||
snapshot.Role == "combo box" && (snapshot.Name == "Key echo speech synthesizer" || snapshot.Name == "Key echo person") { snapshot.Role == "combo box" && (snapshot.Name == "Key echo speech synthesizer" || snapshot.Name == "Key echo person") {
+27
View File
@@ -544,6 +544,33 @@ AtspiAccessible *cthulhu_window_ancestor(AtspiAccessible *source) {
return NULL; return NULL;
} }
// Editable documents can retain native focus while their paragraphs emit
// typing/caret/selection events. Stop at the nearest document boundary so an
// unrelated outer editor cannot claim events from a nested read-only document.
AtspiAccessible *cthulhu_editable_document(AtspiAccessible *source) {
AtspiAccessible *current = source != NULL ? g_object_ref(source) : NULL;
for (gint depth = 0; current != NULL && depth < 64; depth++) {
AtspiRole role = atspi_accessible_get_role(current, NULL);
if (role == ATSPI_ROLE_DOCUMENT_WEB || role == ATSPI_ROLE_DOCUMENT_FRAME) {
AtspiStateSet *states = atspi_accessible_get_state_set(current);
gboolean editable = states != NULL &&
atspi_state_set_contains(states, ATSPI_STATE_EDITABLE) &&
!atspi_state_set_contains(states, ATSPI_STATE_READ_ONLY) &&
!atspi_state_set_contains(states, ATSPI_STATE_DEFUNCT);
if (states != NULL) g_object_unref(states);
if (editable) return current;
g_object_unref(current);
return NULL;
}
if (role == ATSPI_ROLE_FRAME || role == ATSPI_ROLE_WINDOW || role == ATSPI_ROLE_APPLICATION) break;
AtspiAccessible *parent = atspi_accessible_get_parent(current, NULL);
g_object_unref(current);
current = parent;
}
if (current != NULL) g_object_unref(current);
return NULL;
}
// Chromium can detach an existing document from its frame's AT-SPI parent // Chromium can detach an existing document from its frame's AT-SPI parent
// chain while returning to a tab. Its document identity remains usable. // chain while returning to a tab. Its document identity remains usable.
AtspiAccessible *cthulhu_document_ancestor(AtspiAccessible *source) { AtspiAccessible *cthulhu_document_ancestor(AtspiAccessible *source) {
+95 -3
View File
@@ -84,10 +84,9 @@ func (adapter *Adapter) matchForeground(window *C.AtspiAccessible, id ObjectID,
} }
matched := foregroundMenuPopupMatches(state, adapter.foregroundWindow, id, pid, matched := foregroundMenuPopupMatches(state, adapter.foregroundWindow, id, pid,
focusedMenuItem || C.cthulhu_active_menu_window(window) != 0) focusedMenuItem || C.cthulhu_active_menu_window(window) != 0)
if !matched && check != nil { if matched {
check.Reason = "different accessible window from correlated foreground" return true
} }
return matched
} }
// Same-process maximized windows can have identical bounds. As in // Same-process maximized windows can have identical bounds. As in
// Python canBeActiveWindow(clearCache=True), only correlate a freshly // Python canBeActiveWindow(clearCache=True), only correlate a freshly
@@ -144,11 +143,104 @@ func (adapter *Adapter) matchForeground(window *C.AtspiAccessible, id ObjectID,
} }
return false return false
} }
// Mozilla can replace its GTK application root with a Gecko root without
// changing the X11 client. Only inspect its window list after fresh state
// and geometry match. Existing siblings cannot retire the previous owner.
if adapter.foregroundWindow.Valid() && !adapter.replacesForegroundWindow(window, id, state.PID, pid) {
if check != nil {
check.Reason = "different accessible window from correlated foreground"
}
return false
}
if adapter.foregroundWindow.Valid() {
adapter.forgetNativeWindowFocus(adapter.foregroundWindow)
adapter.forgetWindowDocuments(adapter.foregroundWindow)
}
adapter.foregroundWindow = id adapter.foregroundWindow = id
adapter.foregroundRecovery = 0 adapter.foregroundRecovery = 0
return true return true
} }
func (adapter *Adapter) replacesForegroundWindow(window *C.AtspiAccessible, id ObjectID, foregroundPID, pid int) bool {
if foregroundPID <= 0 || pid != foregroundPID || id.BusName != adapter.foregroundWindow.BusName {
return false
}
application := C.atspi_accessible_get_parent(window, nil)
if application == nil {
return false
}
defer C.g_object_unref(C.gpointer(application))
if C.atspi_accessible_get_role(application, nil) != C.ATSPI_ROLE_APPLICATION {
return false
}
// A retired GTK bridge can still answer queries after Gecko registers
// its replacement application. Do not let that detached root reclaim
// the client. GetApplication itself can use libatspi's old root cache.
desktop := C.atspi_get_desktop(0)
if desktop == nil {
return false
}
defer C.g_object_unref(C.gpointer(desktop))
C.atspi_accessible_clear_cache_single(desktop)
var queryError *C.GError
applications := int(C.atspi_accessible_get_child_count(desktop, &queryError))
if queryError != nil {
C.g_error_free(queryError)
return false
}
if applications <= 0 || applications > 128 {
return false
}
registered := false
for index := 0; index < applications; index++ {
child := C.atspi_accessible_get_child_at_index(desktop, C.gint(index), &queryError)
if queryError != nil {
C.g_error_free(queryError)
}
if child == nil {
return false
}
matches := nativeObjectID(child) == nativeObjectID(application)
C.g_object_unref(C.gpointer(child))
if queryError != nil {
return false
}
if matches {
registered = true
break
}
}
if !registered {
return false
}
C.atspi_accessible_clear_cache_single(application)
count := int(C.atspi_accessible_get_child_count(application, &queryError))
if queryError != nil {
C.g_error_free(queryError)
return false
}
if count <= 0 || count > 128 {
return false
}
windows := make([]ObjectID, 0, count)
for index := 0; index < count; index++ {
child := C.atspi_accessible_get_child_at_index(application, C.gint(index), &queryError)
if queryError != nil {
C.g_error_free(queryError)
}
if child == nil {
return false
}
childID := nativeObjectID(child)
C.g_object_unref(C.gpointer(child))
if queryError != nil || !childID.Valid() {
return false
}
windows = append(windows, childID)
}
return foregroundWindowReplaced(adapter.foregroundWindow, id, windows)
}
// Recovery runs only at foreground transitions, off the event consumer. It // Recovery runs only at foreground transitions, off the event consumer. It
// handles returning from an empty workspace when a toolkit retains focus and // handles returning from an empty workspace when a toolkit retains focus and
// emits no new focused-child event. Searches and retries are bounded. // emits no new focused-child event. Searches and retries are bounded.
+1
View File
@@ -120,6 +120,7 @@ const gchar *cthulhu_accessible_path(AtspiAccessible *source);
const gchar *cthulhu_accessible_bus_name(AtspiAccessible *source); const gchar *cthulhu_accessible_bus_name(AtspiAccessible *source);
AtspiAccessible *cthulhu_window_ancestor(AtspiAccessible *source); AtspiAccessible *cthulhu_window_ancestor(AtspiAccessible *source);
AtspiAccessible *cthulhu_document_ancestor(AtspiAccessible *source); AtspiAccessible *cthulhu_document_ancestor(AtspiAccessible *source);
AtspiAccessible *cthulhu_editable_document(AtspiAccessible *source);
AtspiEventListenerCB cthulhu_event_callback_pointer(void); AtspiEventListenerCB cthulhu_event_callback_pointer(void);
gboolean cthulhu_request_event_loop_quit(void); gboolean cthulhu_request_event_loop_quit(void);
gboolean cthulhu_request_tree_capture(gpointer userData); gboolean cthulhu_request_tree_capture(gpointer userData);
@@ -158,6 +158,26 @@ func (adapter *Adapter) rememberNativeFocus(source *C.AtspiAccessible, id Object
adapter.nativeFocus = append(adapter.nativeFocus, nativeFocusReference{id: id, source: owned}) adapter.nativeFocus = append(adapter.nativeFocus, nativeFocusReference{id: id, source: owned})
} }
// An unregistered bridge window must not remain a focused chrome candidate
// after its replacement has acquired the same X11 client. Other windows keep
// their retained focus references. This runs only on the GLib owner.
func (adapter *Adapter) forgetNativeWindowFocus(windowID ObjectID) {
retained := adapter.nativeFocus[:0]
for _, item := range adapter.nativeFocus {
window := C.cthulhu_window_ancestor(item.source)
retired := nativeObjectID(window) == windowID
if window != nil {
C.g_object_unref(C.gpointer(window))
}
if retired {
C.g_object_unref(C.gpointer(item.source))
} else {
retained = append(retained, item)
}
}
adapter.nativeFocus = retained
}
// Keep an exact document-to-frame association across Chromium tab switches. // Keep an exact document-to-frame association across Chromium tab switches.
// Some returned tabs temporarily lose their AT-SPI parent chain, although // Some returned tabs temporarily lose their AT-SPI parent chain, although
// their document identity and the already correlated frame remain live. // their document identity and the already correlated frame remain live.
+6
View File
@@ -149,6 +149,12 @@ chromeFocus:
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
for _, name := range []string{"Firefox baseline heading", "Firefox baseline link"} {
item, err := page.Resolve(findNamedReference(t, page, name))
if err != nil || item.CharacterCount != len([]rune(name)) {
t.Errorf("Firefox document text metadata for %q: characters=%d err=%v", name, item.CharacterCount, err)
}
}
for _, node := range captured.Nodes { for _, node := range captured.Nodes {
if node.Snapshot.Role == "paragraph" { if node.Snapshot.Role == "paragraph" {
paragraph = node.Reference paragraph = node.Reference
+9
View File
@@ -6,10 +6,19 @@ package atspi
import ( import (
"math" "math"
"slices"
"git.stormux.org/storm/cthulhu/internal/platform/xorg" "git.stormux.org/storm/cthulhu/internal/platform/xorg"
) )
// The native caller supplies a complete, fresh, bounded application window
// list, and separately checks the candidate's process, state and geometry.
func foregroundWindowReplaced(previous, candidate ObjectID, windows []ObjectID) bool {
return previous.Valid() && candidate.Valid() && previous != candidate &&
previous.BusName == candidate.BusName && slices.Contains(windows, candidate) &&
!slices.Contains(windows, previous)
}
// Epoch changes with the foreground client, including departure to an empty // Epoch changes with the foreground client, including departure to an empty
// workspace. Moving/resizing the same client does not reset reading ownership. // workspace. Moving/resizing the same client does not reset reading ownership.
type ForegroundState struct { type ForegroundState struct {
+26
View File
@@ -72,3 +72,29 @@ func TestForegroundMenuPopupCorrelation(t *testing.T) {
}) })
} }
} }
func TestForegroundWindowReplacement(t *testing.T) {
previous := NewObjectID(":1.40", "/gtk_frame")
candidate := NewObjectID(":1.40", "/gecko_frame")
sibling := NewObjectID(":1.40", "/other_window")
for _, test := range []struct {
name string
previous, candidate ObjectID
windows []ObjectID
want bool
}{
{"replaced application root", previous, candidate, []ObjectID{candidate, sibling}, true},
{"existing sibling cannot take ownership", previous, candidate, []ObjectID{previous, candidate}, false},
{"unregistered candidate", previous, candidate, []ObjectID{sibling}, false},
{"missing window list", previous, candidate, nil, false},
{"other provider", previous, NewObjectID(":1.41", "/frame"), []ObjectID{NewObjectID(":1.41", "/frame")}, false},
{"no previous owner", ObjectID{}, candidate, []ObjectID{candidate}, false},
{"unchanged owner", candidate, candidate, []ObjectID{candidate}, false},
} {
t.Run(test.name, func(t *testing.T) {
if got := foregroundWindowReplaced(test.previous, test.candidate, test.windows); got != test.want {
t.Fatalf("replacement = %v, want %v", got, test.want)
}
})
}
}
+3
View File
@@ -86,6 +86,9 @@ type Snapshot struct {
Checked bool Checked bool
Defunct bool Defunct bool
Editable bool Editable bool
// EditableRoot identifies the nearest editable web document, copied while
// capturing native events. Paragraph events can belong to its native focus.
EditableRoot ObjectID
Expandable bool Expandable bool
Expanded bool Expanded bool
Indeterminate bool Indeterminate bool
+2
View File
@@ -59,3 +59,5 @@ steam-shader-progress,Steam shader dialog AT-SPI events,web,Report Steam Vulkan
wayland-shared-keyboard-access,src/cthulhu/input_event_manager.py,input,Monitor keys and grab reader shortcuts through the shared accessibility keyboard interface,Storm Dragon,internal/platform/wayland internal/app internal/sessionbus,in-progress,Private dbus-run-session race tests cover wire protocol runtime dispatch configuration rollback secure transitions and cleanup; go test ./... -count=1; python3 tests/check_speech.py,Existing native_x11.py keyboard harness passed; browser tabs and editing passed; browser modes passed on isolated rerun after one failure whose cause was not established.,Shared D-Bus fixture passed without an X display. Live niri and physical key suppression remain unverified.,Pending live listening on Wayland,Pending physical Braille on Wayland,No compositor-specific code. See WAYLAND_SUPPORT.md for supported commands limitations and live acceptance. X11 production listeners are unchanged; the suspension probe now links its existing xkbcommon dependency. wayland-shared-keyboard-access,src/cthulhu/input_event_manager.py,input,Monitor keys and grab reader shortcuts through the shared accessibility keyboard interface,Storm Dragon,internal/platform/wayland internal/app internal/sessionbus,in-progress,Private dbus-run-session race tests cover wire protocol runtime dispatch configuration rollback secure transitions and cleanup; go test ./... -count=1; python3 tests/check_speech.py,Existing native_x11.py keyboard harness passed; browser tabs and editing passed; browser modes passed on isolated rerun after one failure whose cause was not established.,Shared D-Bus fixture passed without an X display. Live niri and physical key suppression remain unverified.,Pending live listening on Wayland,Pending physical Braille on Wayland,No compositor-specific code. See WAYLAND_SUPPORT.md for supported commands limitations and live acceptance. X11 production listeners are unchanged; the suspension probe now links its existing xkbcommon dependency.
web-browse-recovery-activation,Orca src/orca/scripts/web/script_utilities.py; master:src/cthulhu/scripts/web/script.py,web,Recover dynamic browse positions and activate custom clickables silently,,internal/atspi internal/review internal/app,in-progress,go test -race ./... -count=1; python3 tests/native_browser.py; python3 tests/native_browser_navigation.py; python3 tests/native_firefox.py,Private Chromium and Firefox fixtures; real-reader changing-page and activation checks use private Xvfb and D-Bus.,,Recorded SSIP checks verify page-only activation feedback; audible listening remains pending.,Selection-collapse regression and native mode frames covered; physical routing remains pending.,See docs/testing/browser-release-review.md for findings and live acceptance. No full Orca parity claim. web-browse-recovery-activation,Orca src/orca/scripts/web/script_utilities.py; master:src/cthulhu/scripts/web/script.py,web,Recover dynamic browse positions and activate custom clickables silently,,internal/atspi internal/review internal/app,in-progress,go test -race ./... -count=1; python3 tests/native_browser.py; python3 tests/native_browser_navigation.py; python3 tests/native_firefox.py,Private Chromium and Firefox fixtures; real-reader changing-page and activation checks use private Xvfb and D-Bus.,,Recorded SSIP checks verify page-only activation feedback; audible listening remains pending.,Selection-collapse regression and native mode frames covered; physical routing remains pending.,See docs/testing/browser-release-review.md for findings and live acceptance. No full Orca parity claim.
web-structural-lists,Orca src/orca/structural_navigator.py; src/orca/structural_navigator_command_definitions.py,web,List browser elements and move to a selected item,,internal/input internal/review internal/interaction internal/app,in-progress,"go test -race ./... -count=1; CTHULHU_NATIVE_BROWSER_TESTS=lists,keys,document,modes,editing python3 tests/native_browser.py; python3 tests/native_internal_ui.py",Private Brave/Xvfb list workflow covers 302 links hidden content current row selection Escape clickables heading level and empty list.,,Live listening pending.,Shared internal UI frames covered; physical Braille acceptance pending.,25 browse shortcuts use Alt+Shift. All Orca list types are registered including unbound annotations internal frames and math. Exact targets and page ownership are validated before selection. See docs/testing/browser-lists.md for manual review and regression checks. web-structural-lists,Orca src/orca/structural_navigator.py; src/orca/structural_navigator_command_definitions.py,web,List browser elements and move to a selected item,,internal/input internal/review internal/interaction internal/app,in-progress,"go test -race ./... -count=1; CTHULHU_NATIVE_BROWSER_TESTS=lists,keys,document,modes,editing python3 tests/native_browser.py; python3 tests/native_internal_ui.py",Private Brave/Xvfb list workflow covers 302 links hidden content current row selection Escape clickables heading level and empty list.,,Live listening pending.,Shared internal UI frames covered; physical Braille acceptance pending.,25 browse shortcuts use Alt+Shift. All Orca list types are registered including unbound annotations internal frames and math. Exact targets and page ownership are validated before selection. See docs/testing/browser-lists.md for manual review and regression checks.
mozilla-leaf-text-reading,Orca src/orca/scripts/web/script_utilities.py,web,Read native heading and link text leaves in Firefox and Thunderbird,,internal/atspi tests/native_mozilla.py,in-progress,python3 tests/native_firefox.py; CTHULHU_BINARY=/tmp/cthulhu-mozilla-review /usr/bin/python3 tests/native_mozilla.py; go test -race ./internal/atspi ./internal/review ./internal/presentation ./internal/app -count=1 -timeout=120s,Private Firefox 158.0 and Thunderbird 158.0 sessions each cover 17 navigation commands and Read All with zero native caret synchronization errors.,,Recorded SSIP includes both headings and the link in forward/reverse reading and Read All; audible listening pending.,Shared Chromium selection and routing checks passed; physical Mozilla Braille acceptance pending.,Shared document text metrics now include heading and link roles. Native Firefox metadata assertion failed before the fix. See docs/testing/mozilla-reading.md. Thunderbird folders message lists and composition remain outside this reading baseline.
thunderbird-local-mail-composition,Orca src/orca/scripts/apps/Thunderbird/script.py;src/orca/script_utilities.py,application,Navigate local folders and message tabs; select messages; compose replies and forwards; save and add attachments,,internal/atspi internal/app tests/native_thunderbird.py,in-progress,go test -race ./internal/atspi ./internal/app ./internal/review ./internal/presentation -count=1 -timeout=120s; CTHULHU_BINARY=/tmp/cthulhu-mozilla-review /usr/bin/python3 tests/native_thunderbird.py,Private Thunderbird 158.0 HTML and plain-text local mailbox checks cover message-tab return; Subject and body editing; reply and reply-all recipients with Cc; inline forwarding; exact selected rows after Shift and Ctrl navigation; received attachment saving with content verification; and adding a local file to an unsent draft.,,SSIP verifies focus handoff and row roles; selection toggling; Subject and body echo; reply and forward editing; attachment menus and picker return; audible acceptance pending.,Event-loop regression checks paragraph cursor and routing metadata; physical display acceptance pending.,Foreground replacement requires a registered application and fresh complete window membership. Editable paragraph events belong only to their focused editable document. Title echoes require owned editable focus and recent editing plus unchanged title prefix/suffix; unrelated status titles still announce. See docs/testing/thunderbird-workflows.md; no sending remote mailbox or page-load-option acceptance claimed.
1 id python_source subsystem behavior owner go_destination status automated_evidence x11_evidence wayland_evidence speech_evidence braille_evidence notes
59 wayland-shared-keyboard-access src/cthulhu/input_event_manager.py input Monitor keys and grab reader shortcuts through the shared accessibility keyboard interface Storm Dragon internal/platform/wayland internal/app internal/sessionbus in-progress Private dbus-run-session race tests cover wire protocol runtime dispatch configuration rollback secure transitions and cleanup; go test ./... -count=1; python3 tests/check_speech.py Existing native_x11.py keyboard harness passed; browser tabs and editing passed; browser modes passed on isolated rerun after one failure whose cause was not established. Shared D-Bus fixture passed without an X display. Live niri and physical key suppression remain unverified. Pending live listening on Wayland Pending physical Braille on Wayland No compositor-specific code. See WAYLAND_SUPPORT.md for supported commands limitations and live acceptance. X11 production listeners are unchanged; the suspension probe now links its existing xkbcommon dependency.
60 web-browse-recovery-activation Orca src/orca/scripts/web/script_utilities.py; master:src/cthulhu/scripts/web/script.py web Recover dynamic browse positions and activate custom clickables silently internal/atspi internal/review internal/app in-progress go test -race ./... -count=1; python3 tests/native_browser.py; python3 tests/native_browser_navigation.py; python3 tests/native_firefox.py Private Chromium and Firefox fixtures; real-reader changing-page and activation checks use private Xvfb and D-Bus. Recorded SSIP checks verify page-only activation feedback; audible listening remains pending. Selection-collapse regression and native mode frames covered; physical routing remains pending. See docs/testing/browser-release-review.md for findings and live acceptance. No full Orca parity claim.
61 web-structural-lists Orca src/orca/structural_navigator.py; src/orca/structural_navigator_command_definitions.py web List browser elements and move to a selected item internal/input internal/review internal/interaction internal/app in-progress go test -race ./... -count=1; CTHULHU_NATIVE_BROWSER_TESTS=lists,keys,document,modes,editing python3 tests/native_browser.py; python3 tests/native_internal_ui.py Private Brave/Xvfb list workflow covers 302 links hidden content current row selection Escape clickables heading level and empty list. Live listening pending. Shared internal UI frames covered; physical Braille acceptance pending. 25 browse shortcuts use Alt+Shift. All Orca list types are registered including unbound annotations internal frames and math. Exact targets and page ownership are validated before selection. See docs/testing/browser-lists.md for manual review and regression checks.
62 mozilla-leaf-text-reading Orca src/orca/scripts/web/script_utilities.py web Read native heading and link text leaves in Firefox and Thunderbird internal/atspi tests/native_mozilla.py in-progress python3 tests/native_firefox.py; CTHULHU_BINARY=/tmp/cthulhu-mozilla-review /usr/bin/python3 tests/native_mozilla.py; go test -race ./internal/atspi ./internal/review ./internal/presentation ./internal/app -count=1 -timeout=120s Private Firefox 158.0 and Thunderbird 158.0 sessions each cover 17 navigation commands and Read All with zero native caret synchronization errors. Recorded SSIP includes both headings and the link in forward/reverse reading and Read All; audible listening pending. Shared Chromium selection and routing checks passed; physical Mozilla Braille acceptance pending. Shared document text metrics now include heading and link roles. Native Firefox metadata assertion failed before the fix. See docs/testing/mozilla-reading.md. Thunderbird folders message lists and composition remain outside this reading baseline.
63 thunderbird-local-mail-composition Orca src/orca/scripts/apps/Thunderbird/script.py;src/orca/script_utilities.py application Navigate local folders and message tabs; select messages; compose replies and forwards; save and add attachments internal/atspi internal/app tests/native_thunderbird.py in-progress go test -race ./internal/atspi ./internal/app ./internal/review ./internal/presentation -count=1 -timeout=120s; CTHULHU_BINARY=/tmp/cthulhu-mozilla-review /usr/bin/python3 tests/native_thunderbird.py Private Thunderbird 158.0 HTML and plain-text local mailbox checks cover message-tab return; Subject and body editing; reply and reply-all recipients with Cc; inline forwarding; exact selected rows after Shift and Ctrl navigation; received attachment saving with content verification; and adding a local file to an unsent draft. SSIP verifies focus handoff and row roles; selection toggling; Subject and body echo; reply and forward editing; attachment menus and picker return; audible acceptance pending. Event-loop regression checks paragraph cursor and routing metadata; physical display acceptance pending. Foreground replacement requires a registered application and fresh complete window membership. Editable paragraph events belong only to their focused editable document. Title echoes require owned editable focus and recent editing plus unchanged title prefix/suffix; unrelated status titles still announce. See docs/testing/thunderbird-workflows.md; no sending remote mailbox or page-load-option acceptance claimed.
+206
View File
@@ -0,0 +1,206 @@
#!/usr/bin/env python3
"""Firefox/Thunderbird reading with a private reader, X11, profile and SSIP."""
import argparse
import json
import os
from pathlib import Path
import shutil
import subprocess
import sys
import tempfile
sys.dont_write_bytecode = True
from native_echo_preview import PreviewSpeechServer
from native_x11 import wait_for
TITLE = "Cthulhu Mozilla Reading Probe"
CONTENT = ["Reading start", "First readable paragraph.", "Reading section",
"Second readable paragraph.", "Local destination link", "Final readable paragraph."]
HTML = f'''<!doctype html><html lang="en"><meta charset="utf-8"><title>{TITLE}</title>
<body><h1>{CONTENT[0]}</h1><p>{CONTENT[1]}</p><h2>{CONTENT[2]}</h2>
<p>{CONTENT[3]}</p><a href="#destination">{CONTENT[4]}</a>
<p id="destination">{CONTENT[5]}</p></body></html>'''
def write_profile(profile, application):
profile.mkdir()
prefs = {"browser.startup.homepage_override.mstone": "ignore",
"datareporting.policy.firstRunURL": "",
"browser.shell.checkDefaultBrowser": False,
"browser.startup.page": 0,
"browser.newtabpage.enabled": False,
"datareporting.policy.dataSubmissionEnabled": False}
if application == "thunderbird":
(profile / "Mail" / "Local Folders").mkdir(parents=True)
prefs.update({"mail.shell.checkDefaultClient": False,
"mailnews.start_page.enabled": False,
"mail.provider.enabled": False,
"mail.rights.override": True,
"mailnews.database.global.indexer.enabled": False,
"mail.accountmanager.accounts": "account1",
"mail.accountmanager.defaultaccount": "account1",
"mail.accountmanager.localfoldersserver": "server1",
"mail.account.account1.server": "server1",
"mail.account.account1.identities": "id1",
"mail.server.server1.type": "none",
"mail.server.server1.hostname": "Local Folders",
"mail.server.server1.userName": "nobody",
"mail.server.server1.name": "Local Folders",
"mail.server.server1.directory": str(profile / "Mail" / "Local Folders"),
"mail.server.server1.directory-rel": "[ProfD]Mail/Local Folders",
"mail.identity.id1.fullName": "Username",
"mail.identity.id1.useremail": "username@example.invalid",
"mail.identity.id1.valid": True,
"mail.compose.autosave": False})
(profile / "user.js").write_text("".join(
f"user_pref({json.dumps(key)}, {json.dumps(value)});\n" for key, value in prefs.items()))
def walk(node):
if node is not None:
yield node
for child in node:
yield from walk(child)
def run_inside(application):
import pyatspi
directory = Path(os.environ["CTHULHU_MOZILLA_EVIDENCE"])
executable = shutil.which(application)
if not executable:
raise RuntimeError(f"{application} executable missing")
profile = directory / "profile"
write_profile(profile, application)
fixture = directory / "reading.html"
fixture.write_text(HTML)
args = [executable, "--no-remote", "--profile", str(profile), "--new-window", fixture.as_uri()]
if application == "thunderbird":
fixture = directory / "message.eml"
fixture.write_text("From: Username <username@example.invalid>\n"
"To: Recipient <recipient@example.invalid>\n"
f"Subject: {TITLE}\nMIME-Version: 1.0\n"
"Content-Type: text/html; charset=UTF-8\n\n" + HTML)
args = [executable, "--no-remote", "--profile", str(profile), "-file", str(fixture)]
config = directory / "settings.toml"
config.write_text('''schema-version = 2
active-profile = "default"
[profiles.default.speech]
output-module = "main"
[profiles.default.braille]
enabled = false
[profiles.default.sound]
enabled = false
[profiles.default.key-echo]
enabled = false
''')
wmConfig = directory / "i3.config"
wmConfig.write_text(f"font pango:monospace 10\nipc-socket {directory / 'i3.sock'}\n")
server = PreviewSpeechServer(directory / "speech.sock")
processes, report = [], []
logPath = directory / "reader.log"
def key(chord):
subprocess.run(["xdotool", "key", "--clearmodifiers", chord], check=True)
def find_document():
for app in pyatspi.Registry.getDesktop(0):
if app.name.lower() == application:
for node in walk(app):
if node.getRoleName() == "document web" and node.name == TITLE:
return node
def check(chord, expected):
offset = len(server.snapshot())
key(chord)
wait_for(lambda: any(row["text"] == expected
for row in server.snapshot()[offset:]),
f"{application}: {chord} did not speak {expected!r}", timeout=8)
# Normal presentation queues several chunks without SSIP end events.
# The recorder completes the final chunk; earlier chunks can remain open.
wait_for(lambda: server.snapshot()[-1]["finished"], "presentation did not settle")
rows = server.snapshot()[offset:]
actual = [row["text"] for row in rows if row["text"] in CONTENT]
assert actual == [expected], (chord, expected, rows)
assert not any("Could not" in row["text"] for row in rows), (chord, rows)
report.append({"key": chord, "expected": expected, "speech": rows})
try:
processes.append(subprocess.Popen(["i3", "-c", str(wmConfig)],
stdout=subprocess.DEVNULL, stderr=(directory / "wm.log").open("w")))
wait_for(lambda: "0x" in subprocess.check_output(
["xprop", "-root", "_NET_SUPPORTING_WM_CHECK"], text=True), "private WM missing")
processes.append(subprocess.Popen([
os.environ["CTHULHU_BINARY"], "--config", str(config), "--no-tentacles",
"--speech-socket", str(directory / "speech.sock"), "--debug-file", str(logPath)],
stdout=subprocess.DEVNULL, stderr=(directory / "reader-stderr.log").open("w")))
wait_for(lambda: logPath.exists() and "remote D-Bus service ready" in logPath.read_text(),
"private reader missing")
processes.append(subprocess.Popen(args, env=dict(os.environ, MOZ_ACCESSIBILITY_ENABLED="1"),
stdout=subprocess.DEVNULL, stderr=(directory / "application.log").open("w")))
document = wait_for(find_document, "fixture document missing", timeout=25)
window = subprocess.check_output(["xdotool", "search", "--onlyvisible", "--name", TITLE],
text=True).splitlines()[0]
subprocess.run(["xdotool", "windowactivate", "--sync", window], check=True)
assert document.queryComponent().grabFocus(), "document refused focus"
wait_for(lambda: document.getState().contains(pyatspi.STATE_FOCUSED), "document not focused")
wait_for(lambda: server.snapshot() and server.snapshot()[-1]["finished"],
"initial focus presentation did not settle")
check("ctrl+Home", CONTENT[0])
for text in CONTENT[1:]:
check("Down", text)
for text in reversed(CONTENT[:-1]):
check("Up", text)
check("h", CONTENT[2])
check("Down", CONTENT[3])
check("shift+h", CONTENT[2])
check("shift+h", CONTENT[0])
check("k", CONTENT[4])
check("ctrl+Home", CONTENT[0])
offset = len(server.snapshot())
key("KP_Add")
wait_for(lambda: any(row["text"] == CONTENT[-1] and row["finished"]
for row in server.snapshot()[offset:]), "Read All did not finish", timeout=12)
rows = server.snapshot()[offset:]
assert [row["text"] for row in rows if row["text"] in CONTENT] == CONTENT, rows
assert not any("Could not" in row["text"] for row in rows), rows
report.append({"key": "KP_Add", "speech": rows})
print(f"PASS: {application} forward/reverse lines, heading-to-arrow, link and Read All", flush=True)
finally:
for process in reversed(processes):
process.terminate()
try:
process.wait(timeout=5)
except subprocess.TimeoutExpired:
process.kill()
process.wait(timeout=5)
server.close()
(directory / "report.json").write_text(json.dumps(report, indent=2))
(directory / "speech.json").write_text(json.dumps(server.snapshot(), indent=2))
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("application", choices=("firefox", "thunderbird", "both"), nargs="?", default="both")
parser.add_argument("--inside", action="store_true", help=argparse.SUPPRESS)
args = parser.parse_args()
if not os.environ.get("CTHULHU_BINARY"):
parser.error("Set CTHULHU_BINARY to a freshly built reader")
if args.inside:
run_inside(args.application)
return
applications = ("firefox", "thunderbird") if args.application == "both" else (args.application,)
for application in applications:
directory = Path(tempfile.mkdtemp(prefix=f"cthulhu-{application}-reading-"))
print(f"{application} evidence: {directory}", flush=True)
environment = dict(os.environ, XDG_SESSION_TYPE="x11", CTHULHU_MOZILLA_EVIDENCE=str(directory))
for name in ("DBUS_SESSION_BUS_ADDRESS", "AT_SPI_BUS_ADDRESS", "WAYLAND_DISPLAY"):
environment.pop(name, None)
with (directory / "session.log").open("w") as log:
subprocess.run(["xvfb-run", "-a", "-s", "-screen 0 1280x900x24", "dbus-run-session", "--",
sys.executable, str(Path(__file__).resolve()), application, "--inside"],
env=environment, stderr=log, check=True, timeout=120)
if __name__ == "__main__":
main()
+381
View File
@@ -0,0 +1,381 @@
#!/usr/bin/env python3
"""Local Thunderbird folders, message lists and composition with a private reader."""
import argparse
from email.message import EmailMessage
import json
import os
from pathlib import Path
import shutil
import subprocess
import sys
import tempfile
import time
sys.dont_write_bytecode = True
from native_echo_preview import PreviewSpeechServer
from native_mozilla import walk, write_profile
from native_x11 import wait_for
SUBJECTS = ["First test message", "Second test message"]
APPLICATIONS = ("thunderbird", "org.mozilla.thunderbird", "xdg-desktop-portal-gtk")
def seed_mailbox(profile, html):
write_profile(profile, "thunderbird")
with (profile / "user.js").open("a") as stream:
stream.write(f'user_pref("mail.identity.id1.compose_html", {str(html).lower()});\n')
stream.write('user_pref("mailnews.default_sort_order", 1);\n')
stream.write('user_pref("mailnews.default_view_flags", 0);\n')
stream.write('user_pref("mail.openMessageBehavior", 2);\n')
stream.write('user_pref("mail.forward_message_mode", 2);\n')
for folder, subjects in {"Inbox": SUBJECTS, "Archive": ["Archived test message"]}.items():
messages = []
for index, subject in enumerate(subjects):
message = EmailMessage()
message["From"] = "Sender <sender@example.invalid>"
message["To"] = "Username <username@example.invalid>"
message["Cc"] = "Other Recipient <other@example.invalid>"
message["Subject"] = subject
message["Date"] = f"Sat, 17 Oct 2026 12:0{index}:00 +0000"
message["Message-ID"] = f"<{folder}-{index}@example.invalid>"
message.set_content(f"Body of {subject}.\n")
if folder == "Inbox" and index == 0:
message.add_attachment("Synthetic attachment contents.\n", subtype="plain",
filename="test-attachment.txt")
messages.append("From sender@example.invalid Sat Oct 17 12:00:00 2026\n"
+ message.as_string() + "\n")
(profile / "Mail" / "Local Folders" / folder).write_text("".join(messages))
def run_inside(html):
import pyatspi
from gi.repository import GLib
directory = Path(os.environ["CTHULHU_THUNDERBIRD_EVIDENCE"])
executable = shutil.which("thunderbird")
if not executable:
raise RuntimeError("Thunderbird executable missing")
profile = directory / "profile"
seed_mailbox(profile, html)
config = directory / "settings.toml"
config.write_text('''schema-version = 2
active-profile = "default"
[profiles.default.speech]
output-module = "main"
[profiles.default.braille]
enabled = false
[profiles.default.sound]
enabled = false
[profiles.default.key-echo]
enabled = true
''')
wmConfig = directory / "i3.config"
wmConfig.write_text(f"font pango:monospace 10\nipc-socket {directory / 'i3.sock'}\n")
server = PreviewSpeechServer(directory / "speech.sock")
processes, report = [], []
logPath = directory / "reader.log"
def key(chord):
subprocess.run(["xdotool", "key", "--clearmodifiers", chord], check=True)
def type_text(text):
subprocess.run(["xdotool", "type", "--clearmodifiers", "--delay", "35", "--", text], check=True)
def find_node(role, name=None, focused=False):
for app in pyatspi.Registry.getDesktop(0):
try:
if app.name.lower() not in APPLICATIONS:
continue
for node in walk(app):
if (node.getRoleName() == role and (name is None or name in node.name)
and node.getState().contains(pyatspi.STATE_SHOWING)
and (not focused or node.getState().contains(pyatspi.STATE_FOCUSED))):
return node
except GLib.Error:
# A closing portal dialog can retire a child during traversal.
# The caller's bounded wait retries against the current tree.
continue
def expect_focus(role, name=None):
return wait_for(lambda: find_node(role, name, focused=True),
f"native focus missing: {role} {name!r}", timeout=8)
def check(action, operation, expected, role=None, name=None, exact=False, selected=False):
offset = len(server.snapshot())
operation()
node = expect_focus(role, name) if role else None
wait_for(lambda: any((expected == row["text"] if exact else expected in row["text"])
for row in server.snapshot()[offset:]),
f"{action}: missing speech {expected!r}", timeout=8)
wait_for(lambda: server.snapshot()[-1]["finished"], f"{action}: speech did not settle")
rows = server.snapshot()[offset:]
assert not any("Could not" in row["text"] for row in rows), (action, rows)
if node and role in ("tree item", "list item"):
if selected or role == "tree item":
assert node.getState().contains(pyatspi.STATE_SELECTED), (action, "native selection missing")
assert any(row["text"] == role for row in rows), (action, "spoken row role missing", rows)
report.append({"action": action, "expected": expected,
"focus": {"role": node.getRoleName(), "name": node.name} if node else None,
"speech": rows})
print(f"PASS: {action}", flush=True)
return node
def read_text(node):
# HTML editors expose their paragraphs as children; plain text can be
# exposed directly on the document. Ignore embedded-object placeholders.
try:
text = node.queryText().getText(0, -1).replace("\ufffc", "")
except NotImplementedError:
text = ""
if text.strip():
return text
return "\n".join(read_text(child) for child in node)
def selected_text(node):
for child in walk(node):
try:
text = child.queryText()
if text.getNSelections():
start, end = text.getSelection(0)
selected = text.getText(start, end).replace("\ufffc", "")
if selected.strip():
return selected
except NotImplementedError:
continue
return ""
def selected_messages():
return [subject for subject in SUBJECTS
if (node := find_node("list item", subject)) is not None
and node.getState().contains(pyatspi.STATE_SELECTED)]
def discard_composer():
key("ctrl+w")
discard = wait_for(lambda: find_node("button", "Discard changes"),
"discard confirmation missing")
check("focus discard button", lambda: discard.queryComponent().grabFocus(),
"Discard changes", "button", "Discard changes")
check("discard unsent message", lambda: key("Return"), SUBJECTS[1],
"list item", SUBJECTS[1], selected=True)
try:
processes.append(subprocess.Popen(["i3", "-c", str(wmConfig)],
stdout=subprocess.DEVNULL, stderr=(directory / "wm.log").open("w")))
wait_for(lambda: "0x" in subprocess.check_output(
["xprop", "-root", "_NET_SUPPORTING_WM_CHECK"], text=True), "private WM missing")
processes.append(subprocess.Popen([
os.environ["CTHULHU_BINARY"], "--config", str(config), "--no-tentacles",
"--speech-socket", str(directory / "speech.sock"), "--debug-file", str(logPath)],
stdout=subprocess.DEVNULL, stderr=(directory / "reader-stderr.log").open("w")))
wait_for(lambda: logPath.exists() and "remote D-Bus service ready" in logPath.read_text(),
"private reader missing")
processes.append(subprocess.Popen([
executable, "--no-remote", "--profile", str(profile)],
env=dict(os.environ, MOZ_ACCESSIBILITY_ENABLED="1"),
stdout=subprocess.DEVNULL, stderr=(directory / "application.log").open("w")))
root = wait_for(lambda: find_node("tree item", "Local Folders"), "folder tree missing", timeout=25)
window = subprocess.check_output([
"xdotool", "search", "--onlyvisible", "--name", "Local Folders - Mozilla Thunderbird"],
text=True).splitlines()[0]
subprocess.run(["xdotool", "windowactivate", "--sync", window], check=True)
check("focus folder tree", lambda: root.queryComponent().grabFocus(),
"Local Folders", "tree item", "Local Folders")
key("Right")
check("next folder", lambda: key("Down"), "Inbox", "tree item", "Inbox")
check("last folder", lambda: key("End"), "Archive", "tree item", "Archive")
check("first folder", lambda: key("Home"), "Local Folders", "tree item", "Local Folders")
check("return to Inbox", lambda: key("Down"), "Inbox", "tree item", "Inbox")
check("folder to messages", lambda: key("F6"), SUBJECTS[0], "list item", SUBJECTS[0])
check("next message", lambda: key("Down"), SUBJECTS[1], "list item", SUBJECTS[1], selected=True)
check("previous message", lambda: key("Up"), SUBJECTS[0], "list item", SUBJECTS[0], selected=True)
check("extend message selection", lambda: key("shift+Down"), SUBJECTS[1],
"list item", SUBJECTS[1], selected=True)
wait_for(lambda: selected_messages() == SUBJECTS, "two-message selection missing")
report[-1]["selectedMessages"] = selected_messages()
check("shrink message selection", lambda: key("shift+Up"), SUBJECTS[0],
"list item", SUBJECTS[0], selected=True)
wait_for(lambda: selected_messages() == SUBJECTS[:1], "selection did not shrink")
report[-1]["selectedMessages"] = selected_messages()
check("move without extending selection", lambda: key("ctrl+Down"), SUBJECTS[1],
"list item", SUBJECTS[1])
wait_for(lambda: selected_messages() == SUBJECTS[:1], "Ctrl+Down changed selection")
check("add focused message to selection", lambda: key("ctrl+space"), "selected", exact=True)
wait_for(lambda: selected_messages() == SUBJECTS, "Ctrl+Space did not add message")
report[-1]["selectedMessages"] = selected_messages()
check("remove focused message from selection", lambda: key("ctrl+space"), "not selected", exact=True)
wait_for(lambda: selected_messages() == SUBJECTS[:1], "Ctrl+Space did not remove message")
report[-1]["selectedMessages"] = selected_messages()
check("return to first selected message", lambda: key("Up"), SUBJECTS[0],
"list item", SUBJECTS[0], selected=True)
check("messages to folders", lambda: key("shift+F6"), "Inbox", "tree item", "Inbox")
check("folders to retained message", lambda: key("F6"), SUBJECTS[0], "list item", SUBJECTS[0])
check("open selected message", lambda: key("Return"), SUBJECTS[0])
check("read opened message", lambda: key("ctrl+Home"), f"Body of {SUBJECTS[0]}.")
check("focus received attachment bar", lambda: key("Tab"),
"test-attachment.txt", "button", "1 attachment")
check("expand received attachments", lambda: key("Return"), "test-attachment.txt",
"list item", "test-attachment", selected=True)
check("received attachment menu", lambda: key("shift+F10"), "menu", "menu")
check("attachment open menu item", lambda: key("Down"), "Open", "menu item", "Open")
check("attachment save menu item", lambda: key("Down"), "Save As", "menu item", "Save As")
check("open attachment save dialog", lambda: key("Return"), "Save Attachment")
key("ctrl+l")
location = expect_focus("text")
savedAttachment = directory / "saved-attachment.txt"
key("ctrl+a")
type_text(str(savedAttachment))
wait_for(lambda: str(savedAttachment) in location.queryText().getText(0, -1),
"save attachment location missing")
check("save received attachment", lambda: key("Return"), "test-attachment",
"list item", "test-attachment", selected=True)
wait_for(savedAttachment.exists, "attachment was not saved")
assert savedAttachment.read_text().strip() == "Synthetic attachment contents."
check("return to message list", lambda: key("ctrl+w"), SUBJECTS[0],
"list item", SUBJECTS[0], selected=True)
check("next message after return", lambda: key("Down"), SUBJECTS[1],
"list item", SUBJECTS[1], selected=True)
for action, chord, prefix in (("reply", "ctrl+r", "Re: "),
("reply all", "ctrl+shift+r", "Re: "),
("forward", "ctrl+l", "Fwd: ")):
replying = action != "forward"
check(f"open {action}", lambda: key(chord), "Message body" if replying else "To",
"document web" if replying else "entry",
"Message body" if replying else "To")
subject = find_node("entry", "Subject")
assert subject.queryText().getText(0, -1) == prefix + SUBJECTS[1], (action, "subject")
report[-1]["subject"] = subject.queryText().getText(0, -1)
if replying:
assert find_node("section", "Sender <sender@example.invalid>: press Enter"), (action, "recipient")
if action == "reply all":
assert find_node("section", "Other Recipient <other@example.invalid>: press Enter"), (action, "Cc")
if action == "forward":
type_text("recipient@example.invalid")
check("commit forward recipient", lambda: key("Tab"), "recipient@example.invalid",
"entry", "To")
check("forward recipient to subject", lambda: key("Tab"), "Subject", "entry", "Subject")
check("forward subject to body", lambda: key("Tab"), "Message body",
"document web", "Message body")
body = expect_focus("document web", "Message body")
wait_for(lambda: f"Body of {SUBJECTS[1]}." in read_text(body), f"{action}: quoted body missing")
originalText = read_text(body)
check(f"{action} character echo", lambda: key("x"), "x", exact=True)
wait_for(lambda: read_text(body) != originalText, f"{action}: typed text missing")
discard_composer()
check("open composer", lambda: key("ctrl+m"), "To", "entry", "To")
recipient = expect_focus("entry", "To")
type_text("recipient@example.invalid")
wait_for(lambda: recipient.queryText().getText(0, -1) == "recipient@example.invalid",
"recipient text did not reach Thunderbird")
check("commit recipient", lambda: key("Tab"), "recipient@example.invalid", "entry", "To")
subject = check("recipient to subject", lambda: key("Tab"), "Subject", "entry", "Subject")
offset = len(server.snapshot())
type_text("Draft subject")
wait_for(lambda: subject.queryText().getText(0, -1) == "Draft subject", "subject typing missing")
# Let the title coalescer and SSIP recorder drain after the last key.
# The check must include the final title, not just the typing interval.
time.sleep(0.4)
rows = server.snapshot()[offset:]
assert any(row["text"] == "D" for row in rows), ("subject character echo missing", rows)
assert not any(row["text"].startswith("Write:") for row in rows), ("subject title echo", rows)
report.append({"action": "subject typing without title echo", "speech": rows})
print("PASS: subject typing without title echo", flush=True)
offset = len(server.snapshot())
check("subject character deletion", lambda: key("BackSpace"), "t", exact=True)
wait_for(lambda: subject.queryText().getText(0, -1) == "Draft subjec", "subject deletion missing")
time.sleep(0.4)
rows = server.snapshot()[offset:]
assert not any(row["text"].startswith("Write:") for row in rows), ("subject deletion title echo", rows)
type_text("t")
check("subject to body", lambda: key("Tab"), "Message body", "document web", "Message body")
body = expect_focus("document web")
assert body.getState().contains(pyatspi.STATE_EDITABLE), "body is not editable"
check("body character echo", lambda: key("h"), "h", exact=True)
check("body character deletion", lambda: key("BackSpace"), "h", exact=True)
wait_for(lambda: not read_text(body).strip(), "body character deletion missing")
type_text("Draft first line.")
key("Return")
type_text("Draft second line.")
wait_for(lambda: "Draft first line." in read_text(body) and "Draft second line." in read_text(body),
"body text did not reach Thunderbird")
check("body first line", lambda: key("ctrl+Home"), "Draft first line.")
check("body next line", lambda: key("Down"), "Draft second line.")
check("select body word", lambda: (key("Home"), key("ctrl+shift+Right")), "Draft selected", exact=True)
wait_for(lambda: selected_text(body).strip() == "Draft", "native body word selection missing")
check("delete selected word", lambda: key("BackSpace"), "Draft")
wait_for(lambda: "Draft second line." not in read_text(body) and "second line." in read_text(body),
"body selection/deletion did not reach Thunderbird")
check("body back to subject", lambda: key("shift+Tab"), "Subject", "entry", "Subject")
check("subject back to body", lambda: key("Tab"), "Message body", "document web", "Message body")
attachment = directory / "outgoing-attachment.txt"
attachment.write_text("Unsent synthetic attachment.\n")
check("open attachment picker", lambda: key("ctrl+shift+a"), "Attach File")
key("ctrl+l")
location = expect_focus("text")
type_text(str(attachment))
wait_for(lambda: str(attachment) in location.queryText().getText(0, -1),
"attachment picker location missing")
check("attach local file", lambda: key("Return"), attachment.name,
"list item", attachment.stem, selected=True)
check("clear attachment selection", lambda: key("Escape"), "not selected", exact=True)
wait_for(lambda: not find_node("list item", attachment.stem).getState().contains(pyatspi.STATE_SELECTED),
"attachment selection did not clear")
check("attachment back to body", lambda: key("Escape"), "Message body",
"document web", "Message body")
# No send command is used. Termination below leaves the unsent editor
# in this disposable profile without interacting with a real mailbox.
print(f"PASS: Thunderbird {'HTML' if html else 'plain-text'} local mail workflow", flush=True)
finally:
tree = []
try:
try:
for app in pyatspi.Registry.getDesktop(0):
if app.name.lower() in APPLICATIONS:
for node in walk(app):
tree.append({"role": node.getRoleName(), "name": node.name,
"showing": node.getState().contains(pyatspi.STATE_SHOWING),
"focused": node.getState().contains(pyatspi.STATE_FOCUSED),
"selected": node.getState().contains(pyatspi.STATE_SELECTED)})
except Exception as error:
tree.append({"error": str(error)})
(directory / "tree.json").write_text(json.dumps(tree, indent=2))
(directory / "report.json").write_text(json.dumps(report, indent=2))
(directory / "speech.json").write_text(json.dumps(server.snapshot(), indent=2))
finally:
for process in reversed(processes):
process.terminate()
try:
process.wait(timeout=5)
except subprocess.TimeoutExpired:
process.kill()
process.wait(timeout=5)
server.close()
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--format", choices=("html", "plain", "both"), default="both")
parser.add_argument("--inside", action="store_true", help=argparse.SUPPRESS)
args = parser.parse_args()
if not os.environ.get("CTHULHU_BINARY"):
parser.error("Set CTHULHU_BINARY to a freshly built reader")
if args.inside:
run_inside(args.format == "html")
return
formats = ("html", "plain") if args.format == "both" else (args.format,)
for composeFormat in formats:
directory = Path(tempfile.mkdtemp(prefix=f"cthulhu-thunderbird-{composeFormat}-"))
print(f"Thunderbird {composeFormat} evidence: {directory}", flush=True)
environment = dict(os.environ, XDG_SESSION_TYPE="x11", CTHULHU_THUNDERBIRD_EVIDENCE=str(directory))
for name in ("DBUS_SESSION_BUS_ADDRESS", "AT_SPI_BUS_ADDRESS", "WAYLAND_DISPLAY"):
environment.pop(name, None)
with (directory / "session.log").open("w") as log:
subprocess.run(["xvfb-run", "-a", "-s", "-screen 0 1280x900x24", "dbus-run-session", "--",
sys.executable, str(Path(__file__).resolve()), "--format", composeFormat, "--inside"],
env=environment, stderr=log, check=True, timeout=120)
if __name__ == "__main__":
main()