More flat review improvement.

This commit is contained in:
Storm Dragon committed 2026-10-21 20:56:18 -04:00
1 parent 50e32f24df
commit e88de5ee66
41 files changed
+2167 -73

No files matched your search

+16
View File
@@ -164,6 +164,22 @@ changes or “cap”. Explicit phonetic review commands retain capital indicator
See [phonetic review verification](docs/testing/phonetic-review.md) for checks
and live listening steps.
## Read the current line
Press **Cthulhu+Shift+Up** in either keyboard layout to read the line at the
application caret in native AT-SPI controls, or at the browse cursor on a webpage
in browse mode.
Press twice within 500 milliseconds to spell the line, or three times for
phonetics. The command leaves the caret and flat-review position in place.
Empty lines say “blank”; password contents are never read. Controls without
a text interface read their accessible name. Cthulhu+Up still increases the
selected speech setting.
This follows NVDA's desktop NVDA+Up current-line command, rather than replaying
the last spoken message. Numpad 8 (laptop: Cthulhu+I) continues to read the
independent flat-review line.
See [current-line verification](docs/testing/current-line.md) for checks.
## Read the current window title
Press **Cthulhu+Numpad Enter** in the desktop keyboard layout, or
@@ -4,7 +4,7 @@
pkgbase=cthulhu-git
pkgname=cthulhu-git
_pkgname=cthulhu
pkgver=2026.10.20.r61.g4643f8e
pkgver=2026.10.20.r62.g50e32f2
pkgrel=1
pkgdesc="Desktop-agnostic screen reader with speech, Braille, and Tentacle support"
url="https://git.stormux.org/storm/cthulhu"
+44
View File
@@ -0,0 +1,44 @@
# Current-line command
Cthulhu+Shift+Up reads the entire current line. Two presses within 500 ms spell
it; three use phonetics. Both desktop and laptop layouts use the same binding.
It follows NVDA's `script_reportCurrentLine` in `source/globalCommands.py`:
native caret outside browse mode, browse cursor inside it. It does not replay
the previous utterance or use the flat-review position.
Automated checks:
```sh
go test ./internal/input ./internal/review ./internal/app -run 'Test(CurrentLine|CommandRuntimeRoutesFocusPresentationRequests|ReviewMultiPressSequences)' -count=1
python3 tests/native_test_session.py env CTHULHU_NATIVE_CURRENT_LINE=1 go test ./internal/atspi -run '^TestNativeCurrentLine$' -count=1 -v
```
The private GTK fixture checks Unicode, a caret in the middle of a line, the
end of text, empty controls, blank lines, a trailing newline, password exclusion,
non-text control labels and rejection of stale focus. It verifies native caret
offsets after reading. Worker tests check whole browse lines and multiple inline
fragments, preserving the browse position. Existing flat-review and Say All tests
cover the shared line-reading and multi-press helpers.
Live acceptance after a normal rebuild and reload:
1. In a native multiline editor, enter `First line`, `Second café`, an empty line
and `Last line`. Put the caret in the middle of the second line. Press
Cthulhu+Shift+Up: expect the whole second line. Press Left afterward to verify
that the caret did not move. Check the blank line and the end of the last line.
2. Double and triple press to check spelling and phonetics. Pause longer than
500 ms and verify the next press reads normally. Check both keyboard layouts.
3. Move flat review to a different line. The new command should still read the
application caret line; Numpad 8 or Cthulhu+I should retain the review position.
4. In a browser, move into the middle of a line using browse arrows. Check the
whole line, including inline text fragments, and verify the next arrow
continues from the previous position. In a focused editable field, check the
native caret instead. Try another tab or window while a report is pending.
5. Check a FastSMRW table cell and a button. Check that password contents remain
silent and unavailable text gives explicit failure feedback. Cthulhu+Up must
still increase the selected speech setting. Check Say All and Numpad 8's
spelling behavior for regressions.
These automated checks do not establish audible output, physical key routing or
Braille behavior. The native implementation uses AT-SPI; Wine Access does not
currently expose this focused-line operation.
+206
View File
@@ -0,0 +1,206 @@
# Flat-review robustness comparison
Review date: 2026-10-21. Cthulhu baseline: `testing` at `50e32f2`.
Orca reference: upstream `main` at
`828eaf2e70d7b88124b10b3a6b2daa792ab266c7`. The local Orca checkout was
older; fetching the current reference confirmed that the three source files
below have no changes between those revisions.
This is a source comparison and a set of verified capture and recovery repairs.
It does not establish complete Orca parity or resolve every user report.
At the initial comparison, the available local log contained one
`accessible object not found` flat-review failure without enough evidence to
identify its cause, and no instance of the reported text-limit failure. Older diagnostic logs describe
provider timeouts already addressed by earlier capture work; they are not proof
of a remaining timeout in the current build.
## What Orca does
Orca's [Flat Review documentation](https://help.gnome.org/orca/howto_flat_review.html)
describes a spatial representation of the active window's visible contents.
The implementation separates finding on-screen objects, finding visible lines,
and building review zones:
- [`flat_review.py`](https://gitlab.gnome.org/GNOME/orca/-/blob/828eaf2e70d7b88124b10b3a6b2daa792ab266c7/src/orca/flat_review.py):
`Context` chooses a dialog/frame or an explicit root and restricts to an active
menu when appropriate. `_get_zones_from_text` clips to a scroll pane/document,
gets visible lines, removes embedded-object characters, and builds text zones.
Word and character objects are created on demand. `_find_zone_with_object`
can use a parent or child when the exact focused object was pruned.
- [`ax_utilities.py`](https://gitlab.gnome.org/GNOME/orca/-/blob/828eaf2e70d7b88124b10b3a6b2daa792ab266c7/src/orca/ax_utilities.py):
`_get_on_screen_objects` checks showing/visible state and window intersection,
treats suitable controls as leaves, enumerates visible table cells, and
suppresses some duplicate container text. Invalid component geometry is not
invariably treated as off-screen.
- [`ax_utilities_text.py`](https://gitlab.gnome.org/GNOME/orca/-/blob/828eaf2e70d7b88124b10b3a6b2daa792ab266c7/src/orca/ax_utilities_text.py):
`find_first_visible_line` searches by line rectangles, then `get_visible_lines`
walks forward until below the viewport. Invalid line offsets have explicit
handling. This differs from requiring usable character geometry to find a
large object's viewport.
These paths do not impose Cthulhu's 256-range/65,536-character object budgets.
That is not evidence that unlimited capture is safe: Cthulhu still needs bounded
native calls, memory use, cancellation, and protection from defective providers.
Orca also has its own tradeoffs: the single-line editable-text path requests all
text, and `Context.is_stale` itself only checks top-level size changes. Presenter
event handling supplies additional context management. Copying one function
alone would not reproduce that behavior.
## Comparison and priorities
| Area | Current Cthulhu behavior | Assessment |
| --- | --- | --- |
| Scope | Captures the focused window/menu/dialog and skips hidden descendants during review-only traversal. | Already aligned in principle. Preserve foreground ownership and popup handling. |
| Large GTK tables | Captures visible selectable-table cells and headers; keeps a selected-cell identity anchor. | Already follows Orca's viewport approach. Existing table checks remain relevant. |
| Clipped text ranges | Previously emitted one range per clipped visible character. | Confirmed defect; repaired here. Count contiguous visible spans against the range budget. |
| Viewport discovery | Uses character rectangles first; large objects can use a bounded binary search over line rectangles. | Private GTK coverage seeks into a 101000-character scrolled object with unavailable character rectangles. Small objects keep their existing bounded path. |
| Long horizontal lines | Oversized viewport intervals use glyph locations and bounded text retrieval; oversized native line responses also enter that path. | A real GTK 100000-character line is captured without increasing the character budget. New text requests contain at most 128 characters. Sparse or defective point providers remain bounded and explicitly limited. |
| Failure scope | Known unavailable text objects and failed child enumeration retain healthy content with explicit completeness counts. | A partial result announces a speech and Braille warning on entry or a changed omission count. Its line boundaries say top/bottom of available content. Root failure, cancellation, stale snapshots, invalid trees, and aggregate layout limits still fail the operation. |
| Focus anchoring | Tries exact focus, displayed descendants, then the nearest displayed ancestor within the review root. | A targeted fixture proves pruned focus stays on its parent text rather than an unrelated first line. Preserved position still uses object generations and native offsets. |
| Updates and cancellation | Cthulhu refreshes owned layout per command and rejects stale results; native capture and queues are bounded. | Preserve these safeguards. Snapshot refresh, provider timeout, and audible responsiveness are separate concerns. |
| Diagnostics | Capture timings include `text_issues` counts for viewport, character, range, tree-byte, query-budget, and provider failures. Partial presentation logs omission counts. | These fields contain no accessible text or names. A wholly limited window retains its existing spoken text-limit message. |
The line-based discovery, long-line retrieval, explicit partial recovery, and
parent anchoring above are implemented in the current working tree. Capture
budgets, provider-call chunking, cancellation, foreground ownership, and stale
result rejection remain in force. Remaining failures need an application,
triggering command, build revision, and fresh failure/timing record before
assigning a cause. Cold-capture latency remains separate from cached navigation.
## Repair and evidence
`internal/atspi/text.go` now accumulates adjacent visible characters with the
same row geometry into one range. It preserves exact Unicode offsets and unions
only their clipped rectangles. A hidden offset, embedded object, or row change
prevents merging across that boundary. The 128-character geometry blocks and
all existing capture budgets remain in force. No new mutable cross-command state
is introduced; span accumulation belongs to one synchronous capture.
Before the change, a real GTK label with 20 horizontally clipped lines produced
`VisibleText{Limited: true}`. After the change, the same fixture captures more
than 256 characters in at most 21 spans. A deterministic fixture verifies 800
Unicode characters in 20 spans, exact offsets, and clipping at both edges. Other
regressions cover a span crossing geometry blocks, disjoint bidirectional runs,
and a genuinely excessive number of separate visible spans. Existing tests
cover blank rows, embedded objects, large terminals, cancellation and malformed
provider data.
The successful four-row capture benchmark, three runs per version, measured
14.6–15.1 microseconds and 180 allocations before, versus 10.4–10.7 microseconds
and 19 allocations after. Allocated bytes fell from 19,664 to 4,048 per capture.
This measures local layout preparation with a synthetic provider; it does not
measure AT-SPI round trips or prove live speech latency.
## Automated checks
```sh
go test ./... -count=1
go test -race ./internal/atspi ./internal/review ./internal/app ./internal/presentation -count=1
go vet ./internal/atspi ./internal/review ./internal/app ./internal/presentation
python3 tests/native_atspi.py
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_CLIPPED=1 CTHULHU_NATIVE_REVIEW_CACHE=1 CTHULHU_NATIVE_REVIEW_BOUNDS=1 go test ./internal/atspi -run '^TestNativeFlatReview(ClippedViewport|PreservesProviderCache|BoundsSlowLongRanges)$' -count=1 -v
go test ./internal/atspi -run '^$' -bench '^BenchmarkCaptureVisibleTextClipped$' -benchmem -count=3
```
Use a writable `GOCACHE` if required by the environment. Native harnesses create
private Xvfb/D-Bus/AT-SPI sessions and do not replace the running reader.
All checks above passed for this change. The existing table harness emitted
GTK `Atk-CRITICAL` object-role warnings while its assertions passed; the cause
of those warnings was not investigated here.
## Follow-up implementation and evidence
Large-object line discovery moves by validated complete line boundaries during
binary search; it never scans terminal scrollback. Oversized horizontal text
uses native offset-at-point and glyph rectangles to discover visible offsets,
then reads text in spans of at most 128 characters. Offsets are sorted before
capture so bidirectional runs retain exact logical identities. Point traversal
has a 4096-query budget; it checks gaps before skipping vertically so an inset
or shorter glyph is not missed. Ordinary clipped capture continues to use bounded
geometry blocks. Unsupported or excessive work yields an explicit limitation,
not guessed text. No global capture limit was increased.
A native provider may return a zero-area rectangle for a nonvisible glyph.
That is distinct from a failed native request. Such glyphs now break visible
spans instead of discarding the entire object's usable text. Unavailable whole
line geometry remains an unavailable capture. A private GTK fixture injects a
zero-area glyph and verifies that the surrounding visible text survives and
the missing offset is excluded. Combining marks and joiners attached to a
visible base retain their Unicode offsets even without independent rectangles.
Bounded endpoint lookup also handles a huge line ending in a zero-area newline.
`ChildrenUnavailable` is capture-owned state copied into the owned tree and
atomic review snapshot. Recovery is restricted to review-only child enumeration;
root enumeration failure and cancellation remain fatal. `FlatCompleteness`
travels with each worker result. No partial object's accessible name is silently
substituted for its missing text. A warning is announced when entering a partial
layout or when its omission counts change; returning to a complete layout clears
that warning state. Ordinary invalidation triggers recapture as before; there
is no broad automatic retry loop. The warning uses the existing system-message
output path for both speech and Braille, while spelling retains its own request.
Seven private native cases cover cache reuse, bounded geometry, clipped spans,
a 100000-character horizontal line, scrolled line-only geometry, unavailable
child enumeration, and zero-area glyphs. Additional worker and presentation
fixtures cover healthy-content recovery, omission counts, recovery to a complete
layout, ancestor anchoring, spelling, Braille warning delivery, and incomplete
boundary feedback. Run native modes in separate processes:
```sh
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_VIEWPORT=1 CTHULHU_NATIVE_REVIEW_VIEWPORT_MODE=long go test ./internal/atspi -run '^TestNativeFlatReviewLargeTextViewport$' -count=1 -v
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_VIEWPORT=1 CTHULHU_NATIVE_REVIEW_VIEWPORT_MODE=lines go test ./internal/atspi -run '^TestNativeFlatReviewLargeTextViewport$' -count=1 -v
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_SUBTREE=1 go test ./internal/atspi -run '^TestNativeFlatReviewUnavailableSubtree$' -count=1 -v
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_GLYPHS=1 go test ./internal/atspi -run '^TestNativeFlatReviewZeroAreaGlyph$' -count=1 -v
```
A read-only probe of the open FastSMRW instance exposed zero-area glyph failures
that previously left visible table-cell text unavailable. After the repair,
27 worker commands across line, word, character, window-start, and window-end
review passed with zero reported omissions. Cached commands took about 0.3-0.7
milliseconds. Initial capture still took about eight seconds, mostly in visible
text. The standalone probe had no retained native focus event and reported
`root_scope=desktop` while filtering capture to FastSMRW's provider; this timing
does not establish normal foreground reader latency. No application keys were
injected and the running reader was not replaced. These results do not establish
audible speech or physical Braille acceptance.
### Follow-up check results
The full Go suite, affected-package race tests, vet, command builds, parity
ledger validation, recorded-speech checks, and all seven targeted native GTK
cases passed. The broad `tests/native_atspi.py` harness initially failed
`TestNativeGUIListTableSpeech` because the launcher description was repeated.
The isolated table checks passed on both an untouched `50e32f2` snapshot and the
changed tree, and a subsequent complete harness run passed. The initial failure
was not explained; passing reruns do not establish that this startup/event-order
behavior is stable. Existing GTK `Atk-CRITICAL` role warnings also remain.
## Human review and live acceptance
Review the generated diff before submission. With an authorized build/restart:
1. In a native editor or a window containing a wide multiline text control,
horizontally scroll so multiple lines are clipped. Review lines, words, and
characters forward and backward; visible text must remain available when
the total exceeds 256 characters. Repeat after resizing and scrolling. Check
the first and last partially visible characters and Unicode offsets.
2. Review ordinary and large terminal screens, blank rows, and a large file in
a terminal editor. Check Numpad 7/8/9, 4/5/6 and 1/2/3, window start/end,
explicit top/bottom feedback, and multi-press spelling.
3. Check a browser with horizontally clipped text and embedded links, a GTK
table such as a timeline, a preferences notebook, and a popup menu. Hidden
letters and embedded-object placeholders must not be spoken as ordinary text.
Password contents must stay excluded. Review must stay in the foreground
window and ordinary focus/caret navigation must still work afterward.
4. Exercise a window with one unavailable text object or child subtree. Healthy
content must remain reviewable with an explicit warning. Repeated navigation
should not repeat the warning continuously; after the provider recovers and
the tree is refreshed, ordinary top/bottom feedback must return.
5. Confirm audible speech and physical Braille separately, including routing
within a merged span. Native capture and Go test success do not establish
these outcomes. Preserve fresh logs for any remaining failure.
The code change affects shared visible-text capture used in owned trees;
terminal, browser, mouse-review, and Braille consumers can therefore be affected
indirectly. It changes neither X11 key grabs nor foreground ownership. Wayland
live acceptance remains separate.
+51
View File
@@ -0,0 +1,51 @@
// Copyright (c) 2026 Storm Dragon
// SPDX-License-Identifier: GPL-3.0-or-later
package app
import (
"git.stormux.org/storm/cthulhu/internal/input"
"git.stormux.org/storm/cthulhu/internal/presentation"
"git.stormux.org/storm/cthulhu/internal/review"
)
func (handler *eventReviewHandler) current_line(command input.Command) {
if handler.focusGate.Locked() || handler.focusGate.Suspended() {
return
}
handler.syncForeground()
if handler.objectReview == nil || !handler.state.current.ID().Valid() {
return
}
handler.state.pending.terminalFocus.clear()
handler.state.pending.terminalDeletion.clear()
style := review.FlatSpeak
if command == input.SpellCurrentLine {
style = review.FlatSpell
} else if command == input.PhoneticCurrentLine {
style = review.FlatPhonetic
}
browse := handler.browser != nil && handler.browser.ready && handler.browser.mode == review.BrowseMode
if err := handler.objectReview.ReadCurrentLine(browse, style); err != nil {
handler.logger.Warn("queue current line", "error", err)
}
}
func (handler *eventReviewHandler) present_current_line(result review.Result) {
if !handler.objectReview.CurrentDocumentResult(result) || result.DocumentFocus != handler.objectReview.Focus() || handler.focusGate.Suspended() {
return
}
if result.CurrentLineBrowse && (handler.browser == nil || !handler.browser.acceptsDocumentResult(review.Result{
DocumentFocus: result.DocumentFocus, DocumentPosition: result.DocumentPosition, DocumentCaretCommand: review.CaretReadLine, Err: result.Err,
})) {
return
}
request := speechRequest{Text: "Could not read current line.", Role: "system", Cause: "current-line"}
if result.Err == nil {
text := result.DocumentCaretText
request.Text = presentation.FlatReviewText(text, false)
request.Role = result.DocumentPosition.Snapshot.Role
set_review_spelling(&request, text, result.FlatSpeech)
}
handler.speech.Enqueue(request)
}
+100
View File
@@ -0,0 +1,100 @@
package app
import (
"context"
"errors"
"log/slog"
"reflect"
"testing"
"time"
"git.stormux.org/storm/cthulhu/internal/atspi"
"git.stormux.org/storm/cthulhu/internal/input"
"git.stormux.org/storm/cthulhu/internal/review"
)
type currentLineAdapter struct{ *fakeAdapter }
func (*currentLineAdapter) ReadFocusedLine(_ context.Context, focus atspi.ObjectID) (atspi.DocumentContent, error) {
return atspi.DocumentContent{Text: "Complete caret line", Snapshot: atspi.Snapshot{BusName: focus.BusName, Path: focus.Path, Role: "entry"}}, nil
}
func TestCurrentLineEventLoopPreservesFocusedItem(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
adapter := &currentLineAdapter{&fakeAdapter{events: make(chan atspi.Event)}}
worker, _ := review.NewWorker(adapter, time.Second)
if err := worker.Start(ctx); err != nil {
t.Fatal(err)
}
queue := &fakeSpeechQueue{spoken: make(chan string, 8)}
requests := make(chan input.Command)
done := make(chan error, 1)
go func() {
done <- processEventsWithSettings(ctx, slog.New(slog.DiscardHandler), queue, adapter, requests, worker, 0, nil)
}()
t.Cleanup(func() {
cancel()
if err := worker.Stop(context.Background()); err != nil {
t.Error(err)
}
<-done
})
waitSpeech := func(want string) {
t.Helper()
select {
case got := <-queue.spoken:
if got != want {
t.Fatalf("speech %q, want %q", got, want)
}
case <-time.After(time.Second):
t.Fatalf("missing speech %q", want)
}
}
focus := atspi.NewSnapshot("/entry", "Original item", "entry", "", "")
focus.BusName = ":1.42"
adapter.events <- atspi.NewEvent(atspi.FocusEvent, focus)
waitSpeech("Original item, entry")
requests <- input.PresentCurrentLine
waitSpeech("Complete caret line")
requests <- input.WhereAmI
waitSpeech("Original item, entry")
}
func TestCurrentLinePresentationAndStaleResults(t *testing.T) {
focus := atspi.ObjectID{BusName: ":1.42", Path: "/entry"}
worker, _ := review.NewWorker(&fakeAdapter{}, time.Second)
worker.SyncFocus(focus)
queue := &fakeSpeechQueue{requests: make(chan speechRequest, 8), spoken: make(chan string, 8)}
handler := eventReviewHandler{objectReview: worker, speech: queue, focusGate: &focusGate{}}
for _, probe := range []struct {
text string
style review.FlatSpeech
err error
want string
spelling []string
}{
{"A book", review.FlatSpeak, nil, "A book", nil},
{"A", review.FlatSpell, nil, "A", []string{"A"}},
{"A", review.FlatPhonetic, nil, "A", []string{"alpha"}},
{"\n", review.FlatSpeak, nil, "blank", nil},
{"", review.FlatSpeak, nil, "blank", nil},
{"", review.FlatSpeak, errors.New("unavailable"), "Could not read current line.", nil},
} {
result := review.Result{CurrentLine: true, DocumentFocus: focus, DocumentCaretText: probe.text, FlatSpeech: probe.style, Err: probe.err}
handler.present_current_line(result)
got := <-queue.requests
if got.Text != probe.want || !reflect.DeepEqual(got.Spelling, probe.spelling) {
t.Fatalf("current line: %+v, want %q %v", got, probe.want, probe.spelling)
}
}
worker.ResetDocument(focus)
handler.present_current_line(review.Result{CurrentLine: true, DocumentFocus: focus})
if len(queue.requests) != 0 {
t.Fatal("stale report was spoken")
}
worker.SyncFocus(atspi.ObjectID{BusName: ":1.42", Path: "/other"})
handler.present_current_line(review.Result{CurrentLine: true, DocumentFocus: focus, DocumentEpoch: worker.DocumentEpoch()})
if len(queue.requests) != 0 {
t.Fatal("report from previous focus was spoken")
}
}
+2
View File
@@ -325,6 +325,8 @@ func processEventsWithAttributes(
speak(event)
case command := <-reviewRequests:
switch command {
case input.PresentCurrentLine, input.SpellCurrentLine, input.PhoneticCurrentLine:
reviewHandler.current_line(command)
case input.PresentWindowTitle:
syncForeground()
titles.present()
+4
View File
@@ -83,6 +83,10 @@ func (handler *eventReviewHandler) handle(result review.Result) {
if !state.current.ID().Valid() {
return
}
if result.CurrentLine {
handler.present_current_line(result)
return
}
if result.ReadAllPosition {
if objectReview.CurrentDocumentResult(result) && result.DocumentFocus == objectReview.Focus() {
state.current = result.DocumentPosition.Snapshot
+13
View File
@@ -72,6 +72,8 @@ func presentFlatReviewResult(logger *slog.Logger, speech speechEnqueuer, result
text = "This window exceeds the current flat review text limit."
case errors.Is(result.Err, review.ErrNoReviewContent):
text = "No visible content."
case errors.Is(result.Err, review.ErrUnavailableReview):
text = "Visible content is unavailable for flat review."
default:
text = "Could not find current location."
}
@@ -82,8 +84,14 @@ func presentFlatReviewResult(logger *slog.Logger, speech speechEnqueuer, result
boundary = true
if result.FlatStep < 0 {
text = "Top of screen."
if result.FlatCompleteness.Incomplete() {
text = "Top of available content."
}
} else {
text = "Bottom of screen."
if result.FlatCompleteness.Incomplete() {
text = "Bottom of available content."
}
}
} else {
text = presentation.FlatReviewText(result.FlatPosition.Text, result.FlatUnit == review.Character || result.FlatUnit == review.LineEnd)
@@ -96,6 +104,11 @@ func presentFlatReviewResult(logger *slog.Logger, speech speechEnqueuer, result
if hasContent {
set_review_spelling(&request, result.FlatPosition.Text, result.FlatSpeech)
}
if result.FlatWarning {
logger.Warn("flat review content incomplete", "omitted_text_objects", result.FlatCompleteness.OmittedObjects,
"unavailable_subtrees", result.FlatCompleteness.UnavailableSubtrees)
speech.Enqueue(speechRequest{Text: "Some visible content is unavailable for flat review.", Role: "system", Cause: "flat-review-warning"})
}
speech.Enqueue(request)
if logger.Enabled(context.Background(), slog.LevelDebug) {
logger.Debug("queued flat review presentation",
+25
View File
@@ -24,10 +24,35 @@ import (
"time"
"git.stormux.org/storm/cthulhu/internal/atspi"
"git.stormux.org/storm/cthulhu/internal/config"
"git.stormux.org/storm/cthulhu/internal/input"
"git.stormux.org/storm/cthulhu/internal/review"
)
func TestFlatReviewPartialWarningPreservesSpellingAndFlashesBraille(t *testing.T) {
live := &liveSettings{}
live.Store(config.DefaultSettings())
braille := &flashRecorder{}
queue := &fakeSpeechQueue{spoken: make(chan string, 4), requests: make(chan speechRequest, 4)}
output := outputQueue{speech: queue, braille: braille, live: live}
result := review.Result{FlatUnit: review.Word, FlatSpeech: review.FlatSpell, FlatWarning: true,
FlatCompleteness: review.FlatCompleteness{OmittedObjects: 1}, FlatPosition: review.FlatPosition{Text: "cat"}}
presentFlatReviewResult(slog.New(slog.DiscardHandler), output, result)
warning, content := <-queue.requests, <-queue.requests
if warning.Cause != "flat-review-warning" || len(warning.Spelling) != 0 || len(content.Spelling) != 3 || content.Text != "cat" {
t.Fatalf("warning/content metadata = %+v / %+v", warning, content)
}
if len(braille.messages) != 1 || braille.messages[0] != warning.Text {
t.Fatalf("Braille warning missing: %v", braille.messages)
}
result.FlatWarning = false
result.FlatUnit, result.FlatStep, result.Moved = review.Line, 1, false
presentFlatReviewResult(slog.New(slog.DiscardHandler), output, result)
if boundary := <-queue.requests; boundary.Text != "Bottom of available content." {
t.Fatalf("partial boundary claimed complete screen: %+v", boundary)
}
}
func TestProcessEventsPresentsFlatReviewThroughSpeechQueue(t *testing.T) {
id := func(path string) atspi.ObjectID { return atspi.NewObjectID(":1.42", path) }
snapshot := func(path, name, role string) atspi.Snapshot {
+2
View File
@@ -172,6 +172,8 @@ func learnCommandDescription(command input.Command) string {
return "Choose an application and open its preferences."
case input.WhereAmI:
return "Speak details of the current item."
case input.PresentCurrentLine:
return "Read the current line. Press twice to spell it, or three times for phonetics."
case input.PresentWindowTitle:
return "Read the current window title."
case input.PresentCellFormula:
+3 -1
View File
@@ -229,13 +229,15 @@ func (worker commandRuntime) run(ctx context.Context) error {
if onNotificationList != nil {
onNotificationList()
}
case input.WhereAmI, input.PresentWindowTitle, input.PresentCellFormula, input.SetColumnHeadersRow, input.ClearColumnHeadersRow, input.SetRowHeadersColumn, input.ClearRowHeadersColumn:
case input.PresentCurrentLine, input.SpellCurrentLine, input.PhoneticCurrentLine, input.WhereAmI, input.PresentWindowTitle, input.PresentCellFormula, input.SetColumnHeadersRow, input.ClearColumnHeadersRow, input.SetRowHeadersColumn, input.ClearRowHeadersColumn:
if command == input.WhereAmI {
logger.Info("where am I command received")
} else if command == input.PresentWindowTitle {
logger.Info("window title command received")
} else if command == input.PresentCellFormula {
logger.Info("cell formula command received")
} else if command == input.PresentCurrentLine || command == input.SpellCurrentLine || command == input.PhoneticCurrentLine {
logger.Debug("current line command received", "command", command)
} else {
logger.Info("Calc header command received", "command", command)
}
+1 -1
View File
@@ -47,7 +47,7 @@ func TestCommandRuntimeRoutesFocusPresentationRequests(t *testing.T) {
t.Error("command worker did not stop")
}
}()
for _, command := range []input.Command{input.WhereAmI, input.PresentWindowTitle, input.PresentCellFormula, input.SetColumnHeadersRow, input.ClearColumnHeadersRow, input.SetRowHeadersColumn, input.ClearRowHeadersColumn} {
for _, command := range []input.Command{input.PresentCurrentLine, input.SpellCurrentLine, input.PhoneticCurrentLine, input.WhereAmI, input.PresentWindowTitle, input.PresentCellFormula, input.SetColumnHeadersRow, input.ClearColumnHeadersRow, input.SetRowHeadersColumn, input.ClearRowHeadersColumn} {
commands.commands <- input.CommandEvent{Command: command}
select {
case got := <-requests:
+1
View File
@@ -211,6 +211,7 @@ type treeCaptureRequest struct {
ocrGeometry bool
ocrZone *Rectangle
readFocusedText bool
readFocusedLine bool
readAttributes bool
setFocusedCaret bool
brailleRoute *BrailleRoute
+28 -12
View File
@@ -127,7 +127,7 @@ func captureNativeTree(ctx context.Context, root *C.AtspiAccessible, reviewBusNa
pending = pending[:last]
sourceID := nativeObjectID(source)
cached, known := cache.nodes[sourceID]
refresh := !known || cached.snapshotInvalid || cached.textInvalid || cached.geometryInvalid || cached.childrenInvalid
refresh := !known || cached.snapshotInvalid || cached.textInvalid || cached.geometryInvalid || cached.childrenInvalid || cached.childrenUnavailable
snapshot := cached.snapshot
trace.begin_stage("snapshot", sourceID, snapshot.Role)
if refresh {
@@ -206,7 +206,7 @@ func captureNativeTree(ctx context.Context, root *C.AtspiAccessible, reviewBusNa
childClip, childClipped = item.clip, item.clipped
}
trace.begin_stage("visible-text", snapshot.ID(), snapshot.Role)
text := VisibleText{Limited: true}
text := VisibleText{Limited: true, Issue: TextTreeLimit}
if (!document || documentText) && textBytes < maxTreeTextBytes {
if known && !cached.textInvalid && !cached.geometryInvalid && !item.clipChanged {
text = cached.text
@@ -224,7 +224,8 @@ func captureNativeTree(ctx context.Context, root *C.AtspiAccessible, reviewBusNa
size += len(r.Text)
}
if textBytes+size > maxTreeTextBytes {
text = VisibleText{Limited: true}
text = VisibleText{Limited: true, Issue: TextTreeLimit}
trace.text_issue(TextTreeLimit)
} else {
textBytes += size
}
@@ -239,7 +240,8 @@ func captureNativeTree(ctx context.Context, root *C.AtspiAccessible, reviewBusNa
(geometry.Visible && geometry.Showing) || snapshot.Role == "application" || snapshot.Role == "desktop frame"
// A visibility change can reveal children omitted by an earlier review
// capture, even when the provider emits no children-changed event.
cachedChildren := known && !cached.childrenInvalid && (!reviewOnly || !cached.geometryInvalid)
cachedChildren := known && !cached.childrenInvalid && !cached.childrenUnavailable && (!reviewOnly || !cached.geometryInvalid)
childrenUnavailable := false
for _, id := range cached.children {
if cache.sources[id] == nil {
cachedChildren = false
@@ -279,15 +281,22 @@ func captureNativeTree(ctx context.Context, root *C.AtspiAccessible, reviewBusNa
err := fmt.Errorf("%w (bus %q path %q role %q captured_nodes %d)",
nativeError("query AT-SPI child count", childCountError), snapshot.BusName, snapshot.Path, snapshot.Role, len(nodes))
C.g_error_free(childCountError)
C.g_object_unref(C.gpointer(source))
releasePending()
return ObjectID{}, nil, err
if !reviewOnly || item.parentIndex < 0 || ctx.Err() != nil {
C.g_object_unref(C.gpointer(source))
releasePending()
return ObjectID{}, nil, err
}
childCount, childrenUnavailable = 0, true
}
if childCount < 0 {
C.g_object_unref(C.gpointer(source))
releasePending()
return ObjectID{}, nil, fmt.Errorf("query AT-SPI child count: negative result (bus %q path %q role %q captured_nodes %d)",
snapshot.BusName, snapshot.Path, snapshot.Role, len(nodes))
if reviewOnly && item.parentIndex >= 0 && ctx.Err() == nil {
childCount, childrenUnavailable = 0, true
} else {
C.g_object_unref(C.gpointer(source))
releasePending()
return ObjectID{}, nil, fmt.Errorf("query AT-SPI child count: negative result (bus %q path %q role %q captured_nodes %d)",
snapshot.BusName, snapshot.Path, snapshot.Role, len(nodes))
}
}
if reviewBusName == "" && len(scheduled)+childCount > maxTreeNodes {
C.g_object_unref(C.gpointer(source))
@@ -318,6 +327,10 @@ func captureNativeTree(ctx context.Context, root *C.AtspiAccessible, reviewBusNa
if child != nil {
C.g_object_unref(C.gpointer(child))
}
if reviewOnly && ctx.Err() == nil {
childrenUnavailable = true
continue
}
C.g_object_unref(C.gpointer(source))
releasePending()
return ObjectID{}, nil, err
@@ -362,7 +375,7 @@ func captureNativeTree(ctx context.Context, root *C.AtspiAccessible, reviewBusNa
if document && refresh {
snapshot.TextChildren = nativeTextChildren(ctx, source, snapshot.CharacterCount, children)
}
nodes = append(nodes, TreeNode{Snapshot: snapshot, Children: children, Geometry: geometry, Text: text})
nodes = append(nodes, TreeNode{Snapshot: snapshot, Children: children, Geometry: geometry, Text: text, ChildrenUnavailable: childrenUnavailable})
trace.progress(snapshot.Role, len(nodes), len(pending), textBytes)
C.g_object_unref(C.gpointer(source))
}
@@ -412,6 +425,9 @@ func cthulhuAtspiTreeCaptureCallback(userData unsafe.Pointer) {
} else if request.setFocusedCaret {
err := adapter.setNativeFocusedTextCaret(request.ctx, request.reviewFocus, request.positionOffset)
request.result <- treeCaptureResult{err: err}
} else if request.readFocusedLine {
content, err := adapter.read_native_focused_line(request.ctx, request.reviewFocus)
request.result <- treeCaptureResult{content: content, err: err}
} else if request.readFocusedText {
content, err := adapter.readNativeFocusedText(request.ctx, request.reviewFocus)
request.result <- treeCaptureResult{content: content, err: err}
+16
View File
@@ -38,10 +38,25 @@ type captureDiagnostics struct {
capturedNodes int
pendingNodes int
textBytes int
textIssues map[TextCaptureIssue]int
timings []captureStageTiming
finished bool
}
func (trace *captureDiagnostics) text_issue(issue TextCaptureIssue) {
if trace == nil {
return
}
trace.mutex.Lock()
defer trace.mutex.Unlock()
if !trace.finished {
if trace.textIssues == nil {
trace.textIssues = make(map[TextCaptureIssue]int)
}
trace.textIssues[issue]++
}
}
func new_capture_diagnostics(ctx context.Context, logger *slog.Logger, focus ObjectID) (context.Context, *captureDiagnostics) {
if logger == nil || !logger.Enabled(ctx, slog.LevelDebug) {
return ctx, nil
@@ -128,5 +143,6 @@ func (trace *captureDiagnostics) finish(err error) {
"bus", trace.object.BusName, "object_path", trace.object.Path, "role", trace.role,
"root_bus", trace.root.BusName, "root_path", trace.root.Path, "root_scope", trace.rootScope,
"captured_nodes", trace.capturedNodes, "pending_nodes", trace.pendingNodes, "text_bytes", trace.textBytes,
"text_issues", trace.textIssues,
"stages", trace.timings)
}
+86
View File
@@ -0,0 +1,86 @@
// Copyright (c) 2026 Storm Dragon
// SPDX-License-Identifier: GPL-3.0-or-later
package atspi
/*
#cgo pkg-config: atspi-2 glib-2.0 gobject-2.0
#include <atspi/atspi.h>
*/
import "C"
import (
"context"
"errors"
"strings"
)
// ReadFocusedLine reads only the line containing the current native caret.
func (adapter *Adapter) ReadFocusedLine(ctx context.Context, focus ObjectID) (DocumentContent, error) {
result, err := adapter.documentOperation(ctx, treeCaptureRequest{reviewFocus: focus, readFocusedLine: true})
return result.content, err
}
func (adapter *Adapter) read_native_focused_line(ctx context.Context, focus ObjectID) (DocumentContent, error) {
if err := ctx.Err(); err != nil {
return DocumentContent{}, err
}
if !adapter.nativeFocusIs(focus) {
return DocumentContent{}, ErrStaleObject
}
source := adapter.nativeFocus[len(adapter.nativeFocus)-1].source
snapshot := snapshotFromNativeWithCaret(source, false, false, false, false)
if snapshot.Role == "password text" {
return DocumentContent{}, ErrNoDocumentCaret
}
content := DocumentContent{Snapshot: snapshot, Text: snapshot.Name}
text := C.atspi_accessible_get_text_iface(source)
if text == nil {
return content, ctx.Err()
}
defer C.g_object_unref(C.gpointer(text))
reader := nativeReviewText{text: text, terminal: snapshot.Role == "terminal"}
count, err := reader.count(ctx)
if err != nil {
return DocumentContent{}, err
}
content.Text = ""
if count == 0 {
return content, ctx.Err()
}
var nativeErr *C.GError
offset := int(C.atspi_text_get_caret_offset(text, &nativeErr))
if nativeErr != nil {
err := nativeError("read current line caret", nativeErr)
C.g_error_free(nativeErr)
return DocumentContent{}, err
}
// A text object without a caret reads from its first line, as NVDA does.
offset = min(max(0, offset), count)
if offset == count {
last := C.atspi_text_get_text(text, C.gint(count-1), C.gint(count), &nativeErr)
if last != nil {
defer C.g_free(C.gpointer(last))
}
if nativeErr != nil {
err := nativeError("read current line end", nativeErr)
C.g_error_free(nativeErr)
return DocumentContent{}, err
}
// A trailing newline puts the caret on an empty final line.
if strings.HasSuffix(C.GoString(last), "\n") {
content.Start, content.End = count, count
return content, ctx.Err()
}
offset--
}
line, err := reader.line(ctx, offset)
if err != nil {
return DocumentContent{}, err
}
if line.Start < 0 || line.Start > offset || line.End <= offset || line.End > count {
return DocumentContent{}, errors.New("read current line: invalid text range")
}
content.Start, content.End, content.Text, content.RawText = line.Start, line.End, line.Text, line.Text
return content, ctx.Err()
}
@@ -0,0 +1,99 @@
package atspi
import (
"bufio"
"context"
"encoding/json"
"fmt"
"os"
"os/exec"
"strings"
"testing"
"time"
)
func TestNativeCurrentLine(t *testing.T) {
if os.Getenv("CTHULHU_NATIVE_CURRENT_LINE") != "1" {
t.Skip("requires private Xvfb and AT-SPI")
}
ctx, cancel := context.WithTimeout(context.Background(), 25*time.Second)
defer cancel()
adapter := New()
if err := adapter.Start(ctx); err != nil {
t.Fatal(err)
}
defer adapter.Stop(context.Background())
fixture := exec.CommandContext(ctx, "/usr/bin/python3", "-u", "../../tests/current_line_fixture.py")
fixture.Stderr = os.Stderr
stdin, _ := fixture.StdinPipe()
stdout, _ := fixture.StdoutPipe()
if err := fixture.Start(); err != nil {
t.Fatal(err)
}
defer func() { stdin.Close(); fixture.Wait() }()
scanner := bufio.NewScanner(stdout)
ready := func() {
t.Helper()
if !scanner.Scan() || scanner.Text() != "ready" {
t.Fatalf("fixture not ready: %q %v", scanner.Text(), scanner.Err())
}
}
ready()
var focus ObjectID
lastControl := ""
for _, probe := range []struct {
control, text string
offset int
want string
password bool
}{
{"entry", "Café 猫 complete", 7, "Café 猫 complete", false},
{"text", "First\nSecond café 猫\nThird", 11, "Second café 猫", false},
{"entry", "Final line", 10, "Final line", false},
{"text", "First\n\nLast", 6, "", false},
{"entry", "", 0, "", false},
{"text", "Final\n", 6, "", false},
{"password", "private test text", 5, "", true},
{"button", "", 0, "Example button", false},
} {
command, _ := json.Marshal(map[string]any{"action": "focus", "control": probe.control, "text": probe.text, "offset": probe.offset})
fmt.Fprintln(stdin, string(command))
ready()
if probe.control != lastControl {
for {
select {
case event := <-adapter.Events():
if event.Type == "object:state-changed:focused" && event.Detail1 == 1 &&
(event.Source.Name == map[string]string{"entry": "Entry", "text": "Text", "password": "Password", "button": "Example button"}[probe.control]) {
focus = event.Source.ID()
goto focused
}
case <-ctx.Done():
t.Fatal(ctx.Err())
}
}
}
focused:
lastControl = probe.control
content, err := adapter.ReadFocusedLine(ctx, focus)
if probe.password {
if err == nil || content.Text != "" || content.Snapshot.Name != "" {
t.Fatalf("password report: %+v %v", content, err)
}
continue
}
if err != nil || strings.TrimRight(content.Text, "\r\n") != probe.want {
t.Fatalf("%s offset %d: %+v %v, want %q", probe.control, probe.offset, content, err, probe.want)
}
if probe.control != "button" {
fmt.Fprintf(stdin, "{\"action\":\"caret\",\"control\":%q}\n", probe.control)
if !scanner.Scan() || scanner.Text() != fmt.Sprint(probe.offset) {
t.Fatalf("caret moved: %q, want %d", scanner.Text(), probe.offset)
}
ready()
}
}
if _, err := adapter.ReadFocusedLine(ctx, ObjectID{BusName: focus.BusName, Path: "/stale"}); err == nil {
t.Fatal("accepted stale native focus")
}
}
+1 -1
View File
@@ -85,7 +85,7 @@ found:
text := entry.text
text.Ranges = append([]TextRange(nil), text.Ranges...)
nodes = append(nodes, TreeNode{Snapshot: entry.snapshot, Geometry: entry.geometry,
Text: text, Children: append([]ObjectID(nil), entry.children...)})
Text: text, Children: append([]ObjectID(nil), entry.children...), ChildrenUnavailable: entry.childrenUnavailable})
references[id] = newObjectReference(id, entry)
for i := len(entry.children) - 1; i >= 0; i-- {
pending = append(pending, entry.children[i])
+89 -4
View File
@@ -17,6 +17,7 @@ import (
"strings"
"testing"
"time"
"unicode/utf8"
"git.stormux.org/storm/cthulhu/internal/logging"
)
@@ -35,6 +36,39 @@ func TestNativeFlatReviewBoundsSlowLongRanges(t *testing.T) {
test_native_review_provider(t, "bounds")
}
func TestNativeFlatReviewClippedViewport(t *testing.T) {
if os.Getenv("CTHULHU_NATIVE_REVIEW_CLIPPED") != "1" {
t.Skip("requires private Xvfb/AT-SPI session")
}
test_native_review_provider(t, "clipped")
}
func TestNativeFlatReviewLargeTextViewport(t *testing.T) {
if os.Getenv("CTHULHU_NATIVE_REVIEW_VIEWPORT") != "1" {
t.Skip("requires private Xvfb/AT-SPI session")
}
// Adapter ownership is process-global; each mode runs in its own process.
mode := os.Getenv("CTHULHU_NATIVE_REVIEW_VIEWPORT_MODE")
if mode != "lines" && mode != "long" {
t.Fatal("set viewport mode to lines or long")
}
test_native_review_provider(t, mode)
}
func TestNativeFlatReviewUnavailableSubtree(t *testing.T) {
if os.Getenv("CTHULHU_NATIVE_REVIEW_SUBTREE") != "1" {
t.Skip("requires private Xvfb/AT-SPI session")
}
test_native_review_provider(t, "children")
}
func TestNativeFlatReviewZeroAreaGlyph(t *testing.T) {
if os.Getenv("CTHULHU_NATIVE_REVIEW_GLYPHS") != "1" {
t.Skip("requires private Xvfb/AT-SPI session")
}
test_native_review_provider(t, "glyphs")
}
func test_native_review_provider(t *testing.T, mode string) {
t.Helper()
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
@@ -96,11 +130,14 @@ func test_native_review_provider(t *testing.T, mode string) {
t.Fatalf("fixture %s: %q, %v", request, scanner.Text(), scanner.Err())
}
}
if mode == "bounds" {
if mode == "bounds" || mode == "clipped" || mode == "long" || mode == "lines" || mode == "glyphs" {
command("measure", "measuring")
} else {
} else if mode != "children" {
command("arm", "armed")
}
if mode == "lines" {
command("scroll", "scrolled")
}
tree, err := adapter.CaptureReviewWindow(ctx, focus)
if err != nil {
t.Fatal(err)
@@ -117,6 +154,33 @@ func test_native_review_provider(t *testing.T, mode string) {
if timing.Message != "flat review capture timing" || timing.RootScope != "window" || timing.CapturedNodes == 0 {
t.Fatalf("missing capture scope/progress: %+v", timing)
}
if mode == "children" {
broken := findNamedReference(t, tree, "Broken subtree")
if !broken.Valid() {
t.Fatal("broken subtree fixture missing")
}
command("arm", "armed")
tree.InvalidateChildren(broken.ID())
tree, err = adapter.CaptureReviewWindow(ctx, focus)
if err != nil {
t.Fatal(err)
}
snapshot, err := tree.ReviewWindow(ctx, focus)
if err != nil {
t.Fatal(err)
}
omitted, healthy := false, false
for _, node := range snapshot.Nodes {
omitted = omitted || node.Snapshot.ID() == broken.ID() && node.ChildrenUnavailable
for _, span := range node.Text.Ranges {
healthy = healthy || span.Text == "Cached review text"
}
}
if !omitted || !healthy {
t.Fatalf("subtree recovery: omission=%v healthy=%v", omitted, healthy)
}
return
}
for _, want := range []string{"queue-wait", "root-selection", "snapshot", "geometry", "visible-text", "table-viewport", "child-count", "child-enumeration", "tree-installation"} {
found := false
for _, stage := range timing.Stages {
@@ -128,7 +192,7 @@ func test_native_review_provider(t *testing.T, mode string) {
t.Errorf("capture timing missing stage %q: %+v", want, timing)
}
}
if mode == "bounds" {
if mode == "bounds" || mode == "clipped" || mode == "long" || mode == "lines" || mode == "glyphs" {
label := findNamedReference(t, tree, "Long review label")
if !label.Valid() {
t.Fatal("long label missing from review")
@@ -137,7 +201,28 @@ func test_native_review_provider(t *testing.T, mode string) {
if err != nil || !text.Known || text.Limited || len(text.Ranges) == 0 {
t.Fatalf("long label text: %+v, %v", text, err)
}
command("largest", "largest 128")
if mode == "bounds" {
command("largest", "largest 128")
}
if mode == "lines" && text.Ranges[0].Start <= maxReviewCharacters {
t.Fatalf("line viewport did not seek scrolled native offsets: %d", text.Ranges[0].Start)
}
if mode == "glyphs" {
for _, span := range text.Ranges {
if span.Start <= 5 && span.End > 5 {
t.Fatalf("zero-area glyph included as visible text: %+v", span)
}
}
}
if mode == "clipped" {
characters := 0
for _, span := range text.Ranges {
characters += utf8.RuneCountInString(span.Text)
}
if characters <= maxReviewRanges || len(text.Ranges) > 21 {
t.Fatalf("fixture must capture hundreds of characters as visual spans: characters=%d ranges=%d", characters, len(text.Ranges))
}
}
return
}
label := findNamedReference(t, tree, "Cached review text")
+136 -13
View File
@@ -22,6 +22,7 @@ import (
"errors"
"math"
"strings"
"unicode"
"unicode/utf8"
)
@@ -37,6 +38,20 @@ const (
var ErrStaleText = errors.New("accessible text requires recapture")
var errReviewLineTooLong = errors.New("review line requires viewport retrieval")
// TextCaptureIssue describes omitted text without retaining provider content.
type TextCaptureIssue string
const (
TextViewportUnavailable TextCaptureIssue = "viewport-unavailable"
TextCharacterLimit TextCaptureIssue = "character-limit"
TextRangeLimit TextCaptureIssue = "range-limit"
TextTreeLimit TextCaptureIssue = "tree-byte-limit"
TextProviderFailure TextCaptureIssue = "text-provider-failure"
TextViewportScanLimit TextCaptureIssue = "viewport-query-limit"
)
// TextRange owns exact UTF-8 text and AT-SPI character offsets [Start, End).
// Bounds are clipped screen coordinates. Offsets count Unicode code points,
// not bytes or grapheme clusters. Embedded objects have their own tree nodes.
@@ -52,6 +67,7 @@ type TextRange struct {
type VisibleText struct {
Known bool
Limited bool
Issue TextCaptureIssue
Ranges []TextRange
CaretKnown bool
CaretOffset int
@@ -61,7 +77,7 @@ func (text VisibleText) valid(geometry Geometry) bool {
if text.CaretKnown && (!text.Known || text.CaretOffset < 0) || !text.CaretKnown && text.CaretOffset != 0 {
return false
}
if text.Known && text.Limited || !text.Known && len(text.Ranges) != 0 || len(text.Ranges) > maxReviewRanges {
if text.Known && (text.Limited || text.Issue != "") || !text.Known && len(text.Ranges) != 0 || len(text.Ranges) > maxReviewRanges {
return false
}
if text.Known && !geometry.BoundsKnown {
@@ -125,6 +141,12 @@ type reviewTextSource interface {
bounds(context.Context, int, int) (Rectangle, error)
}
// Optional spatial retrieval avoids requesting an entire enormous native line.
// Implementations must bound copied characters and ranges before returning.
type reviewViewportSource interface {
viewport(context.Context, int, int, Rectangle) ([]TextRange, error)
}
// bounded_review_bounds preserves the full range's bounding rectangle without
// asking a provider to measure thousands of characters in one synchronous call.
// GTK text cells can exceed libatspi's timeout on such requests. Union every
@@ -208,9 +230,18 @@ func captureVisibleText(ctx context.Context, source reviewTextSource, clip Recta
return unknown, errors.New("negative review character count")
}
start, end, err := visibleTextOffsets(ctx, source, count, clip)
if err != nil && count > maxReviewCharacters {
if ctx.Err() != nil {
return unknown, ctx.Err()
}
start, end, err = visible_line_offsets(ctx, source, count, clip)
}
if err != nil {
if ctx.Err() != nil {
return unknown, ctx.Err()
}
if count > maxReviewCharacters {
return VisibleText{Limited: true}, nil
return capture_review_viewport(ctx, source, count, clip, TextViewportUnavailable)
}
// Some otherwise usable GTK text providers do not expose per-character
// extents. Retain the established bounded line capture for those small
@@ -218,34 +249,80 @@ func captureVisibleText(ctx context.Context, source reviewTextSource, clip Recta
start, end = 0, count
}
if end-start > maxReviewCharacters {
return VisibleText{Limited: true}, nil
if spatial, ok := source.(reviewViewportSource); ok {
ranges, spatialErr := spatial.viewport(ctx, start, end, clip)
if spatialErr == nil && len(ranges) > 0 {
return capture_review_lines(ctx, source, count, 0, count, clip, ranges)
}
if errors.Is(spatialErr, errReviewViewportScanLimit) {
return VisibleText{Limited: true, Issue: TextViewportScanLimit}, ctx.Err()
}
if ctx.Err() != nil {
return unknown, ctx.Err()
}
}
}
if end-start > maxReviewCharacters {
return VisibleText{Limited: true, Issue: TextCharacterLimit}, nil
}
return capture_review_lines(ctx, source, count, start, end, clip, nil)
}
func capture_review_lines(ctx context.Context, source reviewTextSource, count, start, end int, clip Rectangle, spatial []TextRange) (VisibleText, error) {
unknown := VisibleText{}
if spatial != nil {
if err := validate_viewport_ranges(spatial, count); err != nil {
return unknown, err
}
spatial = split_viewport_lines(spatial)
}
result := VisibleText{Known: true}
characters := 0
for offset, lines := start, 0; offset < end; lines++ {
if err := ctx.Err(); err != nil {
return unknown, err
}
if spatial != nil && lines == len(spatial) {
break
}
if lines >= maxReviewRanges {
return VisibleText{Limited: true}, nil
return VisibleText{Limited: true, Issue: TextRangeLimit}, nil
}
var line TextRange
var err error
if spatial != nil {
line = spatial[lines]
if line.Start < offset {
return unknown, errors.New("unordered review viewport ranges")
}
offset = line.Start
} else {
line, err = source.line(ctx, offset)
}
if errors.Is(err, errReviewLineTooLong) {
return capture_review_viewport(ctx, source, count, clip, TextCharacterLimit)
}
line, err := source.line(ctx, offset)
if err != nil {
return unknown, err
}
// Some AT-SPI providers return a line whose reported start is just
// after the requested offset. Python Cthulhu retries one character
// earlier for that provider quirk before rejecting the capture.
if line.Start > offset && offset > 0 {
if spatial == nil && line.Start > offset && offset > 0 {
line, err = source.line(ctx, offset-1)
if err != nil {
return unknown, err
}
}
if line.Start < 0 || line.Start > offset || line.End <= offset || line.End > count || len(line.Text) > maxReviewLineCharacters*utf8.UTFMax || !utf8.ValidString(line.Text) || utf8.RuneCountInString(line.Text) != line.End-line.Start {
if line.Start < 0 || line.Start > offset || line.End <= offset || line.End > count || len(line.Text) > maxReviewCharacters*utf8.UTFMax || !utf8.ValidString(line.Text) || utf8.RuneCountInString(line.Text) != line.End-line.Start {
return unknown, errors.New("invalid review line offsets or content")
}
lineStart := max(line.Start, offset)
lineEnd := min(line.End, end)
characters += lineEnd - lineStart
if characters > maxReviewCharacters {
return VisibleText{Limited: true, Issue: TextCharacterLimit}, nil
}
runes := []rune(line.Text)[lineStart-line.Start : lineEnd-line.Start]
// Python retains newline-only TextZones. Keep their exact offsets and
// native bounds so empty terminal rows remain navigable and speak blank.
@@ -273,14 +350,39 @@ func captureVisibleText(ctx context.Context, source reviewTextSource, clip Recta
if err != nil {
return unknown, err
}
if !bounds.Valid() {
return unknown, errors.New("unavailable review line geometry")
}
clipped, visible := bounds.Intersection(clip)
if visible && clipped == bounds {
result.Ranges = append(result.Ranges, TextRange{lineStart + start, lineStart + end, string(runes[start:end]), bounds})
singleRow := true
if spatial != nil && visible && end-start > 1 {
first, firstErr := source.bounds(ctx, lineStart+start, lineStart+start+1)
last, lastErr := source.bounds(ctx, lineStart+end-1, lineStart+end)
if firstErr != nil || lastErr != nil {
return unknown, errors.Join(firstErr, lastErr)
}
singleRow = first.Valid() && last.Valid() && first.Y == last.Y && first.Height == last.Height
}
if visible && singleRow && (clipped == bounds || clipped.X == bounds.X && clipped.Width == bounds.Width) {
// Vertical clipping of one visual row does not hide letters.
// This also supports providers with only line rectangles.
result.Ranges = append(result.Ranges, TextRange{lineStart + start, lineStart + end, string(runes[start:end]), clipped})
} else if visible {
// A clipped rectangle alone cannot hide offscreen characters. Check each
// visible character, preserving disjoint spans. Reject offscreen blocks
// first so a horizontally clipped long line does not require thousands
// of individual provider calls. Test every block for bidi text.
// Keep consecutive visible characters on the same visual row in
// one range. The budget bounds spans, not individual glyphs.
// Flush at an offset gap so bidi runs never include hidden text.
spanStart, spanEnd := -1, -1
var spanBounds Rectangle
flushSpan := func() {
if spanStart >= 0 {
result.Ranges = append(result.Ranges, TextRange{lineStart + spanStart, lineStart + spanEnd, string(runes[spanStart:spanEnd]), spanBounds})
spanStart = -1
}
}
for blockStart := start; blockStart < end; {
blockEnd := blockStart + min(end-blockStart, maxReviewBoundsCharacters)
if end-start > maxReviewBoundsCharacters {
@@ -301,18 +403,35 @@ func captureVisibleText(ctx context.Context, source reviewTextSource, clip Recta
if err != nil {
return unknown, err
}
if charClip, ok := charBounds.Intersection(clip); ok {
result.Ranges = append(result.Ranges, TextRange{lineStart + i, lineStart + i + 1, string(runes[i]), charClip})
charClip, ok := charBounds.Intersection(clip)
if !charBounds.Valid() && spanEnd == i && spanStart >= 0 && review_cluster_continuation(runes[i]) {
// Combining marks and joiners belong to the visible base
// even when they have no independent glyph rectangle.
charClip, ok = spanBounds, true
}
if ok {
if spanStart >= 0 && (spanEnd != i || spanBounds.Y != charClip.Y || spanBounds.Height != charClip.Height) {
flushSpan()
}
if spanStart < 0 {
spanStart, spanBounds = i, charClip
} else {
left := min(spanBounds.X, charClip.X)
right := max(int64(spanBounds.X)+int64(spanBounds.Width), int64(charClip.X)+int64(charClip.Width))
spanBounds.X, spanBounds.Width = left, int32(right-int64(left))
}
spanEnd = i + 1
}
if len(result.Ranges) > maxReviewRanges {
return VisibleText{Limited: true}, nil
return VisibleText{Limited: true, Issue: TextRangeLimit}, nil
}
}
blockStart = blockEnd
}
flushSpan()
}
if len(result.Ranges) > maxReviewRanges {
return VisibleText{Limited: true}, nil
return VisibleText{Limited: true, Issue: TextRangeLimit}, nil
}
start = end
}
@@ -321,6 +440,10 @@ func captureVisibleText(ctx context.Context, source reviewTextSource, clip Recta
return result, ctx.Err()
}
func review_cluster_continuation(character rune) bool {
return unicode.IsMark(character) || character == '\u200c' || character == '\u200d'
}
// visibleTextOffsets uses character extents only to locate the native viewport;
// the returned text remains line-based. AT-SPI text offsets are ordered by
// visual rows for the providers supported by flat review, allowing bounded
+52 -5
View File
@@ -189,19 +189,60 @@ func (source nativeReviewText) line(ctx context.Context, offset int) (TextRange,
}
length := int(C.review_text_length(line.content, C.int(maxReviewLineCharacters*utf8.UTFMax)))
if length < 0 || length > maxReviewLineCharacters*utf8.UTFMax {
return TextRange{}, errors.New("oversized or missing review line content")
return TextRange{}, errReviewLineTooLong
}
return TextRange{Start: int(line.start_offset), End: int(line.end_offset), Text: C.GoStringN(line.content, C.int(length))}, ctx.Err()
}
func (source nativeReviewText) viewport(ctx context.Context, start, end int, clip Rectangle) ([]TextRange, error) {
return point_review_viewport(ctx, source, start, end, clip)
}
func (source nativeReviewText) offset_at_point(ctx context.Context, x, y int32) (int, error) {
if err := ctx.Err(); err != nil {
return 0, err
}
var nativeErr *C.GError
offset := C.atspi_text_get_offset_at_point(source.text, C.gint(x), C.gint(y), C.ATSPI_COORD_TYPE_SCREEN, &nativeErr)
if nativeErr != nil {
C.g_error_free(nativeErr)
return 0, errors.New("query review offset at point")
}
return int(offset), ctx.Err()
}
func (source nativeReviewText) text_span(ctx context.Context, start, end int) (string, error) {
if err := ctx.Err(); err != nil {
return "", err
}
if end <= start || end-start > maxReviewBoundsCharacters {
return "", errors.New("invalid review span request")
}
var nativeErr *C.GError
content := C.atspi_text_get_text(source.text, C.gint(start), C.gint(end), &nativeErr)
if content != nil {
defer C.g_free(C.gpointer(content))
}
if nativeErr != nil {
C.g_error_free(nativeErr)
return "", errors.New("query review text span")
}
length := int(C.review_text_length(content, C.int((end-start)*utf8.UTFMax)))
if length < 0 || length > (end-start)*utf8.UTFMax {
return "", errors.New("invalid review span length")
}
value := C.GoStringN(content, C.int(length))
if !utf8.ValidString(value) || utf8.RuneCountInString(value) != end-start {
return "", errors.New("invalid review span offsets or content")
}
return value, ctx.Err()
}
func (source nativeReviewText) bounds(ctx context.Context, start, end int) (Rectangle, error) {
rect, err := bounded_review_bounds(ctx, start, end, source.range_bounds)
if err != nil {
return Rectangle{}, err
}
if !rect.Valid() {
return Rectangle{}, errors.New("invalid review range bounds")
}
return rect, ctx.Err()
}
@@ -242,7 +283,13 @@ func textFromNative(ctx context.Context, accessible *C.AtspiAccessible, geometry
defer C.g_object_unref(C.gpointer(text))
result, err := captureVisibleText(ctx, nativeReviewText{text: text, terminal: role == "terminal"}, clip)
if err != nil {
return VisibleText{}, ctx.Err()
if ctx.Err() != nil {
return VisibleText{}, ctx.Err()
}
result = VisibleText{Issue: TextProviderFailure}
}
if result.Issue != "" {
capture_diagnostics(ctx).text_issue(result.Issue)
}
if result.Known && ctx.Err() == nil {
var nativeErr *C.GError
+88 -1
View File
@@ -63,7 +63,7 @@ func TestCaptureVisibleTextUnicodeClippingAndEmbeddedObjects(t *testing.T) {
}}
got, err := captureVisibleText(context.Background(), source, clip)
want := VisibleText{Known: true, Ranges: []TextRange{
{1, 2, "猫", Rectangle{-10, 0, 10, 10}}, {2, 3, "x", Rectangle{0, 0, 10, 10}},
{1, 3, "猫x", Rectangle{-10, 0, 20, 10}},
{4, 5, "a", Rectangle{-10, 10, 5, 10}}, {6, 8, "bz", Rectangle{-5, 10, 10, 10}},
}}
if err != nil || !reflect.DeepEqual(got, want) {
@@ -178,6 +178,93 @@ func TestCaptureVisibleTextLimitsAndEmpty(t *testing.T) {
}
}
// A horizontally clipped viewport has hundreds of visible characters, but
// only a few visual rows. The range budget must count spans, not characters.
type clippedReviewText struct{ rows, columns int }
func (s clippedReviewText) count(context.Context) (int, error) { return s.rows * s.columns, nil }
func (s clippedReviewText) line(_ context.Context, offset int) (TextRange, error) {
start := offset / s.columns * s.columns
return TextRange{Start: start, End: start + s.columns, Text: strings.Repeat("é", s.columns)}, nil
}
func (s clippedReviewText) bounds(_ context.Context, start, end int) (Rectangle, error) {
return Rectangle{X: int32(start%s.columns*10 - 50), Y: int32(start / s.columns * 12), Width: int32((end - start) * 10), Height: 12}, nil
}
func TestCaptureVisibleTextClippedViewportUsesSpans(t *testing.T) {
source := clippedReviewText{rows: 20, columns: 400}
clip := Rectangle{X: 3, Width: 394, Height: int32(source.rows * 12)}
got, err := captureVisibleText(context.Background(), source, clip)
if err != nil || !got.Known || got.Limited || len(got.Ranges) != source.rows {
t.Fatalf("clipped capture: known=%v limited=%v ranges=%d err=%v", got.Known, got.Limited, len(got.Ranges), err)
}
for row, span := range got.Ranges {
want := TextRange{Start: row*source.columns + 5, End: row*source.columns + 45, Text: strings.Repeat("é", 40), Bounds: Rectangle{X: 3, Y: int32(row * 12), Width: 394, Height: 12}}
if span != want {
t.Fatalf("row %d: %+v, want %+v", row, span, want)
}
}
if !got.valid(Geometry{BoundsKnown: true, Bounds: clip}) {
t.Fatal("clipped viewport rejected by tree validation")
}
}
func TestCaptureVisibleTextClippedSpanCrossesGeometryBlocks(t *testing.T) {
source := clippedReviewText{rows: 1, columns: 400}
got, err := captureVisibleText(context.Background(), source, Rectangle{X: 3, Width: 2594, Height: 12})
want := VisibleText{Known: true, Ranges: []TextRange{
{5, 265, strings.Repeat("é", 260), Rectangle{3, 0, 2594, 12}},
}}
if err != nil || !reflect.DeepEqual(got, want) {
t.Fatalf("cross-block capture = %+v %v, want %+v", got, err, want)
}
}
func TestCaptureVisibleTextClippedDisjointSpanLimit(t *testing.T) {
count := (maxReviewRanges + 1) * 2
source := fakeReviewText{characters: count, lines: map[int]TextRange{
0: {Start: 0, End: count, Text: strings.Repeat("x", count)},
}, rectangles: map[[2]int]Rectangle{{0, count}: {-10, 0, 30, 10}}}
for i := 0; i < count; i++ {
source.rectangles[[2]int{i, i + 1}] = Rectangle{X: int32(i % 2 * 20), Width: 10, Height: 10}
}
got, err := captureVisibleText(context.Background(), source, Rectangle{0, 0, 10, 10})
if err != nil || !got.Limited || got.Known || len(got.Ranges) != 0 {
t.Fatalf("disjoint span limit: %+v %v", got, err)
}
}
func TestCaptureVisibleTextClippedBidiKeepsHiddenGaps(t *testing.T) {
source := fakeReviewText{characters: 5, lines: map[int]TextRange{
0: {Start: 0, End: 5, Text: "אבXגד"},
}, rectangles: map[[2]int]Rectangle{
{0, 5}: {-10, 0, 70, 10},
{0, 1}: {30, 0, 10, 10}, {1, 2}: {20, 0, 10, 10},
{2, 3}: {50, 0, 10, 10},
{3, 4}: {10, 0, 10, 10}, {4, 5}: {0, 0, 10, 10},
}}
got, err := captureVisibleText(context.Background(), source, Rectangle{0, 0, 40, 10})
want := VisibleText{Known: true, Ranges: []TextRange{
{0, 2, "אב", Rectangle{20, 0, 20, 10}},
{3, 5, "גד", Rectangle{0, 0, 20, 10}},
}}
if err != nil || !reflect.DeepEqual(got, want) {
t.Fatalf("bidi capture = %+v %v, want %+v", got, err, want)
}
}
func BenchmarkCaptureVisibleTextClipped(b *testing.B) {
source := clippedReviewText{rows: 4, columns: 100}
clip := Rectangle{X: 3, Width: 394, Height: int32(source.rows * 12)}
b.ReportAllocs()
for b.Loop() {
got, err := captureVisibleText(context.Background(), source, clip)
if err != nil || !got.Known || got.Limited {
b.Fatalf("clipped capture: known=%v limited=%v err=%v", got.Known, got.Limited, err)
}
}
}
func textNode() TreeNode {
node := testNode("/root", "Text")
node.Geometry = Geometry{Bounds: Rectangle{-10, 0, 20, 20}, BoundsKnown: true, VisibilityKnown: true, Visible: true, Showing: true}
+282
View File
@@ -0,0 +1,282 @@
// Copyright (c) 2026 Storm Dragon
// Authors: Storm Dragon, Angus Kola
// SPDX-License-Identifier: GPL-3.0-or-later
package atspi
import (
"context"
"errors"
"sort"
"strings"
"unicode/utf8"
)
var errReviewViewportScanLimit = errors.New("review viewport scan limit")
type reviewSpanSource interface {
bounds(context.Context, int, int) (Rectangle, error)
text_span(context.Context, int, int) (string, error)
}
type reviewPointSource interface {
reviewSpanSource
offset_at_point(context.Context, int32, int32) (int, error)
}
// Walk visible glyph rectangles rather than logical offsets. Logical offsets
// need not increase left-to-right (bidi), and a single line can be enormous.
// Check every intervening pixel when a provider cannot identify a glyph. A
// finite query budget rejects sparse or defective providers without omissions.
func point_review_viewport(ctx context.Context, source reviewPointSource, start, end int, clip Rectangle) ([]TextRange, error) {
if end <= start {
return nil, nil
}
first, err := review_endpoint_bounds(ctx, source, start, end, false)
if err != nil {
return nil, err
}
last, err := review_endpoint_bounds(ctx, source, start, end, true)
if err != nil {
return nil, err
}
top := max(int64(clip.Y), int64(first.Y))
bottom := min(int64(clip.Y)+int64(clip.Height), int64(last.Y)+int64(last.Height))
right := int64(clip.X) + int64(clip.Width)
offsets := make(map[int]Rectangle)
queries := 0
for y := top; y < bottom; {
nextRow := bottom
found := false
rowCovered := true
for x := int64(clip.X); x < right; {
if err := ctx.Err(); err != nil {
return nil, err
}
if queries >= 4096 {
return nil, errReviewViewportScanLimit
}
queries++
offset, err := source.offset_at_point(ctx, int32(x), int32(y))
if err != nil {
return nil, err
}
if offset < start || offset >= end {
rowCovered = false
x++
continue
}
bounds, known := offsets[offset]
if !known {
bounds, err = source.bounds(ctx, offset, offset+1)
if err != nil {
return nil, err
}
}
if !bounds.Valid() || x < int64(bounds.X) || x >= int64(bounds.X)+int64(bounds.Width) || y < int64(bounds.Y) || y >= int64(bounds.Y)+int64(bounds.Height) {
rowCovered = false
x++
continue
}
offsets[offset] = bounds
found = true
nextRow = min(nextRow, int64(bounds.Y)+int64(bounds.Height))
x = min(right, int64(bounds.X)+int64(bounds.Width))
}
if found && rowCovered {
y = nextRow
} else {
y++
}
}
ordered := make([]int, 0, len(offsets))
for offset := range offsets {
ordered = append(ordered, offset)
}
sort.Ints(ordered)
var ranges []TextRange
for index := 0; index < len(ordered); {
if len(ranges) >= maxReviewRanges {
return nil, errors.New("review viewport range limit")
}
first := ordered[index]
bounds := offsets[first]
next := index + 1
for next < len(ordered) && ordered[next] == ordered[next-1]+1 && next-index < maxReviewBoundsCharacters && offsets[ordered[next]].Y == bounds.Y && offsets[ordered[next]].Height == bounds.Height {
next++
}
last := ordered[next-1] + 1
content, err := source.text_span(ctx, first, last)
if err != nil {
return nil, err
}
// A mark or joiner may have no point of its own. Retain a bounded
// continuation of a visible base without including hidden letters.
limit := min(end, last+maxReviewBoundsCharacters)
if next < len(ordered) {
limit = min(limit, ordered[next])
}
if last < limit {
tail, err := source.text_span(ctx, last, limit)
if err != nil {
return nil, err
}
for _, character := range tail {
if !review_cluster_continuation(character) {
break
}
bounds, err := source.bounds(ctx, last, last+1)
if err != nil {
return nil, err
}
if _, visible := bounds.Intersection(clip); bounds.Valid() && !visible {
break
}
content += string(character)
last++
}
if last == limit && limit < end && (next == len(ordered) || limit < ordered[next]) {
following, err := source.text_span(ctx, last, last+1)
if err != nil {
return nil, err
}
for _, character := range following {
if review_cluster_continuation(character) {
return nil, errReviewViewportScanLimit
}
}
}
}
ranges = append(ranges, TextRange{Start: first, End: last, Text: content})
index = next
}
return ranges, ctx.Err()
}
// Trailing newlines and formatting characters can have no glyph rectangle.
// Look only within a bounded edge block, rather than discarding a long line
// or scanning arbitrary offscreen text to find an endpoint.
func review_endpoint_bounds(ctx context.Context, source reviewSpanSource, start, end int, backwards bool) (Rectangle, error) {
for index := 0; index < min(end-start, maxReviewBoundsCharacters); index++ {
if err := ctx.Err(); err != nil {
return Rectangle{}, err
}
offset := start + index
if backwards {
offset = end - index - 1
}
bounds, err := source.bounds(ctx, offset, offset+1)
if err != nil {
return Rectangle{}, err
}
if bounds.Valid() {
return bounds, nil
}
}
return Rectangle{}, errors.New("unavailable review viewport endpoint geometry")
}
// Line rectangles can be usable when single-character extents are missing.
// Move by complete line boundaries on each binary-search step; never scan
// scrollback or accept a non-progressing/provider-clamped line as a match.
func visible_line_offsets(ctx context.Context, source reviewTextSource, count int, clip Rectangle) (int, int, error) {
if !clip.Valid() {
return 0, 0, errors.New("invalid review text clip")
}
search := func(bottom bool) (int, error) {
low, high := 0, count
for low < high {
if err := ctx.Err(); err != nil {
return 0, err
}
middle := low + (high-low)/2
line, err := source.line(ctx, middle)
if err != nil {
return 0, err
}
if line.Start < 0 || line.Start > middle || line.End <= middle || line.End > count || line.End-line.Start > maxReviewLineCharacters {
return 0, errors.New("invalid review viewport line offsets")
}
bounds, err := source.bounds(ctx, line.Start, line.End)
if err != nil {
return 0, err
}
if !bounds.Valid() {
return 0, errors.New("invalid review line bounds")
}
after := int64(bounds.Y)+int64(bounds.Height) > int64(clip.Y)
if bottom {
after = int64(bounds.Y) >= int64(clip.Y)+int64(clip.Height)
}
if after {
high = max(low, line.Start)
} else {
low = min(high, line.End)
}
}
return low, nil
}
start, err := search(false)
if err != nil || start == count {
return start, start, err
}
end, err := search(true)
if err == nil && end < start {
err = errors.New("invalid review line viewport order")
}
return start, end, err
}
func capture_review_viewport(ctx context.Context, source reviewTextSource, count int, clip Rectangle, reason TextCaptureIssue) (VisibleText, error) {
if spatial, ok := source.(reviewViewportSource); ok {
ranges, err := spatial.viewport(ctx, 0, count, clip)
if ctx.Err() != nil {
return VisibleText{}, ctx.Err()
}
if err == nil && len(ranges) > 0 {
return capture_review_lines(ctx, source, count, 0, count, clip, ranges)
}
if errors.Is(err, errReviewViewportScanLimit) {
reason = TextViewportScanLimit
}
}
return VisibleText{Limited: true, Issue: reason}, ctx.Err()
}
// Validate all spatial output before accepting any of it. Native offsets can
// be large, but the amount of copied text and the number of spans cannot.
func validate_viewport_ranges(ranges []TextRange, count int) error {
if len(ranges) > maxReviewRanges {
return errors.New("review viewport range limit")
}
previous, characters := 0, 0
for _, span := range ranges {
if span.Start < previous || span.End <= span.Start || span.End > count || !utf8.ValidString(span.Text) || utf8.RuneCountInString(span.Text) != span.End-span.Start {
return errors.New("invalid review viewport text")
}
characters += span.End - span.Start
if characters > maxReviewCharacters {
return errReviewLineTooLong
}
previous = span.End
}
return nil
}
// Provider spatial ranges need not stop at line breaks. Split without making
// another whole-line native request; wrapping is handled by glyph geometry.
func split_viewport_lines(ranges []TextRange) []TextRange {
var lines []TextRange
for _, span := range ranges {
start := span.Start
for _, content := range strings.SplitAfter(span.Text, "\n") {
if content == "" {
continue
}
end := start + utf8.RuneCountInString(content)
lines = append(lines, TextRange{Start: start, End: end, Text: content})
start = end
}
}
return lines
}
+228
View File
@@ -0,0 +1,228 @@
// Copyright (c) 2026 Storm Dragon
// Authors: Storm Dragon, Angus Kola
// SPDX-License-Identifier: GPL-3.0-or-later
package atspi
import (
"context"
"errors"
"reflect"
"strings"
"testing"
)
type lineGeometryText struct {
lineCalls int
}
func (*lineGeometryText) count(context.Context) (int, error) { return 1000000, nil }
func (source *lineGeometryText) line(ctx context.Context, offset int) (TextRange, error) {
source.lineCalls++
start := offset / 100 * 100
return TextRange{Start: start, End: start + 100, Text: strings.Repeat("é", 100)}, ctx.Err()
}
func (*lineGeometryText) bounds(ctx context.Context, start, end int) (Rectangle, error) {
if end-start == 1 {
return Rectangle{}, errors.New("character rectangles unavailable")
}
return Rectangle{Y: int32(start/100-4200) * 10, Width: 100, Height: 10}, ctx.Err()
}
func TestCaptureVisibleTextDiscoversLargeViewportFromLines(t *testing.T) {
source := &lineGeometryText{}
clip := Rectangle{Width: 100, Height: 20}
got, err := captureVisibleText(context.Background(), source, clip)
want := VisibleText{Known: true, Ranges: []TextRange{
{420000, 420100, strings.Repeat("é", 100), Rectangle{Width: 100, Height: 10}},
{420100, 420200, strings.Repeat("é", 100), Rectangle{Y: 10, Width: 100, Height: 10}},
}}
if err != nil || !reflect.DeepEqual(got, want) || source.lineCalls > 40 {
t.Fatalf("line viewport = %+v, %v; calls=%d", got, err, source.lineCalls)
}
}
type spatialReviewText struct {
clippedReviewText
ranges []TextRange
lineCalls, viewportCalls int
cancel context.CancelFunc
}
func (source *spatialReviewText) line(ctx context.Context, offset int) (TextRange, error) {
source.lineCalls++
return TextRange{}, errReviewLineTooLong
}
func (source *spatialReviewText) viewport(ctx context.Context, start, end int, clip Rectangle) ([]TextRange, error) {
source.viewportCalls++
if source.cancel != nil {
source.cancel()
}
return source.ranges, ctx.Err()
}
func TestCaptureVisibleTextRetrievesLongLineViewportWithoutWholeLine(t *testing.T) {
source := &spatialReviewText{clippedReviewText: clippedReviewText{rows: 1, columns: 1000000}, ranges: []TextRange{
{Start: 500005, End: 500045, Text: strings.Repeat("é", 40)},
}}
clip := Rectangle{X: 5000003, Width: 394, Height: 12}
got, err := captureVisibleText(context.Background(), source, clip)
want := VisibleText{Known: true, Ranges: []TextRange{{500005, 500045, strings.Repeat("é", 40), clip}}}
if err != nil || !reflect.DeepEqual(got, want) || source.lineCalls != 0 || source.viewportCalls != 1 {
t.Fatalf("spatial capture = %+v, %v; line=%d viewport=%d", got, err, source.lineCalls, source.viewportCalls)
}
}
func TestCaptureVisibleTextRejectsInvalidSpatialOutput(t *testing.T) {
for _, ranges := range [][]TextRange{
{{Start: 8, End: 10, Text: "x"}},
{{Start: 8, End: 10, Text: "é猫"}, {Start: 7, End: 8, Text: "x"}},
{{Start: 0, End: maxReviewCharacters + 1, Text: strings.Repeat("x", maxReviewCharacters+1)}},
{{Start: 999999, End: 1000001, Text: "xx"}},
} {
source := &spatialReviewText{clippedReviewText: clippedReviewText{rows: 1, columns: 1000000}, ranges: ranges}
got, err := captureVisibleText(context.Background(), source, Rectangle{Width: 100, Height: 12})
if err == nil || got.Known || len(got.Ranges) != 0 {
t.Fatalf("invalid spatial capture accepted: %+v %v", got, err)
}
}
}
func TestCaptureVisibleTextSpatialCancellation(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
source := &spatialReviewText{clippedReviewText: clippedReviewText{rows: 1, columns: 1000000}, cancel: cancel}
got, err := captureVisibleText(ctx, source, Rectangle{Width: 100, Height: 12})
if !errors.Is(err, context.Canceled) || got.Known || got.Limited {
t.Fatalf("spatial cancellation = %+v %v", got, err)
}
}
type pointReviewText struct {
fakeReviewText
pointCalls int
textByOffset map[int]string
}
func (source *pointReviewText) offset_at_point(ctx context.Context, x, y int32) (int, error) {
source.pointCalls++
for span, bounds := range source.rectangles {
if span[1]-span[0] == 1 && int64(x) >= int64(bounds.X) && int64(x) < int64(bounds.X)+int64(bounds.Width) && int64(y) >= int64(bounds.Y) && int64(y) < int64(bounds.Y)+int64(bounds.Height) {
return span[0], ctx.Err()
}
}
return -1, ctx.Err()
}
func (source *pointReviewText) text_span(ctx context.Context, start, end int) (string, error) {
var text strings.Builder
for offset := start; offset < end; offset++ {
character, ok := source.textByOffset[offset]
if !ok {
character = "x"
}
text.WriteString(character)
}
return text.String(), ctx.Err()
}
func TestPointReviewViewportKeepsDisjointBidiOffsets(t *testing.T) {
source := &pointReviewText{fakeReviewText: fakeReviewText{rectangles: map[[2]int]Rectangle{
{0, 1}: {-100, 0, 10, 10}, {99999, 100000}: {100, 0, 10, 10},
{10, 11}: {30, 0, 10, 10}, {70000, 70001}: {10, 0, 10, 10}, {80000, 80001}: {20, 0, 10, 10},
}}, textByOffset: map[int]string{10: "א", 70000: "猫", 80000: "é"}}
ranges, err := point_review_viewport(context.Background(), source, 0, 100000, Rectangle{X: 10, Width: 30, Height: 10})
if err != nil || len(ranges) != 3 || source.pointCalls != 3 {
t.Fatalf("point bidi = %+v, %v; calls=%d", ranges, err, source.pointCalls)
}
for index, offset := range []int{10, 70000, 80000} {
if ranges[index].Start != offset || ranges[index].Text != source.textByOffset[offset] {
t.Fatalf("point bidi lost native offset: %+v", ranges)
}
}
}
func TestPointReviewViewportDoesNotSkipVerticallyInsetGlyph(t *testing.T) {
source := &pointReviewText{fakeReviewText: fakeReviewText{rectangles: map[[2]int]Rectangle{
{0, 1}: {0, 0, 10, 20}, {1, 2}: {10, 5, 10, 10}, {2, 3}: {20, 0, 10, 20},
}}, textByOffset: map[int]string{0: "A", 1: "é", 2: "B"}}
ranges, err := point_review_viewport(context.Background(), source, 0, 3, Rectangle{Width: 30, Height: 20})
if err != nil || len(ranges) != 3 || ranges[1].Text != "é" {
t.Fatalf("inset glyph omitted: %+v %v", ranges, err)
}
}
func TestPointReviewViewportBoundsSparseProviderWork(t *testing.T) {
source := &pointReviewText{fakeReviewText: fakeReviewText{rectangles: map[[2]int]Rectangle{
{0, 1}: {-100, 0, 10, 100}, {99999, 100000}: {1000, 0, 10, 100},
}}}
ranges, err := point_review_viewport(context.Background(), source, 0, 100000, Rectangle{Width: 100, Height: 100})
if !errors.Is(err, errReviewViewportScanLimit) || len(ranges) != 0 || source.pointCalls != 4096 {
t.Fatalf("sparse provider work: ranges=%d calls=%d err=%v", len(ranges), source.pointCalls, err)
}
}
func TestPointReviewViewportRetainsLineWithZeroAreaTrailingNewline(t *testing.T) {
source := &pointReviewText{fakeReviewText: fakeReviewText{rectangles: map[[2]int]Rectangle{
{0, 1}: {Width: 10, Height: 10}, {1, 2}: {X: 10, Width: 10, Height: 10}, {2, 3}: {},
}}, textByOffset: map[int]string{0: "é", 1: "猫", 2: "\n"}}
ranges, err := point_review_viewport(context.Background(), source, 0, 3, Rectangle{Width: 20, Height: 20})
if err != nil || len(ranges) != 1 || ranges[0].Text != "é猫" || ranges[0].End != 2 {
t.Fatalf("trailing newline discarded visible glyphs: %+v %v", ranges, err)
}
}
func TestCaptureVisibleTextSkipsZeroAreaGlyphsWithoutLosingOtherText(t *testing.T) {
source := fakeReviewText{characters: 6, lines: map[int]TextRange{
0: {Start: 0, End: 6, Text: "abcXYZ"},
}, rectangles: map[[2]int]Rectangle{
{0, 6}: {Width: 60, Height: 10},
{0, 1}: {Width: 10, Height: 10}, {1, 2}: {X: 10, Width: 10, Height: 10},
{2, 3}: {}, {3, 4}: {X: 20, Width: 10, Height: 10},
{4, 5}: {X: 40, Width: 10, Height: 10}, {5, 6}: {X: 50, Width: 10, Height: 10},
}}
got, err := captureVisibleText(context.Background(), source, Rectangle{Width: 30, Height: 10})
want := VisibleText{Known: true, Ranges: []TextRange{
{0, 2, "ab", Rectangle{Width: 20, Height: 10}},
{3, 4, "X", Rectangle{X: 20, Width: 10, Height: 10}},
}}
if err != nil || !reflect.DeepEqual(got, want) {
t.Fatalf("zero-area glyph capture = %+v, %v; want %+v", got, err, want)
}
}
func TestCaptureVisibleTextRetainsZeroAreaCombiningMark(t *testing.T) {
source := fakeReviewText{characters: 4, lines: map[int]TextRange{
0: {Start: 0, End: 4, Text: "a\u0301bc"},
}, rectangles: map[[2]int]Rectangle{
{0, 4}: {Width: 40, Height: 10}, {0, 1}: {Width: 10, Height: 10},
{1, 2}: {}, {2, 3}: {X: 10, Width: 10, Height: 10}, {3, 4}: {X: 30, Width: 10, Height: 10},
}}
got, err := captureVisibleText(context.Background(), source, Rectangle{Width: 20, Height: 10})
if err != nil || !got.Known || len(got.Ranges) != 1 || got.Ranges[0].Text != "a\u0301b" || got.Ranges[0].End != 3 {
t.Fatalf("visible accent lost: %+v %v", got, err)
}
}
func TestPointReviewViewportRetainsZeroAreaCombiningMark(t *testing.T) {
source := &pointReviewText{fakeReviewText: fakeReviewText{rectangles: map[[2]int]Rectangle{
{0, 1}: {Width: 10, Height: 10}, {1, 2}: {}, {2, 3}: {X: 10, Width: 10, Height: 10},
}}, textByOffset: map[int]string{0: "a", 1: "\u0301", 2: "b"}}
ranges, err := point_review_viewport(context.Background(), source, 0, 3, Rectangle{Width: 20, Height: 10})
if err != nil || len(ranges) != 2 || ranges[0].Text != "a\u0301" || ranges[0].End != 2 || ranges[1].Text != "b" {
t.Fatalf("point traversal lost accent: %+v %v", ranges, err)
}
}
func TestPointReviewViewportRejectsExcessiveClusterWithoutPartialText(t *testing.T) {
const finalOffset = maxReviewBoundsCharacters + 2
source := &pointReviewText{fakeReviewText: fakeReviewText{rectangles: map[[2]int]Rectangle{
{0, 1}: {Width: 10, Height: 10}, {finalOffset, finalOffset + 1}: {X: 10, Width: 10, Height: 10},
}}, textByOffset: map[int]string{0: "a", finalOffset: "b"}}
for offset := 1; offset < finalOffset; offset++ {
source.textByOffset[offset] = "\u0301"
}
ranges, err := point_review_viewport(context.Background(), source, 0, finalOffset+1, Rectangle{Width: 20, Height: 10})
if !errors.Is(err, errReviewViewportScanLimit) || len(ranges) != 0 {
t.Fatalf("cluster limit silently omitted text: %+v %v", ranges, err)
}
}
+15 -12
View File
@@ -45,6 +45,8 @@ type TreeNode struct {
Text VisibleText
Snapshot Snapshot
Children []ObjectID
// Review-only capture could not enumerate part of this object's children.
ChildrenUnavailable bool
}
// ObjectReference is a generation-checked reference to an owned snapshot. A
@@ -64,17 +66,18 @@ func (reference ObjectReference) Valid() bool {
}
type treeEntry struct {
snapshotInvalid bool
text VisibleText
textInvalid bool
geometry Geometry
geometryInvalid bool
childrenInvalid bool
snapshot Snapshot
children []ObjectID
parent ObjectID
hasParent bool
generation uint64
snapshotInvalid bool
text VisibleText
textInvalid bool
geometry Geometry
geometryInvalid bool
childrenInvalid bool
childrenUnavailable bool
snapshot Snapshot
children []ObjectID
parent ObjectID
hasParent bool
generation uint64
}
// AccessibleTree owns a validated accessibility tree. It contains no native
@@ -485,7 +488,7 @@ func validateTree(root ObjectID, nodes []TreeNode) (map[ObjectID]treeEntry, erro
text := node.Text
text.Ranges = append([]TextRange(nil), text.Ranges...)
children := append([]ObjectID(nil), node.Children...)
entries[id] = treeEntry{snapshot: node.Snapshot, children: children, geometry: node.Geometry, text: text}
entries[id] = treeEntry{snapshot: node.Snapshot, children: children, geometry: node.Geometry, text: text, childrenUnavailable: node.ChildrenUnavailable}
}
if _, ok := entries[root]; !ok {
return nil, fmt.Errorf("%w: root is missing", ErrInvalidTree)
+2
View File
@@ -56,6 +56,8 @@ func CommandLabel(command Command) string {
return "List " + strings.ToLower(binding.Title)
}
switch command {
case PresentCurrentLine:
return "Read the current line"
case PresentWindowTitle:
return "Read the current window title"
case PresentCellFormula:
+4
View File
@@ -14,6 +14,9 @@ const (
PresentTime Command = "present-time"
PresentDate Command = "present-date"
PresentClipboard Command = "present-clipboard"
PresentCurrentLine Command = "present-current-line"
SpellCurrentLine Command = "spell-current-line"
PhoneticCurrentLine Command = "phonetic-current-line"
ShowSpeechHistory Command = "show-speech-history"
ToggleSpeechMonitor Command = "toggle-speech-monitor"
CycleSpeechMode Command = "cycle-speech-mode"
@@ -204,6 +207,7 @@ func DefaultCommandBindings(layout KeyboardLayout) CommandBindings {
{Symbol: rightKeySymbol, CthulhuModifier: true}: SelectNextSpeechSetting,
{Symbol: downKeySymbol, CthulhuModifier: true}: DecreaseSpeechSetting,
{Symbol: upKeySymbol, CthulhuModifier: true}: IncreaseSpeechSetting,
{Symbol: upKeySymbol, Modifiers: 1, CthulhuModifier: true}: PresentCurrentLine,
{Symbol: 's', Modifiers: controlModifier | 1, CthulhuModifier: true}: SaveSpeechSettings,
{Symbol: 't', CthulhuModifier: true}: PresentTime,
{Symbol: 'c', Modifiers: 1, CthulhuModifier: true}: PresentClipboard,
+35
View File
@@ -0,0 +1,35 @@
package input
import (
"testing"
"time"
)
func TestCurrentLineBindingAndPresses(t *testing.T) {
for _, layout := range []KeyboardLayout{DesktopKeyboardLayout, LaptopKeyboardLayout} {
for _, bindings := range []CommandBindings{DefaultCommandBindings(layout), BrowseCommandBindings(layout)} {
router := NewCommandRouter([]uint32{0xff63}, bindings)
now := time.Unix(100, 0)
router.now = func() time.Time { return now }
router.Handle(0xff63, 118, 0, true)
press := func(modifiers uint16, want Command) {
t.Helper()
if consumed, got := router.Handle(upKeySymbol, 111, modifiers, true); !consumed || got != want {
t.Fatalf("layout %d: consumed=%v command=%s want=%s", layout, consumed, got, want)
}
if consumed, got := router.Handle(upKeySymbol, 111, modifiers, true); !consumed || got != "" {
t.Fatal("held arrow repeated command")
}
router.Handle(upKeySymbol, 111, modifiers, false)
}
press(1, PresentCurrentLine)
press(1, SpellCurrentLine)
press(1, PhoneticCurrentLine)
press(1, PresentCurrentLine)
now = now.Add(501 * time.Millisecond)
press(1, PresentCurrentLine)
press(0, IncreaseSpeechSetting)
press(1, PresentCurrentLine)
}
}
}
+10 -1
View File
@@ -10,7 +10,7 @@ import "time"
// updates. Another key/chord, timeout, or the final press retires it. Held-key
// repeats never reach this helper. Reset only clears held keys, as for time/date.
func (router *CommandRouter) review_press(command Command, chord CommandChord) Command {
if command != ReviewCurrentCharacter && command != ReviewCurrentWord && command != ReviewCurrentLine {
if command != ReviewCurrentCharacter && command != ReviewCurrentWord && command != ReviewCurrentLine && command != PresentCurrentLine {
router.lastReviewPress = time.Time{}
return command
}
@@ -44,5 +44,14 @@ func (router *CommandRouter) review_press(command Command, chord CommandChord) C
return ReviewPhoneticLine
}
}
if command == PresentCurrentLine {
switch router.reviewPresses {
case 2:
return SpellCurrentLine
case 3:
router.lastReviewPress = time.Time{}
return PhoneticCurrentLine
}
}
return command
}
+80
View File
@@ -0,0 +1,80 @@
// Copyright (c) 2026 Storm Dragon
// SPDX-License-Identifier: GPL-3.0-or-later
package review
import (
"context"
"time"
"git.stormux.org/storm/cthulhu/internal/atspi"
)
// ReadCurrentLine leaves both the native caret and review cursors untouched.
func (worker *Worker) ReadCurrentLine(browse bool, style FlatSpeech) error {
if style > FlatPhonetic {
return ErrInvalidDirection
}
select {
case worker.requests <- request{currentLine: true, currentLineBrowse: browse, flatSpeech: style,
documentEpoch: worker.documentEpoch.Load(), queuedAt: time.Now()}:
return nil
default:
return ErrRequestQueueFull
}
}
func (worker *Worker) read_current_line(ctx context.Context, cursor *DocumentCursor, browse bool) Result {
result := Result{CurrentLine: true, CurrentLineBrowse: browse, DocumentFocus: worker.focus()}
if !result.DocumentFocus.Valid() {
result.Err = ErrNoCurrent
return result
}
deadline, cancel := context.WithTimeout(ctx, worker.captureTimeout)
defer cancel()
if !browse {
reader, ok := worker.nativeCapturerFor(result.DocumentFocus).(interface {
ReadFocusedLine(context.Context, atspi.ObjectID) (atspi.DocumentContent, error)
})
if !ok {
result.Err = atspi.ErrNoDocumentCaret
return result
}
content, err := reader.ReadFocusedLine(deadline, result.DocumentFocus)
result.Err = err
result.DocumentCaretText, result.DocumentCaretRawText = content.Text, content.RawText
result.DocumentPosition.Snapshot = content.Snapshot
return result
}
// Start a private cursor at the saved browse position. Going to line start
// before reading avoids Say All's deliberate "rest of this line" behavior.
if cursor != nil {
cursor = &DocumentCursor{tree: cursor.tree, active: cursor.active, places: append([]documentPlace(nil), cursor.places...)}
_, result.Err = cursor.syncActionFocus(deadline, result.DocumentFocus)
} else {
result.Err = ErrNoCurrent
}
if documentNeedsCapture(result.Err) {
cursor, result.Err = worker.refreshDocumentCursor(deadline, cursor, result.DocumentFocus)
}
if result.Err != nil {
return result
}
reader, ok := worker.nativeCapturerFor(result.DocumentFocus).(DocumentCaretReader)
if !ok {
result.Err = atspi.ErrNoDocumentCaret
return result
}
_, cursor, result.Err = worker.readAllMove(deadline, cursor, reader, CaretLineStart, result.DocumentFocus)
if result.Err != nil {
return result
}
movement, _, err := worker.readAllMove(deadline, cursor, reader, CaretReadLine, result.DocumentFocus)
result.Err = err
result.DocumentPosition, result.DocumentCaretText = movement.Position, movement.Text
result.DocumentCaretRawText, result.DocumentCaretLine = movement.RawText, movement.Line
if result.DocumentCaretText == "" && wholeCaretNode(movement.Position.Snapshot) {
result.DocumentCaretText = movement.Position.Snapshot.Name
}
return result
}
+109
View File
@@ -0,0 +1,109 @@
package review
import (
"context"
"reflect"
"strings"
"testing"
"time"
"git.stormux.org/storm/cthulhu/internal/atspi"
)
type currentLineCapturer struct {
*readAllCapturer
content atspi.DocumentContent
reads int
}
func (capturer *currentLineCapturer) ReadFocusedLine(context.Context, atspi.ObjectID) (atspi.DocumentContent, error) {
capturer.reads++
return capturer.content, nil
}
func TestCurrentLineUsesNativeFocusOutsideBrowse(t *testing.T) {
capturer := &currentLineCapturer{content: atspi.DocumentContent{Text: "Native caret line"}}
worker, cancel := startWorker(t, capturer, time.Second)
defer cancel()
worker.SyncFocus(webID("/entry"))
for _, style := range []FlatSpeech{FlatSpeak, FlatSpell, FlatPhonetic} {
if err := worker.ReadCurrentLine(false, style); err != nil {
t.Fatal(err)
}
result := waitResult(t, worker)
if !result.CurrentLine || result.CurrentLineBrowse || result.FlatSpeech != style || result.Err != nil || result.DocumentCaretText != "Native caret line" {
t.Fatalf("native current line: %+v", result)
}
}
if capturer.reads != 3 {
t.Fatal("native line was cached")
}
}
func TestCurrentLineReadsWholeBrowseLineWithoutMoving(t *testing.T) {
reader := caretTextReader{webID("/text"): "First line\nSecond café 猫\nThird", webID("/next"): "Next"}
cursor := newCaretTextCursor(t, reader)
capturer := &readAllCapturer{tree: cursor.tree, caretTextReader: reader}
worker, _ := NewWorker(capturer, time.Second)
worker.SyncFocus(webID("/text"))
for range 15 {
if _, err := cursor.MoveCaret(context.Background(), reader, CaretNextCharacter); err != nil {
t.Fatal(err)
}
}
before := append([]documentPlace(nil), cursor.places...)
result := worker.read_current_line(context.Background(), cursor, true)
if result.Err != nil || strings.TrimSpace(result.DocumentCaretText) != "Second café 猫" {
t.Fatalf("whole browse line from middle: %+v", result)
}
if !reflect.DeepEqual(cursor.places, before) || capturer.nativePlacements != 0 {
t.Fatal("report changed browse or native caret")
}
}
func TestCurrentLineJoinsBrowseFragmentsWithoutMoving(t *testing.T) {
cursor, reader := fragment_cursor(t, "paragraph")
if _, err := cursor.MoveCaret(context.Background(), reader, CaretNextWord); err != nil {
t.Fatal(err)
}
capturer := &readAllCapturer{tree: cursor.tree, caretTextReader: reader.caretTextReader}
// Retain the fixture's geometry so the report can collect both fragments.
source := &currentLineFragmentCapturer{readAllCapturer: capturer, fragmentTextReader: reader}
worker, _ := NewWorker(source, time.Second)
worker.SyncFocus(webID("/one"))
before := append([]documentPlace(nil), cursor.places...)
result := worker.read_current_line(context.Background(), cursor, true)
if result.Err != nil || result.DocumentCaretText != "One two." || !reflect.DeepEqual(cursor.places, before) {
t.Fatalf("fragmented current line: %+v", result)
}
}
func TestCurrentLineReadsBrowseControlLabel(t *testing.T) {
doc := atspi.Snapshot{BusName: webID("/doc").BusName, Path: "/doc", Role: "document web"}
button := atspi.Snapshot{BusName: doc.BusName, Path: "/button", Role: "push button", Name: "Example button"}
tree, err := atspi.NewAccessibleTree(doc.ID(), []atspi.TreeNode{
{Snapshot: doc, Children: []atspi.ObjectID{button.ID()}}, {Snapshot: button},
})
if err != nil {
t.Fatal(err)
}
cursor, _ := NewDocumentCursor(tree)
if _, err := cursor.SyncFocus(context.Background(), button.ID()); err != nil {
t.Fatal(err)
}
worker, _ := NewWorker(&readAllCapturer{tree: tree, caretTextReader: caretTextReader{}}, time.Second)
worker.SyncFocus(button.ID())
result := worker.read_current_line(context.Background(), cursor, true)
if result.Err != nil || result.DocumentCaretText != "Example button" {
t.Fatalf("browse control label: %+v", result)
}
}
type currentLineFragmentCapturer struct {
*readAllCapturer
fragmentTextReader
}
func (capturer *currentLineFragmentCapturer) ReadDocumentCaret(ctx context.Context, ref atspi.ObjectReference, offset int, unit atspi.DocumentCaretUnit) (atspi.DocumentCaretText, error) {
return capturer.fragmentTextReader.ReadDocumentCaret(ctx, ref, offset, unit)
}
+22 -3
View File
@@ -67,6 +67,8 @@ type FlatCursor struct {
verticalTargetX int64
hasVerticalTarget bool
started bool
completeness FlatCompleteness
warnedIncomplete bool
}
func (cursor *FlatCursor) Refresh(ctx context.Context, tree *atspi.AccessibleTree, focus atspi.ObjectID) error {
@@ -78,11 +80,11 @@ func (cursor *FlatCursor) Refresh(ctx context.Context, tree *atspi.AccessibleTre
return err
}
nodes := snapshot.Nodes
lines, err := buildFlatLines(ctx, nodes)
lines, completeness, err := build_flat_layout(ctx, nodes)
if err != nil {
return err
}
next := FlatCursor{lines: lines, focus: focus, tree: tree, references: snapshot.References,
next := FlatCursor{lines: lines, focus: focus, tree: tree, references: snapshot.References, completeness: completeness,
parents: make(map[atspi.ObjectID]atspi.ObjectID, len(nodes))}
for _, n := range nodes {
for _, child := range n.Children {
@@ -91,7 +93,9 @@ func (cursor *FlatCursor) Refresh(ctx context.Context, tree *atspi.AccessibleTre
}
// Exact identity takes priority over a displayed descendant of focus.
if !next.seek(focus, 0, false) {
next.seekDescendant(focus)
if !next.seekDescendant(focus) {
next.seek_ancestor(focus)
}
}
for _, n := range nodes {
if n.Snapshot.ID() == focus && n.Text.CaretKnown {
@@ -109,10 +113,25 @@ func (cursor *FlatCursor) Refresh(ctx context.Context, tree *atspi.AccessibleTre
}
}
next.started = cursor.started && cursor.focus == focus
next.warnedIncomplete = cursor.warnedIncomplete && cursor.focus == focus && cursor.tree == tree && cursor.completeness == completeness
*cursor = next
return nil
}
// A focused leaf can have no displayed zone while its parent supplies the
// text. Choose only an actual ancestor zone, never a sibling or window title.
func (cursor *FlatCursor) seek_ancestor(object atspi.ObjectID) bool {
for id := cursor.parents[object]; id.Valid(); id = cursor.parents[id] {
if !cursor.parents[id].Valid() {
return false
}
if cursor.seek(id, 0, false) {
return true
}
}
return false
}
func (cursor *FlatCursor) seekDescendant(object atspi.ObjectID) bool {
for li, line := range cursor.lines {
for zi, zone := range line.zones {
+46 -10
View File
@@ -28,10 +28,22 @@ import (
)
var (
ErrNoReviewContent = errors.New("no visible content for flat review")
ErrLimitedReview = errors.New("visible text exceeds flat review capture limits")
ErrNoReviewContent = errors.New("no visible content for flat review")
ErrLimitedReview = errors.New("visible text exceeds flat review capture limits")
ErrUnavailableReview = errors.New("visible review content is unavailable")
)
// FlatCompleteness reports known omissions, rather than treating a partial
// layout as a complete representation of the window.
type FlatCompleteness struct {
OmittedObjects int
UnavailableSubtrees int
}
func (status FlatCompleteness) Incomplete() bool {
return status.OmittedObjects != 0 || status.UnavailableSubtrees != 0
}
const (
maxFlatZones = 16384
maxFlatTextBytes = 1 << 20
@@ -56,12 +68,27 @@ type flatLine struct{ zones []flatZone }
// vertical extents and centres within five pixels. Compare to the first zone
// so a staircase of overlapping objects cannot merge separate text rows.
func buildFlatLines(ctx context.Context, nodes []atspi.TreeNode) ([]flatLine, error) {
lines, _, err := build_flat_layout(ctx, nodes)
return lines, err
}
func build_flat_layout(ctx context.Context, nodes []atspi.TreeNode) ([]flatLine, FlatCompleteness, error) {
var status FlatCompleteness
parents, nodeClips, hidden := flat_node_visibility(nodes)
zones, err := collect_flat_zones(ctx, nodes, parents, nodeClips, hidden)
zones, err := collect_flat_zones(ctx, nodes, parents, nodeClips, hidden, &status)
if err != nil {
return nil, err
return nil, status, err
}
return arrange_flat_lines(ctx, nodes, zones)
if len(zones) == 0 && status.Incomplete() {
for _, node := range nodes {
if node.Text.Limited && !hidden[node.Snapshot.ID()] {
return nil, status, ErrLimitedReview
}
}
return nil, status, ErrUnavailableReview
}
lines, err := arrange_flat_lines(ctx, nodes, zones)
return lines, status, err
}
func sameFlatLine(a, b flatZone) bool {
@@ -119,7 +146,7 @@ func flat_node_visibility(nodes []atspi.TreeNode) (map[atspi.ObjectID]atspi.Snap
// collect_flat_zones walks children before parents to avoid repeating container
// names and text. Password contents and partly clipped native runs stay excluded.
func collect_flat_zones(ctx context.Context, nodes []atspi.TreeNode, parents map[atspi.ObjectID]atspi.Snapshot, nodeClips map[atspi.ObjectID]atspi.Rectangle, hidden map[atspi.ObjectID]bool) ([]flatZone, error) {
func collect_flat_zones(ctx context.Context, nodes []atspi.TreeNode, parents map[atspi.ObjectID]atspi.Snapshot, nodeClips map[atspi.ObjectID]atspi.Rectangle, hidden map[atspi.ObjectID]bool, status *FlatCompleteness) ([]flatZone, error) {
var zones []flatZone
textBytes := 0
hasContent := make(map[atspi.ObjectID]bool, len(nodes))
@@ -135,11 +162,20 @@ func collect_flat_zones(ctx context.Context, nodes []atspi.TreeNode, parents map
if hasContent[id] {
hasContent[parent.ID()] = true
}
if !n.Geometry.BoundsKnown || hidden[id] {
if hidden[id] {
continue
}
if n.Text.Limited {
return nil, ErrLimitedReview
if n.ChildrenUnavailable {
status.UnavailableSubtrees++
hasContent[parent.ID()] = true
}
if !n.Geometry.BoundsKnown {
continue
}
if n.Text.Limited || n.Text.Issue != "" {
status.OmittedObjects++
hasContent[parent.ID()] = true
continue
}
base := flatZone{object: id, parent: parent.ID(), role: n.Snapshot.Role, parentRole: parent.Role, bounds: nodeClips[id]}
before := len(zones)
@@ -198,7 +234,7 @@ func collect_flat_zones(ctx context.Context, nodes []atspi.TreeNode, parents map
zones = append(zones, z)
}
}
if len(zones) == before && !hasContent[id] {
if len(zones) == before && !hasContent[id] && !n.ChildrenUnavailable {
// Never derive review text from focus-speech Text, ValueText, or
// password contents. Names are synthetic and have no text offset.
base.text = n.Snapshot.Name
+125
View File
@@ -0,0 +1,125 @@
// Copyright (c) 2026 Storm Dragon
// Authors: Storm Dragon, Angus Kola
// SPDX-License-Identifier: GPL-3.0-or-later
package review
import (
"context"
"testing"
"time"
"git.stormux.org/storm/cthulhu/internal/atspi"
)
func TestFlatReviewRetainsHealthyContentAndReportsOmissions(t *testing.T) {
for _, issue := range []atspi.VisibleText{
{Limited: true, Issue: atspi.TextCharacterLimit},
{Issue: atspi.TextProviderFailure},
} {
bad := flatNode("/bad", "", 0, 30)
bad.Snapshot.Name = "Unavailable object name must not replace its text"
bad.Text = issue
good := flatNode("/good", "Readable content", 0, 0)
cursor := &FlatCursor{}
if err := cursor.Refresh(context.Background(), flatTree(t, good, bad), objectID("/good")); err != nil {
t.Fatal(err)
}
position, _, err := cursor.Read(Line, 0)
if err != nil || position.Text != "Readable content" || cursor.completeness.OmittedObjects != 1 {
t.Fatalf("partial review = %+v, %+v, %v", position, cursor.completeness, err)
}
if _, moved, err := cursor.Read(Line, 1); err != nil || moved {
t.Fatalf("omitted object's name became content: moved=%v err=%v", moved, err)
}
}
}
func TestFlatReviewReportsUnavailableChildrenWithoutDroppingParentText(t *testing.T) {
parent := flatNode("/parent", "Parent text", 0, 0)
parent.ChildrenUnavailable = true
cursor := &FlatCursor{}
if err := cursor.Refresh(context.Background(), flatTree(t, parent), parent.Snapshot.ID()); err != nil {
t.Fatal(err)
}
position, _, err := cursor.Read(Line, 0)
if err != nil || position.Text != "Parent text" || cursor.completeness.UnavailableSubtrees != 1 {
t.Fatalf("partial subtree = %+v, %+v, %v", position, cursor.completeness, err)
}
}
func TestFlatReviewReportsUnavailableChildrenWithoutComponentBounds(t *testing.T) {
parent := node("/parent", "")
parent.Geometry = atspi.Geometry{VisibilityKnown: true, Visible: true, Showing: true}
parent.ChildrenUnavailable = true
good := flatNode("/good", "Readable", 0, 0)
cursor := &FlatCursor{}
if err := cursor.Refresh(context.Background(), flatTree(t, parent, good), good.Snapshot.ID()); err != nil {
t.Fatal(err)
}
if cursor.completeness.UnavailableSubtrees != 1 {
t.Fatalf("missing component bounds hid a failed subtree: %+v", cursor.completeness)
}
}
func TestFlatReviewAnchorsPrunedFocusAtDisplayedParent(t *testing.T) {
before := flatNode("/before", "Unrelated first line", 0, 0)
parent := flatNode("/parent", "Focused parent text", 0, 30)
leaf := node("/leaf", "")
parent.Children = []atspi.ObjectID{leaf.Snapshot.ID()}
root := flatNode("/window", "", 0, 0)
root.Text = atspi.VisibleText{}
root.Geometry.Bounds.Height = 100
root.Snapshot.Role = "frame"
root.Children = []atspi.ObjectID{before.Snapshot.ID(), parent.Snapshot.ID()}
tree, err := atspi.NewAccessibleTree(root.Snapshot.ID(), []atspi.TreeNode{root, before, parent, leaf})
if err != nil {
t.Fatal(err)
}
cursor := &FlatCursor{}
if err := cursor.Refresh(context.Background(), tree, leaf.Snapshot.ID()); err != nil {
t.Fatal(err)
}
position, _, err := cursor.Read(Line, 0)
if err != nil || position.Object != parent.Snapshot.ID() || position.Text != "Focused parent text" {
t.Fatalf("pruned focus anchor = %+v, %v", position, err)
}
}
func TestFlatWorkerWarnsOnceAndClearsRecoveredCompleteness(t *testing.T) {
capturer := &fakeCapturer{calls: make(chan captureCall, 1)}
worker, cancel := startWorker(t, capturer, time.Second)
defer cancel()
defer worker.Stop(context.Background())
good := flatNode("/good", "Readable", 0, 0)
bad := flatNode("/bad", "Unavailable", 0, 30)
bad.Text = atspi.VisibleText{Limited: true}
tree := flatTree(t, good, bad)
worker.SyncFocus(good.Snapshot.ID())
worker.ReadFlat(Line, 0)
call := <-capturer.calls
call.result <- captureResult{tree: tree}
for index := 0; index < 2; index++ {
if index != 0 {
worker.ReadFlat(Character, 0)
}
result := waitResult(t, worker)
if result.Err != nil || !result.FlatCompleteness.Incomplete() || result.FlatWarning != (index == 0) {
t.Fatalf("partial result %d: %+v", index, result)
}
}
bad = flatNode("/bad", "Recovered", 0, 30)
fresh := flatTree(t, good, bad)
snapshot, err := fresh.ReviewWindow(context.Background(), good.Snapshot.ID())
if err != nil {
t.Fatal(err)
}
if err := tree.Replace(objectID("/window"), snapshot.Nodes); err != nil {
t.Fatal(err)
}
worker.ReadFlat(Line, 1)
result := waitResult(t, worker)
if result.Err != nil || result.FlatCompleteness.Incomplete() || result.FlatWarning || result.FlatPosition.Text != "Recovered" {
t.Fatalf("recovery result = %+v", result)
}
}
+8 -1
View File
@@ -121,12 +121,19 @@ func (worker *Worker) readFlat(ctx context.Context, cursor *FlatCursor, tree *at
if origin, offset := worker.flatStart(); origin.Valid() {
if !cursor.seek(origin, offset, true) {
if !cursor.seek(origin, 0, false) {
cursor.seekDescendant(origin)
if !cursor.seekDescendant(origin) {
cursor.seek_ancestor(origin)
}
}
}
}
cursor.started = true
}
result.FlatPosition, result.Moved, result.Err = cursor.Read(unit, step)
result.FlatCompleteness = cursor.completeness
if result.Err == nil && cursor.completeness.Incomplete() && !cursor.warnedIncomplete {
result.FlatWarning = true
cursor.warnedIncomplete = true
}
return result, tree
}
+5 -2
View File
@@ -280,7 +280,7 @@ func (worker *Worker) run(ctx context.Context, done chan<- struct{}) {
request.interest.result <- interestReply{interest, err}
continue
}
if (request.brailleRoute != nil || request.nativeCaret || request.readAll != nil || request.line || request.listSelection != nil || request.target != 0 || request.caretCommand != 0 || request.activate || request.fragment != "" || request.unit != 0) && request.documentEpoch != documentEpoch {
if (request.currentLine || request.brailleRoute != nil || request.nativeCaret || request.readAll != nil || request.line || request.listSelection != nil || request.target != 0 || request.caretCommand != 0 || request.activate || request.fragment != "" || request.unit != 0) && request.documentEpoch != documentEpoch {
if request.brailleRoute != nil {
request.brailleRoute.done <- atspi.ErrStaleObject
}
@@ -291,7 +291,10 @@ func (worker *Worker) run(ctx context.Context, done chan<- struct{}) {
continue
}
var result Result
if request.listSelection != nil {
if request.currentLine {
result = worker.read_current_line(ctx, document, request.currentLineBrowse)
result.FlatSpeech = request.flatSpeech
} else if request.listSelection != nil {
result, document = worker.selectDocumentList(ctx, document, request.listSelection)
} else if request.brailleRoute != nil {
result = worker.route_document_braille(ctx, document, request)
+6
View File
@@ -52,6 +52,8 @@ func (worker *Worker) nativeCapturerFor(id atspi.ObjectID) TreeCapturer {
// Result describes one requested navigation step. A boundary has Moved false
// and no error; Position remains the current object in that case.
type Result struct {
CurrentLine bool
CurrentLineBrowse bool
ReadAllContext context.Context
ReadAllLines []atspi.DocumentLine
ReadAllProgress func(int)
@@ -96,6 +98,8 @@ type Result struct {
FlatStep int
FlatPosition FlatPosition
FlatFocus atspi.ObjectID
FlatCompleteness FlatCompleteness
FlatWarning bool
DocumentLine bool
DocumentLineRequest uint64
DocumentLineData atspi.DocumentLine
@@ -108,6 +112,8 @@ type Result struct {
// activation, fragment, flat unit, then object direction. Flags are deliberately
// retained; epoch and timeout retirement depend on their distinct combinations.
type request struct {
currentLine bool
currentLineBrowse bool
documentList *documentListRequest
listSelection *documentListSelection
brailleRoute *documentBrailleRequest
+2 -2
View File
@@ -38,10 +38,10 @@ self-voice-dbus-speak,src/cthulhu/dbus_service.py,remote,Speak external plain te
self-voice-dbus-braille,src/cthulhu/dbus_service.py,remote,Display an external persistent braille message over the session D-Bus service,Storm Dragon,internal/remote internal/app,in-progress,go test -tags brlapi ./internal/remote ./internal/app -count=1,Focused tests confirm connected-worker queueing and rejection when disconnected.,,,Pending physical BrailleMessage delivery and persistent-display acceptance,BrailleMessage queues a persistent frame only after the BrlAPI helper reports ready. It rejects empty and over-10-KiB text; physical display and Wine client acceptance remain open.
self-voice-dbus-cancel,src/cthulhu/dbus_service.py,remote,Cancel current speech at an external application's request over the session D-Bus service,Storm Dragon,internal/remote internal/app,in-progress,go test ./internal/remote ./internal/app -count=1; CTHULHU_NATIVE_DBUS_TEST=1 go test ./internal/remote -run '^TestNativeSessionBusSelfVoice$' -count=1,Private session-bus proof passes same-user dispatch introspection conflict shutdown release and reacquisition.,,Pending CancelSpeech interruption and subsequent-delivery listening acceptance,,CancelSpeech uses the asynchronous owned speech-worker interruption path and returns false when speech is unavailable. Like Python it does not cancel sound output.
nvda-controller-ssml-rejection,src/cthulhu/dbus_service.py,remote,Reject invalid or unsupported NVDA Controller 2 SSML without claiming mark or completion support,Storm Dragon,internal/remote,implemented,go test ./internal/remote -count=1; CTHULHU_NATIVE_DBUS_TEST=1 go test ./internal/remote -run '^TestNativeSessionBusSelfVoice$' -count=1,Private session-bus proof passes the real SpeakSsml signature and unsupported result.,,,,Malformed XML or a non-speak root returns Windows ERROR_INVALID_PARAMETER 87. Valid speak-root SSML returns ERROR_NOT_SUPPORTED 50 matching Python because mark callbacks and completion semantics are unavailable.
flat-review-keys,src/cthulhu/flat_review_presenter.py,review,Navigate and present content with desktop and laptop review keys,Storm Dragon,internal/input internal/review internal/app internal/platform/xorg,in-progress,DISPLAY=:0 go test -race ./internal/atspi ./internal/review ./internal/app ./internal/input ./internal/platform/xorg ./internal/presentation -count=1; DISPLAY=:0 python3 tests/native_atspi.py; DISPLAY=:0 python3 tests/native_x11.py,Private Xvfb tests pass all nine desktop and laptop sequential review commands; both laptop release orders; NumLock numeric keypad typing; laptop keypad pass-through; existing command and shutdown checks.,,Maintainer accepted all currently implemented terminal flat-review behavior on physical Xorg on 2026-09-23; broader application listening remains pending,Pending braille reading gate,Basic previous/current/next line word and code-point character commands are wired to owned window layout and the existing review worker. Window boundaries stop silently as in Python. Capture limits are reported explicitly. Native GTK text navigation and edit recovery pass. Viewport seeking mode toggling same-object caret following after leaving review state/value zones advanced bindings mouse review and full application parity remain pending. See docs/testing/flat-review.md. Parity review corrections and remaining physical gates: docs/testing/parity-review-follow-up.md. Numpad 2 supports phonetic double press; Numpad 5 and 8 support literal and phonetic spelling on second and third presses within 500 ms. Shared desktop/laptop routing plus private X11 consumption and OCR cursor tests pass; live spelling listening remains pending. See docs/testing/phonetic-review.md.
flat-review-keys,src/cthulhu/flat_review_presenter.py,review,Navigate and present content with desktop and laptop review keys,Storm Dragon,internal/input internal/review internal/app internal/platform/xorg,in-progress,DISPLAY=:0 go test -race ./internal/atspi ./internal/review ./internal/app ./internal/input ./internal/platform/xorg ./internal/presentation -count=1; DISPLAY=:0 python3 tests/native_atspi.py; DISPLAY=:0 python3 tests/native_x11.py,Private Xvfb tests pass all nine desktop and laptop sequential review commands; both laptop release orders; NumLock numeric keypad typing; laptop keypad pass-through; existing command and shutdown checks.,,Maintainer accepted all currently implemented terminal flat-review behavior on physical Xorg on 2026-09-23; broader application listening remains pending,Pending braille reading gate,Basic previous/current/next line word and code-point character commands are wired to owned window layout and the existing review worker. Sequential line boundaries announce top or bottom of screen. Capture limits are reported explicitly. Native GTK text navigation and edit recovery pass. Mode toggling same-object caret following after leaving review state/value zones advanced bindings mouse review and full application parity remain pending. See docs/testing/flat-review-robustness.md for the current capture comparison and clipped-text regression. Parity review corrections and remaining physical gates: docs/testing/parity-review-follow-up.md. Numpad 2 supports phonetic double press; Numpad 5 and 8 support literal and phonetic spelling on second and third presses within 500 ms. Shared desktop/laptop routing plus private X11 consumption and OCR cursor tests pass; live spelling listening remains pending. See docs/testing/phonetic-review.md.
preferences-basic-speech,src/cthulhu/cthulhu_gui_prefs.py,settings,Open accessible basic speech preferences and apply or save settings,Storm Dragon,internal/preferences internal/app internal/config,in-progress,go test -race ./... -count=1; python3 tests/native_preferences.py; python3 tests/native_x11.py,Private Xvfb GTK and command checks pass; maintainer accepted save shutdown restart and restored settings on physical Xorg on 2026-09-13; broader desktop traversal pending,,Pending human listening for immediate rate pitch and volume previews plus punctuation and independent capital cues,,Apply and OK update the active schema-v2 profile while preserving other profiles and application mappings; Apply stays open and OK closes. Default-voice rate pitch volume and server-default toggles preview immediately without committing settings; Cancel restores the last applied voice configuration. Independent capital pitch beep and cap checkboxes replace the capitalization style dropdown; default pitch is enabled and all combinations apply to custom echo and spelling. Private GTK traversal and Apply/reload plus SSIP and persistence checks pass; live listening and Braille remain pending. Explicitly selected legacy files remain session-only and cannot be overwritten. Advanced preferences and importer remain pending. Phonetic character hints is an unchecked Speech Reading option; plain Left/Right character navigation can append a cancellable 300 ms phonetic hint. Private GTK default toggle Apply/reload and cancellation checks pass; physical listening remains pending. See docs/testing/phonetic-review.md.
review-component-geometry,src/cthulhu/flat_review.py,review,Capture component screen bounds and visibility for flat-review clipping,Storm Dragon,internal/atspi,implemented,DISPLAY=:0 go test ./internal/atspi -count=1; DISPLAY=:0 python3 tests/native_atspi.py,Private Xvfb D-Bus GtkBox tests verify screen bounds resize invalidation changed dimensions hide invalidation and recapture. Physical acceptance deferred by maintainer on 2026-09-14.,,,,Geometry queries run only during owned-tree capture. Bounds and visibility events invalidate geometry without removing identities. Negative coordinates overflow-safe clipping malformed geometry and concurrent invalidation have unit tests. Component bounds do not establish occlusion text ranges zones reading order or flat-review key support.
review-visible-text,src/cthulhu/flat_review.py,review,Capture bounded visible text with character offsets and screen clipping,Storm Dragon,internal/atspi,in-progress,DISPLAY=:0 go test -race ./internal/atspi ./internal/review ./internal/app ./internal/parity -count=1; DISPLAY=:0 python3 tests/native_atspi.py,Private ListBox and GtkBox capture edit clipping hidden-text and password-exclusion checks pass. Physical acceptance deferred.,,Pending navigation and listening gate,,Exact Unicode offsets and owned ranges now include captured caret offsets. Text caret and geometry events make cached text stale without removing identities. Maximum 4096 characters and 256 lines or ranges per object and 1 MiB per tree. Oversized objects remain limited with no partial text. Basic zones and review commands now consume this data; large-document viewport seeking and physical listening remain pending.
review-visible-text,src/cthulhu/flat_review.py,review,Capture bounded visible text with character offsets and screen clipping,Storm Dragon,internal/atspi,in-progress,DISPLAY=:0 go test -race ./internal/atspi ./internal/review ./internal/app ./internal/parity -count=1; DISPLAY=:0 python3 tests/native_atspi.py,Private ListBox and GtkBox capture edit clipping hidden-text and password-exclusion checks pass. Physical acceptance deferred.,,Pending navigation and listening gate,,Exact Unicode offsets and owned ranges include captured caret offsets. Text caret and geometry events invalidate cached text without retiring identities. Capture budgets are 65536 characters and 256 visible spans per object and 1 MiB per tree. Bounded line-rectangle discovery supports large objects without character rectangles; oversized lines use viewport glyph locations and bounded text spans. Zero-area glyphs no longer discard other visible text. Known unavailable objects and child subtrees leave healthy content reviewable with an explicit speech and Braille warning. Root failures and aggregate layout budgets remain fatal. Private GTK fixtures and worker tests pass; physical speech and Braille acceptance remain pending. See docs/testing/flat-review-robustness.md.
date-time-commands,src/cthulhu/date_and_time_presenter.py,command,Speak local time with Cthulhu+T and date with a double tap using saved formats,Storm Dragon,internal/input internal/app internal/preferences internal/config,in-progress,go test ./... -count=1; focused race checks; native keyboard double-tap checks,Private Xvfb press-time and both-release-order checks passed; physical Xorg pending,,Pending human listening,Pending braille,All 7 Python time and 16 Python date format choices are wired including locale seconds word numeric full and abbreviated variants; see docs/testing/time-date.md.
notifications,src/cthulhu/notification_presenter.py;src/cthulhu/scripts/apps/notification-daemon/script.py;src/cthulhu/scripts/apps/notify-osd/script.py;src/cthulhu/mako_notification_monitor.py,notification,Present provider notifications save history expose mako live actions speak the last notification and open the notification-list GUI,Storm Dragon,internal/notification internal/app internal/notificationui internal/atspi internal/input,in-progress,NO_COLOR=1 GOCACHE=/tmp/cthulhu-build-cache go test ./... -count=1; NO_COLOR=1 python3 tests/native_notifications.py,,,Automated speech-queue history list-state and list-action assertions passed; headless Xvfb/AT-SPI GUI proof passed; human listening and physical Xorg use not verified,Braille not implemented,Go registers window-create and accessible-name events maps notification-daemon mate and xfce4 providers handles notify-osd gauge and name events keeps a 55-entry history and adds Cthulhu+N plus Cthulhu+Ctrl+N. The modal accessible list opens in a GTK helper process with sortable rows copy dismiss clear close dynamic actions and mako live refresh. Live provider Xorg acceptance remains pending. Parity review corrections and remaining physical gates: docs/testing/parity-review-follow-up.md.
ordinary-key-speech-interrupt,src/cthulhu/input_event.py,input,Interrupt speech on an ordinary key press even if focus does not move,,internal/input internal/app,implemented,go test ./internal/input ./internal/app -run TestProcessInputEventsInterruptsSpeechOnKeyPress -count=1,,,,,Python KeyboardEvent._present interrupts on press outside game mode. Go now interrupts on every observed press; physical FastSMRW table-boundary listening remains pending.
1 id python_source subsystem behavior owner go_destination status automated_evidence x11_evidence wayland_evidence speech_evidence braille_evidence notes
38 self-voice-dbus-braille src/cthulhu/dbus_service.py remote Display an external persistent braille message over the session D-Bus service Storm Dragon internal/remote internal/app in-progress go test -tags brlapi ./internal/remote ./internal/app -count=1 Focused tests confirm connected-worker queueing and rejection when disconnected. Pending physical BrailleMessage delivery and persistent-display acceptance BrailleMessage queues a persistent frame only after the BrlAPI helper reports ready. It rejects empty and over-10-KiB text; physical display and Wine client acceptance remain open.
39 self-voice-dbus-cancel src/cthulhu/dbus_service.py remote Cancel current speech at an external application's request over the session D-Bus service Storm Dragon internal/remote internal/app in-progress go test ./internal/remote ./internal/app -count=1; CTHULHU_NATIVE_DBUS_TEST=1 go test ./internal/remote -run '^TestNativeSessionBusSelfVoice$' -count=1 Private session-bus proof passes same-user dispatch introspection conflict shutdown release and reacquisition. Pending CancelSpeech interruption and subsequent-delivery listening acceptance CancelSpeech uses the asynchronous owned speech-worker interruption path and returns false when speech is unavailable. Like Python it does not cancel sound output.
40 nvda-controller-ssml-rejection src/cthulhu/dbus_service.py remote Reject invalid or unsupported NVDA Controller 2 SSML without claiming mark or completion support Storm Dragon internal/remote implemented go test ./internal/remote -count=1; CTHULHU_NATIVE_DBUS_TEST=1 go test ./internal/remote -run '^TestNativeSessionBusSelfVoice$' -count=1 Private session-bus proof passes the real SpeakSsml signature and unsupported result. Malformed XML or a non-speak root returns Windows ERROR_INVALID_PARAMETER 87. Valid speak-root SSML returns ERROR_NOT_SUPPORTED 50 matching Python because mark callbacks and completion semantics are unavailable.
41 flat-review-keys src/cthulhu/flat_review_presenter.py review Navigate and present content with desktop and laptop review keys Storm Dragon internal/input internal/review internal/app internal/platform/xorg in-progress DISPLAY=:0 go test -race ./internal/atspi ./internal/review ./internal/app ./internal/input ./internal/platform/xorg ./internal/presentation -count=1; DISPLAY=:0 python3 tests/native_atspi.py; DISPLAY=:0 python3 tests/native_x11.py Private Xvfb tests pass all nine desktop and laptop sequential review commands; both laptop release orders; NumLock numeric keypad typing; laptop keypad pass-through; existing command and shutdown checks. Maintainer accepted all currently implemented terminal flat-review behavior on physical Xorg on 2026-09-23; broader application listening remains pending Pending braille reading gate Basic previous/current/next line word and code-point character commands are wired to owned window layout and the existing review worker. Window boundaries stop silently as in Python. Capture limits are reported explicitly. Native GTK text navigation and edit recovery pass. Viewport seeking mode toggling same-object caret following after leaving review state/value zones advanced bindings mouse review and full application parity remain pending. See docs/testing/flat-review.md. Parity review corrections and remaining physical gates: docs/testing/parity-review-follow-up.md. Numpad 2 supports phonetic double press; Numpad 5 and 8 support literal and phonetic spelling on second and third presses within 500 ms. Shared desktop/laptop routing plus private X11 consumption and OCR cursor tests pass; live spelling listening remains pending. See docs/testing/phonetic-review.md. Basic previous/current/next line word and code-point character commands are wired to owned window layout and the existing review worker. Sequential line boundaries announce top or bottom of screen. Capture limits are reported explicitly. Native GTK text navigation and edit recovery pass. Mode toggling same-object caret following after leaving review state/value zones advanced bindings mouse review and full application parity remain pending. See docs/testing/flat-review-robustness.md for the current capture comparison and clipped-text regression. Parity review corrections and remaining physical gates: docs/testing/parity-review-follow-up.md. Numpad 2 supports phonetic double press; Numpad 5 and 8 support literal and phonetic spelling on second and third presses within 500 ms. Shared desktop/laptop routing plus private X11 consumption and OCR cursor tests pass; live spelling listening remains pending. See docs/testing/phonetic-review.md.
42 preferences-basic-speech src/cthulhu/cthulhu_gui_prefs.py settings Open accessible basic speech preferences and apply or save settings Storm Dragon internal/preferences internal/app internal/config in-progress go test -race ./... -count=1; python3 tests/native_preferences.py; python3 tests/native_x11.py Private Xvfb GTK and command checks pass; maintainer accepted save shutdown restart and restored settings on physical Xorg on 2026-09-13; broader desktop traversal pending Pending human listening for immediate rate pitch and volume previews plus punctuation and independent capital cues Apply and OK update the active schema-v2 profile while preserving other profiles and application mappings; Apply stays open and OK closes. Default-voice rate pitch volume and server-default toggles preview immediately without committing settings; Cancel restores the last applied voice configuration. Independent capital pitch beep and cap checkboxes replace the capitalization style dropdown; default pitch is enabled and all combinations apply to custom echo and spelling. Private GTK traversal and Apply/reload plus SSIP and persistence checks pass; live listening and Braille remain pending. Explicitly selected legacy files remain session-only and cannot be overwritten. Advanced preferences and importer remain pending. Phonetic character hints is an unchecked Speech Reading option; plain Left/Right character navigation can append a cancellable 300 ms phonetic hint. Private GTK default toggle Apply/reload and cancellation checks pass; physical listening remains pending. See docs/testing/phonetic-review.md.
43 review-component-geometry src/cthulhu/flat_review.py review Capture component screen bounds and visibility for flat-review clipping Storm Dragon internal/atspi implemented DISPLAY=:0 go test ./internal/atspi -count=1; DISPLAY=:0 python3 tests/native_atspi.py Private Xvfb D-Bus GtkBox tests verify screen bounds resize invalidation changed dimensions hide invalidation and recapture. Physical acceptance deferred by maintainer on 2026-09-14. Geometry queries run only during owned-tree capture. Bounds and visibility events invalidate geometry without removing identities. Negative coordinates overflow-safe clipping malformed geometry and concurrent invalidation have unit tests. Component bounds do not establish occlusion text ranges zones reading order or flat-review key support.
44 review-visible-text src/cthulhu/flat_review.py review Capture bounded visible text with character offsets and screen clipping Storm Dragon internal/atspi in-progress DISPLAY=:0 go test -race ./internal/atspi ./internal/review ./internal/app ./internal/parity -count=1; DISPLAY=:0 python3 tests/native_atspi.py Private ListBox and GtkBox capture edit clipping hidden-text and password-exclusion checks pass. Physical acceptance deferred. Pending navigation and listening gate Exact Unicode offsets and owned ranges now include captured caret offsets. Text caret and geometry events make cached text stale without removing identities. Maximum 4096 characters and 256 lines or ranges per object and 1 MiB per tree. Oversized objects remain limited with no partial text. Basic zones and review commands now consume this data; large-document viewport seeking and physical listening remain pending. Exact Unicode offsets and owned ranges include captured caret offsets. Text caret and geometry events invalidate cached text without retiring identities. Capture budgets are 65536 characters and 256 visible spans per object and 1 MiB per tree. Bounded line-rectangle discovery supports large objects without character rectangles; oversized lines use viewport glyph locations and bounded text spans. Zero-area glyphs no longer discard other visible text. Known unavailable objects and child subtrees leave healthy content reviewable with an explicit speech and Braille warning. Root failures and aggregate layout budgets remain fatal. Private GTK fixtures and worker tests pass; physical speech and Braille acceptance remain pending. See docs/testing/flat-review-robustness.md.
45 date-time-commands src/cthulhu/date_and_time_presenter.py command Speak local time with Cthulhu+T and date with a double tap using saved formats Storm Dragon internal/input internal/app internal/preferences internal/config in-progress go test ./... -count=1; focused race checks; native keyboard double-tap checks Private Xvfb press-time and both-release-order checks passed; physical Xorg pending Pending human listening Pending braille All 7 Python time and 16 Python date format choices are wired including locale seconds word numeric full and abbreviated variants; see docs/testing/time-date.md.
46 notifications src/cthulhu/notification_presenter.py;src/cthulhu/scripts/apps/notification-daemon/script.py;src/cthulhu/scripts/apps/notify-osd/script.py;src/cthulhu/mako_notification_monitor.py notification Present provider notifications save history expose mako live actions speak the last notification and open the notification-list GUI Storm Dragon internal/notification internal/app internal/notificationui internal/atspi internal/input in-progress NO_COLOR=1 GOCACHE=/tmp/cthulhu-build-cache go test ./... -count=1; NO_COLOR=1 python3 tests/native_notifications.py Automated speech-queue history list-state and list-action assertions passed; headless Xvfb/AT-SPI GUI proof passed; human listening and physical Xorg use not verified Braille not implemented Go registers window-create and accessible-name events maps notification-daemon mate and xfce4 providers handles notify-osd gauge and name events keeps a 55-entry history and adds Cthulhu+N plus Cthulhu+Ctrl+N. The modal accessible list opens in a GTK helper process with sortable rows copy dismiss clear close dynamic actions and mako live refresh. Live provider Xorg acceptance remains pending. Parity review corrections and remaining physical gates: docs/testing/parity-review-follow-up.md.
47 ordinary-key-speech-interrupt src/cthulhu/input_event.py input Interrupt speech on an ordinary key press even if focus does not move internal/input internal/app implemented go test ./internal/input ./internal/app -run TestProcessInputEventsInterruptsSpeechOnKeyPress -count=1 Python KeyboardEvent._present interrupts on press outside game mode. Go now interrupts on every observed press; physical FastSMRW table-boundary listening remains pending.
+69
View File
@@ -0,0 +1,69 @@
#!/usr/bin/env python3
"""Private GTK controls for current-line AT-SPI checks."""
import json
import sys
import gi
gi.require_version("Gtk", "3.0")
from gi.repository import GLib, Gtk
window = Gtk.Window(title="Current line fixture")
box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL)
window.add(box)
entry = Gtk.Entry()
entry.get_accessible().set_name("Entry")
textView = Gtk.TextView()
textView.get_accessible().set_name("Text")
password = Gtk.Entry()
password.set_visibility(False)
password.get_accessible().set_name("Password")
button = Gtk.Button(label="Example button")
controls = {"entry": entry, "text": textView, "password": password, "button": button}
for control in controls.values():
box.pack_start(control, True, True, 0)
window.show_all()
def command_ready(source, condition):
line = sys.stdin.readline()
if not line:
Gtk.main_quit()
return False
command = json.loads(line)
control = controls[command["control"]]
if command["action"] == "focus":
if isinstance(control, Gtk.Entry):
control.set_text(command["text"])
control.set_position(command["offset"])
elif isinstance(control, Gtk.TextView):
buffer = control.get_buffer()
buffer.set_text(command["text"])
buffer.place_cursor(buffer.get_iter_at_offset(command["offset"]))
window.present()
control.grab_focus()
def finish_focus():
if isinstance(control, Gtk.Entry):
control.set_position(command["offset"])
elif isinstance(control, Gtk.TextView):
buffer = control.get_buffer()
buffer.place_cursor(buffer.get_iter_at_offset(command["offset"]))
print("ready", flush=True)
return False
GLib.timeout_add(50, finish_focus)
return True
elif command["action"] == "caret":
if isinstance(control, Gtk.Entry):
offset = control.get_position()
else:
buffer = control.get_buffer()
offset = buffer.get_iter_at_mark(buffer.get_insert()).get_offset()
print(offset, flush=True)
print("ready", flush=True)
return True
GLib.io_add_watch(sys.stdin, GLib.IO_IN | GLib.IO_HUP, command_ready)
print("ready", flush=True)
Gtk.main()
+55 -3
View File
@@ -10,9 +10,22 @@ static gboolean armed;
static unsigned childQueries;
static gboolean slowRanges;
static gint largestRange;
static gboolean lineGeometryOnly;
static gboolean missingGlyph;
static GtkWidget *reviewScroller;
static AtkObject *brokenSubtree;
static gint (*originalChildCount)(AtkObject *);
static void (*originalRangeExtents)(AtkText *, gint, gint, AtkCoordType, AtkTextRectangle *);
static void range_extents(AtkText *text, gint start, gint end, AtkCoordType coordinates, AtkTextRectangle *rectangle) {
if (missingGlyph && start == 5 && end == 6) {
memset(rectangle, 0, sizeof(*rectangle));
return;
}
if (lineGeometryOnly && end - start == 1) {
memset(rectangle, 0, sizeof(*rectangle));
return;
}
if (slowRanges) {
largestRange = MAX(largestRange, end - start);
if (end - start > 128) g_usleep(1200000);
@@ -31,6 +44,11 @@ static gint count_children(AtkObject *object) {
return 0;
}
static gint subtree_child_count(AtkObject *object) {
if (object == brokenSubtree && armed) return -1;
return originalChildCount(object);
}
static gboolean command_ready(GIOChannel *channel, GIOCondition condition, gpointer label) {
if (condition & G_IO_HUP) {
gtk_main_quit();
@@ -49,6 +67,11 @@ static gboolean command_ready(GIOChannel *channel, GIOCondition condition, gpoin
puts("measuring");
} else if (g_str_has_prefix(line, "largest")) {
printf("largest %d\n", largestRange);
} else if (g_str_has_prefix(line, "scroll")) {
GtkAdjustment *adjustment = gtk_scrolled_window_get_vadjustment(GTK_SCROLLED_WINDOW(reviewScroller));
gtk_adjustment_set_value(adjustment, gtk_adjustment_get_upper(adjustment) * 0.75);
while (gtk_events_pending()) gtk_main_iteration();
puts("scrolled");
} else if (g_str_has_prefix(line, "change")) {
armed = FALSE;
gtk_label_set_text(GTK_LABEL(label), "Changed review text");
@@ -61,6 +84,13 @@ static gboolean command_ready(GIOChannel *channel, GIOCondition condition, gpoin
int main(int argc, char **argv) {
gboolean boundsMode = argc > 1 && strcmp(argv[1], "bounds") == 0;
gboolean clippedMode = argc > 1 && strcmp(argv[1], "clipped") == 0;
gboolean longMode = argc > 1 && strcmp(argv[1], "long") == 0;
gboolean linesMode = argc > 1 && strcmp(argv[1], "lines") == 0;
gboolean childrenMode = argc > 1 && strcmp(argv[1], "children") == 0;
gboolean glyphsMode = argc > 1 && strcmp(argv[1], "glyphs") == 0;
lineGeometryOnly = linesMode;
missingGlyph = glyphsMode;
gtk_init(&argc, &argv);
GtkWidget *window = gtk_window_new(GTK_WINDOW_TOPLEVEL);
gtk_window_set_title(GTK_WINDOW(window), "Review cache probe");
@@ -69,9 +99,30 @@ int main(int argc, char **argv) {
GtkWidget *label = gtk_label_new("Cached review text");
// This fixture has one label. Replace its class vfunc before the bridge
// builds the initial cache, then arm the delay explicitly over stdin.
ATK_OBJECT_GET_CLASS(gtk_widget_get_accessible(label))->get_n_children = count_children;
if (boundsMode) {
gchar *content = g_strnfill(3424, 'x');
if (!childrenMode)
ATK_OBJECT_GET_CLASS(gtk_widget_get_accessible(label))->get_n_children = count_children;
if (childrenMode) {
GtkWidget *box = gtk_box_new(GTK_ORIENTATION_VERTICAL, 0);
GtkWidget *panel = gtk_event_box_new();
brokenSubtree = gtk_widget_get_accessible(panel);
atk_object_set_name(brokenSubtree, "Broken subtree");
originalChildCount = ATK_OBJECT_GET_CLASS(brokenSubtree)->get_n_children;
ATK_OBJECT_GET_CLASS(brokenSubtree)->get_n_children = subtree_child_count;
gtk_container_add(GTK_CONTAINER(panel), gtk_label_new("Subtree text"));
gtk_container_add(GTK_CONTAINER(box), label);
gtk_container_add(GTK_CONTAINER(box), panel);
gtk_container_add(GTK_CONTAINER(window), box);
} else if (boundsMode || clippedMode || longMode || linesMode || glyphsMode) {
gchar *content = g_strnfill(linesMode ? 101000 : longMode ? 100001 : clippedMode ? 2020 : 3424, 'x');
if (longMode) content[100000] = '\n';
if (linesMode) {
for (int row = 0; row < 1000; row++) content[row * 101 + 100] = '\n';
gtk_window_set_default_size(GTK_WINDOW(window), 1000, 100);
}
if (clippedMode) {
for (int row = 0; row < 20; row++) content[row * 101 + 100] = '\n';
gtk_window_set_default_size(GTK_WINDOW(window), 300, 450);
}
gtk_label_set_text(GTK_LABEL(label), content);
g_free(content);
gtk_label_set_xalign(GTK_LABEL(label), 0);
@@ -81,6 +132,7 @@ int main(int argc, char **argv) {
if (originalRangeExtents == NULL) g_error("fixture needs native GTK range extents");
ATK_TEXT_GET_IFACE(accessible)->get_range_extents = range_extents;
GtkWidget *scroller = gtk_scrolled_window_new(NULL, NULL);
reviewScroller = scroller;
gtk_container_add(GTK_CONTAINER(scroller), label);
gtk_container_add(GTK_CONTAINER(window), scroller);
} else {