More flat review improvement.
This commit is contained in:
1 parent
50e32f24df
commit
e88de5ee66
41 files changed
+2167
-73
No files matched your search
@@ -164,6 +164,22 @@ changes or “cap”. Explicit phonetic review commands retain capital indicator
|
||||
See [phonetic review verification](docs/testing/phonetic-review.md) for checks
|
||||
and live listening steps.
|
||||
|
||||
## Read the current line
|
||||
|
||||
Press **Cthulhu+Shift+Up** in either keyboard layout to read the line at the
|
||||
application caret in native AT-SPI controls, or at the browse cursor on a webpage
|
||||
in browse mode.
|
||||
Press twice within 500 milliseconds to spell the line, or three times for
|
||||
phonetics. The command leaves the caret and flat-review position in place.
|
||||
Empty lines say “blank”; password contents are never read. Controls without
|
||||
a text interface read their accessible name. Cthulhu+Up still increases the
|
||||
selected speech setting.
|
||||
|
||||
This follows NVDA's desktop NVDA+Up current-line command, rather than replaying
|
||||
the last spoken message. Numpad 8 (laptop: Cthulhu+I) continues to read the
|
||||
independent flat-review line.
|
||||
See [current-line verification](docs/testing/current-line.md) for checks.
|
||||
|
||||
## Read the current window title
|
||||
|
||||
Press **Cthulhu+Numpad Enter** in the desktop keyboard layout, or
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
pkgbase=cthulhu-git
|
||||
pkgname=cthulhu-git
|
||||
_pkgname=cthulhu
|
||||
pkgver=2026.10.20.r61.g4643f8e
|
||||
pkgver=2026.10.20.r62.g50e32f2
|
||||
pkgrel=1
|
||||
pkgdesc="Desktop-agnostic screen reader with speech, Braille, and Tentacle support"
|
||||
url="https://git.stormux.org/storm/cthulhu"
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
# Current-line command
|
||||
|
||||
Cthulhu+Shift+Up reads the entire current line. Two presses within 500 ms spell
|
||||
it; three use phonetics. Both desktop and laptop layouts use the same binding.
|
||||
It follows NVDA's `script_reportCurrentLine` in `source/globalCommands.py`:
|
||||
native caret outside browse mode, browse cursor inside it. It does not replay
|
||||
the previous utterance or use the flat-review position.
|
||||
|
||||
Automated checks:
|
||||
|
||||
```sh
|
||||
go test ./internal/input ./internal/review ./internal/app -run 'Test(CurrentLine|CommandRuntimeRoutesFocusPresentationRequests|ReviewMultiPressSequences)' -count=1
|
||||
python3 tests/native_test_session.py env CTHULHU_NATIVE_CURRENT_LINE=1 go test ./internal/atspi -run '^TestNativeCurrentLine$' -count=1 -v
|
||||
```
|
||||
|
||||
The private GTK fixture checks Unicode, a caret in the middle of a line, the
|
||||
end of text, empty controls, blank lines, a trailing newline, password exclusion,
|
||||
non-text control labels and rejection of stale focus. It verifies native caret
|
||||
offsets after reading. Worker tests check whole browse lines and multiple inline
|
||||
fragments, preserving the browse position. Existing flat-review and Say All tests
|
||||
cover the shared line-reading and multi-press helpers.
|
||||
|
||||
Live acceptance after a normal rebuild and reload:
|
||||
|
||||
1. In a native multiline editor, enter `First line`, `Second café`, an empty line
|
||||
and `Last line`. Put the caret in the middle of the second line. Press
|
||||
Cthulhu+Shift+Up: expect the whole second line. Press Left afterward to verify
|
||||
that the caret did not move. Check the blank line and the end of the last line.
|
||||
2. Double and triple press to check spelling and phonetics. Pause longer than
|
||||
500 ms and verify the next press reads normally. Check both keyboard layouts.
|
||||
3. Move flat review to a different line. The new command should still read the
|
||||
application caret line; Numpad 8 or Cthulhu+I should retain the review position.
|
||||
4. In a browser, move into the middle of a line using browse arrows. Check the
|
||||
whole line, including inline text fragments, and verify the next arrow
|
||||
continues from the previous position. In a focused editable field, check the
|
||||
native caret instead. Try another tab or window while a report is pending.
|
||||
5. Check a FastSMRW table cell and a button. Check that password contents remain
|
||||
silent and unavailable text gives explicit failure feedback. Cthulhu+Up must
|
||||
still increase the selected speech setting. Check Say All and Numpad 8's
|
||||
spelling behavior for regressions.
|
||||
|
||||
These automated checks do not establish audible output, physical key routing or
|
||||
Braille behavior. The native implementation uses AT-SPI; Wine Access does not
|
||||
currently expose this focused-line operation.
|
||||
@@ -0,0 +1,206 @@
|
||||
# Flat-review robustness comparison
|
||||
|
||||
Review date: 2026-10-21. Cthulhu baseline: `testing` at `50e32f2`.
|
||||
Orca reference: upstream `main` at
|
||||
`828eaf2e70d7b88124b10b3a6b2daa792ab266c7`. The local Orca checkout was
|
||||
older; fetching the current reference confirmed that the three source files
|
||||
below have no changes between those revisions.
|
||||
|
||||
This is a source comparison and a set of verified capture and recovery repairs.
|
||||
It does not establish complete Orca parity or resolve every user report.
|
||||
At the initial comparison, the available local log contained one
|
||||
`accessible object not found` flat-review failure without enough evidence to
|
||||
identify its cause, and no instance of the reported text-limit failure. Older diagnostic logs describe
|
||||
provider timeouts already addressed by earlier capture work; they are not proof
|
||||
of a remaining timeout in the current build.
|
||||
|
||||
## What Orca does
|
||||
|
||||
Orca's [Flat Review documentation](https://help.gnome.org/orca/howto_flat_review.html)
|
||||
describes a spatial representation of the active window's visible contents.
|
||||
The implementation separates finding on-screen objects, finding visible lines,
|
||||
and building review zones:
|
||||
|
||||
- [`flat_review.py`](https://gitlab.gnome.org/GNOME/orca/-/blob/828eaf2e70d7b88124b10b3a6b2daa792ab266c7/src/orca/flat_review.py):
|
||||
`Context` chooses a dialog/frame or an explicit root and restricts to an active
|
||||
menu when appropriate. `_get_zones_from_text` clips to a scroll pane/document,
|
||||
gets visible lines, removes embedded-object characters, and builds text zones.
|
||||
Word and character objects are created on demand. `_find_zone_with_object`
|
||||
can use a parent or child when the exact focused object was pruned.
|
||||
- [`ax_utilities.py`](https://gitlab.gnome.org/GNOME/orca/-/blob/828eaf2e70d7b88124b10b3a6b2daa792ab266c7/src/orca/ax_utilities.py):
|
||||
`_get_on_screen_objects` checks showing/visible state and window intersection,
|
||||
treats suitable controls as leaves, enumerates visible table cells, and
|
||||
suppresses some duplicate container text. Invalid component geometry is not
|
||||
invariably treated as off-screen.
|
||||
- [`ax_utilities_text.py`](https://gitlab.gnome.org/GNOME/orca/-/blob/828eaf2e70d7b88124b10b3a6b2daa792ab266c7/src/orca/ax_utilities_text.py):
|
||||
`find_first_visible_line` searches by line rectangles, then `get_visible_lines`
|
||||
walks forward until below the viewport. Invalid line offsets have explicit
|
||||
handling. This differs from requiring usable character geometry to find a
|
||||
large object's viewport.
|
||||
|
||||
These paths do not impose Cthulhu's 256-range/65,536-character object budgets.
|
||||
That is not evidence that unlimited capture is safe: Cthulhu still needs bounded
|
||||
native calls, memory use, cancellation, and protection from defective providers.
|
||||
Orca also has its own tradeoffs: the single-line editable-text path requests all
|
||||
text, and `Context.is_stale` itself only checks top-level size changes. Presenter
|
||||
event handling supplies additional context management. Copying one function
|
||||
alone would not reproduce that behavior.
|
||||
|
||||
## Comparison and priorities
|
||||
|
||||
| Area | Current Cthulhu behavior | Assessment |
|
||||
| --- | --- | --- |
|
||||
| Scope | Captures the focused window/menu/dialog and skips hidden descendants during review-only traversal. | Already aligned in principle. Preserve foreground ownership and popup handling. |
|
||||
| Large GTK tables | Captures visible selectable-table cells and headers; keeps a selected-cell identity anchor. | Already follows Orca's viewport approach. Existing table checks remain relevant. |
|
||||
| Clipped text ranges | Previously emitted one range per clipped visible character. | Confirmed defect; repaired here. Count contiguous visible spans against the range budget. |
|
||||
| Viewport discovery | Uses character rectangles first; large objects can use a bounded binary search over line rectangles. | Private GTK coverage seeks into a 101000-character scrolled object with unavailable character rectangles. Small objects keep their existing bounded path. |
|
||||
| Long horizontal lines | Oversized viewport intervals use glyph locations and bounded text retrieval; oversized native line responses also enter that path. | A real GTK 100000-character line is captured without increasing the character budget. New text requests contain at most 128 characters. Sparse or defective point providers remain bounded and explicitly limited. |
|
||||
| Failure scope | Known unavailable text objects and failed child enumeration retain healthy content with explicit completeness counts. | A partial result announces a speech and Braille warning on entry or a changed omission count. Its line boundaries say top/bottom of available content. Root failure, cancellation, stale snapshots, invalid trees, and aggregate layout limits still fail the operation. |
|
||||
| Focus anchoring | Tries exact focus, displayed descendants, then the nearest displayed ancestor within the review root. | A targeted fixture proves pruned focus stays on its parent text rather than an unrelated first line. Preserved position still uses object generations and native offsets. |
|
||||
| Updates and cancellation | Cthulhu refreshes owned layout per command and rejects stale results; native capture and queues are bounded. | Preserve these safeguards. Snapshot refresh, provider timeout, and audible responsiveness are separate concerns. |
|
||||
| Diagnostics | Capture timings include `text_issues` counts for viewport, character, range, tree-byte, query-budget, and provider failures. Partial presentation logs omission counts. | These fields contain no accessible text or names. A wholly limited window retains its existing spoken text-limit message. |
|
||||
|
||||
The line-based discovery, long-line retrieval, explicit partial recovery, and
|
||||
parent anchoring above are implemented in the current working tree. Capture
|
||||
budgets, provider-call chunking, cancellation, foreground ownership, and stale
|
||||
result rejection remain in force. Remaining failures need an application,
|
||||
triggering command, build revision, and fresh failure/timing record before
|
||||
assigning a cause. Cold-capture latency remains separate from cached navigation.
|
||||
|
||||
## Repair and evidence
|
||||
|
||||
`internal/atspi/text.go` now accumulates adjacent visible characters with the
|
||||
same row geometry into one range. It preserves exact Unicode offsets and unions
|
||||
only their clipped rectangles. A hidden offset, embedded object, or row change
|
||||
prevents merging across that boundary. The 128-character geometry blocks and
|
||||
all existing capture budgets remain in force. No new mutable cross-command state
|
||||
is introduced; span accumulation belongs to one synchronous capture.
|
||||
|
||||
Before the change, a real GTK label with 20 horizontally clipped lines produced
|
||||
`VisibleText{Limited: true}`. After the change, the same fixture captures more
|
||||
than 256 characters in at most 21 spans. A deterministic fixture verifies 800
|
||||
Unicode characters in 20 spans, exact offsets, and clipping at both edges. Other
|
||||
regressions cover a span crossing geometry blocks, disjoint bidirectional runs,
|
||||
and a genuinely excessive number of separate visible spans. Existing tests
|
||||
cover blank rows, embedded objects, large terminals, cancellation and malformed
|
||||
provider data.
|
||||
|
||||
The successful four-row capture benchmark, three runs per version, measured
|
||||
14.6–15.1 microseconds and 180 allocations before, versus 10.4–10.7 microseconds
|
||||
and 19 allocations after. Allocated bytes fell from 19,664 to 4,048 per capture.
|
||||
This measures local layout preparation with a synthetic provider; it does not
|
||||
measure AT-SPI round trips or prove live speech latency.
|
||||
|
||||
## Automated checks
|
||||
|
||||
```sh
|
||||
go test ./... -count=1
|
||||
go test -race ./internal/atspi ./internal/review ./internal/app ./internal/presentation -count=1
|
||||
go vet ./internal/atspi ./internal/review ./internal/app ./internal/presentation
|
||||
python3 tests/native_atspi.py
|
||||
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_CLIPPED=1 CTHULHU_NATIVE_REVIEW_CACHE=1 CTHULHU_NATIVE_REVIEW_BOUNDS=1 go test ./internal/atspi -run '^TestNativeFlatReview(ClippedViewport|PreservesProviderCache|BoundsSlowLongRanges)$' -count=1 -v
|
||||
go test ./internal/atspi -run '^$' -bench '^BenchmarkCaptureVisibleTextClipped$' -benchmem -count=3
|
||||
```
|
||||
|
||||
Use a writable `GOCACHE` if required by the environment. Native harnesses create
|
||||
private Xvfb/D-Bus/AT-SPI sessions and do not replace the running reader.
|
||||
All checks above passed for this change. The existing table harness emitted
|
||||
GTK `Atk-CRITICAL` object-role warnings while its assertions passed; the cause
|
||||
of those warnings was not investigated here.
|
||||
|
||||
## Follow-up implementation and evidence
|
||||
|
||||
Large-object line discovery moves by validated complete line boundaries during
|
||||
binary search; it never scans terminal scrollback. Oversized horizontal text
|
||||
uses native offset-at-point and glyph rectangles to discover visible offsets,
|
||||
then reads text in spans of at most 128 characters. Offsets are sorted before
|
||||
capture so bidirectional runs retain exact logical identities. Point traversal
|
||||
has a 4096-query budget; it checks gaps before skipping vertically so an inset
|
||||
or shorter glyph is not missed. Ordinary clipped capture continues to use bounded
|
||||
geometry blocks. Unsupported or excessive work yields an explicit limitation,
|
||||
not guessed text. No global capture limit was increased.
|
||||
|
||||
A native provider may return a zero-area rectangle for a nonvisible glyph.
|
||||
That is distinct from a failed native request. Such glyphs now break visible
|
||||
spans instead of discarding the entire object's usable text. Unavailable whole
|
||||
line geometry remains an unavailable capture. A private GTK fixture injects a
|
||||
zero-area glyph and verifies that the surrounding visible text survives and
|
||||
the missing offset is excluded. Combining marks and joiners attached to a
|
||||
visible base retain their Unicode offsets even without independent rectangles.
|
||||
Bounded endpoint lookup also handles a huge line ending in a zero-area newline.
|
||||
|
||||
`ChildrenUnavailable` is capture-owned state copied into the owned tree and
|
||||
atomic review snapshot. Recovery is restricted to review-only child enumeration;
|
||||
root enumeration failure and cancellation remain fatal. `FlatCompleteness`
|
||||
travels with each worker result. No partial object's accessible name is silently
|
||||
substituted for its missing text. A warning is announced when entering a partial
|
||||
layout or when its omission counts change; returning to a complete layout clears
|
||||
that warning state. Ordinary invalidation triggers recapture as before; there
|
||||
is no broad automatic retry loop. The warning uses the existing system-message
|
||||
output path for both speech and Braille, while spelling retains its own request.
|
||||
|
||||
Seven private native cases cover cache reuse, bounded geometry, clipped spans,
|
||||
a 100000-character horizontal line, scrolled line-only geometry, unavailable
|
||||
child enumeration, and zero-area glyphs. Additional worker and presentation
|
||||
fixtures cover healthy-content recovery, omission counts, recovery to a complete
|
||||
layout, ancestor anchoring, spelling, Braille warning delivery, and incomplete
|
||||
boundary feedback. Run native modes in separate processes:
|
||||
|
||||
```sh
|
||||
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_VIEWPORT=1 CTHULHU_NATIVE_REVIEW_VIEWPORT_MODE=long go test ./internal/atspi -run '^TestNativeFlatReviewLargeTextViewport$' -count=1 -v
|
||||
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_VIEWPORT=1 CTHULHU_NATIVE_REVIEW_VIEWPORT_MODE=lines go test ./internal/atspi -run '^TestNativeFlatReviewLargeTextViewport$' -count=1 -v
|
||||
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_SUBTREE=1 go test ./internal/atspi -run '^TestNativeFlatReviewUnavailableSubtree$' -count=1 -v
|
||||
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_GLYPHS=1 go test ./internal/atspi -run '^TestNativeFlatReviewZeroAreaGlyph$' -count=1 -v
|
||||
```
|
||||
|
||||
A read-only probe of the open FastSMRW instance exposed zero-area glyph failures
|
||||
that previously left visible table-cell text unavailable. After the repair,
|
||||
27 worker commands across line, word, character, window-start, and window-end
|
||||
review passed with zero reported omissions. Cached commands took about 0.3-0.7
|
||||
milliseconds. Initial capture still took about eight seconds, mostly in visible
|
||||
text. The standalone probe had no retained native focus event and reported
|
||||
`root_scope=desktop` while filtering capture to FastSMRW's provider; this timing
|
||||
does not establish normal foreground reader latency. No application keys were
|
||||
injected and the running reader was not replaced. These results do not establish
|
||||
audible speech or physical Braille acceptance.
|
||||
|
||||
### Follow-up check results
|
||||
|
||||
The full Go suite, affected-package race tests, vet, command builds, parity
|
||||
ledger validation, recorded-speech checks, and all seven targeted native GTK
|
||||
cases passed. The broad `tests/native_atspi.py` harness initially failed
|
||||
`TestNativeGUIListTableSpeech` because the launcher description was repeated.
|
||||
The isolated table checks passed on both an untouched `50e32f2` snapshot and the
|
||||
changed tree, and a subsequent complete harness run passed. The initial failure
|
||||
was not explained; passing reruns do not establish that this startup/event-order
|
||||
behavior is stable. Existing GTK `Atk-CRITICAL` role warnings also remain.
|
||||
|
||||
## Human review and live acceptance
|
||||
|
||||
Review the generated diff before submission. With an authorized build/restart:
|
||||
|
||||
1. In a native editor or a window containing a wide multiline text control,
|
||||
horizontally scroll so multiple lines are clipped. Review lines, words, and
|
||||
characters forward and backward; visible text must remain available when
|
||||
the total exceeds 256 characters. Repeat after resizing and scrolling. Check
|
||||
the first and last partially visible characters and Unicode offsets.
|
||||
2. Review ordinary and large terminal screens, blank rows, and a large file in
|
||||
a terminal editor. Check Numpad 7/8/9, 4/5/6 and 1/2/3, window start/end,
|
||||
explicit top/bottom feedback, and multi-press spelling.
|
||||
3. Check a browser with horizontally clipped text and embedded links, a GTK
|
||||
table such as a timeline, a preferences notebook, and a popup menu. Hidden
|
||||
letters and embedded-object placeholders must not be spoken as ordinary text.
|
||||
Password contents must stay excluded. Review must stay in the foreground
|
||||
window and ordinary focus/caret navigation must still work afterward.
|
||||
4. Exercise a window with one unavailable text object or child subtree. Healthy
|
||||
content must remain reviewable with an explicit warning. Repeated navigation
|
||||
should not repeat the warning continuously; after the provider recovers and
|
||||
the tree is refreshed, ordinary top/bottom feedback must return.
|
||||
5. Confirm audible speech and physical Braille separately, including routing
|
||||
within a merged span. Native capture and Go test success do not establish
|
||||
these outcomes. Preserve fresh logs for any remaining failure.
|
||||
|
||||
The code change affects shared visible-text capture used in owned trees;
|
||||
terminal, browser, mouse-review, and Braille consumers can therefore be affected
|
||||
indirectly. It changes neither X11 key grabs nor foreground ownership. Wayland
|
||||
live acceptance remains separate.
|
||||
@@ -0,0 +1,51 @@
|
||||
// Copyright (c) 2026 Storm Dragon
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package app
|
||||
|
||||
import (
|
||||
"git.stormux.org/storm/cthulhu/internal/input"
|
||||
"git.stormux.org/storm/cthulhu/internal/presentation"
|
||||
"git.stormux.org/storm/cthulhu/internal/review"
|
||||
)
|
||||
|
||||
func (handler *eventReviewHandler) current_line(command input.Command) {
|
||||
if handler.focusGate.Locked() || handler.focusGate.Suspended() {
|
||||
return
|
||||
}
|
||||
handler.syncForeground()
|
||||
if handler.objectReview == nil || !handler.state.current.ID().Valid() {
|
||||
return
|
||||
}
|
||||
handler.state.pending.terminalFocus.clear()
|
||||
handler.state.pending.terminalDeletion.clear()
|
||||
style := review.FlatSpeak
|
||||
if command == input.SpellCurrentLine {
|
||||
style = review.FlatSpell
|
||||
} else if command == input.PhoneticCurrentLine {
|
||||
style = review.FlatPhonetic
|
||||
}
|
||||
browse := handler.browser != nil && handler.browser.ready && handler.browser.mode == review.BrowseMode
|
||||
if err := handler.objectReview.ReadCurrentLine(browse, style); err != nil {
|
||||
handler.logger.Warn("queue current line", "error", err)
|
||||
}
|
||||
}
|
||||
|
||||
func (handler *eventReviewHandler) present_current_line(result review.Result) {
|
||||
if !handler.objectReview.CurrentDocumentResult(result) || result.DocumentFocus != handler.objectReview.Focus() || handler.focusGate.Suspended() {
|
||||
return
|
||||
}
|
||||
if result.CurrentLineBrowse && (handler.browser == nil || !handler.browser.acceptsDocumentResult(review.Result{
|
||||
DocumentFocus: result.DocumentFocus, DocumentPosition: result.DocumentPosition, DocumentCaretCommand: review.CaretReadLine, Err: result.Err,
|
||||
})) {
|
||||
return
|
||||
}
|
||||
request := speechRequest{Text: "Could not read current line.", Role: "system", Cause: "current-line"}
|
||||
if result.Err == nil {
|
||||
text := result.DocumentCaretText
|
||||
request.Text = presentation.FlatReviewText(text, false)
|
||||
request.Role = result.DocumentPosition.Snapshot.Role
|
||||
set_review_spelling(&request, text, result.FlatSpeech)
|
||||
}
|
||||
handler.speech.Enqueue(request)
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"log/slog"
|
||||
"reflect"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.stormux.org/storm/cthulhu/internal/atspi"
|
||||
"git.stormux.org/storm/cthulhu/internal/input"
|
||||
"git.stormux.org/storm/cthulhu/internal/review"
|
||||
)
|
||||
|
||||
type currentLineAdapter struct{ *fakeAdapter }
|
||||
|
||||
func (*currentLineAdapter) ReadFocusedLine(_ context.Context, focus atspi.ObjectID) (atspi.DocumentContent, error) {
|
||||
return atspi.DocumentContent{Text: "Complete caret line", Snapshot: atspi.Snapshot{BusName: focus.BusName, Path: focus.Path, Role: "entry"}}, nil
|
||||
}
|
||||
|
||||
func TestCurrentLineEventLoopPreservesFocusedItem(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
adapter := ¤tLineAdapter{&fakeAdapter{events: make(chan atspi.Event)}}
|
||||
worker, _ := review.NewWorker(adapter, time.Second)
|
||||
if err := worker.Start(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
queue := &fakeSpeechQueue{spoken: make(chan string, 8)}
|
||||
requests := make(chan input.Command)
|
||||
done := make(chan error, 1)
|
||||
go func() {
|
||||
done <- processEventsWithSettings(ctx, slog.New(slog.DiscardHandler), queue, adapter, requests, worker, 0, nil)
|
||||
}()
|
||||
t.Cleanup(func() {
|
||||
cancel()
|
||||
if err := worker.Stop(context.Background()); err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
<-done
|
||||
})
|
||||
waitSpeech := func(want string) {
|
||||
t.Helper()
|
||||
select {
|
||||
case got := <-queue.spoken:
|
||||
if got != want {
|
||||
t.Fatalf("speech %q, want %q", got, want)
|
||||
}
|
||||
case <-time.After(time.Second):
|
||||
t.Fatalf("missing speech %q", want)
|
||||
}
|
||||
}
|
||||
focus := atspi.NewSnapshot("/entry", "Original item", "entry", "", "")
|
||||
focus.BusName = ":1.42"
|
||||
adapter.events <- atspi.NewEvent(atspi.FocusEvent, focus)
|
||||
waitSpeech("Original item, entry")
|
||||
requests <- input.PresentCurrentLine
|
||||
waitSpeech("Complete caret line")
|
||||
requests <- input.WhereAmI
|
||||
waitSpeech("Original item, entry")
|
||||
}
|
||||
|
||||
func TestCurrentLinePresentationAndStaleResults(t *testing.T) {
|
||||
focus := atspi.ObjectID{BusName: ":1.42", Path: "/entry"}
|
||||
worker, _ := review.NewWorker(&fakeAdapter{}, time.Second)
|
||||
worker.SyncFocus(focus)
|
||||
queue := &fakeSpeechQueue{requests: make(chan speechRequest, 8), spoken: make(chan string, 8)}
|
||||
handler := eventReviewHandler{objectReview: worker, speech: queue, focusGate: &focusGate{}}
|
||||
for _, probe := range []struct {
|
||||
text string
|
||||
style review.FlatSpeech
|
||||
err error
|
||||
want string
|
||||
spelling []string
|
||||
}{
|
||||
{"A book", review.FlatSpeak, nil, "A book", nil},
|
||||
{"A", review.FlatSpell, nil, "A", []string{"A"}},
|
||||
{"A", review.FlatPhonetic, nil, "A", []string{"alpha"}},
|
||||
{"\n", review.FlatSpeak, nil, "blank", nil},
|
||||
{"", review.FlatSpeak, nil, "blank", nil},
|
||||
{"", review.FlatSpeak, errors.New("unavailable"), "Could not read current line.", nil},
|
||||
} {
|
||||
result := review.Result{CurrentLine: true, DocumentFocus: focus, DocumentCaretText: probe.text, FlatSpeech: probe.style, Err: probe.err}
|
||||
handler.present_current_line(result)
|
||||
got := <-queue.requests
|
||||
if got.Text != probe.want || !reflect.DeepEqual(got.Spelling, probe.spelling) {
|
||||
t.Fatalf("current line: %+v, want %q %v", got, probe.want, probe.spelling)
|
||||
}
|
||||
}
|
||||
worker.ResetDocument(focus)
|
||||
handler.present_current_line(review.Result{CurrentLine: true, DocumentFocus: focus})
|
||||
if len(queue.requests) != 0 {
|
||||
t.Fatal("stale report was spoken")
|
||||
}
|
||||
worker.SyncFocus(atspi.ObjectID{BusName: ":1.42", Path: "/other"})
|
||||
handler.present_current_line(review.Result{CurrentLine: true, DocumentFocus: focus, DocumentEpoch: worker.DocumentEpoch()})
|
||||
if len(queue.requests) != 0 {
|
||||
t.Fatal("report from previous focus was spoken")
|
||||
}
|
||||
}
|
||||
@@ -325,6 +325,8 @@ func processEventsWithAttributes(
|
||||
speak(event)
|
||||
case command := <-reviewRequests:
|
||||
switch command {
|
||||
case input.PresentCurrentLine, input.SpellCurrentLine, input.PhoneticCurrentLine:
|
||||
reviewHandler.current_line(command)
|
||||
case input.PresentWindowTitle:
|
||||
syncForeground()
|
||||
titles.present()
|
||||
|
||||
@@ -83,6 +83,10 @@ func (handler *eventReviewHandler) handle(result review.Result) {
|
||||
if !state.current.ID().Valid() {
|
||||
return
|
||||
}
|
||||
if result.CurrentLine {
|
||||
handler.present_current_line(result)
|
||||
return
|
||||
}
|
||||
if result.ReadAllPosition {
|
||||
if objectReview.CurrentDocumentResult(result) && result.DocumentFocus == objectReview.Focus() {
|
||||
state.current = result.DocumentPosition.Snapshot
|
||||
|
||||
@@ -72,6 +72,8 @@ func presentFlatReviewResult(logger *slog.Logger, speech speechEnqueuer, result
|
||||
text = "This window exceeds the current flat review text limit."
|
||||
case errors.Is(result.Err, review.ErrNoReviewContent):
|
||||
text = "No visible content."
|
||||
case errors.Is(result.Err, review.ErrUnavailableReview):
|
||||
text = "Visible content is unavailable for flat review."
|
||||
default:
|
||||
text = "Could not find current location."
|
||||
}
|
||||
@@ -82,8 +84,14 @@ func presentFlatReviewResult(logger *slog.Logger, speech speechEnqueuer, result
|
||||
boundary = true
|
||||
if result.FlatStep < 0 {
|
||||
text = "Top of screen."
|
||||
if result.FlatCompleteness.Incomplete() {
|
||||
text = "Top of available content."
|
||||
}
|
||||
} else {
|
||||
text = "Bottom of screen."
|
||||
if result.FlatCompleteness.Incomplete() {
|
||||
text = "Bottom of available content."
|
||||
}
|
||||
}
|
||||
} else {
|
||||
text = presentation.FlatReviewText(result.FlatPosition.Text, result.FlatUnit == review.Character || result.FlatUnit == review.LineEnd)
|
||||
@@ -96,6 +104,11 @@ func presentFlatReviewResult(logger *slog.Logger, speech speechEnqueuer, result
|
||||
if hasContent {
|
||||
set_review_spelling(&request, result.FlatPosition.Text, result.FlatSpeech)
|
||||
}
|
||||
if result.FlatWarning {
|
||||
logger.Warn("flat review content incomplete", "omitted_text_objects", result.FlatCompleteness.OmittedObjects,
|
||||
"unavailable_subtrees", result.FlatCompleteness.UnavailableSubtrees)
|
||||
speech.Enqueue(speechRequest{Text: "Some visible content is unavailable for flat review.", Role: "system", Cause: "flat-review-warning"})
|
||||
}
|
||||
speech.Enqueue(request)
|
||||
if logger.Enabled(context.Background(), slog.LevelDebug) {
|
||||
logger.Debug("queued flat review presentation",
|
||||
|
||||
@@ -24,10 +24,35 @@ import (
|
||||
"time"
|
||||
|
||||
"git.stormux.org/storm/cthulhu/internal/atspi"
|
||||
"git.stormux.org/storm/cthulhu/internal/config"
|
||||
"git.stormux.org/storm/cthulhu/internal/input"
|
||||
"git.stormux.org/storm/cthulhu/internal/review"
|
||||
)
|
||||
|
||||
func TestFlatReviewPartialWarningPreservesSpellingAndFlashesBraille(t *testing.T) {
|
||||
live := &liveSettings{}
|
||||
live.Store(config.DefaultSettings())
|
||||
braille := &flashRecorder{}
|
||||
queue := &fakeSpeechQueue{spoken: make(chan string, 4), requests: make(chan speechRequest, 4)}
|
||||
output := outputQueue{speech: queue, braille: braille, live: live}
|
||||
result := review.Result{FlatUnit: review.Word, FlatSpeech: review.FlatSpell, FlatWarning: true,
|
||||
FlatCompleteness: review.FlatCompleteness{OmittedObjects: 1}, FlatPosition: review.FlatPosition{Text: "cat"}}
|
||||
presentFlatReviewResult(slog.New(slog.DiscardHandler), output, result)
|
||||
warning, content := <-queue.requests, <-queue.requests
|
||||
if warning.Cause != "flat-review-warning" || len(warning.Spelling) != 0 || len(content.Spelling) != 3 || content.Text != "cat" {
|
||||
t.Fatalf("warning/content metadata = %+v / %+v", warning, content)
|
||||
}
|
||||
if len(braille.messages) != 1 || braille.messages[0] != warning.Text {
|
||||
t.Fatalf("Braille warning missing: %v", braille.messages)
|
||||
}
|
||||
result.FlatWarning = false
|
||||
result.FlatUnit, result.FlatStep, result.Moved = review.Line, 1, false
|
||||
presentFlatReviewResult(slog.New(slog.DiscardHandler), output, result)
|
||||
if boundary := <-queue.requests; boundary.Text != "Bottom of available content." {
|
||||
t.Fatalf("partial boundary claimed complete screen: %+v", boundary)
|
||||
}
|
||||
}
|
||||
|
||||
func TestProcessEventsPresentsFlatReviewThroughSpeechQueue(t *testing.T) {
|
||||
id := func(path string) atspi.ObjectID { return atspi.NewObjectID(":1.42", path) }
|
||||
snapshot := func(path, name, role string) atspi.Snapshot {
|
||||
|
||||
@@ -172,6 +172,8 @@ func learnCommandDescription(command input.Command) string {
|
||||
return "Choose an application and open its preferences."
|
||||
case input.WhereAmI:
|
||||
return "Speak details of the current item."
|
||||
case input.PresentCurrentLine:
|
||||
return "Read the current line. Press twice to spell it, or three times for phonetics."
|
||||
case input.PresentWindowTitle:
|
||||
return "Read the current window title."
|
||||
case input.PresentCellFormula:
|
||||
|
||||
@@ -229,13 +229,15 @@ func (worker commandRuntime) run(ctx context.Context) error {
|
||||
if onNotificationList != nil {
|
||||
onNotificationList()
|
||||
}
|
||||
case input.WhereAmI, input.PresentWindowTitle, input.PresentCellFormula, input.SetColumnHeadersRow, input.ClearColumnHeadersRow, input.SetRowHeadersColumn, input.ClearRowHeadersColumn:
|
||||
case input.PresentCurrentLine, input.SpellCurrentLine, input.PhoneticCurrentLine, input.WhereAmI, input.PresentWindowTitle, input.PresentCellFormula, input.SetColumnHeadersRow, input.ClearColumnHeadersRow, input.SetRowHeadersColumn, input.ClearRowHeadersColumn:
|
||||
if command == input.WhereAmI {
|
||||
logger.Info("where am I command received")
|
||||
} else if command == input.PresentWindowTitle {
|
||||
logger.Info("window title command received")
|
||||
} else if command == input.PresentCellFormula {
|
||||
logger.Info("cell formula command received")
|
||||
} else if command == input.PresentCurrentLine || command == input.SpellCurrentLine || command == input.PhoneticCurrentLine {
|
||||
logger.Debug("current line command received", "command", command)
|
||||
} else {
|
||||
logger.Info("Calc header command received", "command", command)
|
||||
}
|
||||
|
||||
@@ -47,7 +47,7 @@ func TestCommandRuntimeRoutesFocusPresentationRequests(t *testing.T) {
|
||||
t.Error("command worker did not stop")
|
||||
}
|
||||
}()
|
||||
for _, command := range []input.Command{input.WhereAmI, input.PresentWindowTitle, input.PresentCellFormula, input.SetColumnHeadersRow, input.ClearColumnHeadersRow, input.SetRowHeadersColumn, input.ClearRowHeadersColumn} {
|
||||
for _, command := range []input.Command{input.PresentCurrentLine, input.SpellCurrentLine, input.PhoneticCurrentLine, input.WhereAmI, input.PresentWindowTitle, input.PresentCellFormula, input.SetColumnHeadersRow, input.ClearColumnHeadersRow, input.SetRowHeadersColumn, input.ClearRowHeadersColumn} {
|
||||
commands.commands <- input.CommandEvent{Command: command}
|
||||
select {
|
||||
case got := <-requests:
|
||||
|
||||
@@ -211,6 +211,7 @@ type treeCaptureRequest struct {
|
||||
ocrGeometry bool
|
||||
ocrZone *Rectangle
|
||||
readFocusedText bool
|
||||
readFocusedLine bool
|
||||
readAttributes bool
|
||||
setFocusedCaret bool
|
||||
brailleRoute *BrailleRoute
|
||||
|
||||
@@ -127,7 +127,7 @@ func captureNativeTree(ctx context.Context, root *C.AtspiAccessible, reviewBusNa
|
||||
pending = pending[:last]
|
||||
sourceID := nativeObjectID(source)
|
||||
cached, known := cache.nodes[sourceID]
|
||||
refresh := !known || cached.snapshotInvalid || cached.textInvalid || cached.geometryInvalid || cached.childrenInvalid
|
||||
refresh := !known || cached.snapshotInvalid || cached.textInvalid || cached.geometryInvalid || cached.childrenInvalid || cached.childrenUnavailable
|
||||
snapshot := cached.snapshot
|
||||
trace.begin_stage("snapshot", sourceID, snapshot.Role)
|
||||
if refresh {
|
||||
@@ -206,7 +206,7 @@ func captureNativeTree(ctx context.Context, root *C.AtspiAccessible, reviewBusNa
|
||||
childClip, childClipped = item.clip, item.clipped
|
||||
}
|
||||
trace.begin_stage("visible-text", snapshot.ID(), snapshot.Role)
|
||||
text := VisibleText{Limited: true}
|
||||
text := VisibleText{Limited: true, Issue: TextTreeLimit}
|
||||
if (!document || documentText) && textBytes < maxTreeTextBytes {
|
||||
if known && !cached.textInvalid && !cached.geometryInvalid && !item.clipChanged {
|
||||
text = cached.text
|
||||
@@ -224,7 +224,8 @@ func captureNativeTree(ctx context.Context, root *C.AtspiAccessible, reviewBusNa
|
||||
size += len(r.Text)
|
||||
}
|
||||
if textBytes+size > maxTreeTextBytes {
|
||||
text = VisibleText{Limited: true}
|
||||
text = VisibleText{Limited: true, Issue: TextTreeLimit}
|
||||
trace.text_issue(TextTreeLimit)
|
||||
} else {
|
||||
textBytes += size
|
||||
}
|
||||
@@ -239,7 +240,8 @@ func captureNativeTree(ctx context.Context, root *C.AtspiAccessible, reviewBusNa
|
||||
(geometry.Visible && geometry.Showing) || snapshot.Role == "application" || snapshot.Role == "desktop frame"
|
||||
// A visibility change can reveal children omitted by an earlier review
|
||||
// capture, even when the provider emits no children-changed event.
|
||||
cachedChildren := known && !cached.childrenInvalid && (!reviewOnly || !cached.geometryInvalid)
|
||||
cachedChildren := known && !cached.childrenInvalid && !cached.childrenUnavailable && (!reviewOnly || !cached.geometryInvalid)
|
||||
childrenUnavailable := false
|
||||
for _, id := range cached.children {
|
||||
if cache.sources[id] == nil {
|
||||
cachedChildren = false
|
||||
@@ -279,15 +281,22 @@ func captureNativeTree(ctx context.Context, root *C.AtspiAccessible, reviewBusNa
|
||||
err := fmt.Errorf("%w (bus %q path %q role %q captured_nodes %d)",
|
||||
nativeError("query AT-SPI child count", childCountError), snapshot.BusName, snapshot.Path, snapshot.Role, len(nodes))
|
||||
C.g_error_free(childCountError)
|
||||
C.g_object_unref(C.gpointer(source))
|
||||
releasePending()
|
||||
return ObjectID{}, nil, err
|
||||
if !reviewOnly || item.parentIndex < 0 || ctx.Err() != nil {
|
||||
C.g_object_unref(C.gpointer(source))
|
||||
releasePending()
|
||||
return ObjectID{}, nil, err
|
||||
}
|
||||
childCount, childrenUnavailable = 0, true
|
||||
}
|
||||
if childCount < 0 {
|
||||
C.g_object_unref(C.gpointer(source))
|
||||
releasePending()
|
||||
return ObjectID{}, nil, fmt.Errorf("query AT-SPI child count: negative result (bus %q path %q role %q captured_nodes %d)",
|
||||
snapshot.BusName, snapshot.Path, snapshot.Role, len(nodes))
|
||||
if reviewOnly && item.parentIndex >= 0 && ctx.Err() == nil {
|
||||
childCount, childrenUnavailable = 0, true
|
||||
} else {
|
||||
C.g_object_unref(C.gpointer(source))
|
||||
releasePending()
|
||||
return ObjectID{}, nil, fmt.Errorf("query AT-SPI child count: negative result (bus %q path %q role %q captured_nodes %d)",
|
||||
snapshot.BusName, snapshot.Path, snapshot.Role, len(nodes))
|
||||
}
|
||||
}
|
||||
if reviewBusName == "" && len(scheduled)+childCount > maxTreeNodes {
|
||||
C.g_object_unref(C.gpointer(source))
|
||||
@@ -318,6 +327,10 @@ func captureNativeTree(ctx context.Context, root *C.AtspiAccessible, reviewBusNa
|
||||
if child != nil {
|
||||
C.g_object_unref(C.gpointer(child))
|
||||
}
|
||||
if reviewOnly && ctx.Err() == nil {
|
||||
childrenUnavailable = true
|
||||
continue
|
||||
}
|
||||
C.g_object_unref(C.gpointer(source))
|
||||
releasePending()
|
||||
return ObjectID{}, nil, err
|
||||
@@ -362,7 +375,7 @@ func captureNativeTree(ctx context.Context, root *C.AtspiAccessible, reviewBusNa
|
||||
if document && refresh {
|
||||
snapshot.TextChildren = nativeTextChildren(ctx, source, snapshot.CharacterCount, children)
|
||||
}
|
||||
nodes = append(nodes, TreeNode{Snapshot: snapshot, Children: children, Geometry: geometry, Text: text})
|
||||
nodes = append(nodes, TreeNode{Snapshot: snapshot, Children: children, Geometry: geometry, Text: text, ChildrenUnavailable: childrenUnavailable})
|
||||
trace.progress(snapshot.Role, len(nodes), len(pending), textBytes)
|
||||
C.g_object_unref(C.gpointer(source))
|
||||
}
|
||||
@@ -412,6 +425,9 @@ func cthulhuAtspiTreeCaptureCallback(userData unsafe.Pointer) {
|
||||
} else if request.setFocusedCaret {
|
||||
err := adapter.setNativeFocusedTextCaret(request.ctx, request.reviewFocus, request.positionOffset)
|
||||
request.result <- treeCaptureResult{err: err}
|
||||
} else if request.readFocusedLine {
|
||||
content, err := adapter.read_native_focused_line(request.ctx, request.reviewFocus)
|
||||
request.result <- treeCaptureResult{content: content, err: err}
|
||||
} else if request.readFocusedText {
|
||||
content, err := adapter.readNativeFocusedText(request.ctx, request.reviewFocus)
|
||||
request.result <- treeCaptureResult{content: content, err: err}
|
||||
|
||||
@@ -38,10 +38,25 @@ type captureDiagnostics struct {
|
||||
capturedNodes int
|
||||
pendingNodes int
|
||||
textBytes int
|
||||
textIssues map[TextCaptureIssue]int
|
||||
timings []captureStageTiming
|
||||
finished bool
|
||||
}
|
||||
|
||||
func (trace *captureDiagnostics) text_issue(issue TextCaptureIssue) {
|
||||
if trace == nil {
|
||||
return
|
||||
}
|
||||
trace.mutex.Lock()
|
||||
defer trace.mutex.Unlock()
|
||||
if !trace.finished {
|
||||
if trace.textIssues == nil {
|
||||
trace.textIssues = make(map[TextCaptureIssue]int)
|
||||
}
|
||||
trace.textIssues[issue]++
|
||||
}
|
||||
}
|
||||
|
||||
func new_capture_diagnostics(ctx context.Context, logger *slog.Logger, focus ObjectID) (context.Context, *captureDiagnostics) {
|
||||
if logger == nil || !logger.Enabled(ctx, slog.LevelDebug) {
|
||||
return ctx, nil
|
||||
@@ -128,5 +143,6 @@ func (trace *captureDiagnostics) finish(err error) {
|
||||
"bus", trace.object.BusName, "object_path", trace.object.Path, "role", trace.role,
|
||||
"root_bus", trace.root.BusName, "root_path", trace.root.Path, "root_scope", trace.rootScope,
|
||||
"captured_nodes", trace.capturedNodes, "pending_nodes", trace.pendingNodes, "text_bytes", trace.textBytes,
|
||||
"text_issues", trace.textIssues,
|
||||
"stages", trace.timings)
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
// Copyright (c) 2026 Storm Dragon
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package atspi
|
||||
|
||||
/*
|
||||
#cgo pkg-config: atspi-2 glib-2.0 gobject-2.0
|
||||
#include <atspi/atspi.h>
|
||||
*/
|
||||
import "C"
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ReadFocusedLine reads only the line containing the current native caret.
|
||||
func (adapter *Adapter) ReadFocusedLine(ctx context.Context, focus ObjectID) (DocumentContent, error) {
|
||||
result, err := adapter.documentOperation(ctx, treeCaptureRequest{reviewFocus: focus, readFocusedLine: true})
|
||||
return result.content, err
|
||||
}
|
||||
|
||||
func (adapter *Adapter) read_native_focused_line(ctx context.Context, focus ObjectID) (DocumentContent, error) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return DocumentContent{}, err
|
||||
}
|
||||
if !adapter.nativeFocusIs(focus) {
|
||||
return DocumentContent{}, ErrStaleObject
|
||||
}
|
||||
source := adapter.nativeFocus[len(adapter.nativeFocus)-1].source
|
||||
snapshot := snapshotFromNativeWithCaret(source, false, false, false, false)
|
||||
if snapshot.Role == "password text" {
|
||||
return DocumentContent{}, ErrNoDocumentCaret
|
||||
}
|
||||
content := DocumentContent{Snapshot: snapshot, Text: snapshot.Name}
|
||||
text := C.atspi_accessible_get_text_iface(source)
|
||||
if text == nil {
|
||||
return content, ctx.Err()
|
||||
}
|
||||
defer C.g_object_unref(C.gpointer(text))
|
||||
reader := nativeReviewText{text: text, terminal: snapshot.Role == "terminal"}
|
||||
count, err := reader.count(ctx)
|
||||
if err != nil {
|
||||
return DocumentContent{}, err
|
||||
}
|
||||
content.Text = ""
|
||||
if count == 0 {
|
||||
return content, ctx.Err()
|
||||
}
|
||||
var nativeErr *C.GError
|
||||
offset := int(C.atspi_text_get_caret_offset(text, &nativeErr))
|
||||
if nativeErr != nil {
|
||||
err := nativeError("read current line caret", nativeErr)
|
||||
C.g_error_free(nativeErr)
|
||||
return DocumentContent{}, err
|
||||
}
|
||||
// A text object without a caret reads from its first line, as NVDA does.
|
||||
offset = min(max(0, offset), count)
|
||||
if offset == count {
|
||||
last := C.atspi_text_get_text(text, C.gint(count-1), C.gint(count), &nativeErr)
|
||||
if last != nil {
|
||||
defer C.g_free(C.gpointer(last))
|
||||
}
|
||||
if nativeErr != nil {
|
||||
err := nativeError("read current line end", nativeErr)
|
||||
C.g_error_free(nativeErr)
|
||||
return DocumentContent{}, err
|
||||
}
|
||||
// A trailing newline puts the caret on an empty final line.
|
||||
if strings.HasSuffix(C.GoString(last), "\n") {
|
||||
content.Start, content.End = count, count
|
||||
return content, ctx.Err()
|
||||
}
|
||||
offset--
|
||||
}
|
||||
line, err := reader.line(ctx, offset)
|
||||
if err != nil {
|
||||
return DocumentContent{}, err
|
||||
}
|
||||
if line.Start < 0 || line.Start > offset || line.End <= offset || line.End > count {
|
||||
return DocumentContent{}, errors.New("read current line: invalid text range")
|
||||
}
|
||||
content.Start, content.End, content.Text, content.RawText = line.Start, line.End, line.Text, line.Text
|
||||
return content, ctx.Err()
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
package atspi
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestNativeCurrentLine(t *testing.T) {
|
||||
if os.Getenv("CTHULHU_NATIVE_CURRENT_LINE") != "1" {
|
||||
t.Skip("requires private Xvfb and AT-SPI")
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 25*time.Second)
|
||||
defer cancel()
|
||||
adapter := New()
|
||||
if err := adapter.Start(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer adapter.Stop(context.Background())
|
||||
fixture := exec.CommandContext(ctx, "/usr/bin/python3", "-u", "../../tests/current_line_fixture.py")
|
||||
fixture.Stderr = os.Stderr
|
||||
stdin, _ := fixture.StdinPipe()
|
||||
stdout, _ := fixture.StdoutPipe()
|
||||
if err := fixture.Start(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer func() { stdin.Close(); fixture.Wait() }()
|
||||
scanner := bufio.NewScanner(stdout)
|
||||
ready := func() {
|
||||
t.Helper()
|
||||
if !scanner.Scan() || scanner.Text() != "ready" {
|
||||
t.Fatalf("fixture not ready: %q %v", scanner.Text(), scanner.Err())
|
||||
}
|
||||
}
|
||||
ready()
|
||||
var focus ObjectID
|
||||
lastControl := ""
|
||||
for _, probe := range []struct {
|
||||
control, text string
|
||||
offset int
|
||||
want string
|
||||
password bool
|
||||
}{
|
||||
{"entry", "Café 猫 complete", 7, "Café 猫 complete", false},
|
||||
{"text", "First\nSecond café 猫\nThird", 11, "Second café 猫", false},
|
||||
{"entry", "Final line", 10, "Final line", false},
|
||||
{"text", "First\n\nLast", 6, "", false},
|
||||
{"entry", "", 0, "", false},
|
||||
{"text", "Final\n", 6, "", false},
|
||||
{"password", "private test text", 5, "", true},
|
||||
{"button", "", 0, "Example button", false},
|
||||
} {
|
||||
command, _ := json.Marshal(map[string]any{"action": "focus", "control": probe.control, "text": probe.text, "offset": probe.offset})
|
||||
fmt.Fprintln(stdin, string(command))
|
||||
ready()
|
||||
if probe.control != lastControl {
|
||||
for {
|
||||
select {
|
||||
case event := <-adapter.Events():
|
||||
if event.Type == "object:state-changed:focused" && event.Detail1 == 1 &&
|
||||
(event.Source.Name == map[string]string{"entry": "Entry", "text": "Text", "password": "Password", "button": "Example button"}[probe.control]) {
|
||||
focus = event.Source.ID()
|
||||
goto focused
|
||||
}
|
||||
case <-ctx.Done():
|
||||
t.Fatal(ctx.Err())
|
||||
}
|
||||
}
|
||||
}
|
||||
focused:
|
||||
lastControl = probe.control
|
||||
content, err := adapter.ReadFocusedLine(ctx, focus)
|
||||
if probe.password {
|
||||
if err == nil || content.Text != "" || content.Snapshot.Name != "" {
|
||||
t.Fatalf("password report: %+v %v", content, err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if err != nil || strings.TrimRight(content.Text, "\r\n") != probe.want {
|
||||
t.Fatalf("%s offset %d: %+v %v, want %q", probe.control, probe.offset, content, err, probe.want)
|
||||
}
|
||||
if probe.control != "button" {
|
||||
fmt.Fprintf(stdin, "{\"action\":\"caret\",\"control\":%q}\n", probe.control)
|
||||
if !scanner.Scan() || scanner.Text() != fmt.Sprint(probe.offset) {
|
||||
t.Fatalf("caret moved: %q, want %d", scanner.Text(), probe.offset)
|
||||
}
|
||||
ready()
|
||||
}
|
||||
}
|
||||
if _, err := adapter.ReadFocusedLine(ctx, ObjectID{BusName: focus.BusName, Path: "/stale"}); err == nil {
|
||||
t.Fatal("accepted stale native focus")
|
||||
}
|
||||
}
|
||||
@@ -85,7 +85,7 @@ found:
|
||||
text := entry.text
|
||||
text.Ranges = append([]TextRange(nil), text.Ranges...)
|
||||
nodes = append(nodes, TreeNode{Snapshot: entry.snapshot, Geometry: entry.geometry,
|
||||
Text: text, Children: append([]ObjectID(nil), entry.children...)})
|
||||
Text: text, Children: append([]ObjectID(nil), entry.children...), ChildrenUnavailable: entry.childrenUnavailable})
|
||||
references[id] = newObjectReference(id, entry)
|
||||
for i := len(entry.children) - 1; i >= 0; i-- {
|
||||
pending = append(pending, entry.children[i])
|
||||
|
||||
@@ -17,6 +17,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"git.stormux.org/storm/cthulhu/internal/logging"
|
||||
)
|
||||
@@ -35,6 +36,39 @@ func TestNativeFlatReviewBoundsSlowLongRanges(t *testing.T) {
|
||||
test_native_review_provider(t, "bounds")
|
||||
}
|
||||
|
||||
func TestNativeFlatReviewClippedViewport(t *testing.T) {
|
||||
if os.Getenv("CTHULHU_NATIVE_REVIEW_CLIPPED") != "1" {
|
||||
t.Skip("requires private Xvfb/AT-SPI session")
|
||||
}
|
||||
test_native_review_provider(t, "clipped")
|
||||
}
|
||||
|
||||
func TestNativeFlatReviewLargeTextViewport(t *testing.T) {
|
||||
if os.Getenv("CTHULHU_NATIVE_REVIEW_VIEWPORT") != "1" {
|
||||
t.Skip("requires private Xvfb/AT-SPI session")
|
||||
}
|
||||
// Adapter ownership is process-global; each mode runs in its own process.
|
||||
mode := os.Getenv("CTHULHU_NATIVE_REVIEW_VIEWPORT_MODE")
|
||||
if mode != "lines" && mode != "long" {
|
||||
t.Fatal("set viewport mode to lines or long")
|
||||
}
|
||||
test_native_review_provider(t, mode)
|
||||
}
|
||||
|
||||
func TestNativeFlatReviewUnavailableSubtree(t *testing.T) {
|
||||
if os.Getenv("CTHULHU_NATIVE_REVIEW_SUBTREE") != "1" {
|
||||
t.Skip("requires private Xvfb/AT-SPI session")
|
||||
}
|
||||
test_native_review_provider(t, "children")
|
||||
}
|
||||
|
||||
func TestNativeFlatReviewZeroAreaGlyph(t *testing.T) {
|
||||
if os.Getenv("CTHULHU_NATIVE_REVIEW_GLYPHS") != "1" {
|
||||
t.Skip("requires private Xvfb/AT-SPI session")
|
||||
}
|
||||
test_native_review_provider(t, "glyphs")
|
||||
}
|
||||
|
||||
func test_native_review_provider(t *testing.T, mode string) {
|
||||
t.Helper()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
|
||||
@@ -96,11 +130,14 @@ func test_native_review_provider(t *testing.T, mode string) {
|
||||
t.Fatalf("fixture %s: %q, %v", request, scanner.Text(), scanner.Err())
|
||||
}
|
||||
}
|
||||
if mode == "bounds" {
|
||||
if mode == "bounds" || mode == "clipped" || mode == "long" || mode == "lines" || mode == "glyphs" {
|
||||
command("measure", "measuring")
|
||||
} else {
|
||||
} else if mode != "children" {
|
||||
command("arm", "armed")
|
||||
}
|
||||
if mode == "lines" {
|
||||
command("scroll", "scrolled")
|
||||
}
|
||||
tree, err := adapter.CaptureReviewWindow(ctx, focus)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
@@ -117,6 +154,33 @@ func test_native_review_provider(t *testing.T, mode string) {
|
||||
if timing.Message != "flat review capture timing" || timing.RootScope != "window" || timing.CapturedNodes == 0 {
|
||||
t.Fatalf("missing capture scope/progress: %+v", timing)
|
||||
}
|
||||
if mode == "children" {
|
||||
broken := findNamedReference(t, tree, "Broken subtree")
|
||||
if !broken.Valid() {
|
||||
t.Fatal("broken subtree fixture missing")
|
||||
}
|
||||
command("arm", "armed")
|
||||
tree.InvalidateChildren(broken.ID())
|
||||
tree, err = adapter.CaptureReviewWindow(ctx, focus)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
snapshot, err := tree.ReviewWindow(ctx, focus)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
omitted, healthy := false, false
|
||||
for _, node := range snapshot.Nodes {
|
||||
omitted = omitted || node.Snapshot.ID() == broken.ID() && node.ChildrenUnavailable
|
||||
for _, span := range node.Text.Ranges {
|
||||
healthy = healthy || span.Text == "Cached review text"
|
||||
}
|
||||
}
|
||||
if !omitted || !healthy {
|
||||
t.Fatalf("subtree recovery: omission=%v healthy=%v", omitted, healthy)
|
||||
}
|
||||
return
|
||||
}
|
||||
for _, want := range []string{"queue-wait", "root-selection", "snapshot", "geometry", "visible-text", "table-viewport", "child-count", "child-enumeration", "tree-installation"} {
|
||||
found := false
|
||||
for _, stage := range timing.Stages {
|
||||
@@ -128,7 +192,7 @@ func test_native_review_provider(t *testing.T, mode string) {
|
||||
t.Errorf("capture timing missing stage %q: %+v", want, timing)
|
||||
}
|
||||
}
|
||||
if mode == "bounds" {
|
||||
if mode == "bounds" || mode == "clipped" || mode == "long" || mode == "lines" || mode == "glyphs" {
|
||||
label := findNamedReference(t, tree, "Long review label")
|
||||
if !label.Valid() {
|
||||
t.Fatal("long label missing from review")
|
||||
@@ -137,7 +201,28 @@ func test_native_review_provider(t *testing.T, mode string) {
|
||||
if err != nil || !text.Known || text.Limited || len(text.Ranges) == 0 {
|
||||
t.Fatalf("long label text: %+v, %v", text, err)
|
||||
}
|
||||
command("largest", "largest 128")
|
||||
if mode == "bounds" {
|
||||
command("largest", "largest 128")
|
||||
}
|
||||
if mode == "lines" && text.Ranges[0].Start <= maxReviewCharacters {
|
||||
t.Fatalf("line viewport did not seek scrolled native offsets: %d", text.Ranges[0].Start)
|
||||
}
|
||||
if mode == "glyphs" {
|
||||
for _, span := range text.Ranges {
|
||||
if span.Start <= 5 && span.End > 5 {
|
||||
t.Fatalf("zero-area glyph included as visible text: %+v", span)
|
||||
}
|
||||
}
|
||||
}
|
||||
if mode == "clipped" {
|
||||
characters := 0
|
||||
for _, span := range text.Ranges {
|
||||
characters += utf8.RuneCountInString(span.Text)
|
||||
}
|
||||
if characters <= maxReviewRanges || len(text.Ranges) > 21 {
|
||||
t.Fatalf("fixture must capture hundreds of characters as visual spans: characters=%d ranges=%d", characters, len(text.Ranges))
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
label := findNamedReference(t, tree, "Cached review text")
|
||||
|
||||
+136
-13
@@ -22,6 +22,7 @@ import (
|
||||
"errors"
|
||||
"math"
|
||||
"strings"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
@@ -37,6 +38,20 @@ const (
|
||||
|
||||
var ErrStaleText = errors.New("accessible text requires recapture")
|
||||
|
||||
var errReviewLineTooLong = errors.New("review line requires viewport retrieval")
|
||||
|
||||
// TextCaptureIssue describes omitted text without retaining provider content.
|
||||
type TextCaptureIssue string
|
||||
|
||||
const (
|
||||
TextViewportUnavailable TextCaptureIssue = "viewport-unavailable"
|
||||
TextCharacterLimit TextCaptureIssue = "character-limit"
|
||||
TextRangeLimit TextCaptureIssue = "range-limit"
|
||||
TextTreeLimit TextCaptureIssue = "tree-byte-limit"
|
||||
TextProviderFailure TextCaptureIssue = "text-provider-failure"
|
||||
TextViewportScanLimit TextCaptureIssue = "viewport-query-limit"
|
||||
)
|
||||
|
||||
// TextRange owns exact UTF-8 text and AT-SPI character offsets [Start, End).
|
||||
// Bounds are clipped screen coordinates. Offsets count Unicode code points,
|
||||
// not bytes or grapheme clusters. Embedded objects have their own tree nodes.
|
||||
@@ -52,6 +67,7 @@ type TextRange struct {
|
||||
type VisibleText struct {
|
||||
Known bool
|
||||
Limited bool
|
||||
Issue TextCaptureIssue
|
||||
Ranges []TextRange
|
||||
CaretKnown bool
|
||||
CaretOffset int
|
||||
@@ -61,7 +77,7 @@ func (text VisibleText) valid(geometry Geometry) bool {
|
||||
if text.CaretKnown && (!text.Known || text.CaretOffset < 0) || !text.CaretKnown && text.CaretOffset != 0 {
|
||||
return false
|
||||
}
|
||||
if text.Known && text.Limited || !text.Known && len(text.Ranges) != 0 || len(text.Ranges) > maxReviewRanges {
|
||||
if text.Known && (text.Limited || text.Issue != "") || !text.Known && len(text.Ranges) != 0 || len(text.Ranges) > maxReviewRanges {
|
||||
return false
|
||||
}
|
||||
if text.Known && !geometry.BoundsKnown {
|
||||
@@ -125,6 +141,12 @@ type reviewTextSource interface {
|
||||
bounds(context.Context, int, int) (Rectangle, error)
|
||||
}
|
||||
|
||||
// Optional spatial retrieval avoids requesting an entire enormous native line.
|
||||
// Implementations must bound copied characters and ranges before returning.
|
||||
type reviewViewportSource interface {
|
||||
viewport(context.Context, int, int, Rectangle) ([]TextRange, error)
|
||||
}
|
||||
|
||||
// bounded_review_bounds preserves the full range's bounding rectangle without
|
||||
// asking a provider to measure thousands of characters in one synchronous call.
|
||||
// GTK text cells can exceed libatspi's timeout on such requests. Union every
|
||||
@@ -208,9 +230,18 @@ func captureVisibleText(ctx context.Context, source reviewTextSource, clip Recta
|
||||
return unknown, errors.New("negative review character count")
|
||||
}
|
||||
start, end, err := visibleTextOffsets(ctx, source, count, clip)
|
||||
if err != nil && count > maxReviewCharacters {
|
||||
if ctx.Err() != nil {
|
||||
return unknown, ctx.Err()
|
||||
}
|
||||
start, end, err = visible_line_offsets(ctx, source, count, clip)
|
||||
}
|
||||
if err != nil {
|
||||
if ctx.Err() != nil {
|
||||
return unknown, ctx.Err()
|
||||
}
|
||||
if count > maxReviewCharacters {
|
||||
return VisibleText{Limited: true}, nil
|
||||
return capture_review_viewport(ctx, source, count, clip, TextViewportUnavailable)
|
||||
}
|
||||
// Some otherwise usable GTK text providers do not expose per-character
|
||||
// extents. Retain the established bounded line capture for those small
|
||||
@@ -218,34 +249,80 @@ func captureVisibleText(ctx context.Context, source reviewTextSource, clip Recta
|
||||
start, end = 0, count
|
||||
}
|
||||
if end-start > maxReviewCharacters {
|
||||
return VisibleText{Limited: true}, nil
|
||||
if spatial, ok := source.(reviewViewportSource); ok {
|
||||
ranges, spatialErr := spatial.viewport(ctx, start, end, clip)
|
||||
if spatialErr == nil && len(ranges) > 0 {
|
||||
return capture_review_lines(ctx, source, count, 0, count, clip, ranges)
|
||||
}
|
||||
if errors.Is(spatialErr, errReviewViewportScanLimit) {
|
||||
return VisibleText{Limited: true, Issue: TextViewportScanLimit}, ctx.Err()
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
return unknown, ctx.Err()
|
||||
}
|
||||
}
|
||||
}
|
||||
if end-start > maxReviewCharacters {
|
||||
return VisibleText{Limited: true, Issue: TextCharacterLimit}, nil
|
||||
}
|
||||
return capture_review_lines(ctx, source, count, start, end, clip, nil)
|
||||
}
|
||||
|
||||
func capture_review_lines(ctx context.Context, source reviewTextSource, count, start, end int, clip Rectangle, spatial []TextRange) (VisibleText, error) {
|
||||
unknown := VisibleText{}
|
||||
if spatial != nil {
|
||||
if err := validate_viewport_ranges(spatial, count); err != nil {
|
||||
return unknown, err
|
||||
}
|
||||
spatial = split_viewport_lines(spatial)
|
||||
}
|
||||
result := VisibleText{Known: true}
|
||||
characters := 0
|
||||
for offset, lines := start, 0; offset < end; lines++ {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return unknown, err
|
||||
}
|
||||
if spatial != nil && lines == len(spatial) {
|
||||
break
|
||||
}
|
||||
if lines >= maxReviewRanges {
|
||||
return VisibleText{Limited: true}, nil
|
||||
return VisibleText{Limited: true, Issue: TextRangeLimit}, nil
|
||||
}
|
||||
var line TextRange
|
||||
var err error
|
||||
if spatial != nil {
|
||||
line = spatial[lines]
|
||||
if line.Start < offset {
|
||||
return unknown, errors.New("unordered review viewport ranges")
|
||||
}
|
||||
offset = line.Start
|
||||
} else {
|
||||
line, err = source.line(ctx, offset)
|
||||
}
|
||||
if errors.Is(err, errReviewLineTooLong) {
|
||||
return capture_review_viewport(ctx, source, count, clip, TextCharacterLimit)
|
||||
}
|
||||
line, err := source.line(ctx, offset)
|
||||
if err != nil {
|
||||
return unknown, err
|
||||
}
|
||||
// Some AT-SPI providers return a line whose reported start is just
|
||||
// after the requested offset. Python Cthulhu retries one character
|
||||
// earlier for that provider quirk before rejecting the capture.
|
||||
if line.Start > offset && offset > 0 {
|
||||
if spatial == nil && line.Start > offset && offset > 0 {
|
||||
line, err = source.line(ctx, offset-1)
|
||||
if err != nil {
|
||||
return unknown, err
|
||||
}
|
||||
}
|
||||
if line.Start < 0 || line.Start > offset || line.End <= offset || line.End > count || len(line.Text) > maxReviewLineCharacters*utf8.UTFMax || !utf8.ValidString(line.Text) || utf8.RuneCountInString(line.Text) != line.End-line.Start {
|
||||
if line.Start < 0 || line.Start > offset || line.End <= offset || line.End > count || len(line.Text) > maxReviewCharacters*utf8.UTFMax || !utf8.ValidString(line.Text) || utf8.RuneCountInString(line.Text) != line.End-line.Start {
|
||||
return unknown, errors.New("invalid review line offsets or content")
|
||||
}
|
||||
lineStart := max(line.Start, offset)
|
||||
lineEnd := min(line.End, end)
|
||||
characters += lineEnd - lineStart
|
||||
if characters > maxReviewCharacters {
|
||||
return VisibleText{Limited: true, Issue: TextCharacterLimit}, nil
|
||||
}
|
||||
runes := []rune(line.Text)[lineStart-line.Start : lineEnd-line.Start]
|
||||
// Python retains newline-only TextZones. Keep their exact offsets and
|
||||
// native bounds so empty terminal rows remain navigable and speak blank.
|
||||
@@ -273,14 +350,39 @@ func captureVisibleText(ctx context.Context, source reviewTextSource, clip Recta
|
||||
if err != nil {
|
||||
return unknown, err
|
||||
}
|
||||
if !bounds.Valid() {
|
||||
return unknown, errors.New("unavailable review line geometry")
|
||||
}
|
||||
clipped, visible := bounds.Intersection(clip)
|
||||
if visible && clipped == bounds {
|
||||
result.Ranges = append(result.Ranges, TextRange{lineStart + start, lineStart + end, string(runes[start:end]), bounds})
|
||||
singleRow := true
|
||||
if spatial != nil && visible && end-start > 1 {
|
||||
first, firstErr := source.bounds(ctx, lineStart+start, lineStart+start+1)
|
||||
last, lastErr := source.bounds(ctx, lineStart+end-1, lineStart+end)
|
||||
if firstErr != nil || lastErr != nil {
|
||||
return unknown, errors.Join(firstErr, lastErr)
|
||||
}
|
||||
singleRow = first.Valid() && last.Valid() && first.Y == last.Y && first.Height == last.Height
|
||||
}
|
||||
if visible && singleRow && (clipped == bounds || clipped.X == bounds.X && clipped.Width == bounds.Width) {
|
||||
// Vertical clipping of one visual row does not hide letters.
|
||||
// This also supports providers with only line rectangles.
|
||||
result.Ranges = append(result.Ranges, TextRange{lineStart + start, lineStart + end, string(runes[start:end]), clipped})
|
||||
} else if visible {
|
||||
// A clipped rectangle alone cannot hide offscreen characters. Check each
|
||||
// visible character, preserving disjoint spans. Reject offscreen blocks
|
||||
// first so a horizontally clipped long line does not require thousands
|
||||
// of individual provider calls. Test every block for bidi text.
|
||||
// Keep consecutive visible characters on the same visual row in
|
||||
// one range. The budget bounds spans, not individual glyphs.
|
||||
// Flush at an offset gap so bidi runs never include hidden text.
|
||||
spanStart, spanEnd := -1, -1
|
||||
var spanBounds Rectangle
|
||||
flushSpan := func() {
|
||||
if spanStart >= 0 {
|
||||
result.Ranges = append(result.Ranges, TextRange{lineStart + spanStart, lineStart + spanEnd, string(runes[spanStart:spanEnd]), spanBounds})
|
||||
spanStart = -1
|
||||
}
|
||||
}
|
||||
for blockStart := start; blockStart < end; {
|
||||
blockEnd := blockStart + min(end-blockStart, maxReviewBoundsCharacters)
|
||||
if end-start > maxReviewBoundsCharacters {
|
||||
@@ -301,18 +403,35 @@ func captureVisibleText(ctx context.Context, source reviewTextSource, clip Recta
|
||||
if err != nil {
|
||||
return unknown, err
|
||||
}
|
||||
if charClip, ok := charBounds.Intersection(clip); ok {
|
||||
result.Ranges = append(result.Ranges, TextRange{lineStart + i, lineStart + i + 1, string(runes[i]), charClip})
|
||||
charClip, ok := charBounds.Intersection(clip)
|
||||
if !charBounds.Valid() && spanEnd == i && spanStart >= 0 && review_cluster_continuation(runes[i]) {
|
||||
// Combining marks and joiners belong to the visible base
|
||||
// even when they have no independent glyph rectangle.
|
||||
charClip, ok = spanBounds, true
|
||||
}
|
||||
if ok {
|
||||
if spanStart >= 0 && (spanEnd != i || spanBounds.Y != charClip.Y || spanBounds.Height != charClip.Height) {
|
||||
flushSpan()
|
||||
}
|
||||
if spanStart < 0 {
|
||||
spanStart, spanBounds = i, charClip
|
||||
} else {
|
||||
left := min(spanBounds.X, charClip.X)
|
||||
right := max(int64(spanBounds.X)+int64(spanBounds.Width), int64(charClip.X)+int64(charClip.Width))
|
||||
spanBounds.X, spanBounds.Width = left, int32(right-int64(left))
|
||||
}
|
||||
spanEnd = i + 1
|
||||
}
|
||||
if len(result.Ranges) > maxReviewRanges {
|
||||
return VisibleText{Limited: true}, nil
|
||||
return VisibleText{Limited: true, Issue: TextRangeLimit}, nil
|
||||
}
|
||||
}
|
||||
blockStart = blockEnd
|
||||
}
|
||||
flushSpan()
|
||||
}
|
||||
if len(result.Ranges) > maxReviewRanges {
|
||||
return VisibleText{Limited: true}, nil
|
||||
return VisibleText{Limited: true, Issue: TextRangeLimit}, nil
|
||||
}
|
||||
start = end
|
||||
}
|
||||
@@ -321,6 +440,10 @@ func captureVisibleText(ctx context.Context, source reviewTextSource, clip Recta
|
||||
return result, ctx.Err()
|
||||
}
|
||||
|
||||
func review_cluster_continuation(character rune) bool {
|
||||
return unicode.IsMark(character) || character == '\u200c' || character == '\u200d'
|
||||
}
|
||||
|
||||
// visibleTextOffsets uses character extents only to locate the native viewport;
|
||||
// the returned text remains line-based. AT-SPI text offsets are ordered by
|
||||
// visual rows for the providers supported by flat review, allowing bounded
|
||||
|
||||
@@ -189,19 +189,60 @@ func (source nativeReviewText) line(ctx context.Context, offset int) (TextRange,
|
||||
}
|
||||
length := int(C.review_text_length(line.content, C.int(maxReviewLineCharacters*utf8.UTFMax)))
|
||||
if length < 0 || length > maxReviewLineCharacters*utf8.UTFMax {
|
||||
return TextRange{}, errors.New("oversized or missing review line content")
|
||||
return TextRange{}, errReviewLineTooLong
|
||||
}
|
||||
return TextRange{Start: int(line.start_offset), End: int(line.end_offset), Text: C.GoStringN(line.content, C.int(length))}, ctx.Err()
|
||||
}
|
||||
|
||||
func (source nativeReviewText) viewport(ctx context.Context, start, end int, clip Rectangle) ([]TextRange, error) {
|
||||
return point_review_viewport(ctx, source, start, end, clip)
|
||||
}
|
||||
|
||||
func (source nativeReviewText) offset_at_point(ctx context.Context, x, y int32) (int, error) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
var nativeErr *C.GError
|
||||
offset := C.atspi_text_get_offset_at_point(source.text, C.gint(x), C.gint(y), C.ATSPI_COORD_TYPE_SCREEN, &nativeErr)
|
||||
if nativeErr != nil {
|
||||
C.g_error_free(nativeErr)
|
||||
return 0, errors.New("query review offset at point")
|
||||
}
|
||||
return int(offset), ctx.Err()
|
||||
}
|
||||
|
||||
func (source nativeReviewText) text_span(ctx context.Context, start, end int) (string, error) {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if end <= start || end-start > maxReviewBoundsCharacters {
|
||||
return "", errors.New("invalid review span request")
|
||||
}
|
||||
var nativeErr *C.GError
|
||||
content := C.atspi_text_get_text(source.text, C.gint(start), C.gint(end), &nativeErr)
|
||||
if content != nil {
|
||||
defer C.g_free(C.gpointer(content))
|
||||
}
|
||||
if nativeErr != nil {
|
||||
C.g_error_free(nativeErr)
|
||||
return "", errors.New("query review text span")
|
||||
}
|
||||
length := int(C.review_text_length(content, C.int((end-start)*utf8.UTFMax)))
|
||||
if length < 0 || length > (end-start)*utf8.UTFMax {
|
||||
return "", errors.New("invalid review span length")
|
||||
}
|
||||
value := C.GoStringN(content, C.int(length))
|
||||
if !utf8.ValidString(value) || utf8.RuneCountInString(value) != end-start {
|
||||
return "", errors.New("invalid review span offsets or content")
|
||||
}
|
||||
return value, ctx.Err()
|
||||
}
|
||||
|
||||
func (source nativeReviewText) bounds(ctx context.Context, start, end int) (Rectangle, error) {
|
||||
rect, err := bounded_review_bounds(ctx, start, end, source.range_bounds)
|
||||
if err != nil {
|
||||
return Rectangle{}, err
|
||||
}
|
||||
if !rect.Valid() {
|
||||
return Rectangle{}, errors.New("invalid review range bounds")
|
||||
}
|
||||
return rect, ctx.Err()
|
||||
}
|
||||
|
||||
@@ -242,7 +283,13 @@ func textFromNative(ctx context.Context, accessible *C.AtspiAccessible, geometry
|
||||
defer C.g_object_unref(C.gpointer(text))
|
||||
result, err := captureVisibleText(ctx, nativeReviewText{text: text, terminal: role == "terminal"}, clip)
|
||||
if err != nil {
|
||||
return VisibleText{}, ctx.Err()
|
||||
if ctx.Err() != nil {
|
||||
return VisibleText{}, ctx.Err()
|
||||
}
|
||||
result = VisibleText{Issue: TextProviderFailure}
|
||||
}
|
||||
if result.Issue != "" {
|
||||
capture_diagnostics(ctx).text_issue(result.Issue)
|
||||
}
|
||||
if result.Known && ctx.Err() == nil {
|
||||
var nativeErr *C.GError
|
||||
|
||||
@@ -63,7 +63,7 @@ func TestCaptureVisibleTextUnicodeClippingAndEmbeddedObjects(t *testing.T) {
|
||||
}}
|
||||
got, err := captureVisibleText(context.Background(), source, clip)
|
||||
want := VisibleText{Known: true, Ranges: []TextRange{
|
||||
{1, 2, "猫", Rectangle{-10, 0, 10, 10}}, {2, 3, "x", Rectangle{0, 0, 10, 10}},
|
||||
{1, 3, "猫x", Rectangle{-10, 0, 20, 10}},
|
||||
{4, 5, "a", Rectangle{-10, 10, 5, 10}}, {6, 8, "bz", Rectangle{-5, 10, 10, 10}},
|
||||
}}
|
||||
if err != nil || !reflect.DeepEqual(got, want) {
|
||||
@@ -178,6 +178,93 @@ func TestCaptureVisibleTextLimitsAndEmpty(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// A horizontally clipped viewport has hundreds of visible characters, but
|
||||
// only a few visual rows. The range budget must count spans, not characters.
|
||||
type clippedReviewText struct{ rows, columns int }
|
||||
|
||||
func (s clippedReviewText) count(context.Context) (int, error) { return s.rows * s.columns, nil }
|
||||
func (s clippedReviewText) line(_ context.Context, offset int) (TextRange, error) {
|
||||
start := offset / s.columns * s.columns
|
||||
return TextRange{Start: start, End: start + s.columns, Text: strings.Repeat("é", s.columns)}, nil
|
||||
}
|
||||
func (s clippedReviewText) bounds(_ context.Context, start, end int) (Rectangle, error) {
|
||||
return Rectangle{X: int32(start%s.columns*10 - 50), Y: int32(start / s.columns * 12), Width: int32((end - start) * 10), Height: 12}, nil
|
||||
}
|
||||
|
||||
func TestCaptureVisibleTextClippedViewportUsesSpans(t *testing.T) {
|
||||
source := clippedReviewText{rows: 20, columns: 400}
|
||||
clip := Rectangle{X: 3, Width: 394, Height: int32(source.rows * 12)}
|
||||
got, err := captureVisibleText(context.Background(), source, clip)
|
||||
if err != nil || !got.Known || got.Limited || len(got.Ranges) != source.rows {
|
||||
t.Fatalf("clipped capture: known=%v limited=%v ranges=%d err=%v", got.Known, got.Limited, len(got.Ranges), err)
|
||||
}
|
||||
for row, span := range got.Ranges {
|
||||
want := TextRange{Start: row*source.columns + 5, End: row*source.columns + 45, Text: strings.Repeat("é", 40), Bounds: Rectangle{X: 3, Y: int32(row * 12), Width: 394, Height: 12}}
|
||||
if span != want {
|
||||
t.Fatalf("row %d: %+v, want %+v", row, span, want)
|
||||
}
|
||||
}
|
||||
if !got.valid(Geometry{BoundsKnown: true, Bounds: clip}) {
|
||||
t.Fatal("clipped viewport rejected by tree validation")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCaptureVisibleTextClippedSpanCrossesGeometryBlocks(t *testing.T) {
|
||||
source := clippedReviewText{rows: 1, columns: 400}
|
||||
got, err := captureVisibleText(context.Background(), source, Rectangle{X: 3, Width: 2594, Height: 12})
|
||||
want := VisibleText{Known: true, Ranges: []TextRange{
|
||||
{5, 265, strings.Repeat("é", 260), Rectangle{3, 0, 2594, 12}},
|
||||
}}
|
||||
if err != nil || !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("cross-block capture = %+v %v, want %+v", got, err, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCaptureVisibleTextClippedDisjointSpanLimit(t *testing.T) {
|
||||
count := (maxReviewRanges + 1) * 2
|
||||
source := fakeReviewText{characters: count, lines: map[int]TextRange{
|
||||
0: {Start: 0, End: count, Text: strings.Repeat("x", count)},
|
||||
}, rectangles: map[[2]int]Rectangle{{0, count}: {-10, 0, 30, 10}}}
|
||||
for i := 0; i < count; i++ {
|
||||
source.rectangles[[2]int{i, i + 1}] = Rectangle{X: int32(i % 2 * 20), Width: 10, Height: 10}
|
||||
}
|
||||
got, err := captureVisibleText(context.Background(), source, Rectangle{0, 0, 10, 10})
|
||||
if err != nil || !got.Limited || got.Known || len(got.Ranges) != 0 {
|
||||
t.Fatalf("disjoint span limit: %+v %v", got, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCaptureVisibleTextClippedBidiKeepsHiddenGaps(t *testing.T) {
|
||||
source := fakeReviewText{characters: 5, lines: map[int]TextRange{
|
||||
0: {Start: 0, End: 5, Text: "אבXגד"},
|
||||
}, rectangles: map[[2]int]Rectangle{
|
||||
{0, 5}: {-10, 0, 70, 10},
|
||||
{0, 1}: {30, 0, 10, 10}, {1, 2}: {20, 0, 10, 10},
|
||||
{2, 3}: {50, 0, 10, 10},
|
||||
{3, 4}: {10, 0, 10, 10}, {4, 5}: {0, 0, 10, 10},
|
||||
}}
|
||||
got, err := captureVisibleText(context.Background(), source, Rectangle{0, 0, 40, 10})
|
||||
want := VisibleText{Known: true, Ranges: []TextRange{
|
||||
{0, 2, "אב", Rectangle{20, 0, 20, 10}},
|
||||
{3, 5, "גד", Rectangle{0, 0, 20, 10}},
|
||||
}}
|
||||
if err != nil || !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("bidi capture = %+v %v, want %+v", got, err, want)
|
||||
}
|
||||
}
|
||||
|
||||
func BenchmarkCaptureVisibleTextClipped(b *testing.B) {
|
||||
source := clippedReviewText{rows: 4, columns: 100}
|
||||
clip := Rectangle{X: 3, Width: 394, Height: int32(source.rows * 12)}
|
||||
b.ReportAllocs()
|
||||
for b.Loop() {
|
||||
got, err := captureVisibleText(context.Background(), source, clip)
|
||||
if err != nil || !got.Known || got.Limited {
|
||||
b.Fatalf("clipped capture: known=%v limited=%v err=%v", got.Known, got.Limited, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func textNode() TreeNode {
|
||||
node := testNode("/root", "Text")
|
||||
node.Geometry = Geometry{Bounds: Rectangle{-10, 0, 20, 20}, BoundsKnown: true, VisibilityKnown: true, Visible: true, Showing: true}
|
||||
|
||||
@@ -0,0 +1,282 @@
|
||||
// Copyright (c) 2026 Storm Dragon
|
||||
// Authors: Storm Dragon, Angus Kola
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package atspi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sort"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
var errReviewViewportScanLimit = errors.New("review viewport scan limit")
|
||||
|
||||
type reviewSpanSource interface {
|
||||
bounds(context.Context, int, int) (Rectangle, error)
|
||||
text_span(context.Context, int, int) (string, error)
|
||||
}
|
||||
|
||||
type reviewPointSource interface {
|
||||
reviewSpanSource
|
||||
offset_at_point(context.Context, int32, int32) (int, error)
|
||||
}
|
||||
|
||||
// Walk visible glyph rectangles rather than logical offsets. Logical offsets
|
||||
// need not increase left-to-right (bidi), and a single line can be enormous.
|
||||
// Check every intervening pixel when a provider cannot identify a glyph. A
|
||||
// finite query budget rejects sparse or defective providers without omissions.
|
||||
func point_review_viewport(ctx context.Context, source reviewPointSource, start, end int, clip Rectangle) ([]TextRange, error) {
|
||||
if end <= start {
|
||||
return nil, nil
|
||||
}
|
||||
first, err := review_endpoint_bounds(ctx, source, start, end, false)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
last, err := review_endpoint_bounds(ctx, source, start, end, true)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
top := max(int64(clip.Y), int64(first.Y))
|
||||
bottom := min(int64(clip.Y)+int64(clip.Height), int64(last.Y)+int64(last.Height))
|
||||
right := int64(clip.X) + int64(clip.Width)
|
||||
offsets := make(map[int]Rectangle)
|
||||
queries := 0
|
||||
for y := top; y < bottom; {
|
||||
nextRow := bottom
|
||||
found := false
|
||||
rowCovered := true
|
||||
for x := int64(clip.X); x < right; {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if queries >= 4096 {
|
||||
return nil, errReviewViewportScanLimit
|
||||
}
|
||||
queries++
|
||||
offset, err := source.offset_at_point(ctx, int32(x), int32(y))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if offset < start || offset >= end {
|
||||
rowCovered = false
|
||||
x++
|
||||
continue
|
||||
}
|
||||
bounds, known := offsets[offset]
|
||||
if !known {
|
||||
bounds, err = source.bounds(ctx, offset, offset+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
if !bounds.Valid() || x < int64(bounds.X) || x >= int64(bounds.X)+int64(bounds.Width) || y < int64(bounds.Y) || y >= int64(bounds.Y)+int64(bounds.Height) {
|
||||
rowCovered = false
|
||||
x++
|
||||
continue
|
||||
}
|
||||
offsets[offset] = bounds
|
||||
found = true
|
||||
nextRow = min(nextRow, int64(bounds.Y)+int64(bounds.Height))
|
||||
x = min(right, int64(bounds.X)+int64(bounds.Width))
|
||||
}
|
||||
if found && rowCovered {
|
||||
y = nextRow
|
||||
} else {
|
||||
y++
|
||||
}
|
||||
}
|
||||
ordered := make([]int, 0, len(offsets))
|
||||
for offset := range offsets {
|
||||
ordered = append(ordered, offset)
|
||||
}
|
||||
sort.Ints(ordered)
|
||||
var ranges []TextRange
|
||||
for index := 0; index < len(ordered); {
|
||||
if len(ranges) >= maxReviewRanges {
|
||||
return nil, errors.New("review viewport range limit")
|
||||
}
|
||||
first := ordered[index]
|
||||
bounds := offsets[first]
|
||||
next := index + 1
|
||||
for next < len(ordered) && ordered[next] == ordered[next-1]+1 && next-index < maxReviewBoundsCharacters && offsets[ordered[next]].Y == bounds.Y && offsets[ordered[next]].Height == bounds.Height {
|
||||
next++
|
||||
}
|
||||
last := ordered[next-1] + 1
|
||||
content, err := source.text_span(ctx, first, last)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// A mark or joiner may have no point of its own. Retain a bounded
|
||||
// continuation of a visible base without including hidden letters.
|
||||
limit := min(end, last+maxReviewBoundsCharacters)
|
||||
if next < len(ordered) {
|
||||
limit = min(limit, ordered[next])
|
||||
}
|
||||
if last < limit {
|
||||
tail, err := source.text_span(ctx, last, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, character := range tail {
|
||||
if !review_cluster_continuation(character) {
|
||||
break
|
||||
}
|
||||
bounds, err := source.bounds(ctx, last, last+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, visible := bounds.Intersection(clip); bounds.Valid() && !visible {
|
||||
break
|
||||
}
|
||||
content += string(character)
|
||||
last++
|
||||
}
|
||||
if last == limit && limit < end && (next == len(ordered) || limit < ordered[next]) {
|
||||
following, err := source.text_span(ctx, last, last+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, character := range following {
|
||||
if review_cluster_continuation(character) {
|
||||
return nil, errReviewViewportScanLimit
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
ranges = append(ranges, TextRange{Start: first, End: last, Text: content})
|
||||
index = next
|
||||
}
|
||||
return ranges, ctx.Err()
|
||||
}
|
||||
|
||||
// Trailing newlines and formatting characters can have no glyph rectangle.
|
||||
// Look only within a bounded edge block, rather than discarding a long line
|
||||
// or scanning arbitrary offscreen text to find an endpoint.
|
||||
func review_endpoint_bounds(ctx context.Context, source reviewSpanSource, start, end int, backwards bool) (Rectangle, error) {
|
||||
for index := 0; index < min(end-start, maxReviewBoundsCharacters); index++ {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return Rectangle{}, err
|
||||
}
|
||||
offset := start + index
|
||||
if backwards {
|
||||
offset = end - index - 1
|
||||
}
|
||||
bounds, err := source.bounds(ctx, offset, offset+1)
|
||||
if err != nil {
|
||||
return Rectangle{}, err
|
||||
}
|
||||
if bounds.Valid() {
|
||||
return bounds, nil
|
||||
}
|
||||
}
|
||||
return Rectangle{}, errors.New("unavailable review viewport endpoint geometry")
|
||||
}
|
||||
|
||||
// Line rectangles can be usable when single-character extents are missing.
|
||||
// Move by complete line boundaries on each binary-search step; never scan
|
||||
// scrollback or accept a non-progressing/provider-clamped line as a match.
|
||||
func visible_line_offsets(ctx context.Context, source reviewTextSource, count int, clip Rectangle) (int, int, error) {
|
||||
if !clip.Valid() {
|
||||
return 0, 0, errors.New("invalid review text clip")
|
||||
}
|
||||
search := func(bottom bool) (int, error) {
|
||||
low, high := 0, count
|
||||
for low < high {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
middle := low + (high-low)/2
|
||||
line, err := source.line(ctx, middle)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if line.Start < 0 || line.Start > middle || line.End <= middle || line.End > count || line.End-line.Start > maxReviewLineCharacters {
|
||||
return 0, errors.New("invalid review viewport line offsets")
|
||||
}
|
||||
bounds, err := source.bounds(ctx, line.Start, line.End)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if !bounds.Valid() {
|
||||
return 0, errors.New("invalid review line bounds")
|
||||
}
|
||||
after := int64(bounds.Y)+int64(bounds.Height) > int64(clip.Y)
|
||||
if bottom {
|
||||
after = int64(bounds.Y) >= int64(clip.Y)+int64(clip.Height)
|
||||
}
|
||||
if after {
|
||||
high = max(low, line.Start)
|
||||
} else {
|
||||
low = min(high, line.End)
|
||||
}
|
||||
}
|
||||
return low, nil
|
||||
}
|
||||
start, err := search(false)
|
||||
if err != nil || start == count {
|
||||
return start, start, err
|
||||
}
|
||||
end, err := search(true)
|
||||
if err == nil && end < start {
|
||||
err = errors.New("invalid review line viewport order")
|
||||
}
|
||||
return start, end, err
|
||||
}
|
||||
|
||||
func capture_review_viewport(ctx context.Context, source reviewTextSource, count int, clip Rectangle, reason TextCaptureIssue) (VisibleText, error) {
|
||||
if spatial, ok := source.(reviewViewportSource); ok {
|
||||
ranges, err := spatial.viewport(ctx, 0, count, clip)
|
||||
if ctx.Err() != nil {
|
||||
return VisibleText{}, ctx.Err()
|
||||
}
|
||||
if err == nil && len(ranges) > 0 {
|
||||
return capture_review_lines(ctx, source, count, 0, count, clip, ranges)
|
||||
}
|
||||
if errors.Is(err, errReviewViewportScanLimit) {
|
||||
reason = TextViewportScanLimit
|
||||
}
|
||||
}
|
||||
return VisibleText{Limited: true, Issue: reason}, ctx.Err()
|
||||
}
|
||||
|
||||
// Validate all spatial output before accepting any of it. Native offsets can
|
||||
// be large, but the amount of copied text and the number of spans cannot.
|
||||
func validate_viewport_ranges(ranges []TextRange, count int) error {
|
||||
if len(ranges) > maxReviewRanges {
|
||||
return errors.New("review viewport range limit")
|
||||
}
|
||||
previous, characters := 0, 0
|
||||
for _, span := range ranges {
|
||||
if span.Start < previous || span.End <= span.Start || span.End > count || !utf8.ValidString(span.Text) || utf8.RuneCountInString(span.Text) != span.End-span.Start {
|
||||
return errors.New("invalid review viewport text")
|
||||
}
|
||||
characters += span.End - span.Start
|
||||
if characters > maxReviewCharacters {
|
||||
return errReviewLineTooLong
|
||||
}
|
||||
previous = span.End
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Provider spatial ranges need not stop at line breaks. Split without making
|
||||
// another whole-line native request; wrapping is handled by glyph geometry.
|
||||
func split_viewport_lines(ranges []TextRange) []TextRange {
|
||||
var lines []TextRange
|
||||
for _, span := range ranges {
|
||||
start := span.Start
|
||||
for _, content := range strings.SplitAfter(span.Text, "\n") {
|
||||
if content == "" {
|
||||
continue
|
||||
}
|
||||
end := start + utf8.RuneCountInString(content)
|
||||
lines = append(lines, TextRange{Start: start, End: end, Text: content})
|
||||
start = end
|
||||
}
|
||||
}
|
||||
return lines
|
||||
}
|
||||
@@ -0,0 +1,228 @@
|
||||
// Copyright (c) 2026 Storm Dragon
|
||||
// Authors: Storm Dragon, Angus Kola
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package atspi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type lineGeometryText struct {
|
||||
lineCalls int
|
||||
}
|
||||
|
||||
func (*lineGeometryText) count(context.Context) (int, error) { return 1000000, nil }
|
||||
func (source *lineGeometryText) line(ctx context.Context, offset int) (TextRange, error) {
|
||||
source.lineCalls++
|
||||
start := offset / 100 * 100
|
||||
return TextRange{Start: start, End: start + 100, Text: strings.Repeat("é", 100)}, ctx.Err()
|
||||
}
|
||||
func (*lineGeometryText) bounds(ctx context.Context, start, end int) (Rectangle, error) {
|
||||
if end-start == 1 {
|
||||
return Rectangle{}, errors.New("character rectangles unavailable")
|
||||
}
|
||||
return Rectangle{Y: int32(start/100-4200) * 10, Width: 100, Height: 10}, ctx.Err()
|
||||
}
|
||||
|
||||
func TestCaptureVisibleTextDiscoversLargeViewportFromLines(t *testing.T) {
|
||||
source := &lineGeometryText{}
|
||||
clip := Rectangle{Width: 100, Height: 20}
|
||||
got, err := captureVisibleText(context.Background(), source, clip)
|
||||
want := VisibleText{Known: true, Ranges: []TextRange{
|
||||
{420000, 420100, strings.Repeat("é", 100), Rectangle{Width: 100, Height: 10}},
|
||||
{420100, 420200, strings.Repeat("é", 100), Rectangle{Y: 10, Width: 100, Height: 10}},
|
||||
}}
|
||||
if err != nil || !reflect.DeepEqual(got, want) || source.lineCalls > 40 {
|
||||
t.Fatalf("line viewport = %+v, %v; calls=%d", got, err, source.lineCalls)
|
||||
}
|
||||
}
|
||||
|
||||
type spatialReviewText struct {
|
||||
clippedReviewText
|
||||
ranges []TextRange
|
||||
lineCalls, viewportCalls int
|
||||
cancel context.CancelFunc
|
||||
}
|
||||
|
||||
func (source *spatialReviewText) line(ctx context.Context, offset int) (TextRange, error) {
|
||||
source.lineCalls++
|
||||
return TextRange{}, errReviewLineTooLong
|
||||
}
|
||||
func (source *spatialReviewText) viewport(ctx context.Context, start, end int, clip Rectangle) ([]TextRange, error) {
|
||||
source.viewportCalls++
|
||||
if source.cancel != nil {
|
||||
source.cancel()
|
||||
}
|
||||
return source.ranges, ctx.Err()
|
||||
}
|
||||
|
||||
func TestCaptureVisibleTextRetrievesLongLineViewportWithoutWholeLine(t *testing.T) {
|
||||
source := &spatialReviewText{clippedReviewText: clippedReviewText{rows: 1, columns: 1000000}, ranges: []TextRange{
|
||||
{Start: 500005, End: 500045, Text: strings.Repeat("é", 40)},
|
||||
}}
|
||||
clip := Rectangle{X: 5000003, Width: 394, Height: 12}
|
||||
got, err := captureVisibleText(context.Background(), source, clip)
|
||||
want := VisibleText{Known: true, Ranges: []TextRange{{500005, 500045, strings.Repeat("é", 40), clip}}}
|
||||
if err != nil || !reflect.DeepEqual(got, want) || source.lineCalls != 0 || source.viewportCalls != 1 {
|
||||
t.Fatalf("spatial capture = %+v, %v; line=%d viewport=%d", got, err, source.lineCalls, source.viewportCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCaptureVisibleTextRejectsInvalidSpatialOutput(t *testing.T) {
|
||||
for _, ranges := range [][]TextRange{
|
||||
{{Start: 8, End: 10, Text: "x"}},
|
||||
{{Start: 8, End: 10, Text: "é猫"}, {Start: 7, End: 8, Text: "x"}},
|
||||
{{Start: 0, End: maxReviewCharacters + 1, Text: strings.Repeat("x", maxReviewCharacters+1)}},
|
||||
{{Start: 999999, End: 1000001, Text: "xx"}},
|
||||
} {
|
||||
source := &spatialReviewText{clippedReviewText: clippedReviewText{rows: 1, columns: 1000000}, ranges: ranges}
|
||||
got, err := captureVisibleText(context.Background(), source, Rectangle{Width: 100, Height: 12})
|
||||
if err == nil || got.Known || len(got.Ranges) != 0 {
|
||||
t.Fatalf("invalid spatial capture accepted: %+v %v", got, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCaptureVisibleTextSpatialCancellation(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
source := &spatialReviewText{clippedReviewText: clippedReviewText{rows: 1, columns: 1000000}, cancel: cancel}
|
||||
got, err := captureVisibleText(ctx, source, Rectangle{Width: 100, Height: 12})
|
||||
if !errors.Is(err, context.Canceled) || got.Known || got.Limited {
|
||||
t.Fatalf("spatial cancellation = %+v %v", got, err)
|
||||
}
|
||||
}
|
||||
|
||||
type pointReviewText struct {
|
||||
fakeReviewText
|
||||
pointCalls int
|
||||
textByOffset map[int]string
|
||||
}
|
||||
|
||||
func (source *pointReviewText) offset_at_point(ctx context.Context, x, y int32) (int, error) {
|
||||
source.pointCalls++
|
||||
for span, bounds := range source.rectangles {
|
||||
if span[1]-span[0] == 1 && int64(x) >= int64(bounds.X) && int64(x) < int64(bounds.X)+int64(bounds.Width) && int64(y) >= int64(bounds.Y) && int64(y) < int64(bounds.Y)+int64(bounds.Height) {
|
||||
return span[0], ctx.Err()
|
||||
}
|
||||
}
|
||||
return -1, ctx.Err()
|
||||
}
|
||||
func (source *pointReviewText) text_span(ctx context.Context, start, end int) (string, error) {
|
||||
var text strings.Builder
|
||||
for offset := start; offset < end; offset++ {
|
||||
character, ok := source.textByOffset[offset]
|
||||
if !ok {
|
||||
character = "x"
|
||||
}
|
||||
text.WriteString(character)
|
||||
}
|
||||
return text.String(), ctx.Err()
|
||||
}
|
||||
|
||||
func TestPointReviewViewportKeepsDisjointBidiOffsets(t *testing.T) {
|
||||
source := &pointReviewText{fakeReviewText: fakeReviewText{rectangles: map[[2]int]Rectangle{
|
||||
{0, 1}: {-100, 0, 10, 10}, {99999, 100000}: {100, 0, 10, 10},
|
||||
{10, 11}: {30, 0, 10, 10}, {70000, 70001}: {10, 0, 10, 10}, {80000, 80001}: {20, 0, 10, 10},
|
||||
}}, textByOffset: map[int]string{10: "א", 70000: "猫", 80000: "é"}}
|
||||
ranges, err := point_review_viewport(context.Background(), source, 0, 100000, Rectangle{X: 10, Width: 30, Height: 10})
|
||||
if err != nil || len(ranges) != 3 || source.pointCalls != 3 {
|
||||
t.Fatalf("point bidi = %+v, %v; calls=%d", ranges, err, source.pointCalls)
|
||||
}
|
||||
for index, offset := range []int{10, 70000, 80000} {
|
||||
if ranges[index].Start != offset || ranges[index].Text != source.textByOffset[offset] {
|
||||
t.Fatalf("point bidi lost native offset: %+v", ranges)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPointReviewViewportDoesNotSkipVerticallyInsetGlyph(t *testing.T) {
|
||||
source := &pointReviewText{fakeReviewText: fakeReviewText{rectangles: map[[2]int]Rectangle{
|
||||
{0, 1}: {0, 0, 10, 20}, {1, 2}: {10, 5, 10, 10}, {2, 3}: {20, 0, 10, 20},
|
||||
}}, textByOffset: map[int]string{0: "A", 1: "é", 2: "B"}}
|
||||
ranges, err := point_review_viewport(context.Background(), source, 0, 3, Rectangle{Width: 30, Height: 20})
|
||||
if err != nil || len(ranges) != 3 || ranges[1].Text != "é" {
|
||||
t.Fatalf("inset glyph omitted: %+v %v", ranges, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPointReviewViewportBoundsSparseProviderWork(t *testing.T) {
|
||||
source := &pointReviewText{fakeReviewText: fakeReviewText{rectangles: map[[2]int]Rectangle{
|
||||
{0, 1}: {-100, 0, 10, 100}, {99999, 100000}: {1000, 0, 10, 100},
|
||||
}}}
|
||||
ranges, err := point_review_viewport(context.Background(), source, 0, 100000, Rectangle{Width: 100, Height: 100})
|
||||
if !errors.Is(err, errReviewViewportScanLimit) || len(ranges) != 0 || source.pointCalls != 4096 {
|
||||
t.Fatalf("sparse provider work: ranges=%d calls=%d err=%v", len(ranges), source.pointCalls, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPointReviewViewportRetainsLineWithZeroAreaTrailingNewline(t *testing.T) {
|
||||
source := &pointReviewText{fakeReviewText: fakeReviewText{rectangles: map[[2]int]Rectangle{
|
||||
{0, 1}: {Width: 10, Height: 10}, {1, 2}: {X: 10, Width: 10, Height: 10}, {2, 3}: {},
|
||||
}}, textByOffset: map[int]string{0: "é", 1: "猫", 2: "\n"}}
|
||||
ranges, err := point_review_viewport(context.Background(), source, 0, 3, Rectangle{Width: 20, Height: 20})
|
||||
if err != nil || len(ranges) != 1 || ranges[0].Text != "é猫" || ranges[0].End != 2 {
|
||||
t.Fatalf("trailing newline discarded visible glyphs: %+v %v", ranges, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCaptureVisibleTextSkipsZeroAreaGlyphsWithoutLosingOtherText(t *testing.T) {
|
||||
source := fakeReviewText{characters: 6, lines: map[int]TextRange{
|
||||
0: {Start: 0, End: 6, Text: "abcXYZ"},
|
||||
}, rectangles: map[[2]int]Rectangle{
|
||||
{0, 6}: {Width: 60, Height: 10},
|
||||
{0, 1}: {Width: 10, Height: 10}, {1, 2}: {X: 10, Width: 10, Height: 10},
|
||||
{2, 3}: {}, {3, 4}: {X: 20, Width: 10, Height: 10},
|
||||
{4, 5}: {X: 40, Width: 10, Height: 10}, {5, 6}: {X: 50, Width: 10, Height: 10},
|
||||
}}
|
||||
got, err := captureVisibleText(context.Background(), source, Rectangle{Width: 30, Height: 10})
|
||||
want := VisibleText{Known: true, Ranges: []TextRange{
|
||||
{0, 2, "ab", Rectangle{Width: 20, Height: 10}},
|
||||
{3, 4, "X", Rectangle{X: 20, Width: 10, Height: 10}},
|
||||
}}
|
||||
if err != nil || !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("zero-area glyph capture = %+v, %v; want %+v", got, err, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCaptureVisibleTextRetainsZeroAreaCombiningMark(t *testing.T) {
|
||||
source := fakeReviewText{characters: 4, lines: map[int]TextRange{
|
||||
0: {Start: 0, End: 4, Text: "a\u0301bc"},
|
||||
}, rectangles: map[[2]int]Rectangle{
|
||||
{0, 4}: {Width: 40, Height: 10}, {0, 1}: {Width: 10, Height: 10},
|
||||
{1, 2}: {}, {2, 3}: {X: 10, Width: 10, Height: 10}, {3, 4}: {X: 30, Width: 10, Height: 10},
|
||||
}}
|
||||
got, err := captureVisibleText(context.Background(), source, Rectangle{Width: 20, Height: 10})
|
||||
if err != nil || !got.Known || len(got.Ranges) != 1 || got.Ranges[0].Text != "a\u0301b" || got.Ranges[0].End != 3 {
|
||||
t.Fatalf("visible accent lost: %+v %v", got, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPointReviewViewportRetainsZeroAreaCombiningMark(t *testing.T) {
|
||||
source := &pointReviewText{fakeReviewText: fakeReviewText{rectangles: map[[2]int]Rectangle{
|
||||
{0, 1}: {Width: 10, Height: 10}, {1, 2}: {}, {2, 3}: {X: 10, Width: 10, Height: 10},
|
||||
}}, textByOffset: map[int]string{0: "a", 1: "\u0301", 2: "b"}}
|
||||
ranges, err := point_review_viewport(context.Background(), source, 0, 3, Rectangle{Width: 20, Height: 10})
|
||||
if err != nil || len(ranges) != 2 || ranges[0].Text != "a\u0301" || ranges[0].End != 2 || ranges[1].Text != "b" {
|
||||
t.Fatalf("point traversal lost accent: %+v %v", ranges, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPointReviewViewportRejectsExcessiveClusterWithoutPartialText(t *testing.T) {
|
||||
const finalOffset = maxReviewBoundsCharacters + 2
|
||||
source := &pointReviewText{fakeReviewText: fakeReviewText{rectangles: map[[2]int]Rectangle{
|
||||
{0, 1}: {Width: 10, Height: 10}, {finalOffset, finalOffset + 1}: {X: 10, Width: 10, Height: 10},
|
||||
}}, textByOffset: map[int]string{0: "a", finalOffset: "b"}}
|
||||
for offset := 1; offset < finalOffset; offset++ {
|
||||
source.textByOffset[offset] = "\u0301"
|
||||
}
|
||||
ranges, err := point_review_viewport(context.Background(), source, 0, finalOffset+1, Rectangle{Width: 20, Height: 10})
|
||||
if !errors.Is(err, errReviewViewportScanLimit) || len(ranges) != 0 {
|
||||
t.Fatalf("cluster limit silently omitted text: %+v %v", ranges, err)
|
||||
}
|
||||
}
|
||||
+15
-12
@@ -45,6 +45,8 @@ type TreeNode struct {
|
||||
Text VisibleText
|
||||
Snapshot Snapshot
|
||||
Children []ObjectID
|
||||
// Review-only capture could not enumerate part of this object's children.
|
||||
ChildrenUnavailable bool
|
||||
}
|
||||
|
||||
// ObjectReference is a generation-checked reference to an owned snapshot. A
|
||||
@@ -64,17 +66,18 @@ func (reference ObjectReference) Valid() bool {
|
||||
}
|
||||
|
||||
type treeEntry struct {
|
||||
snapshotInvalid bool
|
||||
text VisibleText
|
||||
textInvalid bool
|
||||
geometry Geometry
|
||||
geometryInvalid bool
|
||||
childrenInvalid bool
|
||||
snapshot Snapshot
|
||||
children []ObjectID
|
||||
parent ObjectID
|
||||
hasParent bool
|
||||
generation uint64
|
||||
snapshotInvalid bool
|
||||
text VisibleText
|
||||
textInvalid bool
|
||||
geometry Geometry
|
||||
geometryInvalid bool
|
||||
childrenInvalid bool
|
||||
childrenUnavailable bool
|
||||
snapshot Snapshot
|
||||
children []ObjectID
|
||||
parent ObjectID
|
||||
hasParent bool
|
||||
generation uint64
|
||||
}
|
||||
|
||||
// AccessibleTree owns a validated accessibility tree. It contains no native
|
||||
@@ -485,7 +488,7 @@ func validateTree(root ObjectID, nodes []TreeNode) (map[ObjectID]treeEntry, erro
|
||||
text := node.Text
|
||||
text.Ranges = append([]TextRange(nil), text.Ranges...)
|
||||
children := append([]ObjectID(nil), node.Children...)
|
||||
entries[id] = treeEntry{snapshot: node.Snapshot, children: children, geometry: node.Geometry, text: text}
|
||||
entries[id] = treeEntry{snapshot: node.Snapshot, children: children, geometry: node.Geometry, text: text, childrenUnavailable: node.ChildrenUnavailable}
|
||||
}
|
||||
if _, ok := entries[root]; !ok {
|
||||
return nil, fmt.Errorf("%w: root is missing", ErrInvalidTree)
|
||||
|
||||
@@ -56,6 +56,8 @@ func CommandLabel(command Command) string {
|
||||
return "List " + strings.ToLower(binding.Title)
|
||||
}
|
||||
switch command {
|
||||
case PresentCurrentLine:
|
||||
return "Read the current line"
|
||||
case PresentWindowTitle:
|
||||
return "Read the current window title"
|
||||
case PresentCellFormula:
|
||||
|
||||
@@ -14,6 +14,9 @@ const (
|
||||
PresentTime Command = "present-time"
|
||||
PresentDate Command = "present-date"
|
||||
PresentClipboard Command = "present-clipboard"
|
||||
PresentCurrentLine Command = "present-current-line"
|
||||
SpellCurrentLine Command = "spell-current-line"
|
||||
PhoneticCurrentLine Command = "phonetic-current-line"
|
||||
ShowSpeechHistory Command = "show-speech-history"
|
||||
ToggleSpeechMonitor Command = "toggle-speech-monitor"
|
||||
CycleSpeechMode Command = "cycle-speech-mode"
|
||||
@@ -204,6 +207,7 @@ func DefaultCommandBindings(layout KeyboardLayout) CommandBindings {
|
||||
{Symbol: rightKeySymbol, CthulhuModifier: true}: SelectNextSpeechSetting,
|
||||
{Symbol: downKeySymbol, CthulhuModifier: true}: DecreaseSpeechSetting,
|
||||
{Symbol: upKeySymbol, CthulhuModifier: true}: IncreaseSpeechSetting,
|
||||
{Symbol: upKeySymbol, Modifiers: 1, CthulhuModifier: true}: PresentCurrentLine,
|
||||
{Symbol: 's', Modifiers: controlModifier | 1, CthulhuModifier: true}: SaveSpeechSettings,
|
||||
{Symbol: 't', CthulhuModifier: true}: PresentTime,
|
||||
{Symbol: 'c', Modifiers: 1, CthulhuModifier: true}: PresentClipboard,
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
package input
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestCurrentLineBindingAndPresses(t *testing.T) {
|
||||
for _, layout := range []KeyboardLayout{DesktopKeyboardLayout, LaptopKeyboardLayout} {
|
||||
for _, bindings := range []CommandBindings{DefaultCommandBindings(layout), BrowseCommandBindings(layout)} {
|
||||
router := NewCommandRouter([]uint32{0xff63}, bindings)
|
||||
now := time.Unix(100, 0)
|
||||
router.now = func() time.Time { return now }
|
||||
router.Handle(0xff63, 118, 0, true)
|
||||
press := func(modifiers uint16, want Command) {
|
||||
t.Helper()
|
||||
if consumed, got := router.Handle(upKeySymbol, 111, modifiers, true); !consumed || got != want {
|
||||
t.Fatalf("layout %d: consumed=%v command=%s want=%s", layout, consumed, got, want)
|
||||
}
|
||||
if consumed, got := router.Handle(upKeySymbol, 111, modifiers, true); !consumed || got != "" {
|
||||
t.Fatal("held arrow repeated command")
|
||||
}
|
||||
router.Handle(upKeySymbol, 111, modifiers, false)
|
||||
}
|
||||
press(1, PresentCurrentLine)
|
||||
press(1, SpellCurrentLine)
|
||||
press(1, PhoneticCurrentLine)
|
||||
press(1, PresentCurrentLine)
|
||||
now = now.Add(501 * time.Millisecond)
|
||||
press(1, PresentCurrentLine)
|
||||
press(0, IncreaseSpeechSetting)
|
||||
press(1, PresentCurrentLine)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -10,7 +10,7 @@ import "time"
|
||||
// updates. Another key/chord, timeout, or the final press retires it. Held-key
|
||||
// repeats never reach this helper. Reset only clears held keys, as for time/date.
|
||||
func (router *CommandRouter) review_press(command Command, chord CommandChord) Command {
|
||||
if command != ReviewCurrentCharacter && command != ReviewCurrentWord && command != ReviewCurrentLine {
|
||||
if command != ReviewCurrentCharacter && command != ReviewCurrentWord && command != ReviewCurrentLine && command != PresentCurrentLine {
|
||||
router.lastReviewPress = time.Time{}
|
||||
return command
|
||||
}
|
||||
@@ -44,5 +44,14 @@ func (router *CommandRouter) review_press(command Command, chord CommandChord) C
|
||||
return ReviewPhoneticLine
|
||||
}
|
||||
}
|
||||
if command == PresentCurrentLine {
|
||||
switch router.reviewPresses {
|
||||
case 2:
|
||||
return SpellCurrentLine
|
||||
case 3:
|
||||
router.lastReviewPress = time.Time{}
|
||||
return PhoneticCurrentLine
|
||||
}
|
||||
}
|
||||
return command
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
// Copyright (c) 2026 Storm Dragon
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package review
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
"git.stormux.org/storm/cthulhu/internal/atspi"
|
||||
)
|
||||
|
||||
// ReadCurrentLine leaves both the native caret and review cursors untouched.
|
||||
func (worker *Worker) ReadCurrentLine(browse bool, style FlatSpeech) error {
|
||||
if style > FlatPhonetic {
|
||||
return ErrInvalidDirection
|
||||
}
|
||||
select {
|
||||
case worker.requests <- request{currentLine: true, currentLineBrowse: browse, flatSpeech: style,
|
||||
documentEpoch: worker.documentEpoch.Load(), queuedAt: time.Now()}:
|
||||
return nil
|
||||
default:
|
||||
return ErrRequestQueueFull
|
||||
}
|
||||
}
|
||||
|
||||
func (worker *Worker) read_current_line(ctx context.Context, cursor *DocumentCursor, browse bool) Result {
|
||||
result := Result{CurrentLine: true, CurrentLineBrowse: browse, DocumentFocus: worker.focus()}
|
||||
if !result.DocumentFocus.Valid() {
|
||||
result.Err = ErrNoCurrent
|
||||
return result
|
||||
}
|
||||
deadline, cancel := context.WithTimeout(ctx, worker.captureTimeout)
|
||||
defer cancel()
|
||||
if !browse {
|
||||
reader, ok := worker.nativeCapturerFor(result.DocumentFocus).(interface {
|
||||
ReadFocusedLine(context.Context, atspi.ObjectID) (atspi.DocumentContent, error)
|
||||
})
|
||||
if !ok {
|
||||
result.Err = atspi.ErrNoDocumentCaret
|
||||
return result
|
||||
}
|
||||
content, err := reader.ReadFocusedLine(deadline, result.DocumentFocus)
|
||||
result.Err = err
|
||||
result.DocumentCaretText, result.DocumentCaretRawText = content.Text, content.RawText
|
||||
result.DocumentPosition.Snapshot = content.Snapshot
|
||||
return result
|
||||
}
|
||||
// Start a private cursor at the saved browse position. Going to line start
|
||||
// before reading avoids Say All's deliberate "rest of this line" behavior.
|
||||
if cursor != nil {
|
||||
cursor = &DocumentCursor{tree: cursor.tree, active: cursor.active, places: append([]documentPlace(nil), cursor.places...)}
|
||||
_, result.Err = cursor.syncActionFocus(deadline, result.DocumentFocus)
|
||||
} else {
|
||||
result.Err = ErrNoCurrent
|
||||
}
|
||||
if documentNeedsCapture(result.Err) {
|
||||
cursor, result.Err = worker.refreshDocumentCursor(deadline, cursor, result.DocumentFocus)
|
||||
}
|
||||
if result.Err != nil {
|
||||
return result
|
||||
}
|
||||
reader, ok := worker.nativeCapturerFor(result.DocumentFocus).(DocumentCaretReader)
|
||||
if !ok {
|
||||
result.Err = atspi.ErrNoDocumentCaret
|
||||
return result
|
||||
}
|
||||
_, cursor, result.Err = worker.readAllMove(deadline, cursor, reader, CaretLineStart, result.DocumentFocus)
|
||||
if result.Err != nil {
|
||||
return result
|
||||
}
|
||||
movement, _, err := worker.readAllMove(deadline, cursor, reader, CaretReadLine, result.DocumentFocus)
|
||||
result.Err = err
|
||||
result.DocumentPosition, result.DocumentCaretText = movement.Position, movement.Text
|
||||
result.DocumentCaretRawText, result.DocumentCaretLine = movement.RawText, movement.Line
|
||||
if result.DocumentCaretText == "" && wholeCaretNode(movement.Position.Snapshot) {
|
||||
result.DocumentCaretText = movement.Position.Snapshot.Name
|
||||
}
|
||||
return result
|
||||
}
|
||||
@@ -0,0 +1,109 @@
|
||||
package review
|
||||
|
||||
import (
|
||||
"context"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.stormux.org/storm/cthulhu/internal/atspi"
|
||||
)
|
||||
|
||||
type currentLineCapturer struct {
|
||||
*readAllCapturer
|
||||
content atspi.DocumentContent
|
||||
reads int
|
||||
}
|
||||
|
||||
func (capturer *currentLineCapturer) ReadFocusedLine(context.Context, atspi.ObjectID) (atspi.DocumentContent, error) {
|
||||
capturer.reads++
|
||||
return capturer.content, nil
|
||||
}
|
||||
|
||||
func TestCurrentLineUsesNativeFocusOutsideBrowse(t *testing.T) {
|
||||
capturer := ¤tLineCapturer{content: atspi.DocumentContent{Text: "Native caret line"}}
|
||||
worker, cancel := startWorker(t, capturer, time.Second)
|
||||
defer cancel()
|
||||
worker.SyncFocus(webID("/entry"))
|
||||
for _, style := range []FlatSpeech{FlatSpeak, FlatSpell, FlatPhonetic} {
|
||||
if err := worker.ReadCurrentLine(false, style); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
result := waitResult(t, worker)
|
||||
if !result.CurrentLine || result.CurrentLineBrowse || result.FlatSpeech != style || result.Err != nil || result.DocumentCaretText != "Native caret line" {
|
||||
t.Fatalf("native current line: %+v", result)
|
||||
}
|
||||
}
|
||||
if capturer.reads != 3 {
|
||||
t.Fatal("native line was cached")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCurrentLineReadsWholeBrowseLineWithoutMoving(t *testing.T) {
|
||||
reader := caretTextReader{webID("/text"): "First line\nSecond café 猫\nThird", webID("/next"): "Next"}
|
||||
cursor := newCaretTextCursor(t, reader)
|
||||
capturer := &readAllCapturer{tree: cursor.tree, caretTextReader: reader}
|
||||
worker, _ := NewWorker(capturer, time.Second)
|
||||
worker.SyncFocus(webID("/text"))
|
||||
for range 15 {
|
||||
if _, err := cursor.MoveCaret(context.Background(), reader, CaretNextCharacter); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
before := append([]documentPlace(nil), cursor.places...)
|
||||
result := worker.read_current_line(context.Background(), cursor, true)
|
||||
if result.Err != nil || strings.TrimSpace(result.DocumentCaretText) != "Second café 猫" {
|
||||
t.Fatalf("whole browse line from middle: %+v", result)
|
||||
}
|
||||
if !reflect.DeepEqual(cursor.places, before) || capturer.nativePlacements != 0 {
|
||||
t.Fatal("report changed browse or native caret")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCurrentLineJoinsBrowseFragmentsWithoutMoving(t *testing.T) {
|
||||
cursor, reader := fragment_cursor(t, "paragraph")
|
||||
if _, err := cursor.MoveCaret(context.Background(), reader, CaretNextWord); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
capturer := &readAllCapturer{tree: cursor.tree, caretTextReader: reader.caretTextReader}
|
||||
// Retain the fixture's geometry so the report can collect both fragments.
|
||||
source := ¤tLineFragmentCapturer{readAllCapturer: capturer, fragmentTextReader: reader}
|
||||
worker, _ := NewWorker(source, time.Second)
|
||||
worker.SyncFocus(webID("/one"))
|
||||
before := append([]documentPlace(nil), cursor.places...)
|
||||
result := worker.read_current_line(context.Background(), cursor, true)
|
||||
if result.Err != nil || result.DocumentCaretText != "One two." || !reflect.DeepEqual(cursor.places, before) {
|
||||
t.Fatalf("fragmented current line: %+v", result)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCurrentLineReadsBrowseControlLabel(t *testing.T) {
|
||||
doc := atspi.Snapshot{BusName: webID("/doc").BusName, Path: "/doc", Role: "document web"}
|
||||
button := atspi.Snapshot{BusName: doc.BusName, Path: "/button", Role: "push button", Name: "Example button"}
|
||||
tree, err := atspi.NewAccessibleTree(doc.ID(), []atspi.TreeNode{
|
||||
{Snapshot: doc, Children: []atspi.ObjectID{button.ID()}}, {Snapshot: button},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cursor, _ := NewDocumentCursor(tree)
|
||||
if _, err := cursor.SyncFocus(context.Background(), button.ID()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
worker, _ := NewWorker(&readAllCapturer{tree: tree, caretTextReader: caretTextReader{}}, time.Second)
|
||||
worker.SyncFocus(button.ID())
|
||||
result := worker.read_current_line(context.Background(), cursor, true)
|
||||
if result.Err != nil || result.DocumentCaretText != "Example button" {
|
||||
t.Fatalf("browse control label: %+v", result)
|
||||
}
|
||||
}
|
||||
|
||||
type currentLineFragmentCapturer struct {
|
||||
*readAllCapturer
|
||||
fragmentTextReader
|
||||
}
|
||||
|
||||
func (capturer *currentLineFragmentCapturer) ReadDocumentCaret(ctx context.Context, ref atspi.ObjectReference, offset int, unit atspi.DocumentCaretUnit) (atspi.DocumentCaretText, error) {
|
||||
return capturer.fragmentTextReader.ReadDocumentCaret(ctx, ref, offset, unit)
|
||||
}
|
||||
+22
-3
@@ -67,6 +67,8 @@ type FlatCursor struct {
|
||||
verticalTargetX int64
|
||||
hasVerticalTarget bool
|
||||
started bool
|
||||
completeness FlatCompleteness
|
||||
warnedIncomplete bool
|
||||
}
|
||||
|
||||
func (cursor *FlatCursor) Refresh(ctx context.Context, tree *atspi.AccessibleTree, focus atspi.ObjectID) error {
|
||||
@@ -78,11 +80,11 @@ func (cursor *FlatCursor) Refresh(ctx context.Context, tree *atspi.AccessibleTre
|
||||
return err
|
||||
}
|
||||
nodes := snapshot.Nodes
|
||||
lines, err := buildFlatLines(ctx, nodes)
|
||||
lines, completeness, err := build_flat_layout(ctx, nodes)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
next := FlatCursor{lines: lines, focus: focus, tree: tree, references: snapshot.References,
|
||||
next := FlatCursor{lines: lines, focus: focus, tree: tree, references: snapshot.References, completeness: completeness,
|
||||
parents: make(map[atspi.ObjectID]atspi.ObjectID, len(nodes))}
|
||||
for _, n := range nodes {
|
||||
for _, child := range n.Children {
|
||||
@@ -91,7 +93,9 @@ func (cursor *FlatCursor) Refresh(ctx context.Context, tree *atspi.AccessibleTre
|
||||
}
|
||||
// Exact identity takes priority over a displayed descendant of focus.
|
||||
if !next.seek(focus, 0, false) {
|
||||
next.seekDescendant(focus)
|
||||
if !next.seekDescendant(focus) {
|
||||
next.seek_ancestor(focus)
|
||||
}
|
||||
}
|
||||
for _, n := range nodes {
|
||||
if n.Snapshot.ID() == focus && n.Text.CaretKnown {
|
||||
@@ -109,10 +113,25 @@ func (cursor *FlatCursor) Refresh(ctx context.Context, tree *atspi.AccessibleTre
|
||||
}
|
||||
}
|
||||
next.started = cursor.started && cursor.focus == focus
|
||||
next.warnedIncomplete = cursor.warnedIncomplete && cursor.focus == focus && cursor.tree == tree && cursor.completeness == completeness
|
||||
*cursor = next
|
||||
return nil
|
||||
}
|
||||
|
||||
// A focused leaf can have no displayed zone while its parent supplies the
|
||||
// text. Choose only an actual ancestor zone, never a sibling or window title.
|
||||
func (cursor *FlatCursor) seek_ancestor(object atspi.ObjectID) bool {
|
||||
for id := cursor.parents[object]; id.Valid(); id = cursor.parents[id] {
|
||||
if !cursor.parents[id].Valid() {
|
||||
return false
|
||||
}
|
||||
if cursor.seek(id, 0, false) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (cursor *FlatCursor) seekDescendant(object atspi.ObjectID) bool {
|
||||
for li, line := range cursor.lines {
|
||||
for zi, zone := range line.zones {
|
||||
|
||||
@@ -28,10 +28,22 @@ import (
|
||||
)
|
||||
|
||||
var (
|
||||
ErrNoReviewContent = errors.New("no visible content for flat review")
|
||||
ErrLimitedReview = errors.New("visible text exceeds flat review capture limits")
|
||||
ErrNoReviewContent = errors.New("no visible content for flat review")
|
||||
ErrLimitedReview = errors.New("visible text exceeds flat review capture limits")
|
||||
ErrUnavailableReview = errors.New("visible review content is unavailable")
|
||||
)
|
||||
|
||||
// FlatCompleteness reports known omissions, rather than treating a partial
|
||||
// layout as a complete representation of the window.
|
||||
type FlatCompleteness struct {
|
||||
OmittedObjects int
|
||||
UnavailableSubtrees int
|
||||
}
|
||||
|
||||
func (status FlatCompleteness) Incomplete() bool {
|
||||
return status.OmittedObjects != 0 || status.UnavailableSubtrees != 0
|
||||
}
|
||||
|
||||
const (
|
||||
maxFlatZones = 16384
|
||||
maxFlatTextBytes = 1 << 20
|
||||
@@ -56,12 +68,27 @@ type flatLine struct{ zones []flatZone }
|
||||
// vertical extents and centres within five pixels. Compare to the first zone
|
||||
// so a staircase of overlapping objects cannot merge separate text rows.
|
||||
func buildFlatLines(ctx context.Context, nodes []atspi.TreeNode) ([]flatLine, error) {
|
||||
lines, _, err := build_flat_layout(ctx, nodes)
|
||||
return lines, err
|
||||
}
|
||||
|
||||
func build_flat_layout(ctx context.Context, nodes []atspi.TreeNode) ([]flatLine, FlatCompleteness, error) {
|
||||
var status FlatCompleteness
|
||||
parents, nodeClips, hidden := flat_node_visibility(nodes)
|
||||
zones, err := collect_flat_zones(ctx, nodes, parents, nodeClips, hidden)
|
||||
zones, err := collect_flat_zones(ctx, nodes, parents, nodeClips, hidden, &status)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
return nil, status, err
|
||||
}
|
||||
return arrange_flat_lines(ctx, nodes, zones)
|
||||
if len(zones) == 0 && status.Incomplete() {
|
||||
for _, node := range nodes {
|
||||
if node.Text.Limited && !hidden[node.Snapshot.ID()] {
|
||||
return nil, status, ErrLimitedReview
|
||||
}
|
||||
}
|
||||
return nil, status, ErrUnavailableReview
|
||||
}
|
||||
lines, err := arrange_flat_lines(ctx, nodes, zones)
|
||||
return lines, status, err
|
||||
}
|
||||
|
||||
func sameFlatLine(a, b flatZone) bool {
|
||||
@@ -119,7 +146,7 @@ func flat_node_visibility(nodes []atspi.TreeNode) (map[atspi.ObjectID]atspi.Snap
|
||||
|
||||
// collect_flat_zones walks children before parents to avoid repeating container
|
||||
// names and text. Password contents and partly clipped native runs stay excluded.
|
||||
func collect_flat_zones(ctx context.Context, nodes []atspi.TreeNode, parents map[atspi.ObjectID]atspi.Snapshot, nodeClips map[atspi.ObjectID]atspi.Rectangle, hidden map[atspi.ObjectID]bool) ([]flatZone, error) {
|
||||
func collect_flat_zones(ctx context.Context, nodes []atspi.TreeNode, parents map[atspi.ObjectID]atspi.Snapshot, nodeClips map[atspi.ObjectID]atspi.Rectangle, hidden map[atspi.ObjectID]bool, status *FlatCompleteness) ([]flatZone, error) {
|
||||
var zones []flatZone
|
||||
textBytes := 0
|
||||
hasContent := make(map[atspi.ObjectID]bool, len(nodes))
|
||||
@@ -135,11 +162,20 @@ func collect_flat_zones(ctx context.Context, nodes []atspi.TreeNode, parents map
|
||||
if hasContent[id] {
|
||||
hasContent[parent.ID()] = true
|
||||
}
|
||||
if !n.Geometry.BoundsKnown || hidden[id] {
|
||||
if hidden[id] {
|
||||
continue
|
||||
}
|
||||
if n.Text.Limited {
|
||||
return nil, ErrLimitedReview
|
||||
if n.ChildrenUnavailable {
|
||||
status.UnavailableSubtrees++
|
||||
hasContent[parent.ID()] = true
|
||||
}
|
||||
if !n.Geometry.BoundsKnown {
|
||||
continue
|
||||
}
|
||||
if n.Text.Limited || n.Text.Issue != "" {
|
||||
status.OmittedObjects++
|
||||
hasContent[parent.ID()] = true
|
||||
continue
|
||||
}
|
||||
base := flatZone{object: id, parent: parent.ID(), role: n.Snapshot.Role, parentRole: parent.Role, bounds: nodeClips[id]}
|
||||
before := len(zones)
|
||||
@@ -198,7 +234,7 @@ func collect_flat_zones(ctx context.Context, nodes []atspi.TreeNode, parents map
|
||||
zones = append(zones, z)
|
||||
}
|
||||
}
|
||||
if len(zones) == before && !hasContent[id] {
|
||||
if len(zones) == before && !hasContent[id] && !n.ChildrenUnavailable {
|
||||
// Never derive review text from focus-speech Text, ValueText, or
|
||||
// password contents. Names are synthetic and have no text offset.
|
||||
base.text = n.Snapshot.Name
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
// Copyright (c) 2026 Storm Dragon
|
||||
// Authors: Storm Dragon, Angus Kola
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package review
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.stormux.org/storm/cthulhu/internal/atspi"
|
||||
)
|
||||
|
||||
func TestFlatReviewRetainsHealthyContentAndReportsOmissions(t *testing.T) {
|
||||
for _, issue := range []atspi.VisibleText{
|
||||
{Limited: true, Issue: atspi.TextCharacterLimit},
|
||||
{Issue: atspi.TextProviderFailure},
|
||||
} {
|
||||
bad := flatNode("/bad", "", 0, 30)
|
||||
bad.Snapshot.Name = "Unavailable object name must not replace its text"
|
||||
bad.Text = issue
|
||||
good := flatNode("/good", "Readable content", 0, 0)
|
||||
cursor := &FlatCursor{}
|
||||
if err := cursor.Refresh(context.Background(), flatTree(t, good, bad), objectID("/good")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
position, _, err := cursor.Read(Line, 0)
|
||||
if err != nil || position.Text != "Readable content" || cursor.completeness.OmittedObjects != 1 {
|
||||
t.Fatalf("partial review = %+v, %+v, %v", position, cursor.completeness, err)
|
||||
}
|
||||
if _, moved, err := cursor.Read(Line, 1); err != nil || moved {
|
||||
t.Fatalf("omitted object's name became content: moved=%v err=%v", moved, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFlatReviewReportsUnavailableChildrenWithoutDroppingParentText(t *testing.T) {
|
||||
parent := flatNode("/parent", "Parent text", 0, 0)
|
||||
parent.ChildrenUnavailable = true
|
||||
cursor := &FlatCursor{}
|
||||
if err := cursor.Refresh(context.Background(), flatTree(t, parent), parent.Snapshot.ID()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
position, _, err := cursor.Read(Line, 0)
|
||||
if err != nil || position.Text != "Parent text" || cursor.completeness.UnavailableSubtrees != 1 {
|
||||
t.Fatalf("partial subtree = %+v, %+v, %v", position, cursor.completeness, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFlatReviewReportsUnavailableChildrenWithoutComponentBounds(t *testing.T) {
|
||||
parent := node("/parent", "")
|
||||
parent.Geometry = atspi.Geometry{VisibilityKnown: true, Visible: true, Showing: true}
|
||||
parent.ChildrenUnavailable = true
|
||||
good := flatNode("/good", "Readable", 0, 0)
|
||||
cursor := &FlatCursor{}
|
||||
if err := cursor.Refresh(context.Background(), flatTree(t, parent, good), good.Snapshot.ID()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cursor.completeness.UnavailableSubtrees != 1 {
|
||||
t.Fatalf("missing component bounds hid a failed subtree: %+v", cursor.completeness)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFlatReviewAnchorsPrunedFocusAtDisplayedParent(t *testing.T) {
|
||||
before := flatNode("/before", "Unrelated first line", 0, 0)
|
||||
parent := flatNode("/parent", "Focused parent text", 0, 30)
|
||||
leaf := node("/leaf", "")
|
||||
parent.Children = []atspi.ObjectID{leaf.Snapshot.ID()}
|
||||
root := flatNode("/window", "", 0, 0)
|
||||
root.Text = atspi.VisibleText{}
|
||||
root.Geometry.Bounds.Height = 100
|
||||
root.Snapshot.Role = "frame"
|
||||
root.Children = []atspi.ObjectID{before.Snapshot.ID(), parent.Snapshot.ID()}
|
||||
tree, err := atspi.NewAccessibleTree(root.Snapshot.ID(), []atspi.TreeNode{root, before, parent, leaf})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cursor := &FlatCursor{}
|
||||
if err := cursor.Refresh(context.Background(), tree, leaf.Snapshot.ID()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
position, _, err := cursor.Read(Line, 0)
|
||||
if err != nil || position.Object != parent.Snapshot.ID() || position.Text != "Focused parent text" {
|
||||
t.Fatalf("pruned focus anchor = %+v, %v", position, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFlatWorkerWarnsOnceAndClearsRecoveredCompleteness(t *testing.T) {
|
||||
capturer := &fakeCapturer{calls: make(chan captureCall, 1)}
|
||||
worker, cancel := startWorker(t, capturer, time.Second)
|
||||
defer cancel()
|
||||
defer worker.Stop(context.Background())
|
||||
good := flatNode("/good", "Readable", 0, 0)
|
||||
bad := flatNode("/bad", "Unavailable", 0, 30)
|
||||
bad.Text = atspi.VisibleText{Limited: true}
|
||||
tree := flatTree(t, good, bad)
|
||||
worker.SyncFocus(good.Snapshot.ID())
|
||||
worker.ReadFlat(Line, 0)
|
||||
call := <-capturer.calls
|
||||
call.result <- captureResult{tree: tree}
|
||||
for index := 0; index < 2; index++ {
|
||||
if index != 0 {
|
||||
worker.ReadFlat(Character, 0)
|
||||
}
|
||||
result := waitResult(t, worker)
|
||||
if result.Err != nil || !result.FlatCompleteness.Incomplete() || result.FlatWarning != (index == 0) {
|
||||
t.Fatalf("partial result %d: %+v", index, result)
|
||||
}
|
||||
}
|
||||
bad = flatNode("/bad", "Recovered", 0, 30)
|
||||
fresh := flatTree(t, good, bad)
|
||||
snapshot, err := fresh.ReviewWindow(context.Background(), good.Snapshot.ID())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := tree.Replace(objectID("/window"), snapshot.Nodes); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
worker.ReadFlat(Line, 1)
|
||||
result := waitResult(t, worker)
|
||||
if result.Err != nil || result.FlatCompleteness.Incomplete() || result.FlatWarning || result.FlatPosition.Text != "Recovered" {
|
||||
t.Fatalf("recovery result = %+v", result)
|
||||
}
|
||||
}
|
||||
@@ -121,12 +121,19 @@ func (worker *Worker) readFlat(ctx context.Context, cursor *FlatCursor, tree *at
|
||||
if origin, offset := worker.flatStart(); origin.Valid() {
|
||||
if !cursor.seek(origin, offset, true) {
|
||||
if !cursor.seek(origin, 0, false) {
|
||||
cursor.seekDescendant(origin)
|
||||
if !cursor.seekDescendant(origin) {
|
||||
cursor.seek_ancestor(origin)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
cursor.started = true
|
||||
}
|
||||
result.FlatPosition, result.Moved, result.Err = cursor.Read(unit, step)
|
||||
result.FlatCompleteness = cursor.completeness
|
||||
if result.Err == nil && cursor.completeness.Incomplete() && !cursor.warnedIncomplete {
|
||||
result.FlatWarning = true
|
||||
cursor.warnedIncomplete = true
|
||||
}
|
||||
return result, tree
|
||||
}
|
||||
@@ -280,7 +280,7 @@ func (worker *Worker) run(ctx context.Context, done chan<- struct{}) {
|
||||
request.interest.result <- interestReply{interest, err}
|
||||
continue
|
||||
}
|
||||
if (request.brailleRoute != nil || request.nativeCaret || request.readAll != nil || request.line || request.listSelection != nil || request.target != 0 || request.caretCommand != 0 || request.activate || request.fragment != "" || request.unit != 0) && request.documentEpoch != documentEpoch {
|
||||
if (request.currentLine || request.brailleRoute != nil || request.nativeCaret || request.readAll != nil || request.line || request.listSelection != nil || request.target != 0 || request.caretCommand != 0 || request.activate || request.fragment != "" || request.unit != 0) && request.documentEpoch != documentEpoch {
|
||||
if request.brailleRoute != nil {
|
||||
request.brailleRoute.done <- atspi.ErrStaleObject
|
||||
}
|
||||
@@ -291,7 +291,10 @@ func (worker *Worker) run(ctx context.Context, done chan<- struct{}) {
|
||||
continue
|
||||
}
|
||||
var result Result
|
||||
if request.listSelection != nil {
|
||||
if request.currentLine {
|
||||
result = worker.read_current_line(ctx, document, request.currentLineBrowse)
|
||||
result.FlatSpeech = request.flatSpeech
|
||||
} else if request.listSelection != nil {
|
||||
result, document = worker.selectDocumentList(ctx, document, request.listSelection)
|
||||
} else if request.brailleRoute != nil {
|
||||
result = worker.route_document_braille(ctx, document, request)
|
||||
|
||||
@@ -52,6 +52,8 @@ func (worker *Worker) nativeCapturerFor(id atspi.ObjectID) TreeCapturer {
|
||||
// Result describes one requested navigation step. A boundary has Moved false
|
||||
// and no error; Position remains the current object in that case.
|
||||
type Result struct {
|
||||
CurrentLine bool
|
||||
CurrentLineBrowse bool
|
||||
ReadAllContext context.Context
|
||||
ReadAllLines []atspi.DocumentLine
|
||||
ReadAllProgress func(int)
|
||||
@@ -96,6 +98,8 @@ type Result struct {
|
||||
FlatStep int
|
||||
FlatPosition FlatPosition
|
||||
FlatFocus atspi.ObjectID
|
||||
FlatCompleteness FlatCompleteness
|
||||
FlatWarning bool
|
||||
DocumentLine bool
|
||||
DocumentLineRequest uint64
|
||||
DocumentLineData atspi.DocumentLine
|
||||
@@ -108,6 +112,8 @@ type Result struct {
|
||||
// activation, fragment, flat unit, then object direction. Flags are deliberately
|
||||
// retained; epoch and timeout retirement depend on their distinct combinations.
|
||||
type request struct {
|
||||
currentLine bool
|
||||
currentLineBrowse bool
|
||||
documentList *documentListRequest
|
||||
listSelection *documentListSelection
|
||||
brailleRoute *documentBrailleRequest
|
||||
|
||||
Vendored
+2
-2
@@ -38,10 +38,10 @@ self-voice-dbus-speak,src/cthulhu/dbus_service.py,remote,Speak external plain te
|
||||
self-voice-dbus-braille,src/cthulhu/dbus_service.py,remote,Display an external persistent braille message over the session D-Bus service,Storm Dragon,internal/remote internal/app,in-progress,go test -tags brlapi ./internal/remote ./internal/app -count=1,Focused tests confirm connected-worker queueing and rejection when disconnected.,,,Pending physical BrailleMessage delivery and persistent-display acceptance,BrailleMessage queues a persistent frame only after the BrlAPI helper reports ready. It rejects empty and over-10-KiB text; physical display and Wine client acceptance remain open.
|
||||
self-voice-dbus-cancel,src/cthulhu/dbus_service.py,remote,Cancel current speech at an external application's request over the session D-Bus service,Storm Dragon,internal/remote internal/app,in-progress,go test ./internal/remote ./internal/app -count=1; CTHULHU_NATIVE_DBUS_TEST=1 go test ./internal/remote -run '^TestNativeSessionBusSelfVoice$' -count=1,Private session-bus proof passes same-user dispatch introspection conflict shutdown release and reacquisition.,,Pending CancelSpeech interruption and subsequent-delivery listening acceptance,,CancelSpeech uses the asynchronous owned speech-worker interruption path and returns false when speech is unavailable. Like Python it does not cancel sound output.
|
||||
nvda-controller-ssml-rejection,src/cthulhu/dbus_service.py,remote,Reject invalid or unsupported NVDA Controller 2 SSML without claiming mark or completion support,Storm Dragon,internal/remote,implemented,go test ./internal/remote -count=1; CTHULHU_NATIVE_DBUS_TEST=1 go test ./internal/remote -run '^TestNativeSessionBusSelfVoice$' -count=1,Private session-bus proof passes the real SpeakSsml signature and unsupported result.,,,,Malformed XML or a non-speak root returns Windows ERROR_INVALID_PARAMETER 87. Valid speak-root SSML returns ERROR_NOT_SUPPORTED 50 matching Python because mark callbacks and completion semantics are unavailable.
|
||||
flat-review-keys,src/cthulhu/flat_review_presenter.py,review,Navigate and present content with desktop and laptop review keys,Storm Dragon,internal/input internal/review internal/app internal/platform/xorg,in-progress,DISPLAY=:0 go test -race ./internal/atspi ./internal/review ./internal/app ./internal/input ./internal/platform/xorg ./internal/presentation -count=1; DISPLAY=:0 python3 tests/native_atspi.py; DISPLAY=:0 python3 tests/native_x11.py,Private Xvfb tests pass all nine desktop and laptop sequential review commands; both laptop release orders; NumLock numeric keypad typing; laptop keypad pass-through; existing command and shutdown checks.,,Maintainer accepted all currently implemented terminal flat-review behavior on physical Xorg on 2026-09-23; broader application listening remains pending,Pending braille reading gate,Basic previous/current/next line word and code-point character commands are wired to owned window layout and the existing review worker. Window boundaries stop silently as in Python. Capture limits are reported explicitly. Native GTK text navigation and edit recovery pass. Viewport seeking mode toggling same-object caret following after leaving review state/value zones advanced bindings mouse review and full application parity remain pending. See docs/testing/flat-review.md. Parity review corrections and remaining physical gates: docs/testing/parity-review-follow-up.md. Numpad 2 supports phonetic double press; Numpad 5 and 8 support literal and phonetic spelling on second and third presses within 500 ms. Shared desktop/laptop routing plus private X11 consumption and OCR cursor tests pass; live spelling listening remains pending. See docs/testing/phonetic-review.md.
|
||||
flat-review-keys,src/cthulhu/flat_review_presenter.py,review,Navigate and present content with desktop and laptop review keys,Storm Dragon,internal/input internal/review internal/app internal/platform/xorg,in-progress,DISPLAY=:0 go test -race ./internal/atspi ./internal/review ./internal/app ./internal/input ./internal/platform/xorg ./internal/presentation -count=1; DISPLAY=:0 python3 tests/native_atspi.py; DISPLAY=:0 python3 tests/native_x11.py,Private Xvfb tests pass all nine desktop and laptop sequential review commands; both laptop release orders; NumLock numeric keypad typing; laptop keypad pass-through; existing command and shutdown checks.,,Maintainer accepted all currently implemented terminal flat-review behavior on physical Xorg on 2026-09-23; broader application listening remains pending,Pending braille reading gate,Basic previous/current/next line word and code-point character commands are wired to owned window layout and the existing review worker. Sequential line boundaries announce top or bottom of screen. Capture limits are reported explicitly. Native GTK text navigation and edit recovery pass. Mode toggling same-object caret following after leaving review state/value zones advanced bindings mouse review and full application parity remain pending. See docs/testing/flat-review-robustness.md for the current capture comparison and clipped-text regression. Parity review corrections and remaining physical gates: docs/testing/parity-review-follow-up.md. Numpad 2 supports phonetic double press; Numpad 5 and 8 support literal and phonetic spelling on second and third presses within 500 ms. Shared desktop/laptop routing plus private X11 consumption and OCR cursor tests pass; live spelling listening remains pending. See docs/testing/phonetic-review.md.
|
||||
preferences-basic-speech,src/cthulhu/cthulhu_gui_prefs.py,settings,Open accessible basic speech preferences and apply or save settings,Storm Dragon,internal/preferences internal/app internal/config,in-progress,go test -race ./... -count=1; python3 tests/native_preferences.py; python3 tests/native_x11.py,Private Xvfb GTK and command checks pass; maintainer accepted save shutdown restart and restored settings on physical Xorg on 2026-09-13; broader desktop traversal pending,,Pending human listening for immediate rate pitch and volume previews plus punctuation and independent capital cues,,Apply and OK update the active schema-v2 profile while preserving other profiles and application mappings; Apply stays open and OK closes. Default-voice rate pitch volume and server-default toggles preview immediately without committing settings; Cancel restores the last applied voice configuration. Independent capital pitch beep and cap checkboxes replace the capitalization style dropdown; default pitch is enabled and all combinations apply to custom echo and spelling. Private GTK traversal and Apply/reload plus SSIP and persistence checks pass; live listening and Braille remain pending. Explicitly selected legacy files remain session-only and cannot be overwritten. Advanced preferences and importer remain pending. Phonetic character hints is an unchecked Speech Reading option; plain Left/Right character navigation can append a cancellable 300 ms phonetic hint. Private GTK default toggle Apply/reload and cancellation checks pass; physical listening remains pending. See docs/testing/phonetic-review.md.
|
||||
review-component-geometry,src/cthulhu/flat_review.py,review,Capture component screen bounds and visibility for flat-review clipping,Storm Dragon,internal/atspi,implemented,DISPLAY=:0 go test ./internal/atspi -count=1; DISPLAY=:0 python3 tests/native_atspi.py,Private Xvfb D-Bus GtkBox tests verify screen bounds resize invalidation changed dimensions hide invalidation and recapture. Physical acceptance deferred by maintainer on 2026-09-14.,,,,Geometry queries run only during owned-tree capture. Bounds and visibility events invalidate geometry without removing identities. Negative coordinates overflow-safe clipping malformed geometry and concurrent invalidation have unit tests. Component bounds do not establish occlusion text ranges zones reading order or flat-review key support.
|
||||
review-visible-text,src/cthulhu/flat_review.py,review,Capture bounded visible text with character offsets and screen clipping,Storm Dragon,internal/atspi,in-progress,DISPLAY=:0 go test -race ./internal/atspi ./internal/review ./internal/app ./internal/parity -count=1; DISPLAY=:0 python3 tests/native_atspi.py,Private ListBox and GtkBox capture edit clipping hidden-text and password-exclusion checks pass. Physical acceptance deferred.,,Pending navigation and listening gate,,Exact Unicode offsets and owned ranges now include captured caret offsets. Text caret and geometry events make cached text stale without removing identities. Maximum 4096 characters and 256 lines or ranges per object and 1 MiB per tree. Oversized objects remain limited with no partial text. Basic zones and review commands now consume this data; large-document viewport seeking and physical listening remain pending.
|
||||
review-visible-text,src/cthulhu/flat_review.py,review,Capture bounded visible text with character offsets and screen clipping,Storm Dragon,internal/atspi,in-progress,DISPLAY=:0 go test -race ./internal/atspi ./internal/review ./internal/app ./internal/parity -count=1; DISPLAY=:0 python3 tests/native_atspi.py,Private ListBox and GtkBox capture edit clipping hidden-text and password-exclusion checks pass. Physical acceptance deferred.,,Pending navigation and listening gate,,Exact Unicode offsets and owned ranges include captured caret offsets. Text caret and geometry events invalidate cached text without retiring identities. Capture budgets are 65536 characters and 256 visible spans per object and 1 MiB per tree. Bounded line-rectangle discovery supports large objects without character rectangles; oversized lines use viewport glyph locations and bounded text spans. Zero-area glyphs no longer discard other visible text. Known unavailable objects and child subtrees leave healthy content reviewable with an explicit speech and Braille warning. Root failures and aggregate layout budgets remain fatal. Private GTK fixtures and worker tests pass; physical speech and Braille acceptance remain pending. See docs/testing/flat-review-robustness.md.
|
||||
date-time-commands,src/cthulhu/date_and_time_presenter.py,command,Speak local time with Cthulhu+T and date with a double tap using saved formats,Storm Dragon,internal/input internal/app internal/preferences internal/config,in-progress,go test ./... -count=1; focused race checks; native keyboard double-tap checks,Private Xvfb press-time and both-release-order checks passed; physical Xorg pending,,Pending human listening,Pending braille,All 7 Python time and 16 Python date format choices are wired including locale seconds word numeric full and abbreviated variants; see docs/testing/time-date.md.
|
||||
notifications,src/cthulhu/notification_presenter.py;src/cthulhu/scripts/apps/notification-daemon/script.py;src/cthulhu/scripts/apps/notify-osd/script.py;src/cthulhu/mako_notification_monitor.py,notification,Present provider notifications save history expose mako live actions speak the last notification and open the notification-list GUI,Storm Dragon,internal/notification internal/app internal/notificationui internal/atspi internal/input,in-progress,NO_COLOR=1 GOCACHE=/tmp/cthulhu-build-cache go test ./... -count=1; NO_COLOR=1 python3 tests/native_notifications.py,,,Automated speech-queue history list-state and list-action assertions passed; headless Xvfb/AT-SPI GUI proof passed; human listening and physical Xorg use not verified,Braille not implemented,Go registers window-create and accessible-name events maps notification-daemon mate and xfce4 providers handles notify-osd gauge and name events keeps a 55-entry history and adds Cthulhu+N plus Cthulhu+Ctrl+N. The modal accessible list opens in a GTK helper process with sortable rows copy dismiss clear close dynamic actions and mako live refresh. Live provider Xorg acceptance remains pending. Parity review corrections and remaining physical gates: docs/testing/parity-review-follow-up.md.
|
||||
ordinary-key-speech-interrupt,src/cthulhu/input_event.py,input,Interrupt speech on an ordinary key press even if focus does not move,,internal/input internal/app,implemented,go test ./internal/input ./internal/app -run TestProcessInputEventsInterruptsSpeechOnKeyPress -count=1,,,,,Python KeyboardEvent._present interrupts on press outside game mode. Go now interrupts on every observed press; physical FastSMRW table-boundary listening remains pending.
|
||||
|
||||
|
@@ -0,0 +1,69 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Private GTK controls for current-line AT-SPI checks."""
|
||||
import json
|
||||
import sys
|
||||
|
||||
import gi
|
||||
gi.require_version("Gtk", "3.0")
|
||||
from gi.repository import GLib, Gtk
|
||||
|
||||
window = Gtk.Window(title="Current line fixture")
|
||||
box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL)
|
||||
window.add(box)
|
||||
entry = Gtk.Entry()
|
||||
entry.get_accessible().set_name("Entry")
|
||||
textView = Gtk.TextView()
|
||||
textView.get_accessible().set_name("Text")
|
||||
password = Gtk.Entry()
|
||||
password.set_visibility(False)
|
||||
password.get_accessible().set_name("Password")
|
||||
button = Gtk.Button(label="Example button")
|
||||
controls = {"entry": entry, "text": textView, "password": password, "button": button}
|
||||
for control in controls.values():
|
||||
box.pack_start(control, True, True, 0)
|
||||
window.show_all()
|
||||
|
||||
|
||||
def command_ready(source, condition):
|
||||
line = sys.stdin.readline()
|
||||
if not line:
|
||||
Gtk.main_quit()
|
||||
return False
|
||||
command = json.loads(line)
|
||||
control = controls[command["control"]]
|
||||
if command["action"] == "focus":
|
||||
if isinstance(control, Gtk.Entry):
|
||||
control.set_text(command["text"])
|
||||
control.set_position(command["offset"])
|
||||
elif isinstance(control, Gtk.TextView):
|
||||
buffer = control.get_buffer()
|
||||
buffer.set_text(command["text"])
|
||||
buffer.place_cursor(buffer.get_iter_at_offset(command["offset"]))
|
||||
window.present()
|
||||
control.grab_focus()
|
||||
|
||||
def finish_focus():
|
||||
if isinstance(control, Gtk.Entry):
|
||||
control.set_position(command["offset"])
|
||||
elif isinstance(control, Gtk.TextView):
|
||||
buffer = control.get_buffer()
|
||||
buffer.place_cursor(buffer.get_iter_at_offset(command["offset"]))
|
||||
print("ready", flush=True)
|
||||
return False
|
||||
|
||||
GLib.timeout_add(50, finish_focus)
|
||||
return True
|
||||
elif command["action"] == "caret":
|
||||
if isinstance(control, Gtk.Entry):
|
||||
offset = control.get_position()
|
||||
else:
|
||||
buffer = control.get_buffer()
|
||||
offset = buffer.get_iter_at_mark(buffer.get_insert()).get_offset()
|
||||
print(offset, flush=True)
|
||||
print("ready", flush=True)
|
||||
return True
|
||||
|
||||
|
||||
GLib.io_add_watch(sys.stdin, GLib.IO_IN | GLib.IO_HUP, command_ready)
|
||||
print("ready", flush=True)
|
||||
Gtk.main()
|
||||
@@ -10,9 +10,22 @@ static gboolean armed;
|
||||
static unsigned childQueries;
|
||||
static gboolean slowRanges;
|
||||
static gint largestRange;
|
||||
static gboolean lineGeometryOnly;
|
||||
static gboolean missingGlyph;
|
||||
static GtkWidget *reviewScroller;
|
||||
static AtkObject *brokenSubtree;
|
||||
static gint (*originalChildCount)(AtkObject *);
|
||||
static void (*originalRangeExtents)(AtkText *, gint, gint, AtkCoordType, AtkTextRectangle *);
|
||||
|
||||
static void range_extents(AtkText *text, gint start, gint end, AtkCoordType coordinates, AtkTextRectangle *rectangle) {
|
||||
if (missingGlyph && start == 5 && end == 6) {
|
||||
memset(rectangle, 0, sizeof(*rectangle));
|
||||
return;
|
||||
}
|
||||
if (lineGeometryOnly && end - start == 1) {
|
||||
memset(rectangle, 0, sizeof(*rectangle));
|
||||
return;
|
||||
}
|
||||
if (slowRanges) {
|
||||
largestRange = MAX(largestRange, end - start);
|
||||
if (end - start > 128) g_usleep(1200000);
|
||||
@@ -31,6 +44,11 @@ static gint count_children(AtkObject *object) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
static gint subtree_child_count(AtkObject *object) {
|
||||
if (object == brokenSubtree && armed) return -1;
|
||||
return originalChildCount(object);
|
||||
}
|
||||
|
||||
static gboolean command_ready(GIOChannel *channel, GIOCondition condition, gpointer label) {
|
||||
if (condition & G_IO_HUP) {
|
||||
gtk_main_quit();
|
||||
@@ -49,6 +67,11 @@ static gboolean command_ready(GIOChannel *channel, GIOCondition condition, gpoin
|
||||
puts("measuring");
|
||||
} else if (g_str_has_prefix(line, "largest")) {
|
||||
printf("largest %d\n", largestRange);
|
||||
} else if (g_str_has_prefix(line, "scroll")) {
|
||||
GtkAdjustment *adjustment = gtk_scrolled_window_get_vadjustment(GTK_SCROLLED_WINDOW(reviewScroller));
|
||||
gtk_adjustment_set_value(adjustment, gtk_adjustment_get_upper(adjustment) * 0.75);
|
||||
while (gtk_events_pending()) gtk_main_iteration();
|
||||
puts("scrolled");
|
||||
} else if (g_str_has_prefix(line, "change")) {
|
||||
armed = FALSE;
|
||||
gtk_label_set_text(GTK_LABEL(label), "Changed review text");
|
||||
@@ -61,6 +84,13 @@ static gboolean command_ready(GIOChannel *channel, GIOCondition condition, gpoin
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
gboolean boundsMode = argc > 1 && strcmp(argv[1], "bounds") == 0;
|
||||
gboolean clippedMode = argc > 1 && strcmp(argv[1], "clipped") == 0;
|
||||
gboolean longMode = argc > 1 && strcmp(argv[1], "long") == 0;
|
||||
gboolean linesMode = argc > 1 && strcmp(argv[1], "lines") == 0;
|
||||
gboolean childrenMode = argc > 1 && strcmp(argv[1], "children") == 0;
|
||||
gboolean glyphsMode = argc > 1 && strcmp(argv[1], "glyphs") == 0;
|
||||
lineGeometryOnly = linesMode;
|
||||
missingGlyph = glyphsMode;
|
||||
gtk_init(&argc, &argv);
|
||||
GtkWidget *window = gtk_window_new(GTK_WINDOW_TOPLEVEL);
|
||||
gtk_window_set_title(GTK_WINDOW(window), "Review cache probe");
|
||||
@@ -69,9 +99,30 @@ int main(int argc, char **argv) {
|
||||
GtkWidget *label = gtk_label_new("Cached review text");
|
||||
// This fixture has one label. Replace its class vfunc before the bridge
|
||||
// builds the initial cache, then arm the delay explicitly over stdin.
|
||||
ATK_OBJECT_GET_CLASS(gtk_widget_get_accessible(label))->get_n_children = count_children;
|
||||
if (boundsMode) {
|
||||
gchar *content = g_strnfill(3424, 'x');
|
||||
if (!childrenMode)
|
||||
ATK_OBJECT_GET_CLASS(gtk_widget_get_accessible(label))->get_n_children = count_children;
|
||||
if (childrenMode) {
|
||||
GtkWidget *box = gtk_box_new(GTK_ORIENTATION_VERTICAL, 0);
|
||||
GtkWidget *panel = gtk_event_box_new();
|
||||
brokenSubtree = gtk_widget_get_accessible(panel);
|
||||
atk_object_set_name(brokenSubtree, "Broken subtree");
|
||||
originalChildCount = ATK_OBJECT_GET_CLASS(brokenSubtree)->get_n_children;
|
||||
ATK_OBJECT_GET_CLASS(brokenSubtree)->get_n_children = subtree_child_count;
|
||||
gtk_container_add(GTK_CONTAINER(panel), gtk_label_new("Subtree text"));
|
||||
gtk_container_add(GTK_CONTAINER(box), label);
|
||||
gtk_container_add(GTK_CONTAINER(box), panel);
|
||||
gtk_container_add(GTK_CONTAINER(window), box);
|
||||
} else if (boundsMode || clippedMode || longMode || linesMode || glyphsMode) {
|
||||
gchar *content = g_strnfill(linesMode ? 101000 : longMode ? 100001 : clippedMode ? 2020 : 3424, 'x');
|
||||
if (longMode) content[100000] = '\n';
|
||||
if (linesMode) {
|
||||
for (int row = 0; row < 1000; row++) content[row * 101 + 100] = '\n';
|
||||
gtk_window_set_default_size(GTK_WINDOW(window), 1000, 100);
|
||||
}
|
||||
if (clippedMode) {
|
||||
for (int row = 0; row < 20; row++) content[row * 101 + 100] = '\n';
|
||||
gtk_window_set_default_size(GTK_WINDOW(window), 300, 450);
|
||||
}
|
||||
gtk_label_set_text(GTK_LABEL(label), content);
|
||||
g_free(content);
|
||||
gtk_label_set_xalign(GTK_LABEL(label), 0);
|
||||
@@ -81,6 +132,7 @@ int main(int argc, char **argv) {
|
||||
if (originalRangeExtents == NULL) g_error("fixture needs native GTK range extents");
|
||||
ATK_TEXT_GET_IFACE(accessible)->get_range_extents = range_extents;
|
||||
GtkWidget *scroller = gtk_scrolled_window_new(NULL, NULL);
|
||||
reviewScroller = scroller;
|
||||
gtk_container_add(GTK_CONTAINER(scroller), label);
|
||||
gtk_container_add(GTK_CONTAINER(window), scroller);
|
||||
} else {
|
||||
|
||||
Reference in new issue
Block a user