314 lines
21 KiB
Markdown
314 lines
21 KiB
Markdown
# Flat-review robustness comparison
|
||
|
||
Review date: 2026-10-21. Cthulhu baseline: `testing` at `50e32f2`.
|
||
Orca reference: upstream `main` at
|
||
`828eaf2e70d7b88124b10b3a6b2daa792ab266c7`. The local Orca checkout was
|
||
older; fetching the current reference confirmed that the three source files
|
||
below have no changes between those revisions.
|
||
|
||
This is a source comparison and a set of verified capture and recovery repairs.
|
||
It does not establish complete Orca parity or resolve every user report.
|
||
At the initial comparison, the available local log contained one
|
||
`accessible object not found` flat-review failure without enough evidence to
|
||
identify its cause, and no instance of the reported text-limit failure. Older diagnostic logs describe
|
||
provider timeouts already addressed by earlier capture work; they are not proof
|
||
of a remaining timeout in the current build.
|
||
|
||
## What Orca does
|
||
|
||
Orca's [Flat Review documentation](https://help.gnome.org/orca/howto_flat_review.html)
|
||
describes a spatial representation of the active window's visible contents.
|
||
The implementation separates finding on-screen objects, finding visible lines,
|
||
and building review zones:
|
||
|
||
- [`flat_review.py`](https://gitlab.gnome.org/GNOME/orca/-/blob/828eaf2e70d7b88124b10b3a6b2daa792ab266c7/src/orca/flat_review.py):
|
||
`Context` chooses a dialog/frame or an explicit root and restricts to an active
|
||
menu when appropriate. `_get_zones_from_text` clips to a scroll pane/document,
|
||
gets visible lines, removes embedded-object characters, and builds text zones.
|
||
Word and character objects are created on demand. `_find_zone_with_object`
|
||
can use a parent or child when the exact focused object was pruned.
|
||
- [`ax_utilities.py`](https://gitlab.gnome.org/GNOME/orca/-/blob/828eaf2e70d7b88124b10b3a6b2daa792ab266c7/src/orca/ax_utilities.py):
|
||
`_get_on_screen_objects` checks showing/visible state and window intersection,
|
||
treats suitable controls as leaves, enumerates visible table cells, and
|
||
suppresses some duplicate container text. Invalid component geometry is not
|
||
invariably treated as off-screen.
|
||
- [`ax_utilities_text.py`](https://gitlab.gnome.org/GNOME/orca/-/blob/828eaf2e70d7b88124b10b3a6b2daa792ab266c7/src/orca/ax_utilities_text.py):
|
||
`find_first_visible_line` searches by line rectangles, then `get_visible_lines`
|
||
walks forward until below the viewport. Invalid line offsets have explicit
|
||
handling. This differs from requiring usable character geometry to find a
|
||
large object's viewport.
|
||
|
||
These paths do not impose Cthulhu's 256-range/65,536-character object budgets.
|
||
That is not evidence that unlimited capture is safe: Cthulhu still needs bounded
|
||
native calls, memory use, cancellation, and protection from defective providers.
|
||
Orca also has its own tradeoffs: the single-line editable-text path requests all
|
||
text, and `Context.is_stale` itself only checks top-level size changes. Presenter
|
||
event handling supplies additional context management. Copying one function
|
||
alone would not reproduce that behavior.
|
||
|
||
## Comparison and priorities
|
||
|
||
| Area | Current Cthulhu behavior | Assessment |
|
||
| --- | --- | --- |
|
||
| Scope | Captures the focused window/menu/dialog and skips hidden descendants during review-only traversal. | Already aligned in principle. Preserve foreground ownership and popup handling. |
|
||
| Large GTK tables | Captures visible selectable-table cells and headers; keeps a selected-cell identity anchor. | Already follows Orca's viewport approach. Existing table checks remain relevant. |
|
||
| Clipped text ranges | Previously emitted one range per clipped visible character. | Confirmed defect; repaired here. Count contiguous visible spans against the range budget. |
|
||
| Viewport discovery | Uses character rectangles first; large objects can use a bounded binary search over line rectangles. | Private GTK coverage seeks into a 101000-character scrolled object with unavailable character rectangles. Small objects keep their existing bounded path. |
|
||
| Long horizontal lines | Oversized viewport intervals use glyph locations and bounded text retrieval; oversized native line responses also enter that path. | A real GTK 100000-character line is captured without increasing the character budget. New text requests contain at most 128 characters. Sparse or defective point providers remain bounded and explicitly limited. |
|
||
| Failure scope | Known unavailable text objects and failed child enumeration retain healthy content with explicit completeness counts. | Partial-result counts remain diagnostic log data. Line boundaries always use "Top of screen." and "Bottom of screen."; no extra partial-content speech or Braille message is emitted. Root failure, cancellation, stale snapshots, invalid trees, and aggregate layout limits still fail the operation. |
|
||
| Focus anchoring | Tries exact focus, displayed descendants, then the nearest displayed ancestor within the review root. | A targeted fixture proves pruned focus stays on its parent text rather than an unrelated first line. Preserved position still uses object generations and native offsets. |
|
||
| Updates and cancellation | Cthulhu refreshes owned layout per command and rejects stale results; native capture and queues are bounded. | Preserve these safeguards. Snapshot refresh, provider timeout, and audible responsiveness are separate concerns. |
|
||
| Diagnostics | Capture timings include `text_issues` counts for viewport, character, range, tree-byte, query-budget, and provider failures. Partial presentation logs omission counts. | These fields contain no accessible text or names. A wholly limited window retains its existing spoken text-limit message. |
|
||
|
||
The line-based discovery, long-line retrieval, explicit partial recovery, and
|
||
parent anchoring above are implemented in the current working tree. Capture
|
||
budgets, provider-call chunking, cancellation, foreground ownership, and stale
|
||
result rejection remain in force. Remaining failures need an application,
|
||
triggering command, build revision, and fresh failure/timing record before
|
||
assigning a cause. Cold-capture latency remains separate from cached navigation.
|
||
|
||
## Repair and evidence
|
||
|
||
`internal/atspi/text.go` now accumulates adjacent visible characters with the
|
||
same row geometry into one range. It preserves exact Unicode offsets and unions
|
||
only their clipped rectangles. A hidden offset, embedded object, or row change
|
||
prevents merging across that boundary. The 128-character geometry blocks and
|
||
all existing capture budgets remain in force. No new mutable cross-command state
|
||
is introduced; span accumulation belongs to one synchronous capture.
|
||
|
||
Before the change, a real GTK label with 20 horizontally clipped lines produced
|
||
`VisibleText{Limited: true}`. After the change, the same fixture captures more
|
||
than 256 characters in at most 21 spans. A deterministic fixture verifies 800
|
||
Unicode characters in 20 spans, exact offsets, and clipping at both edges. Other
|
||
regressions cover a span crossing geometry blocks, disjoint bidirectional runs,
|
||
and a genuinely excessive number of separate visible spans. Existing tests
|
||
cover blank rows, embedded objects, large terminals, cancellation and malformed
|
||
provider data.
|
||
|
||
The successful four-row capture benchmark, three runs per version, measured
|
||
14.6–15.1 microseconds and 180 allocations before, versus 10.4–10.7 microseconds
|
||
and 19 allocations after. Allocated bytes fell from 19,664 to 4,048 per capture.
|
||
This measures local layout preparation with a synthetic provider; it does not
|
||
measure AT-SPI round trips or prove live speech latency.
|
||
|
||
## Automated checks
|
||
|
||
```sh
|
||
go test ./... -count=1
|
||
go test -race ./internal/atspi ./internal/review ./internal/app ./internal/presentation -count=1
|
||
go vet ./internal/atspi ./internal/review ./internal/app ./internal/presentation
|
||
python3 tests/native_atspi.py
|
||
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_CLIPPED=1 CTHULHU_NATIVE_REVIEW_CACHE=1 CTHULHU_NATIVE_REVIEW_BOUNDS=1 go test ./internal/atspi -run '^TestNativeFlatReview(ClippedViewport|PreservesProviderCache|BoundsSlowLongRanges)$' -count=1 -v
|
||
go test ./internal/atspi -run '^$' -bench '^BenchmarkCaptureVisibleTextClipped$' -benchmem -count=3
|
||
```
|
||
|
||
Use a writable `GOCACHE` if required by the environment. Native harnesses create
|
||
private Xvfb/D-Bus/AT-SPI sessions and do not replace the running reader.
|
||
All checks above passed for this change. The existing table harness emitted
|
||
GTK `Atk-CRITICAL` object-role warnings while its assertions passed; the cause
|
||
of those warnings was not investigated here.
|
||
|
||
## Follow-up implementation and evidence
|
||
|
||
Large-object line discovery moves by validated complete line boundaries during
|
||
binary search; it never scans terminal scrollback. Oversized horizontal text
|
||
uses native offset-at-point and glyph rectangles to discover visible offsets,
|
||
then reads text in spans of at most 128 characters. Offsets are sorted before
|
||
capture so bidirectional runs retain exact logical identities. Point traversal
|
||
has a 4096-query budget; it checks gaps before skipping vertically so an inset
|
||
or shorter glyph is not missed. Ordinary clipped capture continues to use bounded
|
||
geometry blocks. Unsupported or excessive work yields an explicit limitation,
|
||
not guessed text. No global capture limit was increased.
|
||
|
||
A native provider may return a zero-area rectangle for a nonvisible glyph.
|
||
That is distinct from a failed native request. Such glyphs now break visible
|
||
spans instead of discarding the entire object's usable text. Unavailable whole
|
||
line geometry remains an unavailable capture. A private GTK fixture injects a
|
||
zero-area glyph and verifies that the surrounding visible text survives and
|
||
the missing offset is excluded. Combining marks and joiners attached to a
|
||
visible base retain their Unicode offsets even without independent rectangles.
|
||
Bounded endpoint lookup also handles a huge line ending in a zero-area newline.
|
||
|
||
`ChildrenUnavailable` is capture-owned state copied into the owned tree and
|
||
atomic review snapshot. Recovery is restricted to review-only child enumeration;
|
||
root enumeration failure and cancellation remain fatal. `FlatCompleteness`
|
||
travels with each worker result. No partial object's accessible name is silently
|
||
substituted for its missing text. Incomplete-capture diagnostics are logged on
|
||
entry or when omission counts change; returning to a complete layout clears
|
||
that diagnostic state. Ordinary invalidation triggers recapture as before; there
|
||
is no broad automatic retry loop. Partial layouts add no speech or Braille
|
||
warning, and boundaries retain the standard top/bottom-of-screen messages.
|
||
Spelling retains its own request.
|
||
|
||
Seven private native cases cover cache reuse, bounded geometry, clipped spans,
|
||
a 100000-character horizontal line, scrolled line-only geometry, unavailable
|
||
child enumeration, and zero-area glyphs. Additional worker and presentation
|
||
fixtures cover healthy-content recovery, omission counts, recovery to a complete
|
||
layout, ancestor anchoring, spelling without extra speech/Braille messages,
|
||
and standard boundary feedback on incomplete layouts. Run native modes in separate processes:
|
||
|
||
```sh
|
||
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_VIEWPORT=1 CTHULHU_NATIVE_REVIEW_VIEWPORT_MODE=long go test ./internal/atspi -run '^TestNativeFlatReviewLargeTextViewport$' -count=1 -v
|
||
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_VIEWPORT=1 CTHULHU_NATIVE_REVIEW_VIEWPORT_MODE=lines go test ./internal/atspi -run '^TestNativeFlatReviewLargeTextViewport$' -count=1 -v
|
||
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_SUBTREE=1 go test ./internal/atspi -run '^TestNativeFlatReviewUnavailableSubtree$' -count=1 -v
|
||
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_GLYPHS=1 go test ./internal/atspi -run '^TestNativeFlatReviewZeroAreaGlyph$' -count=1 -v
|
||
```
|
||
|
||
A read-only probe of the open FastSMRW instance exposed zero-area glyph failures
|
||
that previously left visible table-cell text unavailable. After the repair,
|
||
27 worker commands across line, word, character, window-start, and window-end
|
||
review passed with zero reported omissions. Cached commands took about 0.3-0.7
|
||
milliseconds. Initial capture still took about eight seconds, mostly in visible
|
||
text. The standalone probe had no retained native focus event and reported
|
||
`root_scope=desktop` while filtering capture to FastSMRW's provider; this timing
|
||
does not establish normal foreground reader latency. No application keys were
|
||
injected and the running reader was not replaced. These results do not establish
|
||
audible speech or physical Braille acceptance.
|
||
|
||
### Follow-up check results
|
||
|
||
The full Go suite, affected-package race tests, vet, command builds, parity
|
||
ledger validation, recorded-speech checks, and all seven targeted native GTK
|
||
cases passed. The broad `tests/native_atspi.py` harness initially failed
|
||
`TestNativeGUIListTableSpeech` because the launcher description was repeated.
|
||
The isolated table checks passed on both an untouched `50e32f2` snapshot and the
|
||
changed tree, and a subsequent complete harness run passed. The initial failure
|
||
was not explained; passing reruns do not establish that this startup/event-order
|
||
behavior is stable. Existing GTK `Atk-CRITICAL` role warnings also remain.
|
||
|
||
## Dynamic-page visibility cache regression
|
||
|
||
On 2026-10-22, `TestNativeBrowserDynamicNavigation` reproduced a separate
|
||
flat-review failure after browse navigation scrolled a 1000-paragraph page to
|
||
newly inserted content. Window-end review returned "New content", but previous
|
||
line returned the same text with `Moved=false`, including after a text-triggered
|
||
recapture.
|
||
|
||
Temporary capture diagnostics showed one review line: cached AT-SPI `SHOWING`
|
||
states still described the original viewport. Paragraphs now on screen were
|
||
treated as hidden, while the originally showing paragraphs had current bounds
|
||
far above the window. Cursor restoration was not the cause. The 40 omitted
|
||
objects were browser-chrome text-provider failures and remained after the fix;
|
||
they did not account for the missing page paragraphs.
|
||
|
||
For a node entering a review-only native capture, refresh the cached state bits
|
||
before reading its snapshot and geometry. Preserve cached children, interfaces,
|
||
and the existing dirty-node refresh policy. This recovers the current viewport
|
||
without adding retries or changing any text, range, tree, or deadline limits.
|
||
The diagnostic capture then contained 26 review lines. Temporary text-bearing
|
||
diagnostics were removed from the code.
|
||
|
||
```sh
|
||
CTHULHU_NATIVE_BROWSER_TESTS=dynamic python3 tests/native_browser.py
|
||
```
|
||
|
||
The strengthened regression keeps the original window-end/previous-line
|
||
sequence and requires "New content", "End of page", and "Unrelated paragraph
|
||
999" in order. It also checks forward lines, words, characters, the last
|
||
character at window end, and backward recovery after the bottom boundary.
|
||
Private GTK provider-cache and clipped-text regressions and the native AT-SPI
|
||
harness passed, including terminal and table workflows. The table harness still
|
||
emitted GTK `Atk-CRITICAL` role warnings while its assertions passed.
|
||
Race tests and vet for `internal/atspi`, `internal/review`, and `internal/app`,
|
||
plus `go build ./cmd/...`, also passed. The initial ordinary app-package run
|
||
could not create private Unix speech sockets inside the sandbox; the approved
|
||
race run outside that restriction passed all three packages.
|
||
|
||
For live acceptance after an authorized build/restart, browse to the bottom of
|
||
a long dynamically updated page. Press Cthulhu+Numpad 9, then Numpad 7: speech
|
||
and Braille should move from the final visible line to its predecessor. Check
|
||
forward movement back to the final line, bottom feedback, and backward movement
|
||
again. Repeat after scrolling back up and check word/character review and a
|
||
native terminal. These automated checks do not establish physical-key, audible
|
||
speech, or Braille acceptance.
|
||
|
||
## Browser chrome omissions and standard messages
|
||
|
||
A private Chromium-family browser probe reproduced 40 omitted objects. Native
|
||
text queries exposed three distinct provider patterns: containers containing
|
||
only embedded-child placeholders, named controls returning their accessible
|
||
labels through Text but invalid line offsets, and real text with unavailable
|
||
geometry. Both character and range geometry were zero for the failing chrome
|
||
objects. The address entry's native Text contained its URL, not its label.
|
||
|
||
After a provider failure, bounded verification now allows embedded-only
|
||
containers to use child-provided content. A noneditable named control or static
|
||
label can use the existing synthetic-name path only when its native text
|
||
exactly equals its accessible name and its component is fully inside the clip.
|
||
Editable text, passwords, differing names/text, mixed container content, capture
|
||
limits, and clipped labels do not qualify. Recovered names have no native text
|
||
offsets. This is a generic provider-content rule, not a browser-name exception.
|
||
|
||
Tabs and compound menu rows also retain their own label beside independently
|
||
named child actions such as Close. Text/label descendants still suppress duplicate
|
||
parent labels. Ordinary nested-button suppression is preserved.
|
||
|
||
The new browser proof fails with the fixes disabled: Back, Forward, Reload,
|
||
New Tab, and the tab label are absent, with 40 omissions. With the fixes enabled,
|
||
these controls and popup entries are reviewable. The window fixture reports
|
||
two omissions (the address entry and a mixed-text panel); its popup reports
|
||
none. These counts describe this fixture, not every browser or screen.
|
||
The mixed-text panel remains conservatively unresolved even though a static
|
||
child supplies its displayed label. The address URL remains unavailable in flat review because
|
||
its visible character span cannot be established from zero geometry. Neither
|
||
its label nor its entire potentially clipped value is substituted. Ordinary
|
||
address-bar focus and editing remain separate from geometric flat review.
|
||
|
||
The partial-content speech/Braille warning is removed. Both complete and partial
|
||
layouts use "Top of screen." and "Bottom of screen." Omission counts remain in
|
||
diagnostics; fatal capture errors keep their existing messages. Ask before
|
||
adding or replacing established user-facing messages.
|
||
|
||
```sh
|
||
CTHULHU_NATIVE_BROWSER_TESTS=flat-chrome,dynamic,menu python3 tests/native_browser.py
|
||
```
|
||
|
||
The new proof checks actual tab-label zones, toolbar controls, focused/unfocused
|
||
address handling, and popup review. The dynamic-page proof retains its stale
|
||
state-cache fix and backward/forward line, word, and character navigation.
|
||
Focused unit cases reject unsafe component recovery and verify tab/menu labels
|
||
with child actions and text-label deduplication. Existing cache, clipping, and
|
||
zero-area-glyph native checks also remain relevant.
|
||
|
||
After an authorized build/restart, check speech and physical Braille separately:
|
||
review the tab strip and toolbar, including a tab's title and Close control;
|
||
open the browser menu and review its rows and inline controls; return to the
|
||
page and exercise dynamic-page backward review. At the first/last review line,
|
||
Numpad 7/9 must announce only the standard screen boundary. No partial-content
|
||
message should interrupt review or flash Braille. Check normal address-bar
|
||
focus/typing, a narrow/scrolled field, passwords, a native terminal, and native
|
||
text clipping. Address-value flat review remains a known limitation in the
|
||
observed provider. Private tests do not establish audible speech, physical
|
||
keys, or Braille acceptance, and the running reader was not replaced.
|
||
|
||
## Human review and live acceptance
|
||
|
||
Review the generated diff before submission. With an authorized build/restart:
|
||
|
||
1. In a native editor or a window containing a wide multiline text control,
|
||
horizontally scroll so multiple lines are clipped. Review lines, words, and
|
||
characters forward and backward; visible text must remain available when
|
||
the total exceeds 256 characters. Repeat after resizing and scrolling. Check
|
||
the first and last partially visible characters and Unicode offsets.
|
||
2. Review ordinary and large terminal screens, blank rows, and a large file in
|
||
a terminal editor. Check Numpad 7/8/9, 4/5/6 and 1/2/3, window start/end,
|
||
explicit top/bottom feedback, and multi-press spelling.
|
||
3. Check a browser with horizontally clipped text and embedded links, a GTK
|
||
table such as a timeline, a preferences notebook, and a popup menu. Hidden
|
||
letters and embedded-object placeholders must not be spoken as ordinary text.
|
||
Password contents must stay excluded. Review must stay in the foreground
|
||
window and ordinary focus/caret navigation must still work afterward.
|
||
4. Exercise a window with one unavailable text object or child subtree. Healthy
|
||
content must remain reviewable without an additional partial-content
|
||
announcement. Verify omission diagnostics in the log, then refresh after the
|
||
provider recovers. Top/bottom feedback must use the same standard messages
|
||
before and after recovery.
|
||
5. Confirm audible speech and physical Braille separately, including routing
|
||
within a merged span. Native capture and Go test success do not establish
|
||
these outcomes. Preserve fresh logs for any remaining failure.
|
||
|
||
The code change affects shared visible-text capture used in owned trees;
|
||
terminal, browser, mouse-review, and Braille consumers can therefore be affected
|
||
indirectly. It changes neither X11 key grabs nor foreground ownership. Wayland
|
||
live acceptance remains separate.
|