cthulhu/docs/testing/flat-review-robustness.md

Flat-review robustness comparison

Review date: 2026-10-21. Cthulhu baseline: testing at 50e32f2. Orca reference: upstream main at 828eaf2e70d7b88124b10b3a6b2daa792ab266c7. The local Orca checkout was older; fetching the current reference confirmed that the three source files below have no changes between those revisions.

This is a source comparison and a set of verified capture and recovery repairs. It does not establish complete Orca parity or resolve every user report. At the initial comparison, the available local log contained one accessible object not found flat-review failure without enough evidence to identify its cause, and no instance of the reported text-limit failure. Older diagnostic logs describe provider timeouts already addressed by earlier capture work; they are not proof of a remaining timeout in the current build.

What Orca does

Orca's Flat Review documentation describes a spatial representation of the active window's visible contents. The implementation separates finding on-screen objects, finding visible lines, and building review zones:

• flat_review.py: Context chooses a dialog/frame or an explicit root and restricts to an active menu when appropriate. _get_zones_from_text clips to a scroll pane/document, gets visible lines, removes embedded-object characters, and builds text zones. Word and character objects are created on demand. _find_zone_with_object can use a parent or child when the exact focused object was pruned.
• ax_utilities.py: _get_on_screen_objects checks showing/visible state and window intersection, treats suitable controls as leaves, enumerates visible table cells, and suppresses some duplicate container text. Invalid component geometry is not invariably treated as off-screen.
• ax_utilities_text.py: find_first_visible_line searches by line rectangles, then get_visible_lines walks forward until below the viewport. Invalid line offsets have explicit handling. This differs from requiring usable character geometry to find a large object's viewport.

These paths do not impose Cthulhu's 256-range/65,536-character object budgets. That is not evidence that unlimited capture is safe: Cthulhu still needs bounded native calls, memory use, cancellation, and protection from defective providers. Orca also has its own tradeoffs: the single-line editable-text path requests all text, and Context.is_stale itself only checks top-level size changes. Presenter event handling supplies additional context management. Copying one function alone would not reproduce that behavior.

Comparison and priorities

Area	Current Cthulhu behavior	Assessment
Scope	Captures the focused window/menu/dialog and skips hidden descendants during review-only traversal.	Already aligned in principle. Preserve foreground ownership and popup handling.
Large GTK tables	Captures visible selectable-table cells and headers; keeps a selected-cell identity anchor.	Already follows Orca's viewport approach. Existing table checks remain relevant.
Clipped text ranges	Previously emitted one range per clipped visible character.	Confirmed defect; repaired here. Count contiguous visible spans against the range budget.
Viewport discovery	Uses character rectangles first; large objects can use a bounded binary search over line rectangles.	Private GTK coverage seeks into a 101000-character scrolled object with unavailable character rectangles. Small objects keep their existing bounded path.
Long horizontal lines	Oversized viewport intervals use glyph locations and bounded text retrieval; oversized native line responses also enter that path.	A real GTK 100000-character line is captured without increasing the character budget. New text requests contain at most 128 characters. Sparse or defective point providers remain bounded and explicitly limited.
Failure scope	Known unavailable text objects and failed child enumeration retain healthy content with explicit completeness counts.	Partial-result counts remain diagnostic log data. Line boundaries always use "Top of screen." and "Bottom of screen."; no extra partial-content speech or Braille message is emitted. Root failure, cancellation, stale snapshots, invalid trees, and aggregate layout limits still fail the operation.
Focus anchoring	Tries exact focus, displayed descendants, then the nearest displayed ancestor within the review root.	A targeted fixture proves pruned focus stays on its parent text rather than an unrelated first line. Preserved position still uses object generations and native offsets.
Updates and cancellation	Cthulhu refreshes owned layout per command and rejects stale results; native capture and queues are bounded.	Preserve these safeguards. Snapshot refresh, provider timeout, and audible responsiveness are separate concerns.
Diagnostics	Capture timings include text_issues counts for viewport, character, range, tree-byte, query-budget, and provider failures. Partial presentation logs omission counts.	These fields contain no accessible text or names. A wholly limited window retains its existing spoken text-limit message.

The line-based discovery, long-line retrieval, explicit partial recovery, and parent anchoring above are implemented in the current working tree. Capture budgets, provider-call chunking, cancellation, foreground ownership, and stale result rejection remain in force. Remaining failures need an application, triggering command, build revision, and fresh failure/timing record before assigning a cause. Cold-capture latency remains separate from cached navigation.

Repair and evidence

internal/atspi/text.go now accumulates adjacent visible characters with the same row geometry into one range. It preserves exact Unicode offsets and unions only their clipped rectangles. A hidden offset, embedded object, or row change prevents merging across that boundary. The 128-character geometry blocks and all existing capture budgets remain in force. No new mutable cross-command state is introduced; span accumulation belongs to one synchronous capture.

Before the change, a real GTK label with 20 horizontally clipped lines produced VisibleText{Limited: true}. After the change, the same fixture captures more than 256 characters in at most 21 spans. A deterministic fixture verifies 800 Unicode characters in 20 spans, exact offsets, and clipping at both edges. Other regressions cover a span crossing geometry blocks, disjoint bidirectional runs, and a genuinely excessive number of separate visible spans. Existing tests cover blank rows, embedded objects, large terminals, cancellation and malformed provider data.

The successful four-row capture benchmark, three runs per version, measured 14.6–15.1 microseconds and 180 allocations before, versus 10.4–10.7 microseconds and 19 allocations after. Allocated bytes fell from 19,664 to 4,048 per capture. This measures local layout preparation with a synthetic provider; it does not measure AT-SPI round trips or prove live speech latency.

Automated checks

go test ./... -count=1
go test -race ./internal/atspi ./internal/review ./internal/app ./internal/presentation -count=1
go vet ./internal/atspi ./internal/review ./internal/app ./internal/presentation
python3 tests/native_atspi.py
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_CLIPPED=1 CTHULHU_NATIVE_REVIEW_CACHE=1 CTHULHU_NATIVE_REVIEW_BOUNDS=1 go test ./internal/atspi -run '^TestNativeFlatReview(ClippedViewport|PreservesProviderCache|BoundsSlowLongRanges)$' -count=1 -v
go test ./internal/atspi -run '^$' -bench '^BenchmarkCaptureVisibleTextClipped$' -benchmem -count=3

Use a writable GOCACHE if required by the environment. Native harnesses create private Xvfb/D-Bus/AT-SPI sessions and do not replace the running reader. All checks above passed for this change. The existing table harness emitted GTK Atk-CRITICAL object-role warnings while its assertions passed; the cause of those warnings was not investigated here.

Follow-up implementation and evidence

Large-object line discovery moves by validated complete line boundaries during binary search; it never scans terminal scrollback. Oversized horizontal text uses native offset-at-point and glyph rectangles to discover visible offsets, then reads text in spans of at most 128 characters. Offsets are sorted before capture so bidirectional runs retain exact logical identities. Point traversal has a 4096-query budget; it checks gaps before skipping vertically so an inset or shorter glyph is not missed. Ordinary clipped capture continues to use bounded geometry blocks. Unsupported or excessive work yields an explicit limitation, not guessed text. No global capture limit was increased.

A native provider may return a zero-area rectangle for a nonvisible glyph. That is distinct from a failed native request. Such glyphs now break visible spans instead of discarding the entire object's usable text. Unavailable whole line geometry remains an unavailable capture. A private GTK fixture injects a zero-area glyph and verifies that the surrounding visible text survives and the missing offset is excluded. Combining marks and joiners attached to a visible base retain their Unicode offsets even without independent rectangles. Bounded endpoint lookup also handles a huge line ending in a zero-area newline.

ChildrenUnavailable is capture-owned state copied into the owned tree and atomic review snapshot. Recovery is restricted to review-only child enumeration; root enumeration failure and cancellation remain fatal. FlatCompleteness travels with each worker result. No partial object's accessible name is silently substituted for its missing text. Incomplete-capture diagnostics are logged on entry or when omission counts change; returning to a complete layout clears that diagnostic state. Ordinary invalidation triggers recapture as before; there is no broad automatic retry loop. Partial layouts add no speech or Braille warning, and boundaries retain the standard top/bottom-of-screen messages. Spelling retains its own request.

Seven private native cases cover cache reuse, bounded geometry, clipped spans, a 100000-character horizontal line, scrolled line-only geometry, unavailable child enumeration, and zero-area glyphs. Additional worker and presentation fixtures cover healthy-content recovery, omission counts, recovery to a complete layout, ancestor anchoring, spelling without extra speech/Braille messages, and standard boundary feedback on incomplete layouts. Run native modes in separate processes:

python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_VIEWPORT=1 CTHULHU_NATIVE_REVIEW_VIEWPORT_MODE=long go test ./internal/atspi -run '^TestNativeFlatReviewLargeTextViewport$' -count=1 -v
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_VIEWPORT=1 CTHULHU_NATIVE_REVIEW_VIEWPORT_MODE=lines go test ./internal/atspi -run '^TestNativeFlatReviewLargeTextViewport$' -count=1 -v
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_SUBTREE=1 go test ./internal/atspi -run '^TestNativeFlatReviewUnavailableSubtree$' -count=1 -v
python3 tests/native_test_session.py env CTHULHU_NATIVE_REVIEW_GLYPHS=1 go test ./internal/atspi -run '^TestNativeFlatReviewZeroAreaGlyph$' -count=1 -v

A read-only probe of the open FastSMRW instance exposed zero-area glyph failures that previously left visible table-cell text unavailable. After the repair, 27 worker commands across line, word, character, window-start, and window-end review passed with zero reported omissions. Cached commands took about 0.3-0.7 milliseconds. Initial capture still took about eight seconds, mostly in visible text. The standalone probe had no retained native focus event and reported root_scope=desktop while filtering capture to FastSMRW's provider; this timing does not establish normal foreground reader latency. No application keys were injected and the running reader was not replaced. These results do not establish audible speech or physical Braille acceptance.

Follow-up check results

The full Go suite, affected-package race tests, vet, command builds, parity ledger validation, recorded-speech checks, and all seven targeted native GTK cases passed. The broad tests/native_atspi.py harness initially failed TestNativeGUIListTableSpeech because the launcher description was repeated. The isolated table checks passed on both an untouched 50e32f2 snapshot and the changed tree, and a subsequent complete harness run passed. The initial failure was not explained; passing reruns do not establish that this startup/event-order behavior is stable. Existing GTK Atk-CRITICAL role warnings also remain.

Dynamic-page visibility cache regression

On 2026-10-22, TestNativeBrowserDynamicNavigation reproduced a separate flat-review failure after browse navigation scrolled a 1000-paragraph page to newly inserted content. Window-end review returned "New content", but previous line returned the same text with Moved=false, including after a text-triggered recapture.

Temporary capture diagnostics showed one review line: cached AT-SPI SHOWING states still described the original viewport. Paragraphs now on screen were treated as hidden, while the originally showing paragraphs had current bounds far above the window. Cursor restoration was not the cause. The 40 omitted objects were browser-chrome text-provider failures and remained after the fix; they did not account for the missing page paragraphs.

For a node entering a review-only native capture, refresh the cached state bits before reading its snapshot and geometry. Preserve cached children, interfaces, and the existing dirty-node refresh policy. This recovers the current viewport without adding retries or changing any text, range, tree, or deadline limits. The diagnostic capture then contained 26 review lines. Temporary text-bearing diagnostics were removed from the code.

CTHULHU_NATIVE_BROWSER_TESTS=dynamic python3 tests/native_browser.py

The strengthened regression keeps the original window-end/previous-line sequence and requires "New content", "End of page", and "Unrelated paragraph 999" in order. It also checks forward lines, words, characters, the last character at window end, and backward recovery after the bottom boundary. Private GTK provider-cache and clipped-text regressions and the native AT-SPI harness passed, including terminal and table workflows. The table harness still emitted GTK Atk-CRITICAL role warnings while its assertions passed. Race tests and vet for internal/atspi, internal/review, and internal/app, plus go build ./cmd/..., also passed. The initial ordinary app-package run could not create private Unix speech sockets inside the sandbox; the approved race run outside that restriction passed all three packages.

For live acceptance after an authorized build/restart, browse to the bottom of a long dynamically updated page. Press Cthulhu+Numpad 9, then Numpad 7: speech and Braille should move from the final visible line to its predecessor. Check forward movement back to the final line, bottom feedback, and backward movement again. Repeat after scrolling back up and check word/character review and a native terminal. These automated checks do not establish physical-key, audible speech, or Braille acceptance.

Browser chrome omissions and standard messages

A private Chromium-family browser probe reproduced 40 omitted objects. Native text queries exposed three distinct provider patterns: containers containing only embedded-child placeholders, named controls returning their accessible labels through Text but invalid line offsets, and real text with unavailable geometry. Both character and range geometry were zero for the failing chrome objects. The address entry's native Text contained its URL, not its label.

After a provider failure, bounded verification now allows embedded-only containers to use child-provided content. A noneditable named control or static label can use the existing synthetic-name path only when its native text exactly equals its accessible name and its component is fully inside the clip. Editable text, passwords, differing names/text, mixed container content, capture limits, and clipped labels do not qualify. Recovered names have no native text offsets. This is a generic provider-content rule, not a browser-name exception.

Tabs and compound menu rows also retain their own label beside independently named child actions such as Close. Text/label descendants still suppress duplicate parent labels. Ordinary nested-button suppression is preserved.

The new browser proof fails with the fixes disabled: Back, Forward, Reload, New Tab, and the tab label are absent, with 40 omissions. With the fixes enabled, these controls and popup entries are reviewable. The window fixture reports two omissions (the address entry and a mixed-text panel); its popup reports none. These counts describe this fixture, not every browser or screen. The mixed-text panel remains conservatively unresolved even though a static child supplies its displayed label. The address URL remains unavailable in flat review because its visible character span cannot be established from zero geometry. Neither its label nor its entire potentially clipped value is substituted. Ordinary address-bar focus and editing remain separate from geometric flat review.

The partial-content speech/Braille warning is removed. Both complete and partial layouts use "Top of screen." and "Bottom of screen." Omission counts remain in diagnostics; fatal capture errors keep their existing messages. Ask before adding or replacing established user-facing messages.

CTHULHU_NATIVE_BROWSER_TESTS=flat-chrome,dynamic,menu python3 tests/native_browser.py

The new proof checks actual tab-label zones, toolbar controls, focused/unfocused address handling, and popup review. The dynamic-page proof retains its stale state-cache fix and backward/forward line, word, and character navigation. Focused unit cases reject unsafe component recovery and verify tab/menu labels with child actions and text-label deduplication. Existing cache, clipping, and zero-area-glyph native checks also remain relevant.

After an authorized build/restart, check speech and physical Braille separately: review the tab strip and toolbar, including a tab's title and Close control; open the browser menu and review its rows and inline controls; return to the page and exercise dynamic-page backward review. At the first/last review line, Numpad 7/9 must announce only the standard screen boundary. No partial-content message should interrupt review or flash Braille. Check normal address-bar focus/typing, a narrow/scrolled field, passwords, a native terminal, and native text clipping. Address-value flat review remains a known limitation in the observed provider. Private tests do not establish audible speech, physical keys, or Braille acceptance, and the running reader was not replaced.

Human review and live acceptance

Review the generated diff before submission. With an authorized build/restart:

1. In a native editor or a window containing a wide multiline text control, horizontally scroll so multiple lines are clipped. Review lines, words, and characters forward and backward; visible text must remain available when the total exceeds 256 characters. Repeat after resizing and scrolling. Check the first and last partially visible characters and Unicode offsets.
2. Review ordinary and large terminal screens, blank rows, and a large file in a terminal editor. Check Numpad 7/8/9, 4/5/6 and 1/2/3, window start/end, explicit top/bottom feedback, and multi-press spelling.
3. Check a browser with horizontally clipped text and embedded links, a GTK table such as a timeline, a preferences notebook, and a popup menu. Hidden letters and embedded-object placeholders must not be spoken as ordinary text. Password contents must stay excluded. Review must stay in the foreground window and ordinary focus/caret navigation must still work afterward.
4. Exercise a window with one unavailable text object or child subtree. Healthy content must remain reviewable without an additional partial-content announcement. Verify omission diagnostics in the log, then refresh after the provider recovers. Top/bottom feedback must use the same standard messages before and after recovery.
5. Confirm audible speech and physical Braille separately, including routing within a merged span. Native capture and Go test success do not establish these outcomes. Preserve fresh logs for any remaining failure.

The code change affects shared visible-text capture used in owned trees; terminal, browser, mouse-review, and Braille consumers can therefore be affected indirectly. It changes neither X11 key grabs nor foreground ownership. Wayland live acceptance remains separate.