Provide fresh Wine review snapshots and 32-bit Edit hooks
Expose visible Wine window text to the matching Go reader's independent keypad review commands, and extend standard Edit feedback to 32-bit programs such as MUSHclient without requiring 32-bit Linux GLib/Winelib libraries. Add lifetime-aware instance/window/control identities and bounded capture of standard Edit text, button/static captions, native ListBox/ListView items, and usable MSAA text. Preserve focus, caret, selection, scrolling, and the clipboard. Exclude hidden/password content before retrieval, retain both rune and UTF-16 offsets, clip visible ranges, and reject changing or oversized sources rather than returning incomplete text as a full snapshot. Export CaptureWindow on the version-1 WineReview D-Bus interface. Register with the current reader service owner and publish copied focus metadata. Authorize capture only for the current same-user owner. Use a dedicated GLib dispatcher and separate COM worker with a 500 ms deadline, one active job and one pending job, bounded payloads, and explicit stale/busy/limited/ unavailable replies. Keep menu and focus forwarding responsive when an MSAA provider stalls; retain ordinary feedback with older readers. Read the actual X11 geometry associated with Wine HWNDs using bounded XCB queries and __wine_x11_whole_window. Translate virtual-desktop coordinates and recheck physical geometry before returning captured text. Link XCB in the helper/tests and declare libxcb in the Arch package dependencies. Build edit_hook.c into the 32-bit DLL and attach both menu and input hooks to the foreground 32-bit UI thread. Release both hooks on transitions or attachment failure. Keep Scintilla-specific hooks confined to 64-bit builds. Expand Edit integration checks with read-only multiline navigation and unsuccessful deletion, plus 32-bit editor/Windows Forms lifecycle support. Add 32-bit/64-bit native capture fixtures, wire-protocol integration checks, and design/implementation documents. Document exact schema, capture limits, installation dependencies, compatibility, and live acceptance procedures. Validation during commit preparation: - git diff --check passed. - cmake --build build --parallel passed. - CTest could not start because xvfb-run is unavailable in this environment. - Further tests were skipped at the user's request; capture, protocol, and menu/editor integration tests were not run during commit preparation. Live speech and physical braille checks remain necessary, including review without moving the native caret, incoming output, plugin dialogs, and return to Linux/Fenrir. Custom painted output requires an accessible provider; RichEdit/Scintilla capture, semantic object review, Say All, typing echo, and native Wine braille routing remain unsupported by this protocol.
This commit is contained in:
1 parent
872388a59b
commit
a8cb350467
21 files changed
+2401
-38
No files matched your search
@@ -0,0 +1,233 @@
|
||||
# Cthulhu review keys in Wine
|
||||
|
||||
Status: approved, implemented, verified and installed on 2026-10-11. Independent review fixes and matching-binary desktop checks completed; MUSHclient listening and physical braille acceptance remain live checks.
|
||||
|
||||
## Intended outcome
|
||||
|
||||
The user wants the existing Cthulhu review keys to work in Wine, especially
|
||||
MUSHclient running in `~/.local/bard`. Keep the current desktop keyboard
|
||||
bindings. Review must read and navigate independently of application focus,
|
||||
caret, selection, and keyboard input. The user confirmed this scope on
|
||||
2026-10-11.
|
||||
|
||||
Support current/previous/next line, word, and character; current-word spelling
|
||||
and phonetics; current-character phonetics; window start/end; word above/below;
|
||||
and line end. Reuse current speech and braille presentation and boundary
|
||||
behavior. This is flat text review, not semantic parent/child object navigation,
|
||||
Say All, editing, mouse activation, or braille routing into Windows controls.
|
||||
|
||||
Cover exposed visible text and controls in 32-bit and 64-bit Wine applications.
|
||||
The concrete MUSHclient cases are its command Edit and read-only output notepad.
|
||||
Its custom game display does not become readable merely by adding this bridge;
|
||||
the existing output plugin remains the source of reviewable game text.
|
||||
|
||||
## Existing behavior and cause
|
||||
|
||||
`wine-access/main.cpp` supplies normalized focus/menu/editor events to
|
||||
Cthulhu's session D-Bus service. The 32-bit standard Edit feedback change is
|
||||
already implemented and locally installed. Preserve that work.
|
||||
|
||||
`cthulhu-go/internal/app/runtime_commands.go` sends review commands to
|
||||
`review.Worker`. Its `CaptureReviewWindow` provider currently captures only
|
||||
AT-SPI objects. Wine focus events go to a separate speech presenter and never
|
||||
establish a review provider or a text snapshot. A spoken caret line is not a
|
||||
reviewable window.
|
||||
|
||||
`review.FlatCursor` already navigates immutable window trees and preserves a
|
||||
position through stable object identities. `AccessibleTree.Replace` preserves
|
||||
generations of live identities. Reuse these mechanisms instead of implementing
|
||||
a second Wine-specific navigation algorithm or reviewing cached speech.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Wine capture provider
|
||||
|
||||
Add a focused capture module beside `main.cpp`, rather than putting enumeration
|
||||
and serialization in its event loop. The helper publishes a read-only D-Bus
|
||||
capture endpoint on its existing unique session-bus connection. Multiple Wine
|
||||
prefixes therefore have independent endpoints without competing for one name.
|
||||
|
||||
Capture uses a dedicated COM STA worker, independent of the existing native
|
||||
event/menu thread. Permit one active capture and one pending capture; reject
|
||||
additional requests promptly. Native accessibility objects stay on that STA
|
||||
and are released there. Application queries use `SendMessageTimeoutW` where
|
||||
available. A COM call that does not return cannot be forcibly cancelled safely;
|
||||
the Go deadline discards its result, and the busy provider rejects further
|
||||
capture work instead of accumulating it. Helper shutdown remains bounded by
|
||||
the supervisor's existing process-stop policy.
|
||||
|
||||
Collect a complete snapshot of the foreground Wine window/dialog: hierarchy,
|
||||
stable identities, roles, captions, protected states, clipped screen bounds,
|
||||
visible text ranges, and the focused Edit's caret when available. Traverse MSAA
|
||||
objects and use existing standard-control queries when Wine exposes only a
|
||||
generic client. Standard Edit includes Windows Forms Edit subclasses, in both
|
||||
architectures. Preserve mnemonic decoding for captions and literal ampersands
|
||||
in values. Password text is excluded before copying or serializing it.
|
||||
|
||||
For Edit text, collect displayed visual lines, including soft wraps and blank
|
||||
rows, with native character positions and client clipping. Do not scroll,
|
||||
select text, change focus, send navigation keys, or copy to the clipboard.
|
||||
The helper converts UTF-16 offsets to Unicode code-point offsets before
|
||||
serialization, retaining native offsets for diagnostics. Computing absolute
|
||||
offsets requires the preceding source text, so bound full Edit source reads at
|
||||
one million UTF-16 units in total per window and return `limited` above that
|
||||
bound. Only displayed ranges cross D-Bus. No approximate character geometry
|
||||
is presented as native.
|
||||
|
||||
First implementation uses standard Edit and exposed native/MSAA captions and
|
||||
values. Custom painted text, RichEdit, and Scintilla text capture are excluded
|
||||
unless their existing accessibility data supplies usable visible text. Their
|
||||
caret feedback remains as it is today.
|
||||
|
||||
### Additive protocol
|
||||
|
||||
Leave `PresentAccessibleEvent` and NVDA/Tolk contracts unchanged. Add:
|
||||
|
||||
* `RegisterWineReviewProvider(object_path: o, protocol_version: u) -> b` to the Cthulhu
|
||||
service. Version 1 registration is associated with the actual unique D-Bus
|
||||
sender after the existing same-user authorization, never a caller-supplied
|
||||
destination name. Registration supplies no native references.
|
||||
* `PresentWineReviewFocus(instance: s, window: s, focus: s, revision: t,
|
||||
title: s, role: s, protected: b, bounds: (iiii)) -> b` for that registered
|
||||
sender. Window and focus are opaque lifetime identities. Emit on focus/owner
|
||||
changes and retire identities
|
||||
on destruction. The existing bounded foreground poll recovers missed events.
|
||||
* `CaptureWindow(expected_window: s, expected_focus: s, expected_revision: t)
|
||||
-> a{sv}` on
|
||||
`/org/stormux/Cthulhu1/WineReview`, interface
|
||||
`org.stormux.Cthulhu1.WineReview`. Return a versioned owned snapshot with its
|
||||
identities, revision, bounds, nodes, text ranges, and capture status.
|
||||
|
||||
The capture dictionary has required keys `version: u`, `instance: s`,
|
||||
`window: s`, `focus: s`, `revision: t`, `status: s`, `bounds: (iiii)`, and
|
||||
`nodes: aa{sv}`. Status is `ok`, `unavailable`, `stale`, `busy`, or `limited`;
|
||||
non-ok responses contain no nodes. Each node has `id: s`, `parent: s` (empty
|
||||
only at root), `role: s`, `name: s`, `protected: b`, `visible: b`,
|
||||
`bounds: (iiii)`, `text_known: b`, `caret_known: b`, `caret: t`,
|
||||
`caret_utf16: t`, and `ranges: a(tttts(iiii))`. Ranges contain code-point
|
||||
start/end, native UTF-16 start/end, exact UTF-8 text, and bounds. `caret`
|
||||
counts code points. Go validates offsets, rune counts, and the resulting
|
||||
tree before use. Captions and identity fields
|
||||
are individually bounded at 4096 and 256 UTF-8 bytes respectively. Limit
|
||||
traversal depth to 64 and reject cycles or duplicate node identities.
|
||||
|
||||
Revision tracks focus/window lifetime changes, not incoming text. Within a
|
||||
capture, recheck foreground/focus identity before returning and verify that
|
||||
each collected control's text length and visible ranges remain consistent.
|
||||
Return `stale` on a detected capture race, with at most one recapture under
|
||||
the original deadline. Text updates do not retire live control identities.
|
||||
|
||||
Native handles in identities are opaque fixed-width integers. Include a
|
||||
helper-instance identity and a lifetime counter so reused HWNDs cannot inherit
|
||||
an old review position. MSAA child identities remain subordinate to their
|
||||
owning window lifetime; destroy/reorder events retire uncertain child identity.
|
||||
Do not serialize pointers, COM objects, or prefix environment data.
|
||||
|
||||
The capture endpoint authorizes the current Cthulhu service owner under the
|
||||
same-user trust model. Reject invalid versions, identities, dimensions, offsets,
|
||||
UTF-8, and oversized messages at both ends. Older helpers/readers retain normal
|
||||
speech; review reports unavailable when this optional protocol is absent.
|
||||
|
||||
### Go ownership and integration
|
||||
|
||||
Place provider registry, capture client, validation, and snapshot conversion
|
||||
in cohesive `internal/wine` files. The registry owns its session-bus connection
|
||||
and tracks provider disconnection. Copied focus notices go to the event-loop
|
||||
owner through a bounded mailbox; D-Bus callbacks never mutate event-loop state.
|
||||
|
||||
Compose a review capturer around the existing AT-SPI adapter. Wine identities
|
||||
use the registered helper's unique bus name and a reserved `/wine/...` object
|
||||
path namespace. Only this composition layer routes those identities to Wine;
|
||||
they must never be sent to AT-SPI native operations. Ordinary AT-SPI capture
|
||||
keeps its current path.
|
||||
|
||||
The event loop owns which focus source is eligible. On Xorg, corroborate Wine
|
||||
window bounds and metadata with the current foreground client and epoch.
|
||||
Support a child window inside Wine's virtual desktop using the existing
|
||||
foreground/title discovery. Title equality alone cannot establish ownership.
|
||||
When foreground ownership is unknown, reject Wine capture rather than using
|
||||
an earlier Linux focus. Returning to Linux restores its normal review path.
|
||||
|
||||
`review.Worker` remains the only owner of the flat cursor and capture ordering.
|
||||
Before each Wine review command, obtain a fresh bounded snapshot, updating a
|
||||
stable owned tree so live object generations and independent review offsets
|
||||
survive text updates. Refreshing only on focus events would miss streaming
|
||||
MUD output. Reuse existing capture/result queues and presentation methods;
|
||||
native text is not fetched on the input or event loop.
|
||||
|
||||
Accept a result only if the foreground epoch, registered provider, focus,
|
||||
window lifetime, and source revision still match. Focus change, provider death,
|
||||
helper restart, window destruction, or lock activation retires Wine review
|
||||
ownership and queued work. Repeated focus notifications for the same live
|
||||
control preserve review position. Empty focused edits remain valid anchors so
|
||||
review can reach other visible content in the window.
|
||||
|
||||
## Bounds, failures, and presentation
|
||||
|
||||
Use a 500 ms total Wine capture deadline, at most 4096 nodes, 256 visible text
|
||||
ranges per control, 4096 code points per visual line, 65536 captured code points
|
||||
per control, and 1 MiB of text per window. Enforce a 2 MiB serialized snapshot
|
||||
limit including metadata. Exhaustion returns a limited status; never present a
|
||||
partial snapshot as a complete window. The existing review command FIFO remains
|
||||
bounded at 16. Shutdown cancels outstanding D-Bus work and drops late results.
|
||||
|
||||
Unavailable capture, no visible content, and capture limits use existing
|
||||
complete review messages. Keep Python/current Go's silent movement boundaries,
|
||||
spelling and phonetic sequences, and blank/space handling. Review speech and
|
||||
braille go through the normal queues and settings; no direct speech calls from
|
||||
the helper. Debug logs include provider/source identities, epochs, native
|
||||
UTF-16 and converted offsets, capture timing, command, final speech, and braille
|
||||
text. Password contents never enter debug logs.
|
||||
|
||||
## Verification and acceptance
|
||||
|
||||
Write failing behavioral checks before production changes:
|
||||
|
||||
* Helper fixtures, 32-bit and 64-bit: visible wrapped text and blank lines,
|
||||
Unicode/surrogate conversion, read-only multiline output, empty command field,
|
||||
captions/native control fallback, hidden/clipped content, password exclusion,
|
||||
unchanged application caret/selection/focus, and capture limits.
|
||||
* Go protocol and review tests: real private-session D-Bus registration/capture,
|
||||
sender association and malformed payload rejection; all listed review units,
|
||||
spelling/phonetics and quiet boundaries; text updates preserving position;
|
||||
independent provider identities for two prefixes; focus switches, reused
|
||||
handles, provider death, helper restarts, stale replies, timeout/queue limits,
|
||||
and cancellation.
|
||||
* Existing helper menu/editor/list regressions and focused Go review, remote,
|
||||
Wine, speech and braille tests with race checks.
|
||||
* Private Xorg fixture driving physical X11 key events through the actual Go
|
||||
command router and both Wine architectures. Verify key press/release balance,
|
||||
focus switching to Linux, input responsiveness during a delayed capture, and
|
||||
no application caret movement caused by review keys.
|
||||
|
||||
Run private displays sequentially. Recorded speech/braille requests establish
|
||||
queue behavior, not audible speech or physical braille acceptance. Live
|
||||
MUSHclient acceptance must check the user's existing desktop review keys in the
|
||||
command field and output notepad, repeated-word spelling, reading while output
|
||||
arrives, plugin dialogs, and returning to Linux. Install matching tested helper
|
||||
and reader builds only after verification; preserve backups and game state.
|
||||
|
||||
## Work locations and delivery
|
||||
|
||||
Helper changes belong in this standalone repository. Reader integration and
|
||||
its tests belong in `/home/deedra/git/cthulhu-go`, with focused evidence in its
|
||||
Wine testing note and the corresponding gap updated in `MIGRATION_PLAN.md`.
|
||||
The reader directory and installed binaries are outside this session's writable
|
||||
roots, so writes there require tool approval. Existing uncommitted helper
|
||||
changes and `cthulhu.log` must be preserved.
|
||||
|
||||
After user approval of this document, write a reviewable implementation plan
|
||||
and obtain the required plan/execution decision. Keep source changes available
|
||||
for human review; do not submit them automatically.
|
||||
|
||||
## Reference evidence
|
||||
|
||||
Local behavior references: Cthulhu Python `flat_review_presenter.py` for desktop
|
||||
bindings, independent navigation, spelling, braille and silent boundaries;
|
||||
Cthulhu-Go `flat_review.go`, `flat_worker.go`, `flat.go`, `flat_layout.go`,
|
||||
`AccessibleTree.Replace`, foreground/title ownership, and remote authorization.
|
||||
The Python Wine bridge provides speech events, not a reusable review provider.
|
||||
|
||||
Microsoft documents [Edit character positions](https://learn.microsoft.com/en-us/windows/win32/controls/em-posfromchar)
|
||||
and [Edit visible-line/text operations](https://github.com/MicrosoftDocs/win32/blob/docs/desktop-src/Controls/edit-controls-text-operations.md).
|
||||
These establish the native capture approach, not Wine or MUSHclient acceptance.
|
||||
Reference in new issue
Block a user