Files
cthulhu/docs/testing/remote-input-feasibility.md
Chase CichorzandClaude Opus 5.5 c1333c2fdb Route quarantined Remote capture and balance bypass modifiers
Add RemoteCaptureRouter for normalized controller capture. Ordinary keys
and Cthulhu/NVDA modifier chords are forwarded; only a required core
escape chord and local chords derived from resolved bindings stay local.
A local chord first releases forwarded modifiers remotely; a later
forwarded key presses still-held ones again. Releases of local or
withdrawn keys are consumed, and retirement reports physically held keys
whose releases the native owner must swallow. No production caller.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 16:06:38 -05:00

11 KiB

Remote Xorg input feasibility

This is an isolated prototype and a record of limitations, not a Remote input implementation. No reader input behavior or installed executable changes. A subsequent Go destination-key prerequisite now supplies stored-code ownership, retryable cleanup and a quarantined native resolver. It does not inherit this Python prototype's independent EOF/watchdog behavior and is not attached to production input. The source baseline is a47253d697c1aa44dea80646cad99a4fe163c7f1 on testing, with the existing secret-entry work preserved. The branch was checked against its upstream on 2026-11-03 before these tests were added.

Run the private checks

From the reader checkout:

python3 -B tests/native_remote_input.py

Requirements are Python 3, Xvfb/xvfb-run, D-Bus/dbus-run-session, Xlib, XTEST, xdotool, xev, xset, Go and the reader's usual native build dependencies. Missing tools or failed checks fail the command; they do not count as skipped acceptance. The launcher starts a 24-bit Xvfb display before private D-Bus, with disposable runtime/config directories. It never connects the fixture to the desktop display or the active reader. Invoke it without private fixture arguments.

The Python tests create a focused X window, a capture process and independent writer/helper processes. Only the writer retains the private control pipe's write end. The harness kills or stops that writer to simulate a dead or hung reader, while retaining the ability to reap both fixture children. This models descriptor ownership; it does not test the real reader's process launch tree. The prototype helper uses fixed two-byte records, a held-key set and a three-second monotonic watchdog renewed only by explicit heartbeats. It releases owned keys on EOF, timeout, malformed input or an already-held-key collision.

The same private session runs three Go race tests against the real Xorg owners: the new injected-key observation check and the existing internal-form capture and learn-mode application-isolation tests.

Evidence recorded on 2026-11-03

All ten private Python experiments and the three native Go race tests passed. The adjacent Tentacles repository's make check also passed, including existing command-host checks; its unchanged Go package tests used cached results.

  • An exclusive XGrabKeyboard consumed the test press and release. The focused application received neither. Killing the grab owner made the grab available again, and a fresh key reached the application.
  • The existing internal-form and learn-mode tests retained their capture and restoration behavior with the race detector enabled. This does not establish coexistence with a new Remote mode or a desktop window manager.
  • Closing an XTEST client's X connection left its injected Shift down. X connection lifetime alone is therefore insufficient for injected-key cleanup on this server.
  • Killing the writer released the helper's Shift and Control keys and restored application input. The initial passing run observed EOF cleanup in about 0.010 seconds. Stopping the writer without closing its descriptor triggered cleanup in about 3.019 seconds. These are observations from a private Xvfb run, not hard real-time or physical-desktop recovery guarantees; each run prints its own timings and enforces a four-second test bound.
  • Heartbeats extended the watchdog. Duplicate downs, unmatched ups, fragmented input ending at EOF and invalid records did not leave fixture-owned keys held.
  • A key already down through another client was not adopted or released. A collision revoked the prototype and released its other owned keys.
  • A second XTEST client pressing the same code after the helper's down was not separately represented by the global key bitmap: helper cleanup cleared that code. The corresponding test records this limitation, not safe collision handling. No physical key device participates in this test.
  • Injected events had send_event == False, and the reader's current XRecord listener delivered them through the ordinary input event stream. Neither that flag nor the existing event representation proves local physical origin.

Decisions and remaining gates

An independent reader-owned helper with an exclusive inherited control descriptor is a viable cleanup direction for the tested failures. It must not be a child of the network Tentacle or be killed with that Tentacle's process group before releasing keys. Reader shutdown must close/revoke the helper and allow bounded cleanup before forced termination. Prevent inherited write ends in unrelated descendants; otherwise reader death will not produce EOF.

Do not promote this Python fixture to a production helper. It uses synchronous Xlib calls and blocking test acknowledgements, and has no lease, epoch, sequence, input-origin or reader-owner integration. A production Go/native helper needs bounded IPC and X-server failure handling, stale-work invalidation, explicit error reporting, restart policy and lifecycle tests. Simultaneous helper death or a blocked X server is not recovered by this experiment.

Trusted input origin remains a release blocker. A subsequent authorization increment now adds separate XI2 capture for sensitive forms, XTEST/device-node filtering and private rejection tests. Positive physical-keyboard acceptance and production command integration remain pending. The ordinary reader event schema contains keycode, transition and timestamp but no source device or core-owned injection identity. Do not grant local consent based on an XRecord event, send_event, or a timing-only match. Investigate device-aware events and core-owned injection routing, including exact-once target reader commands, echo suppression and exclusion from consent/clipboard/trust actions.

Physical/injected same-key collisions also remain a blocker. Checking the global bitmap before a down only handles an already-visible collision, with a check/inject race; it does not resolve a physical press arriving while a remote key is held. Do not clear all modifiers or assume the synthetic two-client result establishes physical-device behavior.

Capture-side repeat normalization (2026-11-06)

internal/input/remote_repeat.go adds a quarantined RemoteRepeatNormalizer for a future controller capture owner. Remote's outgoing encoder requires one fresh down, repeated downs and one final up; the ordinary RepeatTracker marks repeats but still exposes the release of a classic X11 release/press pair.

The normalizer holds a nonzero-timestamp release until the next observation or an explicit deadline (initially 25 ms, at most 250 ms). A press of the same keycode with the same server timestamp becomes a repeat and the release is dropped. Any other observation, a zero timestamp or the deadline emits the release first, so ordering is preserved. Pairing is decided by event content, not timer accuracy; a late flush adds latency only. Detectable-autorepeat downs pass through as repeats. Releases for keys held before capture are suppressed and never treated as owned. An invalid keycode or more than 32 held keys stops admission; Retire returns one release per owned key, including a pending release, in reverse down order, exactly once. It performs no I/O and has no production caller; it grants no consent or provenance.

TestNativeRemoteRepeatNormalizesHeldKey runs in the harness above. It holds an XTEST key with private Xvfb autorepeat set to 200 ms/40 Hz and feeds the real XRecord listener's events through the normalizer with timer-driven flushes. Five repeated runs on 2026-11-06 each observed 22 raw downs, one raw up and no classic release/press pairs: on this Xvfb, XRecord reported server autorepeat as repeated presses. The normalized stream was one down, 21 repeats and one up, with nothing left owned. The classic-pair path is therefore covered only by unit tests; physical keyboards, other X servers and XGrabKeyboard-based capture may differ and remain unverified. The complete harness, including the existing internal-UI, learn-mode and injected-approval tests, passed with this test added. The xset change must follow the listener's connection because Xvfb resets defaults when its last client exits.

Capture routing and bypass balancing (2026-11-06)

internal/input/remote_capture.go adds a quarantined RemoteCaptureRouter that consumes normalized transitions for one controller capture lifetime. Ordinary keys and Cthulhu/NVDA modifier chords are forwarded. Only a required core escape chord (a non-modifier key; proposed F11) and supplied local chords stay local. Callers derive local chords from ResolveCommandBindings filtered to Remote's commands, so user-assigned shortcuts are honored instead of a hard-coded bypass list; the escape always wins over a local binding. Chords use the same group-zero, level-zero keysym and modifier bits as the command owner.

Modifiers are forwarded when pressed because they cannot be classified yet. When a later key completes a local chord, every forwarded modifier is released remotely first (newest first) and marked withdrawn, so no prefix remains held on the remote. A forwarded key pressed while withdrawn modifiers are still held presses them again before itself, so the remote sees the physical chord. Repeats and releases of local and withdrawn keys are consumed. Releases for keys held before capture are ignored. Orphan repeats, double downs, invalid codes and more than 32 held keys stop routing. Retire returns remote releases for forwarded keys in reverse order plus a Swallow list of every physically held key: the local application never received those presses, so the native owner must consume their releases before ungrabbing.

Limitation: a withdrawn prefix reaches the remote as a tap of those modifiers. On Windows a lone Alt tap can activate an application menu bar, and NVDA treats modifier taps specially; this needs live testing and may need a neutral key between press and release. The router has no production caller, performs no I/O and grants no consent or provenance. Nine unit tests cover forwarding, withdrawal and re-press, consumed releases, escape priority, retirement, inconsistent input, invalid chords, resolved Tentacle bindings and composition with the repeat normalizer.

Still pending: Remote capture within the command owner; local escape and balanced bypass chords; remapping, learn/UI/lock/handoff arbitration; production helper parent-death and blocked-pipe tests; held-key collisions with actual devices; Wayland capability behavior; virtual Braille feasibility; and all live NVDA/Orca, audible-speech and physical-Braille acceptance. Phase A remains open. There is no live Remote command to launch or user desktop test for this increment.