Observe authorized local speech for Remote sessions

This commit is contained in:
ak47 committed 2026-11-04 16:53:16 -05:00
1 parent 9bc74dbb13
commit b60cf5d5f2
12 files changed
+927 -12

No files matched your search

+78 -9
View File
@@ -1,10 +1,10 @@
# Remote speech transport prerequisite
This increment adds typed speech IPC and an explicit reader-owned mailbox. It
does not connect the speech worker, observe local output, or make Remote speak.
The external coordinator now connects this IPC to relay speech; the production
reader still has no presentation owner or output observer. The current increment
adds atomic assembly of split incoming utterances before owner admission.
Typed speech IPC now includes an explicit reader-owned incoming mailbox and
limited production observation of ordinary accessibility-event speech. The
external coordinator connects this IPC to relay speech. Incoming speech still
has no production presentation owner and does not make Remote speak. Atomic
assembly of split incoming utterances precedes owner admission.
Original base reader revision: `4f9b89e`
on `testing`; matching Tentacles base: `0c74038` on `master`, plus the pending
connection-command changes. No active or installed reader was replaced.
@@ -72,6 +72,57 @@ there is no child RPC that publishes an observed event. The method queues
Full IPC queues fail the process session, cancelling its lease. A receiving
Tentacle must recheck lease/epoch and peer readiness before sending over a relay.
## Limited production output observation
The Tentacle runtime now attaches its manager to the existing speech worker.
`Manager.CaptureRemoteSpeechOutput` returns a reader-only ticket for an opened,
live controlled lease with `output-observe`. The worker captures this ticket
when it admits explicitly local speech. Work queued before consent gets no
ticket; delivery never captures a replacement. The original lease, deadline and
context are checked again when publishing. Production admission also holds the
core availability policy lock, so lock/handoff cannot admit new output after
policy cancellation, including across unlock/return. Already accepted IPC or
relay writes cannot be recalled.
The zero speech origin is private. Only the main accessibility event presenter
currently marks requests local. Forms, preferences, connection/status messages,
incoming remote text, command-result Tentacle messages and other unclassified
sources stay private. The presenter classifies the original source and ancestors
before context filtering/reset, so a discarded password ancestor still excludes
its descendant's output. Password roles/ancestors, history exclusion, forced mode
announcements, confirmations and protected messages are independently denied.
This does not export every reading path: document/review commands, Say All,
reading-context announcements, separate key-name echo and notification paths
still need explicit source and sensitivity review. Derived phonetic hints retain
their parent's origin and original observation ticket.
Observation runs after speech-mode suppression, line/indentation processing,
pronunciation, spelling expansion and backend character-path selection. It
records resolved backend **attempts**, before backend I/O, not accepted audio or
audible completion. Character mode is exported only for a single valid CHAR
character; other text uses normal speech. Remote language is the default and
priority is normal; voice, language/profile fidelity and pause/resume remain
unimplemented. No sound cue or backend configuration is sent.
One local backend chunk becomes an observed utterance. Long UTF-8 text splits
conservatively into at most 64 events, each within the encoded speech limit;
only the last ends the utterance. The complete set of IPC frames must fit the
shared session queue and byte budget, including in-flight bytes, before any
prefix is queued. Invalid/oversized output or overflow retires the process
session, matching existing output-event overflow behavior. Local speech proceeds
independently. This atomic IPC admission is not atomic relay or NVDA delivery.
User/input interruption and mode/enable changes publish cancel through the same
speech-owner mutex as observed utterances. An older delivery generation cannot
publish behind that cancel. Remote publication never performs pipe, relay or
backend I/O in the input loop. Incoming cancellation ownership remains pending.
The source command still holds a modal status form for its connection lifetime.
That form captures local keys and suppresses ordinary application presentation;
this increment does not detach it or provide an ordinary desktop workflow.
Keep relay-testing warnings until that UI lifetime and incoming speech ownership
are implemented and live acceptance is recorded.
## Verification and remaining work
Run from this checkout:
@@ -91,10 +142,28 @@ multilingual speech through the actual source command and reader host to one
complete test-owner delivery. Existing lifecycle/SDK tests exercise old 1.0
behavior, process failure, timeout, event overflow and installation contracts.
Next: connect the policy-resolved speech worker with origin ownership,
secure/secret exclusion, delivery-time policy, local
priority and cancellation generations. Atomic IPC assembly is implemented;
it does not establish backend priority/cancel behavior or audible delivery.
The observation increment adds Go/Python SDK tests for captured authority,
UTF-8/escaped splitting, cancel order and stale tickets; isolated host tests for
all-or-none queue admission, in-flight bytes, expired/pending leases and policy
round trips; and speech-worker tests for resolved pronunciation/characters,
chunks/spelling, private origins, modes, generation and lock/handoff retirement.
The source-command/private-relay fixture also exercises production observation
attachment, transformed long output, private exclusion and cancel order with a
test speech backend. It does not drive physical input or audible synthesis.
Queue-level regressions additionally hold the backend across reconnection:
output admitted under old consent or before fresh consent cannot acquire the
replacement lease. An oversized observed utterance retires Remote while both
that local utterance and subsequent local reading still reach the test backend.
Ten repetitions of the observation race cases passed, along with the complete
host/wire/both-SDK race suites (host: about 127 seconds), focused application
speech/Remote/Tentacle race checks, affected-package vet, a separate reader
build, the existing speech regression harness and Tentacles `make check`.
Next: add incoming speech ownership, delivery-time policy, local priority and
remote-only cancellation, and detach connection status from desktop capture.
Review remaining output producers before assigning exportable origins.
Atomic IPC assembly is implemented; it does not establish backend
priority/cancel behavior or audible delivery.
Pause support or cancel-on-pause behavior and NVDA priority fidelity must be
resolved there. The codec transports metadata without claiming backend behavior.
Then verify normal reading, key echo, read-all, spelling, speech modes and local