Export document reading with Remote lease safeguards

This commit is contained in:
ak47 committed 2026-11-04 19:24:58 -05:00
1 parent 228271b43b
commit 622d2f96a1
11 files changed
+299 -17

No files matched your search

+18 -7
View File
@@ -1,7 +1,7 @@
# Remote speech transport prerequisite
Typed speech IPC now includes an explicit reader-owned incoming mailbox and
limited production observation of ordinary accessibility-event speech. The
limited production observation of accessibility-event and document-reading speech. The
external coordinator connects this IPC to relay speech. Incoming speech still
has no production presentation owner and does not make Remote speak. Atomic
assembly of split incoming utterances precedes owner admission.
@@ -84,17 +84,28 @@ core availability policy lock, so lock/handoff cannot admit new output after
policy cancellation, including across unlock/return. Already accepted IPC or
relay writes cannot be recalled.
The zero speech origin is private. Only the main accessibility event presenter
currently marks requests local. Forms, preferences, connection/status messages,
The zero speech origin is private. The main accessibility event presenter and
reviewed document-reading producers mark requests local. Forms, preferences, connection/status messages,
incoming remote text, command-result Tentacle messages and other unclassified
sources stay private. The presenter classifies the original source and ancestors
before context filtering/reset, so a discarded password ancestor still excludes
its descendant's output. Password roles/ancestors, history exclusion, forced mode
announcements, confirmations and protected messages are independently denied.
This does not export every reading path: document/review commands, Say All,
Document caret movement, document navigation and assembled caret lines inspect
their original source snapshots and unfiltered ancestors. Every content object
in an assembled line must have a known role and no password ancestry. Say All
classifies all spoken lines before admission; a sensitive or unclassified line
keeps the entire batch private, including sentence-mode continuations. Local
navigation failures stay private. Derived phonetic hints retain their parent's
origin and original observation ticket. Flat/object review, current-line commands,
reading-context announcements, separate key-name echo and notification paths
still need explicit source and sensitivity review. Derived phonetic hints retain
their parent's origin and original observation ticket.
still need explicit source and sensitivity review.
Say All exports each resolved chunk immediately before its completion-paced
backend call. Sentence boundaries, pronunciation and the original request's
lease/generation are retained. A lease replacement during reading cannot export
later chunks to the new helper; local reading continues. Context announcements
remain private, including labels remembered from an earlier reading context.
Observation runs after speech-mode suppression, line/indentation processing,
pronunciation, spelling expansion and backend character-path selection. It
@@ -164,7 +175,7 @@ build, the existing speech regression harness and Tentacles `make check`.
Next: add incoming speech ownership, delivery-time policy, local priority and
remote-only cancellation. Explicit status detachment is implemented; physical
input capture/release and ordinary reading while connected still need acceptance.
Review remaining output producers before assigning exportable origins.
Review the remaining output producers before assigning exportable origins.
Atomic IPC assembly is implemented; it does not establish backend
priority/cancel behavior or audible delivery.
Pause support or cancel-on-pause behavior and NVDA priority fidelity must be
+7 -3
View File
@@ -166,8 +166,9 @@ The process supervisor still enforces cleanup for a child ignoring cancellation.
Typed speech IPC now extends this transport; see
[the speech contract](tentacle-remote-speech.md). It uses the same ordered event
queue for exported speech and a separate 64-item incoming mailbox. No production
output owner is connected. Keyboard/Braille/clipboard effects remain unsupported.
queue for exported speech and a separate 64-item incoming mailbox. Limited
production speech observation is connected; incoming presentation ownership is
still pending. Keyboard/Braille/clipboard effects remain unsupported.
## Implemented lifecycle operations
@@ -203,7 +204,10 @@ positive sequence numbers. Heartbeat should be sent each second and expires afte
three seconds without refresh. Expiry revokes the lease independently of command
workers and pipe writes. A late heartbeat cannot revive it. A new local action
revokes old access and advances the core epoch. Closing an already retired epoch
is idempotent and cannot revoke a newer one.
is idempotent and cannot revoke a newer one. An authenticated close also succeeds
when expiry or policy cancellation has invalidated the current lease but its
retirement callback has not yet acquired the session lock. It still checks the
lease token, epoch and operation sequence; it never renews authority.
Core `RemoteContext(lease, grant)` returns a cancellation context only for a live
matching lease with that grant. Close, expiry, process failure, disable and parent
+7 -2
View File
@@ -220,7 +220,7 @@ func presentDocumentCaretResult(logger *slog.Logger, speech speechEnqueuer, resu
}
text = "blank"
}
request := speechRequest{Text: text, Role: result.DocumentPosition.Snapshot.Role, Cause: "document-caret"}
request := speechRequest{origin: document_speech_origin(result.DocumentPosition.Snapshot, result.DocumentCaretContext), Text: text, Role: result.DocumentPosition.Snapshot.Role, Cause: "document-caret"}
request.PhoneticHint = result.DocumentCaretCommand == review.CaretNextCharacter || result.DocumentCaretCommand == review.CaretPreviousCharacter
request.Context = result.DocumentCaretContext
if len(request.Context) > 0 && request.Context[0].ID() == result.DocumentPosition.Snapshot.ID() {
@@ -333,6 +333,7 @@ func presentDocumentResult(logger *slog.Logger, speech speechEnqueuer, result re
"braille_read_error", result.DocumentBrailleErr)
}
text, fallbackText, role := "", "", "system"
origin := speechOriginUnknown
var spoken, fallbackSpeech presentation.Speech
var cues []sound.Cue
switch {
@@ -393,6 +394,10 @@ func presentDocumentResult(logger *slog.Logger, speech speechEnqueuer, result re
text = "No more large objects."
}
default:
origin = document_speech_origin(result.DocumentPosition.Snapshot, result.DocumentCaretContext)
if link := result.DocumentPosition.LinkedHeading; link != nil && accessibility_speech_origin(*link) == speechOriginSecret {
origin = speechOriginSecret
}
if link := result.DocumentPosition.LinkedHeading; link != nil {
roleOnly := *link
roleOnly.Name, roleOnly.Text = "", ""
@@ -462,5 +467,5 @@ func presentDocumentResult(logger *slog.Logger, speech speechEnqueuer, result re
if len(ancestors) > 0 && ancestors[0].ID() == result.DocumentPosition.Snapshot.ID() {
ancestors = ancestors[1:]
}
speech.Enqueue(speechRequest{Text: text, Chunks: spoken.WithText(text).Chunks, FallbackText: fallbackText, FallbackChunks: fallbackSpeech.WithText(fallbackText).Chunks, Role: role, Sound: cues, Cause: "document-navigation", Context: ancestors, VoiceType: voiceType})
speech.Enqueue(speechRequest{origin: origin, Text: text, Chunks: spoken.WithText(text).Chunks, FallbackText: fallbackText, FallbackChunks: fallbackSpeech.WithText(fallbackText).Chunks, Role: role, Sound: cues, Cause: "document-navigation", Context: ancestors, VoiceType: voiceType})
}
+1 -1
View File
@@ -144,7 +144,7 @@ func (handler *eventReviewHandler) handle(result review.Result) {
if text == "" && len(cues) == 0 {
return
}
speech.Enqueue(speechRequest{Text: text, Chunks: spoken.Chunks, FallbackText: fallback.Text, FallbackChunks: fallback.Chunks, Sound: cues, Role: "text", Cause: "document-caret-line", Line: document_speech_line(result.DocumentLineData), Context: result.DocumentLineData.Context, VoiceType: document_line_voice(result.DocumentLineData)})
speech.Enqueue(speechRequest{origin: document_line_speech_origin(result.DocumentLineData), Text: text, Chunks: spoken.Chunks, FallbackText: fallback.Text, FallbackChunks: fallback.Chunks, Sound: cues, Role: "text", Cause: "document-caret-line", Line: document_speech_line(result.DocumentLineData), Context: result.DocumentLineData.Context, VoiceType: document_line_voice(result.DocumentLineData)})
return
}
if result.DocumentTarget != 0 || result.DocumentCaretCommand != 0 || result.DocumentFragment {
+9 -1
View File
@@ -19,11 +19,13 @@ func presentReadAllResult(logger *slog.Logger, speech speechEnqueuer, result rev
return
}
text := ""
origin := speechOriginUnknown
var parts []readAllPart
if result.Err != nil {
logger.Warn("read all failed", "error", result.Err)
text = "Could not read from current location."
} else {
origin = speechOriginLocal
// Keep semantic roles audible in continuous reading, without firing all
// the page's sound cues before the speech has reached those objects.
options.RoleSoundPresentation = "sound_and_speech"
@@ -38,6 +40,12 @@ func presentReadAllResult(logger *slog.Logger, speech speechEnqueuer, result rev
options.ReadFullRowInGUITable = false
for index, line := range result.ReadAllLines {
if spoken := presentation.DocumentLineSpeechWithOptions(line, options); spoken.Text != "" {
// One request owns the original observation ticket. Conservatively
// keep the entire reading private if any spoken line is sensitive
// or lacks source metadata, including sentence-mode continuations.
if source := document_line_speech_origin(line); source != speechOriginLocal {
origin = source
}
var began func()
if result.ReadAllProgress != nil {
began = func() { result.ReadAllProgress(index) }
@@ -61,7 +69,7 @@ func presentReadAllResult(logger *slog.Logger, speech speechEnqueuer, result rev
}
}
if text != "" || len(parts) != 0 {
speech.Enqueue(speechRequest{Text: text, ReadAll: parts, Role: "system", Cause: "read-all", context: result.ReadAllContext})
speech.Enqueue(speechRequest{origin: origin, Text: text, ReadAll: parts, Role: "system", Cause: "read-all", context: result.ReadAllContext})
}
}
+1
View File
@@ -93,6 +93,7 @@ func (worker *speechWorker) deliver(ctx context.Context, request speechRequest)
delivery := request
delivery.VoiceType, delivery.CustomEcho = part.VoiceType, part.VoiceType == "echo"
worker.logDelivery(ctx, delivery, text, index)
worker.observe_speech(ctx, delivery, text, false)
if worker.history != nil && !request.ExcludeFromHistory && request.Cause != "speech-history" {
worker.history.add(text, request.Role)
}
@@ -0,0 +1,163 @@
// Copyright (c) 2026 Storm Dragon
// SPDX-License-Identifier: GPL-3.0-or-later
package app
import (
"context"
"encoding/json"
"errors"
"log/slog"
"strings"
"testing"
"time"
"git.stormux.org/storm/cthulhu/internal/atspi"
"git.stormux.org/storm/cthulhu/internal/presentation"
"git.stormux.org/storm/cthulhu/internal/review"
"git.stormux.org/storm/cthulhu/internal/tentaclewire"
)
func document_observation_request(t *testing.T, kind string, result review.Result) speechRequest {
t.Helper()
queue := &fakeSpeechQueue{requests: make(chan speechRequest, 1), spoken: make(chan string, 1)}
logger := slog.New(slog.DiscardHandler)
options := presentation.DefaultOptions()
options.SayAllStyle = "sentence"
switch kind {
case "caret":
presentDocumentCaretResult(logger, queue, result, options)
case "navigation":
presentDocumentResult(logger, queue, result, options)
case "read-all":
presentReadAllResult(logger, queue, result, options)
}
select {
case request := <-queue.requests:
return request
default:
t.Fatal("document producer did not enqueue speech")
return speechRequest{}
}
}
func TestRemoteDocumentSpeechSourcePrivacy(t *testing.T) {
for _, kind := range []string{"caret", "navigation", "read-all"} {
for _, source := range []string{"ordinary", "password", "ancestor", "snapshot-ancestor"} {
t.Run(kind+"/"+source, func(t *testing.T) {
snapshot := atspi.Snapshot{Path: "/text", Role: "text", Name: "Reading. Next sentence."}
ancestors := []atspi.Snapshot{}
switch source {
case "password":
snapshot.Role = "password text"
case "ancestor":
// Same identity is removed from caret/navigation context.
ancestors = []atspi.Snapshot{{Path: "/text", Role: "password text"}}
case "snapshot-ancestor":
snapshot.SpeechContext = []atspi.Snapshot{{Role: "password text"}}
}
result := review.Result{Moved: true, DocumentCaretText: "Reading. Next sentence.",
DocumentCaretCommand: review.CaretNextLine, DocumentCaretContext: ancestors,
DocumentPosition: review.DocumentPosition{Snapshot: snapshot}, ReadAllContext: t.Context(),
ReadAllLines: []atspi.DocumentLine{{Context: ancestors, Contents: []atspi.DocumentContent{{Snapshot: snapshot, Text: "Reading. Next sentence."}}}}}
request := document_observation_request(t, kind, result)
if exportable_speech(request) != (source == "ordinary") {
t.Fatalf("source %s produced exportable=%v", source, exportable_speech(request))
}
})
}
}
for _, line := range []atspi.DocumentLine{
{},
{Contents: []atspi.DocumentContent{{Text: "text without source metadata"}}},
{Contents: []atspi.DocumentContent{{Snapshot: atspi.Snapshot{Role: "text"}}, {Snapshot: atspi.Snapshot{Role: "password text"}}}},
} {
if document_line_speech_origin(line) == speechOriginLocal {
t.Fatal("unclassified or mixed private line was exportable")
}
}
for _, kind := range []string{"caret", "navigation", "read-all"} {
request := document_observation_request(t, kind, review.Result{Err: errors.New("fixture failure"), ReadAllContext: t.Context()})
if exportable_speech(request) {
t.Fatal("local failure status was exported", kind)
}
}
}
func (speaker *observationSpeaker) SpeakAndWait(ctx context.Context, text string, began func()) error {
if began != nil {
began()
}
return speaker.Speak(ctx, text)
}
func TestRemoteReadAllObservationResolvedChunksAndPrivateBatch(t *testing.T) {
worker, session, _, _ := observation_fixture(t)
worker.textTransform = func(text string) string { return strings.ReplaceAll(text, "colour", "color") }
line := atspi.DocumentLine{Contents: []atspi.DocumentContent{{Snapshot: atspi.Snapshot{Role: "text"}, Text: "First colour. Second colour."}}}
request := document_observation_request(t, "read-all", review.Result{ReadAllContext: t.Context(), ReadAllLines: []atspi.DocumentLine{line}})
if err := worker.EnqueueAndWait(t.Context(), request); err != nil {
t.Fatal(err)
}
for _, want := range []string{"First color.", "Second color."} {
value := observed_speech(t, session)
if value.Kind != "speak" || len(value.Parts) != 2 || value.Parts[0].Text != want || !value.Parts[1].End {
t.Fatal("Say All did not export resolved sentence boundaries", value)
}
}
private := line
private.Context = []atspi.Snapshot{{Role: "password text"}}
request = document_observation_request(t, "read-all", review.Result{ReadAllContext: t.Context(), ReadAllLines: []atspi.DocumentLine{line, private, line}})
if err := worker.EnqueueAndWait(t.Context(), request); err != nil {
t.Fatal(err)
}
worker.InterruptForUser()
if value := observed_speech(t, session); value.Kind != "cancel" {
t.Fatal("mixed private Say All batch was exported", value)
}
}
func TestRemoteReadAllObservationKeepsOriginalLease(t *testing.T) {
worker, session, _, _ := observation_fixture(t)
speaker := &continuousSpeaker{started: make(chan string, 2), finish: make(chan struct{}, 2)}
worker.speaker = speaker // No requests have been admitted yet.
worker.Enqueue(speechRequest{origin: speechOriginLocal, ReadAll: []readAllPart{{Text: "First"}, {Text: "Second"}}})
select {
case <-speaker.started:
case <-time.After(time.Second):
t.Fatal("Say All did not start")
}
if value := observed_speech(t, session); value.Parts[0].Text != "First" {
t.Fatal("first line missing", value)
}
session.RevokeRemote()
var event tentaclewire.SessionEvent
if json.Unmarshal(observation_call(t, session, "event"), &event) != nil || event.Kind != "remote.revoked" {
t.Fatal("missing revocation")
}
approval, err := session.BeginRemoteApproval("controlled", []string{"output-observe"})
if err != nil {
t.Fatal(err)
}
if _, err := session.AuthorizeRemoteWithApproval(approval, "controlled", []string{"output-observe"}); err != nil {
t.Fatal(err)
}
observation_call(t, session, "open")
speaker.finish <- struct{}{}
select {
case text := <-speaker.started:
if text != "Second" {
t.Fatal("local reading changed", text)
}
case <-time.After(time.Second):
t.Fatal("local reading stalled across reconnection")
}
speaker.finish <- struct{}{}
if err := worker.EnqueueAndWait(t.Context(), speechRequest{}); err != nil {
t.Fatal(err)
}
worker.InterruptForUser()
if value := observed_speech(t, session); value.Kind != "cancel" {
t.Fatal("old Say All continued under replacement consent", value)
}
}
+30
View File
@@ -36,6 +36,36 @@ func accessibility_speech_origin(source atspi.Snapshot) speechOrigin {
return speechOriginLocal
}
// Document producers may remove the current object from their presentation
// context. Inspect the original sources before that filtering, including every
// content object in an assembled line. A mixed sensitive line stays private.
func document_speech_origin(source atspi.Snapshot, ancestors []atspi.Snapshot) speechOrigin {
if accessibility_speech_origin(source) == speechOriginSecret {
return speechOriginSecret
}
for _, ancestor := range ancestors {
if accessibility_speech_origin(ancestor) == speechOriginSecret {
return speechOriginSecret
}
}
if source.Role == "" {
return speechOriginUnknown
}
return speechOriginLocal
}
func document_line_speech_origin(line atspi.DocumentLine) speechOrigin {
if len(line.Contents) == 0 {
return speechOriginUnknown
}
for _, content := range line.Contents {
if origin := document_speech_origin(content.Snapshot, line.Context); origin != speechOriginLocal {
return origin
}
}
return speechOriginLocal
}
func exportable_speech(request speechRequest) bool {
if request.origin != speechOriginLocal || request.Role == "password text" ||
request.ExcludeFromHistory || request.Confirmation || request.Uninterruptible || request.ForceSpeech {
@@ -22,8 +22,10 @@ import (
"testing"
"time"
"git.stormux.org/storm/cthulhu/internal/atspi"
"git.stormux.org/storm/cthulhu/internal/input"
"git.stormux.org/storm/cthulhu/internal/interaction"
"git.stormux.org/storm/cthulhu/internal/review"
"git.stormux.org/storm/cthulhu/internal/tentacle"
"git.stormux.org/storm/cthulhu/internal/tentaclewire"
)
@@ -308,7 +310,7 @@ func TestRemoteConnectionCommandPrivateRelay(t *testing.T) {
runtime, forms, shortcut := remote_command_fixture(t)
if role == "controlled" {
worker := runtime.output.speech
worker.speaker = &fakeSpeaker{spoken: make(chan string, 16)}
worker.speaker = &observationSpeaker{fakeSpeaker{spoken: make(chan string, 16)}}
worker.textTransform = func(text string) string { return strings.ReplaceAll(text, "colour", "color") }
if err := worker.Start(t.Context()); err != nil {
t.Fatal(err)
@@ -438,6 +440,14 @@ func TestRemoteConnectionCommandPrivateRelay(t *testing.T) {
t.Fatal(err)
}
}
request := document_observation_request(t, "read-all", review.Result{
ReadAllContext: t.Context(), ReadAllLines: []atspi.DocumentLine{{Contents: []atspi.DocumentContent{{
Snapshot: atspi.Snapshot{Role: "text"}, Text: "First colour. Second colour.",
}}}},
})
if err := worker.EnqueueAndWait(t.Context(), request); err != nil {
t.Fatal(err)
}
worker.InterruptForUser()
var received strings.Builder
ends, frames := 0, 0
@@ -473,7 +483,7 @@ func TestRemoteConnectionCommandPrivateRelay(t *testing.T) {
t.Fatal("reader output did not reach the relay in order")
}
}
if received.String() != "color "+remoteText || frames < 2 || ends != 1 {
if received.String() != "color "+remoteText+"First color.Second color." || frames < 4 || ends != 3 {
t.Fatal("observation leaked private text or lost resolved output/boundaries")
}
}
+8 -1
View File
@@ -296,7 +296,14 @@ func (session *Session) remote_call_locked(method string, params json.RawMessage
if method == "remote.close" && request.Epoch <= session.remoteEpoch && (access == nil || access.epoch != request.Epoch) {
return struct{}{}, nil
}
if access == nil || access.lease == "" || access.lease != request.Lease || access.epoch != request.Epoch || request.Sequence <= access.sequence || !time.Now().Before(access.deadline) || access.ctx.Err() != nil {
if access == nil || access.lease == "" || access.lease != request.Lease || access.epoch != request.Epoch || request.Sequence <= access.sequence {
return nil, denied
}
// Closing only removes authority. Allow authenticated cleanup even
// before the expiry/policy callback acquires this lock; rejecting it
// would make the child permanently distrust an otherwise healthy IPC
// session. Heartbeats must never renew that cancelled authority.
if method == "remote.heartbeat" && (!time.Now().Before(access.deadline) || access.ctx.Err() != nil) {
return nil, denied
}
access.sequence = request.Sequence
+43
View File
@@ -4,13 +4,56 @@
package tentacle
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"time"
"git.stormux.org/storm/cthulhu/internal/tentaclewire"
)
func TestRemoteCloseBeforeRetirementCallback(t *testing.T) {
for _, state := range []string{"expired", "cancelled"} {
t.Run(state, func(t *testing.T) {
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
access := &remoteAccess{lease: strings.Repeat("a", 32), epoch: 1,
ctx: ctx, cancel: cancel, deadline: time.Now().Add(time.Minute)}
if state == "expired" {
access.deadline = time.Now().Add(-time.Second)
} else {
cancel()
}
manager := &Manager{}
session := &Session{manager: manager, remote: access, remoteEpoch: 1,
outbound: make(chan outgoing, 4)}
manager.remoteOwner = session
operation := tentaclewire.RemoteOperation{Lease: access.lease, Epoch: 1, Sequence: 1}
data, _ := json.Marshal(operation)
if _, failure := session.remote_call_locked("remote.heartbeat", data); failure == nil {
t.Fatal("heartbeat revived retired authority")
}
wrong := operation
wrong.Lease = strings.Repeat("b", 32)
invalid, _ := json.Marshal(wrong)
if _, failure := session.remote_call_locked("remote.close", invalid); failure == nil {
t.Fatal("foreign token closed pending retirement")
}
if _, failure := session.remote_call_locked("remote.close", data); failure != nil {
t.Fatal("cleanup rejected before retirement callback", failure)
}
if session.remote != nil || manager.remoteOwner != nil {
t.Fatal("close retained remote ownership")
}
if _, failure := session.remote_call_locked("remote.close", data); failure != nil {
t.Fatal("repeated cleanup failed", failure)
}
})
}
}
func TestRemotePolicyRevokesPendingAndLiveSDKAccess(t *testing.T) {
for _, language := range []string{"go", "python"} {
t.Run(language, func(t *testing.T) {