Add Mumble message review shortcuts
This commit is contained in:
1 parent
d5f8a392cf
commit
64440fbb78
18 files changed
+484
-30
No files matched your search
@@ -9,7 +9,7 @@ Cthulhu is a Linux screen reader written in Go.
|
||||
- [Steam keyboard focus](STEAM_FOCUS.md): Xorg workspace focus recovery and verification.
|
||||
- [XTerm handoff](docs/testing/xterm-handoff.md): terminal reader keys, lock state, speech interruption, and verification.
|
||||
- [Graphical terminals](docs/testing/graphical-terminals.md): repaint coalescing, editing feedback, title animation, and verification.
|
||||
- [Messaging support](docs/testing/messaging.md): Pidgin message review/copy shortcuts, application boundaries, and verification.
|
||||
- [Messaging support](docs/testing/messaging.md): Pidgin and Mumble message review/copy shortcuts, application boundaries, and verification.
|
||||
- [Flatpak foreground accessibility](docs/testing/flatpak-foreground.md): process identity and proxy matching, tests, and live acceptance.
|
||||
- [Background event filtering](docs/testing/background-events.md): early foreground checks, notification exceptions, and performance verification.
|
||||
- [Browser Find](docs/testing/browser-find.md): native Ctrl+F focus, typing, window ownership and verification.
|
||||
|
||||
@@ -1,11 +1,19 @@
|
||||
# Messaging support
|
||||
|
||||
Pidgin now uses the shared Go messaging history. This first port follows the
|
||||
Pidgin and Mumble use the shared Go messaging history. The Pidgin port follows the
|
||||
legacy defaults: announce inserted messages from all conversations, keep one
|
||||
combined history for each running Pidgin accessibility provider, omit room-name
|
||||
prefixes, and suppress typing-status announcements. History contains messages
|
||||
received while this reader is running; it does not import an existing transcript.
|
||||
|
||||
Mumble captures insertions into its read-only **Activity log**, matching the
|
||||
legacy application script. It retains twelve messages per provider, strips a
|
||||
leading `[HH:MM:SS]` timestamp, removes Qt noncharacters, and normalizes whitespace
|
||||
including paragraph separators. Timestamp-only insertions stay silent and do not
|
||||
occupy history slots. Identical messages remain separate entries. Log messages
|
||||
can speak in the background without taking focus. Mumble's message entry and
|
||||
other text controls do not supply history.
|
||||
|
||||
The legacy controls for separate conversation histories, room-name prefixes,
|
||||
typing announcements, and message verbosity are not yet ported.
|
||||
|
||||
@@ -25,10 +33,13 @@ Both desktop and laptop layouts use:
|
||||
|
||||
- **Cthulhu+F1–F9**: review the newest through ninth-newest message.
|
||||
- **Cthulhu+Shift+F1–F9**: copy that message and confirm after the write succeeds.
|
||||
- In **Mumble**, both ranges extend through **F12**: F10, F11 and F12 review the
|
||||
tenth-, eleventh- and twelfth-newest messages; adding Shift copies them.
|
||||
- **Cthulhu+H**: enter or exit learn mode. Cthulhu+F1 no longer opens it.
|
||||
|
||||
Empty history slots are silent. Chat commands act only when Pidgin owns the
|
||||
reader's current focus. They are configurable in the existing key-binding page.
|
||||
Empty history slots are silent. Chat commands act only when the corresponding
|
||||
Pidgin or Mumble instance owns the reader's current focus. Pidgin retains its
|
||||
nine-message review range. Commands are configurable in the existing key-binding page.
|
||||
An explicit shortcut assignment takes precedence over an uncustomized chat
|
||||
default, preserving existing configurations; the shortcut list reflects this.
|
||||
Within temporary OCR review, Cthulhu+F5 retains its refresh behavior.
|
||||
@@ -46,8 +57,11 @@ Steam script nor this port provides F-key conversation review for Steam.
|
||||
to a provider whose process name is `pidgin`. Its bounded ancestor scan copies
|
||||
conversation/window identities and the room name. Typing status is identified
|
||||
from the inserted range's smaller/lighter text attributes.
|
||||
- `internal/atspi/mumble_native.go` recognizes Mumble by process identity and the
|
||||
read-only Activity log's text or label role, then copies transcript and window
|
||||
identities with a bounded ancestor walk. `mumble.go` cleans the inserted text.
|
||||
- `internal/messaging` stores copied messages without native handles or output
|
||||
dependencies. It bounds each provider's history to nine messages and the number
|
||||
dependencies. It bounds each provider's history to twelve messages and the number
|
||||
of retained providers to 32. Provider identities prevent cross-instance replay.
|
||||
- `internal/app/messaging.go` coordinates passive speech, one Braille flash,
|
||||
review, and asynchronous clipboard copying on the reader event loop. A bounded
|
||||
@@ -74,6 +88,7 @@ notification text alone does not establish conversation identity.
|
||||
From the repository root:
|
||||
|
||||
```sh
|
||||
python3 -B tests/native_messaging.py
|
||||
GOCACHE=/tmp/cthulhu-cache go test ./...
|
||||
GOCACHE=/tmp/cthulhu-cache go test -race ./internal/messaging ./internal/input ./internal/atspi ./internal/app
|
||||
python3 tests/native_test_session.py env GOCACHE=/tmp/cthulhu-cache CTHULHU_NATIVE_MESSAGING=1 go test ./internal/atspi -run '^TestNativePidginTranscriptCapture$' -count=1 -v
|
||||
@@ -83,9 +98,11 @@ go test ./internal/input ./internal/app -run '^(TestPlainEnterActivityLifetime|T
|
||||
python3 tests/native_test_session.py env CTHULHU_NATIVE_SUBMISSION_ECHO=1 go test ./internal/app -run '^TestNativeSubmissionClearAndSelectedReplacement$' -count=1 -v
|
||||
```
|
||||
|
||||
Native checks use private X11 and D-Bus sessions. The transcript fixture sets its
|
||||
own process name to exercise the Pidgin detector; it is not the actual Pidgin
|
||||
client. These checks verify copied metadata, recorded presentation, synthetic
|
||||
Native checks use private X11 and D-Bus sessions. `native_messaging.py` uses a
|
||||
24-bit display and separate sessions for Mumble/Qt, Pidgin/GTK, clipboard, and
|
||||
X11 shortcut checks. It requires PySide6 and GTK3's Python introspection bindings.
|
||||
The transcript fixtures set their own process names to exercise the detectors;
|
||||
they are not the actual clients. These checks verify copied metadata, recorded presentation, synthetic
|
||||
keys, and clipboard contents. They do not establish audible speech, physical
|
||||
keyboard behavior, or a connected Braille display.
|
||||
|
||||
@@ -129,6 +146,35 @@ The implementation work does not replace the running reader.
|
||||
|
||||
Actual Pidgin listening, physical shortcuts, and Braille acceptance remain pending.
|
||||
|
||||
### Mumble live acceptance
|
||||
|
||||
1. Quit the current reader with **Cthulhu+Q**. Press **Ctrl+Escape**, then press
|
||||
**Shift+G**. These are separate key presses. The launch script rebuilds the
|
||||
reader and Braille helper, starts debug logging, and opens a test terminal.
|
||||
2. Focus Mumble's message entry and receive at least twelve distinct messages.
|
||||
Confirm each incoming message speaks once, without a timestamp-only
|
||||
announcement. On Braille, check one flash per message and return to the
|
||||
message entry afterward. Keyboard focus must remain in the entry.
|
||||
3. Press **Cthulhu+F1**, **F2**, **F10**, **F11**, and **F12**, keeping the Cthulhu
|
||||
modifier for each. Expect the newest, previous, tenth-, eleventh-, and
|
||||
twelfth-newest messages. Receive a thirteenth message and check that the
|
||||
oldest retained message shifts by one. Review must not move focus or the caret.
|
||||
4. Press **Cthulhu+Shift+F12** and paste in an editor. Expect the same message
|
||||
as F12, with a successful copy confirmation. Switch to another application:
|
||||
review keys must not replay Mumble history there. Receive a Mumble message in
|
||||
the background and verify it speaks without changing keyboard focus.
|
||||
5. Return to Mumble and check typing, selection, deletion, Tab, and channel-tree
|
||||
navigation. Ordinary control feedback must remain correct. Verify
|
||||
**Cthulhu+H** learn mode and any saved custom F-key bindings still work.
|
||||
If Pidgin is available, check its existing F1–F9 review and Shift-copy behavior.
|
||||
6. Report the exact speech for F1/F10/F11/F12, any Braille differences, which
|
||||
control retained focus, and any keys that behaved unexpectedly. If necessary,
|
||||
inspect `~/devel/cthulhu-go.log` before restarting again.
|
||||
|
||||
Actual Mumble listening, installed-binary behavior, physical shortcuts, and a
|
||||
physical Braille display remain pending. Review the diff and perform these live
|
||||
checks before submitting the change.
|
||||
|
||||
## Discord message focus
|
||||
|
||||
Named article ancestors are announced when focus enters their controls, matching
|
||||
|
||||
@@ -26,6 +26,17 @@ type messageCopyResult struct {
|
||||
err error
|
||||
}
|
||||
|
||||
func messageHistorySlots(process string) int {
|
||||
switch process {
|
||||
case "pidgin":
|
||||
return 9
|
||||
case "mumble":
|
||||
return 12
|
||||
default:
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
// messagingController belongs to the accessibility event loop. Histories survive
|
||||
// focus changes, retire with their native window/transcript, and clear on lock.
|
||||
// Its one clipboard worker has a bounded queue and never performs native queries.
|
||||
@@ -87,8 +98,8 @@ func (controller *messagingController) handle(event atspi.Event) bool {
|
||||
controller.history.Retire(id.BusName, id.Path)
|
||||
}
|
||||
message := event.ChatMessage
|
||||
if message == nil || event.Type != atspi.TextInsertedEvent || event.Source.ApplicationProcess != "pidgin" ||
|
||||
event.Source.Defunct || event.Source.Role != "text" ||
|
||||
if message == nil || event.Type != atspi.TextInsertedEvent || messageHistorySlots(event.Source.ApplicationProcess) == 0 ||
|
||||
event.Source.Defunct || (event.Source.Role != "text" && !(event.Source.ApplicationProcess == "mumble" && event.Source.Role == "label")) ||
|
||||
message.Provider != event.Source.BusName || message.Conversation != event.Source.Path ||
|
||||
!event.Source.ID().Valid() || event.Window != atspi.NewObjectID(message.Provider, message.Window) {
|
||||
return false
|
||||
@@ -117,7 +128,7 @@ func (controller *messagingController) present(text string, background bool) {
|
||||
|
||||
func (controller *messagingController) review(command input.Command, current atspi.Snapshot) {
|
||||
index, copyMessage, ok := input.MessageCommandIndex(command)
|
||||
if !ok || controller.gate.Locked() || controller.gate.Suspended() || current.ApplicationProcess != "pidgin" {
|
||||
if !ok || controller.gate.Locked() || controller.gate.Suspended() || index >= messageHistorySlots(current.ApplicationProcess) {
|
||||
return
|
||||
}
|
||||
message, exists := controller.history.Previous(current.BusName, index)
|
||||
|
||||
@@ -7,6 +7,7 @@ package app
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"os"
|
||||
"testing"
|
||||
@@ -26,7 +27,71 @@ func pidginMessageEvent(text string) atspi.Event {
|
||||
return event
|
||||
}
|
||||
|
||||
func TestMumbleMessagingReviewBoundsOwnershipAndRetirement(t *testing.T) {
|
||||
queue, braille := &learnSpeechRecorder{}, &flashRecorder{}
|
||||
controller := newMessagingController(context.Background(), queue, nil, nil, func(_ context.Context, text string) error {
|
||||
if text != "Mumble message 3" {
|
||||
return fmt.Errorf("copied wrong message: %q", text)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
controller.braille = braille
|
||||
defer controller.stop()
|
||||
current := atspi.Snapshot{ApplicationProcess: "mumble", BusName: ":1.42"}
|
||||
for i := 0; i < 15; i++ {
|
||||
event := pidginMessageEvent(fmt.Sprintf("Mumble message %d", i))
|
||||
event.Source.ApplicationProcess = "mumble"
|
||||
if i%2 == 0 {
|
||||
event.Source.Role = "label"
|
||||
}
|
||||
if !controller.handle(event) {
|
||||
t.Fatal("Mumble log insertion was not handled")
|
||||
}
|
||||
}
|
||||
for i := 0; i < 12; i++ {
|
||||
controller.review(input.MessageCommand(i, false), current)
|
||||
want := fmt.Sprintf("Mumble message %d", 14-i)
|
||||
if queue.texts[len(queue.texts)-1] != want || braille.messages[len(braille.messages)-1] != want {
|
||||
t.Fatalf("review slot %d did not present %q", i, want)
|
||||
}
|
||||
}
|
||||
controller.review(input.MessageCommand(11, true), current)
|
||||
select {
|
||||
case result := <-controller.results:
|
||||
if result.err != nil {
|
||||
t.Fatal(result.err)
|
||||
}
|
||||
controller.copied(result)
|
||||
case <-time.After(time.Second):
|
||||
t.Fatal("Mumble F12 copy did not complete")
|
||||
}
|
||||
if queue.texts[len(queue.texts)-1] != "Copied Mumble message 3 to clipboard." {
|
||||
t.Fatal("Mumble F12 copy was not confirmed")
|
||||
}
|
||||
count := len(queue.texts)
|
||||
for _, focus := range []atspi.Snapshot{
|
||||
{ApplicationProcess: "mumble", BusName: ":1.43"},
|
||||
{ApplicationProcess: "chromium", BusName: ":1.42"},
|
||||
{ApplicationProcess: "pidgin", BusName: ":1.42"},
|
||||
} {
|
||||
controller.review(input.MessageCommand(11, false), focus)
|
||||
}
|
||||
retired := atspi.NewEvent(atspi.DefunctStateEvent, atspi.Snapshot{BusName: ":1.42", Path: "/transcript", Defunct: true})
|
||||
retired.Detail1 = 1
|
||||
controller.handle(retired)
|
||||
controller.review(input.MessageCommand(0, false), current)
|
||||
if len(queue.texts) != count || len(braille.messages) != count {
|
||||
t.Fatal("unowned or retired history was presented")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMessagingBackgroundPresentationKeepsFocusAndUsesCommands(t *testing.T) {
|
||||
for _, process := range []string{"pidgin", "mumble"} {
|
||||
t.Run(process, func(t *testing.T) { testMessagingBackgroundPresentationKeepsFocusAndUsesCommands(t, process) })
|
||||
}
|
||||
}
|
||||
|
||||
func testMessagingBackgroundPresentationKeepsFocusAndUsesCommands(t *testing.T, process string) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
adapter := &fakeAdapter{events: make(chan atspi.Event)}
|
||||
queue := &fakeSpeechQueue{spoken: make(chan string, 16)}
|
||||
@@ -37,13 +102,18 @@ func TestMessagingBackgroundPresentationKeepsFocusAndUsesCommands(t *testing.T)
|
||||
}()
|
||||
defer func() { cancel(); <-done }()
|
||||
focus := ownerEvent(atspi.FocusedStateEvent, ":1.42", "/window", "/entry")
|
||||
focus.Source.ApplicationProcess, focus.Source.Role, focus.Source.Name = "pidgin", "entry", "Message input"
|
||||
focus.Source.ApplicationProcess, focus.Source.Role, focus.Source.Name = process, "entry", "Message input"
|
||||
focus.Source.Editable, focus.Source.Focused, focus.Detail1 = true, true, 1
|
||||
adapter.events <- focus
|
||||
expectActivationSpeech(t, queue.spoken, "Message input, entry")
|
||||
adapter.events <- pidginMessageEvent("First message")
|
||||
messageEvent := func(text string) atspi.Event {
|
||||
event := pidginMessageEvent(text)
|
||||
event.Source.ApplicationProcess = process
|
||||
return event
|
||||
}
|
||||
adapter.events <- messageEvent("First message")
|
||||
expectActivationSpeech(t, queue.spoken, "First message")
|
||||
adapter.events <- pidginMessageEvent("Second message")
|
||||
adapter.events <- messageEvent("Second message")
|
||||
expectActivationSpeech(t, queue.spoken, "Second message")
|
||||
requests <- input.MessageCommand(0, false)
|
||||
expectActivationSpeech(t, queue.spoken, "Second message")
|
||||
|
||||
@@ -495,12 +495,12 @@ func dispatch_native_event(event *C.AtspiEvent, userData unsafe.Pointer, deferre
|
||||
return
|
||||
}
|
||||
if eventType == DefunctStateEvent && C.cthulhu_event_detail1(event) != 0 &&
|
||||
adapter.applicationProcesses[originID.BusName] == "pidgin" {
|
||||
(adapter.applicationProcesses[originID.BusName] == "pidgin" || adapter.applicationProcesses[originID.BusName] == "mumble") {
|
||||
// Read-only transcripts need not own native focus. Forward their
|
||||
// retirement before the ordinary dead-object early return, without
|
||||
// querying a destroyed accessible or broadening focus ownership.
|
||||
retired := NewEvent(eventType, Snapshot{BusName: originID.BusName, Path: originID.Path,
|
||||
ApplicationProcess: "pidgin", Defunct: true})
|
||||
ApplicationProcess: adapter.applicationProcesses[originID.BusName], Defunct: true})
|
||||
retired.Detail1 = 1
|
||||
adapter.enqueueSecureEvent(retired, secureEpoch)
|
||||
return
|
||||
@@ -518,11 +518,17 @@ func dispatch_native_event(event *C.AtspiEvent, userData unsafe.Pointer, deferre
|
||||
// notification script needs their semantics before that early return,
|
||||
// including when the provider is outside foreground ownership.
|
||||
webNotification := adapter.webNotification(event)
|
||||
if message := adapter.pidginMessage(event); message != nil {
|
||||
message := adapter.pidginMessage(event)
|
||||
if message == nil {
|
||||
message = adapter.mumbleMessage(event)
|
||||
}
|
||||
if message != nil {
|
||||
// A transcript insertion is a passive announcement. Keep it out of
|
||||
// native focus capture, selected-descendant resolution and review.
|
||||
roleName := C.atspi_accessible_get_role_name(event.source, nil)
|
||||
chatEvent := NewEvent(TextInsertedEvent, Snapshot{BusName: message.Provider,
|
||||
Path: message.Conversation, Role: "text", ApplicationProcess: "pidgin"})
|
||||
Path: message.Conversation, Role: C.GoString(roleName), ApplicationProcess: adapter.applicationProcesses[originID.BusName]})
|
||||
C.g_free(C.gpointer(roleName))
|
||||
chatEvent.ChatMessage = message
|
||||
chatEvent.Window = NewObjectID(message.Provider, message.Window)
|
||||
chatEvent.Foreground = adapter.Foreground()
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
// Copyright (c) 2026 Storm Dragon
|
||||
// Authors: Storm Dragon, Angus Kola
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package atspi
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
var mumbleTimestamp = regexp.MustCompile(`^\[\d{2}:\d{2}:\d{2}\]\s*`)
|
||||
|
||||
// Match legacy Mumble's inserted-message cleanup. Qt can insert the timestamp
|
||||
// separately, and uses paragraph separators and noncharacters in its log.
|
||||
func mumbleMessageText(text string) string {
|
||||
text = strings.Map(func(character rune) rune {
|
||||
if character >= '\ufdd0' && character <= '\ufdef' {
|
||||
return -1
|
||||
}
|
||||
return character
|
||||
}, text)
|
||||
text = strings.Join(strings.Fields(text), " ")
|
||||
return mumbleTimestamp.ReplaceAllString(text, "")
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
// Copyright (c) 2026 Storm Dragon
|
||||
// Authors: Storm Dragon, Angus Kola
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package atspi
|
||||
|
||||
/*
|
||||
#include "adapter_native.h"
|
||||
*/
|
||||
import "C"
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"git.stormux.org/storm/cthulhu/internal/messaging"
|
||||
)
|
||||
|
||||
// Only Mumble's named, read-only Activity log supplies message history. The
|
||||
// bounded walk copies its owning window; no native handles survive the callback.
|
||||
func (adapter *Adapter) mumbleMessage(event *C.AtspiEvent) *messaging.Message {
|
||||
if canonicalObjectChangeEventType(C.GoString(event._type)) != TextInsertedEvent || adapter.applicationProcess(event.source) != "mumble" {
|
||||
return nil
|
||||
}
|
||||
role := C.atspi_accessible_get_role(event.source, nil)
|
||||
if role != C.ATSPI_ROLE_TEXT && role != C.ATSPI_ROLE_LABEL {
|
||||
return nil
|
||||
}
|
||||
name := C.atspi_accessible_get_name(event.source, nil)
|
||||
activityLog := C.GoString(name) == "Activity log"
|
||||
C.g_free(C.gpointer(name))
|
||||
if !activityLog {
|
||||
return nil
|
||||
}
|
||||
states := C.atspi_accessible_get_state_set(event.source)
|
||||
if states == nil {
|
||||
return nil
|
||||
}
|
||||
valid := C.atspi_state_set_contains(states, C.ATSPI_STATE_EDITABLE) == 0 &&
|
||||
C.atspi_state_set_contains(states, C.ATSPI_STATE_DEFUNCT) == 0
|
||||
C.g_object_unref(C.gpointer(states))
|
||||
if !valid {
|
||||
return nil
|
||||
}
|
||||
id := nativeObjectID(event.source)
|
||||
message := &messaging.Message{Provider: id.BusName, Conversation: id.Path, RoomName: "Mumble"}
|
||||
current := C.atspi_accessible_get_parent(event.source, nil)
|
||||
deadline := time.Now().Add(50 * time.Millisecond)
|
||||
for depth := 0; current != nil && depth < 32 && time.Now().Before(deadline); depth++ {
|
||||
if isWindowRole(C.atspi_accessible_get_role(current, nil)) {
|
||||
window := nativeObjectID(current)
|
||||
if window.BusName == id.BusName {
|
||||
message.Window = window.Path
|
||||
}
|
||||
C.g_object_unref(C.gpointer(current))
|
||||
current = nil
|
||||
break
|
||||
}
|
||||
parent := C.atspi_accessible_get_parent(current, nil)
|
||||
C.g_object_unref(C.gpointer(current))
|
||||
current = parent
|
||||
}
|
||||
if current != nil {
|
||||
C.g_object_unref(C.gpointer(current))
|
||||
}
|
||||
if !id.Valid() || message.Window == "" {
|
||||
return nil
|
||||
}
|
||||
payload := C.cthulhu_event_payload(event)
|
||||
message.Text = mumbleMessageText(sanitizeTextContext(C.GoString(payload)))
|
||||
C.g_free(C.gpointer(payload))
|
||||
return message
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
// Copyright (c) 2026 Storm Dragon
|
||||
// Authors: Storm Dragon, Angus Kola
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package atspi
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestNativeMumbleTranscriptCapture(t *testing.T) {
|
||||
if os.Getenv("CTHULHU_NATIVE_MUMBLE") != "1" {
|
||||
t.Skip("requires private Qt/Xorg/AT-SPI session")
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
adapter := New()
|
||||
if err := adapter.Start(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer adapter.Stop(context.Background())
|
||||
for _, process := range []string{"mumble", "messaging-test"} {
|
||||
t.Run(process, func(t *testing.T) {
|
||||
command := exec.CommandContext(ctx, "python3", "../../tests/qt_messaging.py", process)
|
||||
input, err := command.StdinPipe()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
output, err := command.StdoutPipe()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
command.Stderr = os.Stderr
|
||||
if err := command.Start(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer func() { input.Close(); command.Process.Kill(); command.Wait() }()
|
||||
line, err := bufio.NewReader(output).ReadString('\n')
|
||||
fields := strings.Fields(line)
|
||||
if err != nil || len(fields) != 2 || fields[0] != "ready" {
|
||||
t.Fatalf("fixture readiness: %q, %v", line, err)
|
||||
}
|
||||
if _, err := adapter.CaptureDesktop(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if out, err := exec.CommandContext(ctx, "xdotool", "windowfocus", "--sync", fields[1]).CombinedOutput(); err != nil {
|
||||
t.Fatalf("focus: %s, %v", out, err)
|
||||
}
|
||||
fmt.Fprintln(input, "messages\nbarrier")
|
||||
var messages []string
|
||||
for {
|
||||
select {
|
||||
case event := <-adapter.Events():
|
||||
if event.Source.ApplicationProcess != process {
|
||||
continue
|
||||
}
|
||||
if message := event.ChatMessage; message != nil {
|
||||
if process != "mumble" {
|
||||
t.Fatal("unrelated process received chat metadata")
|
||||
}
|
||||
if message.Provider != event.Source.BusName || message.Conversation != event.Source.Path || message.Window == "" || message.RoomName != "Mumble" {
|
||||
t.Fatalf("incomplete message identities: %+v", message)
|
||||
}
|
||||
if message.Text != "" {
|
||||
messages = append(messages, message.Text)
|
||||
}
|
||||
}
|
||||
if event.Type == TextInsertedEvent && event.Payload == "Messaging barrier" {
|
||||
want := "Username: hello world 界|Repeated message|Repeated message"
|
||||
if process == "mumble" && strings.Join(messages, "|") != want {
|
||||
t.Fatalf("captured messages = %q", messages)
|
||||
}
|
||||
return
|
||||
}
|
||||
case <-ctx.Done():
|
||||
t.Fatalf("transcript/barrier missing: %v (messages=%q)", ctx.Err(), messages)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
// Copyright (c) 2026 Storm Dragon
|
||||
// Authors: Storm Dragon, Angus Kola
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package atspi
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestMumbleMessageText(t *testing.T) {
|
||||
for _, test := range []struct{ input, want string }{
|
||||
{"[12:34:56]", ""},
|
||||
{" \ufdd0[12:34:56]\ufdef \n", ""},
|
||||
{"[12:34:56] Username: hello\u2028world \ufdd0界\ufdef", "Username: hello world 界"},
|
||||
{"Username: [12:34:56] hello", "Username: [12:34:56] hello"},
|
||||
{"Username: hello\n\tworld", "Username: hello world"},
|
||||
{" \n\ufdd0\ufdef", ""},
|
||||
} {
|
||||
if got := mumbleMessageText(test.input); got != test.want {
|
||||
t.Errorf("cleanup %q = %q, want %q", test.input, got, test.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -237,7 +237,7 @@ func DefaultCommandBindings(layout KeyboardLayout) CommandBindings {
|
||||
{Symbol: leftKeySymbol, Modifiers: controlModifier, CthulhuModifier: true}: ObjectPreviousSibling,
|
||||
{Symbol: rightKeySymbol, Modifiers: controlModifier, CthulhuModifier: true}: ObjectNextSibling,
|
||||
}
|
||||
for index := 0; index < 9; index++ {
|
||||
for index := 0; index < 12; index++ {
|
||||
bindings[CommandChord{Symbol: 0xffbe + uint32(index), CthulhuModifier: true}] = MessageCommand(index, false)
|
||||
bindings[CommandChord{Symbol: 0xffbe + uint32(index), Modifiers: 1, CthulhuModifier: true}] = MessageCommand(index, true)
|
||||
}
|
||||
|
||||
@@ -4,19 +4,22 @@
|
||||
|
||||
package input
|
||||
|
||||
import "strings"
|
||||
import (
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Message commands are independent of the physical key. Configured bindings,
|
||||
// learn mode and both keyboard layouts use the same numbered requests.
|
||||
func MessageCommand(index int, copyMessage bool) Command {
|
||||
if index < 0 || index >= 9 {
|
||||
if index < 0 || index >= 12 {
|
||||
return ""
|
||||
}
|
||||
action := "review"
|
||||
if copyMessage {
|
||||
action = "copy"
|
||||
}
|
||||
return Command(action + "-chat-message-" + string(rune('1'+index)))
|
||||
return Command(action + "-chat-message-" + strconv.Itoa(index+1))
|
||||
}
|
||||
|
||||
func MessageCommandIndex(command Command) (index int, copyMessage, ok bool) {
|
||||
@@ -25,8 +28,9 @@ func MessageCommandIndex(command Command) (index int, copyMessage, ok bool) {
|
||||
number, found = strings.CutPrefix(string(command), "copy-chat-message-")
|
||||
copyMessage = true
|
||||
}
|
||||
if !found || len(number) != 1 || number[0] < '1' || number[0] > '9' {
|
||||
value, err := strconv.Atoi(number)
|
||||
if !found || err != nil || value < 1 || value > 12 || strconv.Itoa(value) != number {
|
||||
return 0, false, false
|
||||
}
|
||||
return int(number[0] - '1'), copyMessage, true
|
||||
return value - 1, copyMessage, true
|
||||
}
|
||||
@@ -16,7 +16,7 @@ func TestMessagingBindingsAndExplicitAssignments(t *testing.T) {
|
||||
if bindings[CommandChord{Symbol: 'h', CthulhuModifier: true}] != ToggleLearnMode {
|
||||
t.Fatal("learn mode shortcut changed")
|
||||
}
|
||||
for index := 0; index < 9; index++ {
|
||||
for index := 0; index < 12; index++ {
|
||||
for _, copyMessage := range []bool{false, true} {
|
||||
modifiers := uint16(0)
|
||||
if copyMessage {
|
||||
@@ -45,7 +45,7 @@ func TestMessagingBindingsAndExplicitAssignments(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, command := range []Command{"review-chat-message-0", "review-chat-message-10", "copy-chat-message-01", ToggleLearnMode} {
|
||||
for _, command := range []Command{"review-chat-message-0", "review-chat-message-13", "copy-chat-message-01", "review-chat-message-+1", "review-chat-message- 1", ToggleLearnMode} {
|
||||
if _, _, ok := MessageCommandIndex(command); ok {
|
||||
t.Fatalf("accepted %q", command)
|
||||
}
|
||||
|
||||
@@ -8,7 +8,7 @@ package messaging
|
||||
|
||||
import "strings"
|
||||
|
||||
const HistorySize = 9
|
||||
const HistorySize = 12
|
||||
const maxProviders = 32
|
||||
|
||||
// Message contains no native objects. Provider is the unique accessibility bus
|
||||
|
||||
@@ -12,17 +12,17 @@ import (
|
||||
|
||||
func TestHistoryOrdersConversationsAndIsolatesApplicationInstances(t *testing.T) {
|
||||
var history History
|
||||
for i := 0; i < 12; i++ {
|
||||
for i := 0; i < 15; i++ {
|
||||
history.Add(Message{Provider: ":1.42", Conversation: fmt.Sprintf("/room%d", i%2), Window: "/window", Text: fmt.Sprintf("Message %d", i)})
|
||||
}
|
||||
history.Add(Message{Provider: ":1.43", Conversation: "/room0", Window: "/window", Text: "Other instance"})
|
||||
for i := 0; i < HistorySize; i++ {
|
||||
message, ok := history.Previous(":1.42", i)
|
||||
if !ok || message.Text != fmt.Sprintf("Message %d", 11-i) {
|
||||
if !ok || message.Text != fmt.Sprintf("Message %d", 14-i) {
|
||||
t.Fatalf("slot %d = %+v, %t", i, message, ok)
|
||||
}
|
||||
}
|
||||
for _, index := range []int{-1, 9} {
|
||||
for _, index := range []int{-1, HistorySize} {
|
||||
if _, ok := history.Previous(":1.42", index); ok {
|
||||
t.Fatal("out-of-range slot exists")
|
||||
}
|
||||
|
||||
@@ -169,6 +169,12 @@ func TestNativeLearnModeCapturesKeysAndRestoresCommands(t *testing.T) {
|
||||
}
|
||||
key("Insert+Shift_L+F9")
|
||||
want(input.MessageCommand(8, true))
|
||||
for index := 9; index < 12; index++ {
|
||||
key(fmt.Sprintf("Insert+F%d", index+1))
|
||||
want(input.MessageCommand(index, false))
|
||||
key(fmt.Sprintf("Insert+Shift_L+F%d", index+1))
|
||||
want(input.MessageCommand(index, true))
|
||||
}
|
||||
for _, opener := range []string{"Insert+h"} {
|
||||
key(opener)
|
||||
want(input.ToggleLearnMode)
|
||||
|
||||
Vendored
+1
@@ -75,3 +75,4 @@ pidgin-message-history,src/cthulhu/chat.py;src/cthulhu/scripts/apps/pidgin/chat.
|
||||
atspi-announcements,Orca src/orca/script.py;src/orca/scripts/default.py;src/orca/event_manager.py,focus,"Present object:announcement payloads from any provider, politely or assertively, without requiring focus or foreground ownership",,internal/atspi internal/app,implemented,"go test ./internal/atspi ./internal/app -run Announcement -count=1; native GTK4 fixture delivers polite detail1=1 and assertive detail1=2 payloads through the adapter; live Niri 26.04 event capture recorded the workspace-switch announcement shape",Native fixture runs on private Xvfb; Xorg reader behavior otherwise unchanged,"Live Niri 26.04 capture on 2026-10-31: workspace switch emits accessible-name then object:announcement with the workspace label about 10ms before window activation. Maintainer confirms workspace switching, AltTab announcements and the exit dialog read out with the built reader on Niri on 2026-10-31.","Maintainer live listening on Niri passed for workspace switching, AltTab and the exit dialog on 2026-10-31.",Braille flash follows the system-role output queue; physical display acceptance pending.,"Announcements bypass background-event filtering and foreground epochs because the provider explicitly requests speech and may never own focus (a compositor's own accessible tree under Niri via AccessKit). Screen-lock and secure-input gates still apply. Trimmed repeats from one provider inside 300ms are deduped; Niri alternates live-node identities and may append a space to force re-emission. There is no Orca workspace-specific behavior to mirror: Orca only presents the generic announcement. See docs/testing/announcements.md."
|
||||
web-message-article-focus,src/cthulhu/scripts/web/speech_generator.py;src/cthulhu/formatting.py,web,Announce named message articles when focus enters their controls,,internal/atspi/speech_metadata_native.go;internal/app/reading_context.go;internal/presentation/reading.go,in-progress,Speech replay covers browse and focus modes plus pending browser discovery; private Chromium article capture regression. See docs/testing/messaging.md.,Private Chromium fixture uses isolated X11 and D-Bus.,,Recorded speech only; live Discord listening pending.,Physical display acceptance pending.,Shared named article handling; no Discord identifiers or child scans. Preserve ordinary container context policy; announce article labels on entry without repeating them on exit.
|
||||
web-disclosure-refresh,src/cthulhu/action_presenter.py; src/cthulhu/scripts/web/script_utilities.py,web,Discover and keep disclosure options open during Enter or Actions-list activation and caret navigation,,internal/atspi internal/app,implemented,"CTHULHU_BROWSER=/usr/bin/chromium CTHULHU_NATIVE_BROWSER_TESTS=actions,disclosure,reveal,positions,ticking,document python3 tests/native_browser.py",Private Chromium/Xorg fixture covers both activation paths; character and line reading; blur and document-scroll dismissal; offscreen text and images. Original public page also checked in a private Chromium profile.,,Original-page audible speech acceptance pending.,Physical Braille acceptance pending.,Preserve target generations and strict ownership; refresh action snapshots and ancestor order. Text caret placement reveals its range without an extra whole-object scroll that dismisses popups; non-text scrolling retained. User installed Brave acceptance pending. Link-sounds native Tab pan baseline fails before and after this fix. See docs/testing/browser-clickables.md.
|
||||
mumble-message-history,src/cthulhu/scripts/apps/Mumble/chat.py;src/cthulhu/chat.py,messaging,Review or copy twelve recent Mumble Activity log messages with Cthulhu+F1 through F12,,internal/atspi internal/messaging internal/app internal/input,in-progress,Message cleanup and twelve-slot ordering tests; focused messaging race checks; python3 -B tests/native_messaging.py.,Private Qt transcript and unrelated-provider checks; synthetic X11 review and copy chords through F12; clipboard checks.,,Recorded presentation tested; actual Mumble listening pending.,Single flash and review text tested; physical display pending.,Legacy combined history defaults. Strip timestamps and Qt noncharacters; ignore timestamp-only insertions; preserve repeated messages and keyboard focus. Pidgin keeps its nine-message review range. History is session-only and clears on lock or transcript retirement. See docs/testing/messaging.md for live acceptance.
|
||||
|
@@ -0,0 +1,29 @@
|
||||
#!/usr/bin/env python3
|
||||
# Copyright (c) 2026 Storm Dragon
|
||||
# Authors: Storm Dragon, Angus Kola
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""Verify messaging adapters and X11 shortcuts without touching the reader."""
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
sys.dont_write_bytecode = True
|
||||
from native_test_session import run_private_session
|
||||
|
||||
environment = dict(os.environ, GOCACHE="/tmp/cthulhu-go-cache", GOPROXY="off",
|
||||
CTHULHU_NATIVE_MUMBLE="1", CTHULHU_NATIVE_MESSAGING="1",
|
||||
CTHULHU_NATIVE_X11="1", QT_QPA_PLATFORM="xcb",
|
||||
QT_LINUX_ACCESSIBILITY_ALWAYS_ON="1", NO_AT_BRIDGE="0")
|
||||
checks = [
|
||||
("./internal/atspi", "TestNativeMumbleTranscriptCapture"),
|
||||
("./internal/atspi", "TestNativePidginTranscriptCapture"),
|
||||
("./internal/app", "TestNativeMessagingCopyPreservesClipboardText"),
|
||||
("./internal/platform/xorg", "TestNativeLearnModeCapturesKeysAndRestoresCommands"),
|
||||
]
|
||||
for package, test in checks:
|
||||
result = run_private_session(
|
||||
["xvfb-run", "-a", "-s", "-screen 0 1024x768x24", "dbus-run-session", "--",
|
||||
"go", "test", package, "-run", "^" + test + "$", "-count=1", "-v", "-timeout=60s"],
|
||||
env=environment, cwd=Path(__file__).resolve().parents[1])
|
||||
if result.returncode:
|
||||
sys.exit(result.returncode)
|
||||
@@ -0,0 +1,74 @@
|
||||
#!/usr/bin/env python3
|
||||
# Copyright (c) 2026 Storm Dragon
|
||||
# Authors: Storm Dragon, Angus Kola
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
"""Private Qt provider matching Mumble's read-only Activity log."""
|
||||
import ctypes
|
||||
import os
|
||||
import sys
|
||||
|
||||
from PySide6 import QtCore, QtGui, QtWidgets
|
||||
|
||||
if ctypes.CDLL(None).prctl(15, sys.argv[1].encode(), 0, 0, 0) != 0:
|
||||
raise RuntimeError("Cannot set fixture process identity")
|
||||
|
||||
application = QtWidgets.QApplication([])
|
||||
chat_window = QtWidgets.QMainWindow()
|
||||
chat_window.setWindowTitle("Chat fixture")
|
||||
chat_content = QtWidgets.QWidget()
|
||||
chat_window.setCentralWidget(chat_content)
|
||||
layout = QtWidgets.QVBoxLayout(chat_content)
|
||||
transcript = QtWidgets.QTextBrowser()
|
||||
transcript.setAccessibleName("Activity log")
|
||||
layout.addWidget(transcript)
|
||||
unrelated = QtWidgets.QTextBrowser()
|
||||
unrelated.setAccessibleName("Other log")
|
||||
layout.addWidget(unrelated)
|
||||
chat_window.show()
|
||||
|
||||
foreground = QtWidgets.QMainWindow()
|
||||
foreground.setWindowTitle("Foreground fixture")
|
||||
foreground_content = QtWidgets.QWidget()
|
||||
foreground.setCentralWidget(foreground_content)
|
||||
foreground_layout = QtWidgets.QVBoxLayout(foreground_content)
|
||||
editor = QtWidgets.QLineEdit()
|
||||
# An editable field with the same name must never count as a transcript.
|
||||
editor.setAccessibleName("Activity log")
|
||||
foreground_layout.addWidget(editor)
|
||||
foreground.show()
|
||||
editor.setFocus()
|
||||
pending_input = ""
|
||||
|
||||
|
||||
def insert(widget, text):
|
||||
cursor = widget.textCursor()
|
||||
cursor.movePosition(QtGui.QTextCursor.MoveOperation.End)
|
||||
cursor.insertText(text)
|
||||
|
||||
|
||||
def receive_command(*_):
|
||||
global pending_input
|
||||
content = os.read(sys.stdin.fileno(), 4096)
|
||||
if not content:
|
||||
application.quit()
|
||||
return
|
||||
pending_input += content.decode()
|
||||
while "\n" in pending_input:
|
||||
command, pending_input = pending_input.split("\n", 1)
|
||||
if command == "messages":
|
||||
insert(transcript, "[12:34:56]")
|
||||
insert(transcript, "[12:34:56] Username: hello\u2028world \ufdd0界\ufdef")
|
||||
for _ in range(2):
|
||||
insert(transcript, "[12:34:57] Repeated message")
|
||||
insert(unrelated, "Unrelated log text")
|
||||
editor.setText("Typed input")
|
||||
elif command == "barrier":
|
||||
editor.setText("Messaging barrier")
|
||||
elif command == "quit":
|
||||
application.quit()
|
||||
|
||||
|
||||
notifier = QtCore.QSocketNotifier(sys.stdin.fileno(), QtCore.QSocketNotifier.Type.Read)
|
||||
notifier.activated.connect(receive_command)
|
||||
QtCore.QTimer.singleShot(0, lambda: print("ready", int(foreground.winId()), flush=True))
|
||||
application.exec()
|
||||
Reference in new issue
Block a user