From 37fdd4e114e709ba5bc8a342aa0a0262c081befa Mon Sep 17 00:00:00 2001 From: Storm Dragon Date: Sat, 17 Oct 2026 01:01:51 -0400 Subject: [PATCH] Element list keys added. --- README.md | 14 + docs/testing/browser-lists.md | 87 ++++++ internal/app/browser_lists.go | 99 +++++++ internal/app/browser_lists_native_test.go | 135 ++++++++++ internal/app/browser_lists_test.go | 97 +++++++ internal/app/browser_menu_native_test.go | 200 ++++++++++++++ internal/app/internal_ui.go | 50 +++- internal/app/learn_mode.go | 3 + internal/app/runtime_commands.go | 3 +- internal/app/runtime_workers.go | 2 +- .../atspi/adapter_event_callback_native.go | 17 ++ internal/atspi/adapter_events.c | 43 +++ internal/atspi/adapter_native.h | 1 + internal/atspi/model.go | 2 +- internal/input/bindings.go | 10 + internal/input/commands.go | 5 + internal/input/document_lists.go | 88 ++++++ internal/input/document_lists_test.go | 46 ++++ internal/interaction/session.go | 16 +- internal/interaction/session_test.go | 18 ++ .../platform/xorg/listener_native_test.go | 6 + internal/review/document.go | 11 +- .../review/document_caret_editable_test.go | 109 ++++++++ internal/review/document_caret_nodes.go | 9 +- internal/review/document_lists.go | 251 ++++++++++++++++++ internal/review/document_lists_test.go | 143 ++++++++++ .../document_visual_line_native_test.go | 33 +++ internal/review/worker.go | 17 +- internal/review/worker_contract.go | 2 + testdata/parity-ledger.csv | 1 + tests/browser-fragments.html | 5 + tests/browser-lists.html | 20 ++ tests/native_browser.py | 2 + 33 files changed, 1530 insertions(+), 15 deletions(-) create mode 100644 docs/testing/browser-lists.md create mode 100644 internal/app/browser_lists.go create mode 100644 internal/app/browser_lists_native_test.go create mode 100644 internal/app/browser_lists_test.go create mode 100644 internal/app/browser_menu_native_test.go create mode 100644 internal/input/document_lists.go create mode 100644 internal/input/document_lists_test.go create mode 100644 internal/review/document_caret_editable_test.go create mode 100644 internal/review/document_lists.go create mode 100644 internal/review/document_lists_test.go create mode 100644 tests/browser-lists.html diff --git a/README.md b/README.md index 6357f79..43e26aa 100644 --- a/README.md +++ b/README.md @@ -156,6 +156,12 @@ without kernel resource quotas; it does not provide a security sandbox. ## Browser navigation +Browser element lists use **Alt+Shift** plus the structural navigation +key: **K** for links, **A** for clickables, **H** for headings, and **1–6** for +headings at a specific level. Use arrows or type a name to select an item, +Enter to move to it, and Escape to cancel. See [browser element lists](docs/testing/browser-lists.md) +for every list shortcut and verification steps. + Heading and browse-caret commands retain the virtual reading position while the browser reports delayed native caret events. Each reader command takes ownership before its result arrives. Native keyboard input, text selection, @@ -181,6 +187,14 @@ Stale native placement gets one refresh and retry at the same target and offset. Braille routing also recognizes the temporary caret produced by clearing a selection, so it cannot reset the routed reading position. +Browse-mode arrows and Read All include text inside embedded editors. Reading +continues through the editor's paragraphs before reaching the following page +content. + +Native menu rows can contain several controls. Up and Down announce those +controls individually, including Brave's Sidebar choices and Zoom controls. +Check this workflow with `CTHULHU_NATIVE_BROWSER_TESTS=menu python3 tests/native_browser.py`. + See [browser release review](docs/testing/browser-release-review.md) for findings, automated checks and remaining live acceptance. diff --git a/docs/testing/browser-lists.md b/docs/testing/browser-lists.md new file mode 100644 index 0000000..b21fdc9 --- /dev/null +++ b/docs/testing/browser-lists.md @@ -0,0 +1,87 @@ +# Browser element lists + +In browse mode, hold **Alt+Shift** and press the key below. These +shortcuts work in both desktop and laptop layouts and can be changed in +Preferences under browser shortcuts. + +| Key | List | +| --- | --- | +| A | Clickables | +| B | Buttons | +| C | Combo boxes | +| E | Text entries | +| F | Form fields | +| G | Images | +| H | Headings | +| I | List items | +| K | Links | +| L | Lists | +| M | Landmarks | +| O | Large objects | +| P | Paragraphs | +| Q | Block quotes | +| R | Radio buttons | +| T | Tables | +| U | Unvisited links | +| V | Visited links | +| X | Check boxes | +| 1–6 | Headings at that level | + +Annotation, internal frame and math lists have no default shortcut, matching +Orca. Assign them a browser shortcut in Preferences. Annotation lists include +content insertions, content deletions and marks. Separators and live regions +have no list command in Orca. + +The list opens with the current item selected when it matches the filter. +Use arrows, Home, End, Page Up and Page Down to select an item, or type the +beginning of its name. Enter closes the list and moves the reading position +to the selected item. It does not activate a link or press a button. Escape +closes the list without moving. An empty list reports zero items and releases +the keyboard. Speech presents rows and Braille shows the selection and count. +The browser retains native keyboard focus while the list is open. + +Lists use the same semantic filters as structural navigation, exclude hidden +content, and stay within the current document, including embedded content. +The native document capture limit also bounds lists. Core navigation lists can +contain more than 256 items; external Tentacle forms retain their existing +256-item limit. List selections retain exact accessible references and page +ownership, so removed targets, replaced documents and changed focus cannot +select a different item. A changed page can require reopening the list. + +Automated checks: + +```sh +go test ./internal/input ./internal/review ./internal/interaction ./internal/app -count=1 +CTHULHU_NATIVE_BROWSER_TESTS=lists,keys,document,modes,editing python3 tests/native_browser.py +python3 tests/native_internal_ui.py +``` + +The private browser list fixture checks 302 links, hidden-link exclusion, +current-row restoration, selection, Escape, clickables, a heading level and an +empty image list with actual X11 grabs and AT-SPI data. These checks inspect +speech requests and synthetic input. They do not establish audible speech or +physical keyboard/Braille behavior. + +Before submission, review the diff and test the rebuilt reader on a real page: + +1. Open links with Alt+Shift+K. Move with arrows, type a name, select + with Enter, and confirm subsequent browse arrows start at that item. Press + Enter again to verify that ordinary link activation still works. +2. Open clickables with Alt+Shift+A and headings with H and 1–6. Check + spoken labels and Braille rows, then cancel and confirm the old reading + position remains. Check lists with several hundred entries. +3. Open a list, change tabs or windows, and confirm capture is dismissed and + typing works normally. Reload or remove a selected page item and confirm + the old list cannot move to a different target. +4. Check ordinary H/Shift+H and K/Shift+K, browse arrows, Tab/Shift+Tab, + editable-field typing, browser chrome, and the existing Cthulhu+Shift+A + actions menu. Verify the new list keys pass through outside browse mode. + +Human review and these live checks remain necessary under the project's +AI-assisted contribution policy. + +The full Go race suite, private browser lists, document navigation, editing, +keyboard transitions and existing internal UI checks passed. The browser mode +check missed a dynamic-text announcement during the combined run, then passed +in isolation. The cause of that initial failure is unverified; its diagnostic +log was retained at `/tmp/cthulhu-browser-modes-2991369198.log`. diff --git a/internal/app/browser_lists.go b/internal/app/browser_lists.go new file mode 100644 index 0000000..0fc8bbb --- /dev/null +++ b/internal/app/browser_lists.go @@ -0,0 +1,99 @@ +// Copyright (c) 2026 Storm Dragon +// Authors: Storm Dragon, Angus Kola +// SPDX-License-Identifier: GPL-3.0-or-later + +package app + +import ( + "context" + "fmt" + "strconv" + "strings" + "unicode" + + "git.stormux.org/storm/cthulhu/internal/input" + "git.stormux.org/storm/cthulhu/internal/interaction" + "git.stormux.org/storm/cthulhu/internal/review" +) + +type documentListSource interface { + ListDocument(context.Context, review.DocumentTarget, int) (review.DocumentListSnapshot, error) + SelectDocumentList(review.DocumentListSnapshot, int) error +} + +func (ui *internalUI) browserListMessage(message string) { + if ui.gate.Locked() || ui.gate.Suspended() { + return + } + ui.speech.Interrupt() + ui.speech.Enqueue(speechRequest{Text: message, Role: "system", Cause: "browser-list"}) + if ui.braille != nil { + ui.braille.EnqueueMessage(message) + } +} + +func documentListTarget(binding input.DocumentListBinding) review.DocumentTarget { + switch binding.Command { + case input.DocumentListAnnotations: + return review.DocumentAnnotation + case input.DocumentListInternalFrames: + return review.DocumentInternalFrame + case input.DocumentListMath: + return review.DocumentMath + } + target, _, _ := documentReviewCommand(binding.Next) + return target +} + +func (ui *internalUI) browserList(ctx context.Context, binding input.DocumentListBinding) (*interaction.Session, review.DocumentListSnapshot, error) { + if ui.documents == nil { + return nil, review.DocumentListSnapshot{}, review.ErrNoCurrent + } + list, err := ui.documents.ListDocument(ctx, documentListTarget(binding), binding.Level) + if err != nil { + return nil, list, err + } + if len(list.Entries) == 0 { + return nil, list, nil + } + items := make([]interaction.Item, 0, len(list.Entries)) + for index, position := range list.Entries { + snapshot := position.Snapshot + label := position.Label + if label == "" { + label = snapshot.Name + } + if label == "" && snapshot.Role != "password text" { + label = snapshot.Text + } + if label == "" { + label = snapshot.Role + } + label = strings.Join(strings.FieldsFunc(label, func(r rune) bool { return unicode.IsControl(r) || unicode.IsSpace(r) }), " ") + // Keep the existing form label bound without splitting UTF-8. + labelRunes := []rune(label) + if len(labelRunes) > 225 { + label = string(labelRunes[:225]) + } + if label == "" { + label = snapshot.Role + } + if snapshot.HeadingLevel > 0 { + label += fmt.Sprintf(", heading level %d", snapshot.HeadingLevel) + } + if snapshot.Role == "check box" || snapshot.Role == "radio button" || snapshot.Role == "toggle button" { + if snapshot.Checked { + label += ", checked" + } else { + label += ", not checked" + } + } + items = append(items, interaction.Item{ID: strconv.Itoa(index), Label: label}) + } + title := fmt.Sprintf("%s: %d items found", binding.Title, len(items)) + if len(items) == 1 { + title = binding.Title + ": 1 item found" + } + session, err := interaction.NewNavigationSession(title, items, list.Selected) + return session, list, err +} diff --git a/internal/app/browser_lists_native_test.go b/internal/app/browser_lists_native_test.go new file mode 100644 index 0000000..0ac44e2 --- /dev/null +++ b/internal/app/browser_lists_native_test.go @@ -0,0 +1,135 @@ +// Copyright (c) 2026 Storm Dragon +// Authors: Storm Dragon, Angus Kola +// SPDX-License-Identifier: GPL-3.0-or-later + +package app + +import ( + "context" + "os" + "os/exec" + "strings" + "testing" + "time" + + "git.stormux.org/storm/cthulhu/internal/atspi" + "git.stormux.org/storm/cthulhu/internal/input" + "git.stormux.org/storm/cthulhu/internal/platform/xorg" + "git.stormux.org/storm/cthulhu/internal/review" +) + +func TestNativeBrowserLists(t *testing.T) { + if os.Getenv("CTHULHU_NATIVE_BROWSER") != "1" { + t.Skip("requires private browser list fixture") + } + ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) + defer cancel() + adapter := atspi.New() + if err := adapter.Start(ctx); err != nil { + t.Fatal(err) + } + defer adapter.Stop(context.Background()) + key := func(chord string) { + t.Helper() + if out, err := exec.CommandContext(ctx, "xdotool", "key", "--clearmodifiers", chord).CombinedOutput(); err != nil { + t.Fatalf("keys: %v %s", err, out) + } + } + var focus atspi.ObjectID + for attempt := 0; attempt < 25 && !focus.Valid(); attempt++ { + key("Tab") + timer := time.NewTimer(150 * time.Millisecond) + loop: + for { + select { + case event := <-adapter.Events(): + if isFocusEvent(event.Type) && event.Source.Name == "Browser foundation" { + focus = event.Source.ID() + break loop + } + case <-timer.C: + break loop + case <-ctx.Done(): + t.Fatal(ctx.Err()) + } + } + timer.Stop() + } + if !focus.Valid() { + t.Fatal("fixture heading focus missing") + } + worker, _ := review.NewWorker(adapter, 2*time.Second) + if err := worker.Start(ctx); err != nil { + t.Fatal(err) + } + defer worker.Stop(context.Background()) + worker.SyncFocus(focus) + keys := xorg.NewCommands([]string{"Insert"}, input.DesktopKeyboardLayout) + if err := keys.Start(ctx); err != nil { + t.Fatal(err) + } + defer keys.Stop(context.Background()) + keys.SetBrowseNavigation(true) + for !keys.BrowseNavigationEnabled() { + if ctx.Err() != nil { + t.Fatal(ctx.Err()) + } + time.Sleep(5 * time.Millisecond) + } + output := &ocrTestOutput{speech: make(chan speechRequest, 32)} + ui := &internalUI{keys: keys, documents: worker, speech: output, gate: newFocusGate(keys), events: make(chan input.CommandEvent, 32), done: make(chan struct{})} + go ui.run(ctx) + done := make(chan struct{}) + go func() { + defer close(done) + for { + select { + case <-ctx.Done(): + return + case event := <-keys.Commands(): + ui.Handle(event) + case <-adapter.Events(): // Keep the native event queue draining. + } + } + }() + defer func() { cancel(); <-ui.done; <-done }() + wait := func(want string) { + t.Helper() + select { + case message := <-output.speech: + if !strings.HasPrefix(message.Text, want) { + t.Fatalf("speech %q, want prefix %q", message.Text, want) + } + case <-ctx.Done(): + t.Fatalf("missing %q: %v", want, ctx.Err()) + } + } + key("alt+shift+k") + wait("Links: 302 items found. First link") + key("Down") + wait("Second link") + key("Return") + select { + case result := <-worker.Results(): + if result.Err != nil || !result.Moved || result.DocumentPosition.Snapshot.Name != "Second link" { + t.Fatalf("list selection: %+v", result) + } + case <-ctx.Done(): + t.Fatal("selection did not reach review worker") + } + key("alt+shift+k") + wait("Links: 302 items found. Second link") + key("Escape") + // The next shortcut follows Escape in the native input stream. + key("alt+shift+a") + wait("Clickables: 1 item found. Custom clickable") + key("Escape") + key("alt+shift+2") + wait("Headings at level 2: 1 item found. Second heading") + key("Escape") + key("alt+shift+g") + wait("Images: 0 items found") + if keys.InternalUISession() != 0 { + t.Fatal("empty list retained keyboard capture") + } +} diff --git a/internal/app/browser_lists_test.go b/internal/app/browser_lists_test.go new file mode 100644 index 0000000..c436b8c --- /dev/null +++ b/internal/app/browser_lists_test.go @@ -0,0 +1,97 @@ +// Copyright (c) 2026 Storm Dragon +// Authors: Storm Dragon, Angus Kola +// SPDX-License-Identifier: GPL-3.0-or-later + +package app + +import ( + "context" + "fmt" + "testing" + "time" + + "git.stormux.org/storm/cthulhu/internal/atspi" + "git.stormux.org/storm/cthulhu/internal/input" + "git.stormux.org/storm/cthulhu/internal/review" +) + +type listTestSource struct { + keys *uiTestKeys + selected chan int + count int +} + +func (source *listTestSource) ListDocument(context.Context, review.DocumentTarget, int) (review.DocumentListSnapshot, error) { + list := review.DocumentListSnapshot{Selected: source.count - 1} + for index := 0; index < source.count; index++ { + list.Entries = append(list.Entries, review.DocumentListEntry{DocumentPosition: review.DocumentPosition{Snapshot: atspi.Snapshot{Name: fmt.Sprintf("Example %03d", index), Role: "link"}}}) + } + return list, nil +} +func (source *listTestSource) SelectDocumentList(_ review.DocumentListSnapshot, index int) error { + if source.keys.InternalUISession() != 0 { + index = -1 + } + source.selected <- index + return nil +} + +func TestBrowserListNavigationSelectionAndCancellation(t *testing.T) { + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + keys := &uiTestKeys{} + source := &listTestSource{keys: keys, selected: make(chan int, 1), count: 300} + output := &ocrTestOutput{speech: make(chan speechRequest, 16)} + ui := &internalUI{keys: keys, documents: source, speech: output, gate: &focusGate{}, events: make(chan input.CommandEvent, 16), done: make(chan struct{})} + go ui.run(ctx) + defer func() { cancel(); <-ui.done }() + wait := func(want string) { + t.Helper() + select { + case message := <-output.speech: + if message.Text != want { + t.Fatalf("got %q want %q", message.Text, want) + } + case <-ctx.Done(): + t.Fatal(ctx.Err()) + } + } + send := func(symbol uint32) { + ui.Handle(input.CommandEvent{Command: input.InternalUIKey, UISession: 1, Chord: input.CommandChord{Symbol: symbol}}) + } + ui.Handle(input.CommandEvent{Command: input.DocumentListLinks}) + wait("Links: 300 items found. Example 299") + send(0xff50) + wait("Example 000") + send(0xff54) + wait("Example 001") + send(0xff0d) + select { + case index := <-source.selected: + if index != 1 { + t.Fatalf("selected %d or keyboard capture retained", index) + } + case <-ctx.Done(): + t.Fatal("selection did not finish") + } + ui.Handle(input.CommandEvent{Command: input.DocumentListLinks}) + wait("Links: 300 items found. Example 299") + send(0xff1b) + // A second request is processed after Escape, establishing cancellation + // without a timing assumption or a selection callback. + ui.Handle(input.CommandEvent{Command: input.DocumentListLinks}) + wait("Links: 300 items found. Example 299") + select { + case <-source.selected: + t.Fatal("Escape selected an item") + default: + } + send(0xff1b) +} + +func TestBrowserListCatalogMapsAllOrcaListTypes(t *testing.T) { + for _, binding := range input.DocumentListBindings() { + if documentListTarget(binding) == 0 { + t.Fatalf("unhandled %s", binding.Command) + } + } +} diff --git a/internal/app/browser_menu_native_test.go b/internal/app/browser_menu_native_test.go new file mode 100644 index 0000000..063eef6 --- /dev/null +++ b/internal/app/browser_menu_native_test.go @@ -0,0 +1,200 @@ +// Copyright (c) 2026 Storm Dragon +// Authors: Storm Dragon, Angus Kola +// SPDX-License-Identifier: GPL-3.0-or-later + +package app + +import ( + "context" + "log/slog" + "os" + "os/exec" + "strings" + "testing" + "time" + + "git.stormux.org/storm/cthulhu/internal/atspi" +) + +type menuRecordingAdapter struct { + *atspi.Adapter + events chan atspi.Event +} + +func (adapter *menuRecordingAdapter) Events() <-chan atspi.Event { return adapter.events } + +func TestNativeBrowserMenuControls(t *testing.T) { + if os.Getenv("CTHULHU_NATIVE_BROWSER") != "1" { + t.Skip("requires a private browser session") + } + ctx, cancel := context.WithTimeout(context.Background(), 25*time.Second) + defer cancel() + native := atspi.New() + if err := native.Start(ctx); err != nil { + t.Fatal(err) + } + defer native.Stop(context.Background()) + adapter := &menuRecordingAdapter{Adapter: native, events: make(chan atspi.Event, 256)} + observed := make(chan atspi.Event, 256) + pumpDone := make(chan struct{}) + go func() { + defer close(pumpDone) + for { + select { + case event := <-native.Events(): + select { + case observed <- event: + case <-ctx.Done(): + return + } + select { + case adapter.events <- event: + case <-ctx.Done(): + return + } + case <-ctx.Done(): + return + } + } + }() + queue := &autocompleteRequestQueue{requests: make(chan speechRequest, 256)} + done := make(chan struct{}) + go func() { + defer close(done) + processEventsWithInput(ctx, slog.New(slog.DiscardHandler), queue, adapter, nil, nil, 0, nil, nil, nil, nil) + }() + defer func() { cancel(); <-done; <-pumpDone }() + keys := func(chord string) { + t.Helper() + if out, err := exec.CommandContext(ctx, "xdotool", "key", "--clearmodifiers", chord).CombinedOutput(); err != nil { + t.Fatalf("keys: %v %s", err, out) + } + } + waitEvent := func(match func(atspi.Event) bool) atspi.Event { + t.Helper() + for { + select { + case event := <-observed: + if match(event) { + t.Logf("event %s (native %s): %s %q, origin %s", event.Type, event.NativeType, event.Source.Role, event.Source.Name, event.OriginPath) + return event + } + case <-ctx.Done(): + t.Fatal("missing menu event: ", ctx.Err()) + return atspi.Event{} + } + } + } + waitSpeech := func(name string) { + t.Helper() + if name == "" { + t.Fatal("menu control has no accessible name") + } + timer := time.NewTimer(time.Second) + defer timer.Stop() + for { + select { + case request := <-queue.requests: + if strings.Contains(request.Text, name) { + t.Logf("speech: %q", request.Text) + return + } + case <-timer.C: + t.Fatalf("missing menu speech %q", name) + case <-ctx.Done(): + t.Fatal(ctx.Err()) + } + } + } + settle := func() { + t.Helper() + timer := time.NewTimer(100 * time.Millisecond) + defer timer.Stop() + for { + select { + case <-observed: + timer.Reset(100 * time.Millisecond) + case request := <-queue.requests: + if request.Cause == atspi.SelectionChangedEvent || request.Cause == atspi.ActiveDescendantEvent { + t.Fatalf("duplicate menu speech without another key: %q", request.Text) + } + case <-timer.C: + return + case <-ctx.Done(): + t.Fatal(ctx.Err()) + } + } + } + keys("F10") + waitEvent(func(e atspi.Event) bool { return isFocusGainEvent(e) && strings.Contains(e.Source.Name, "Brave") }) + keys("Down") + waitEvent(func(e atspi.Event) bool { return e.Type == atspi.WindowActivatedEvent && e.Source.Role == "menu" }) + // The app menu wraps; find the row by its real selection rather than + // relying on a fixed number of entries or the current browser profile. + for steps := 0; ; steps++ { + if steps >= 40 { + t.Fatal("Sidebar row absent from app menu") + } + keys("Down") + event := waitEvent(func(e atspi.Event) bool { return e.Type == atspi.SelectionChangedEvent && e.Source.Role == "menu item" }) + waitSpeech(event.Source.Name) + if event.Source.Name == "Sidebar" { + break + } + settle() + } + inline := func(role, rowName string) atspi.Event { + return waitEvent(func(e atspi.Event) bool { + return (e.Type == atspi.SelectedStateEvent || e.NativeType == atspi.SelectedStateEvent) && + e.Detail1 == 1 && e.Source.Role == role && e.Source.Name != rowName + }) + } + first := inline("button", "Sidebar") + waitSpeech(first.Source.Name) + settle() + for _, chord := range []string{"Down", "Down", "Up", "Up", "Down", "Up"} { + keys(chord) + event := inline("button", "Sidebar") + if event.Type != atspi.ActiveDescendantEvent || event.OriginPath == event.Source.Path { + t.Fatalf("inline control lacks menu ownership: %+v", event) + } + waitSpeech(event.Source.Name) + settle() + } + for range 2 { + keys("Down") + event := inline("button", "Sidebar") + waitSpeech(event.Source.Name) + settle() + } + // Continue through the normal entries to the second compound row. + // Its inner controls use menu-item roles rather than button roles. + for steps := 0; ; steps++ { + if steps >= 25 { + t.Fatal("Zoom row absent from app menu") + } + keys("Down") + event := waitEvent(func(e atspi.Event) bool { return e.Type == atspi.SelectionChangedEvent && e.Source.Role == "menu item" }) + waitSpeech(event.Source.Name) + if event.Source.Name == "Zoom" { + break + } + settle() + } + zoom := inline("menu item", "Zoom") + waitSpeech(zoom.Source.Name) + settle() + for _, chord := range []string{"Down", "Down", "Up", "Up", "Down", "Up"} { + keys(chord) + event := inline("menu item", "Zoom") + if event.Type != atspi.ActiveDescendantEvent || event.OriginPath != zoom.OriginPath { + t.Fatalf("Zoom control lacks the same menu owner: %+v", event) + } + waitSpeech(event.Source.Name) + settle() + } + keys("Escape") + waitEvent(func(e atspi.Event) bool { return e.Type == atspi.WindowActivatedEvent && e.Source.Role != "menu" }) + keys("ctrl+l") + waitSpeech("Address and search bar") +} diff --git a/internal/app/internal_ui.go b/internal/app/internal_ui.go index 3a66f5c..74b8c3b 100644 --- a/internal/app/internal_ui.go +++ b/internal/app/internal_ui.go @@ -9,6 +9,7 @@ import ( "errors" "io" "os/exec" + "strconv" "strings" "time" @@ -50,9 +51,10 @@ type internalUIRequest struct { // keeps AT-SPI/clipboard latency off the native input and command consumers. // Core clients can Open a form and receive its submitted values/cancellation. type internalUI struct { - keys internalUIKeys - actions accessibleActionSource - interest interface { + documents documentListSource + keys internalUIKeys + actions accessibleActionSource + interest interface { ObjectOfInterest(context.Context) (review.Interest, error) } speech speechEnqueuer @@ -104,7 +106,8 @@ func (ui *internalUI) Notice(ctx context.Context, value tentaclewire.Presentatio } } func (ui *internalUI) Handle(event input.CommandEvent) bool { - if ui == nil || event.Command != input.ShowActions && event.Command != input.InternalUIKey && event.Command != input.InternalUIDismiss { + _, browserList := input.LookupDocumentList(event.Command) + if ui == nil || !browserList && event.Command != input.ShowActions && event.Command != input.InternalUIKey && event.Command != input.InternalUIDismiss { return false } select { @@ -137,6 +140,7 @@ func (ui *internalUI) run(ctx context.Context) { var session *interaction.Session var sessionID uint64 var target atspi.ActionTarget + var documentList review.DocumentListSnapshot var result chan internalUIResult var requestDone <-chan struct{} finish := func(reply internalUIResult) error { @@ -163,6 +167,7 @@ func (ui *internalUI) run(ctx context.Context) { } session, sessionID, result, requestDone = nil, 0, nil, nil target = atspi.ActionTarget{} + documentList = review.DocumentListSnapshot{} return closeErr } defer func() { finish(internalUIResult{Cancelled: true}) }() @@ -201,6 +206,33 @@ func (ui *internalUI) run(ctx context.Context) { ui.present(session.Opening()) request.ready <- nil case event := <-ui.events: + if binding, ok := input.LookupDocumentList(event.Command); ok { + if sessionID != 0 { + continue + } + lookupCtx, cancel := context.WithTimeout(ctx, 3*time.Second) + var err error + sessionID, err = ui.keys.BeginInternalUI(lookupCtx) + if err == nil { + session, documentList, err = ui.browserList(lookupCtx, binding) + } + cancel() + if sessionID == 0 || ui.keys.InternalUISession() != sessionID { + finish(internalUIResult{Cancelled: true}) + continue + } + if err != nil || session == nil { + message := binding.Title + ": 0 items found" + if err != nil { + message = "Browser list unavailable" + } + finish(internalUIResult{Err: err}) + ui.browserListMessage(message) + continue + } + ui.present(session.Opening()) + continue + } if event.Command == input.ShowActions { if sessionID != 0 { finish(internalUIResult{Cancelled: true}) @@ -303,7 +335,17 @@ func (ui *internalUI) run(ctx context.Context) { if effect.Activated != "" { values := session.Values() actionTarget := target + selectedList := documentList closeErr := finish(internalUIResult{Activated: effect.Activated, Values: values}) + if len(selectedList.Entries) > 0 && closeErr == nil && !ui.gate.Locked() && !ui.gate.Suspended() { + index, err := strconv.Atoi(effect.Activated) + if err == nil { + err = ui.documents.SelectDocumentList(selectedList, index) + } + if err != nil { + ui.browserListMessage("Browser item unavailable") + } + } if actionTarget.Object.Valid() && closeErr == nil { actionCtx, cancel := context.WithTimeout(ctx, 2*time.Second) err := ui.actions.PerformAccessibleAction(actionCtx, actionTarget, effect.Activated) diff --git a/internal/app/learn_mode.go b/internal/app/learn_mode.go index 71ed9fd..83e0b8e 100644 --- a/internal/app/learn_mode.go +++ b/internal/app/learn_mode.go @@ -150,6 +150,9 @@ func spokenLearnKey(event input.CommandEvent) string { } func learnCommandDescription(command input.Command) string { + if _, ok := input.LookupDocumentList(command); ok { + return input.CommandLabel(command) + } switch command { case input.ShowActions: return "Show the actions available for the current object" diff --git a/internal/app/runtime_commands.go b/internal/app/runtime_commands.go index 0ca3442..fa71e82 100644 --- a/internal/app/runtime_commands.go +++ b/internal/app/runtime_commands.go @@ -141,7 +141,8 @@ func (worker commandRuntime) run(ctx context.Context) error { if worker.internalUI.Handle(event) { continue } - if worker.internalUI != nil && (command == input.ShowActions || command == input.InternalUIKey || command == input.InternalUIDismiss) { + _, browserList := input.LookupDocumentList(command) + if worker.internalUI != nil && (browserList || command == input.ShowActions || command == input.InternalUIKey || command == input.InternalUIDismiss) { stopRun() return errors.New("internal UI event queue overflow") } diff --git a/internal/app/runtime_workers.go b/internal/app/runtime_workers.go index 3c5dc68..9359c91 100644 --- a/internal/app/runtime_workers.go +++ b/internal/app/runtime_workers.go @@ -65,7 +65,7 @@ func startRuntimeWorkers(ctx context.Context, command commandRuntime, inputListe clipboardDone: clipboardDone, } if keys, ok := command.commands.(internalUIKeys); ok { - ui := &internalUI{notices: make(chan internalUINotice, 4), keys: keys, actions: command.actionSource, interest: command.objectReview, speech: speech, braille: braille, gate: command.focusGate, events: make(chan input.CommandEvent, 32), requests: make(chan internalUIRequest, 1), done: make(chan struct{})} + ui := &internalUI{notices: make(chan internalUINotice, 4), keys: keys, documents: command.objectReview, actions: command.actionSource, interest: command.objectReview, speech: speech, braille: braille, gate: command.focusGate, events: make(chan input.CommandEvent, 32), requests: make(chan internalUIRequest, 1), done: make(chan struct{})} command.internalUI = ui if command.tentacles != nil { command.tentacles.forms = ui diff --git a/internal/atspi/adapter_event_callback_native.go b/internal/atspi/adapter_event_callback_native.go index ffe3624..fb2b20a 100644 --- a/internal/atspi/adapter_event_callback_native.go +++ b/internal/atspi/adapter_event_callback_native.go @@ -462,6 +462,20 @@ func cthulhuAtspiEventCallback(event *C.AtspiEvent, userData unsafe.Pointer) { } payload := C.cthulhu_event_payload(event) snapshot, popupFor := adapter.captureNativeEventSnapshot(event, target, eventType, originID) + var inlineMenuID ObjectID + if eventType == SelectedStateEvent && C.cthulhu_event_detail1(event) == 1 && !snapshot.Defunct && + (snapshot.Role == "button" || snapshot.Role == "push button" || snapshot.Role == "toggle button" || + snapshot.Role == "menu item" || snapshot.Role == "check menu item" || snapshot.Role == "radio menu item") { + if menu := C.cthulhu_inline_menu_container(target); menu != nil { + inlineMenuID = nativeObjectID(menu) + C.g_object_unref(C.gpointer(menu)) + if inlineMenuID.Valid() && inlineMenuID.BusName == snapshot.BusName { + // The event's detail is authoritative even when libatspi still + // caches the previous SELECTED state on the first visit. + eventType, snapshot.Selected = ActiveDescendantEvent, true + } + } + } // Chromium can drop ACTIVE from its visible main frame while a tiny // same-process sibling claims it. On X11, the correlated foreground // window is the stronger ownership check performed below. Keep the @@ -525,6 +539,9 @@ func cthulhuAtspiEventCallback(event *C.AtspiEvent, userData unsafe.Pointer) { clear(adapter.navigationEcho) } focusEvent.OriginPath = sanitizeSnapshotField(C.GoString(C.cthulhu_accessible_path(event.source))) + if eventType == ActiveDescendantEvent && inlineMenuID.Valid() { + focusEvent.OriginPath = inlineMenuID.Path + } C.g_free(C.gpointer(payload)) if ownedTarget { C.g_object_unref(C.gpointer(target)) diff --git a/internal/atspi/adapter_events.c b/internal/atspi/adapter_events.c index db011fc..62bfb27 100644 --- a/internal/atspi/adapter_events.c +++ b/internal/atspi/adapter_events.c @@ -454,6 +454,49 @@ AtspiAccessible *cthulhu_popup_menu_target(AtspiAccessible *window) { return menu; } +// Compound native menu rows can report selection on an inner control without +// changing the menu's selected child. Return its menu only when a menu-item +// ancestor precedes that menu, and the chain reaches a native window without +// entering document content. The caller owns the returned reference. +AtspiAccessible *cthulhu_inline_menu_container(AtspiAccessible *source) { + AtspiAccessible *current = atspi_accessible_get_parent(source, NULL); + AtspiAccessible *menu = NULL; + gboolean inRow = FALSE; + for (int depth = 0; current != NULL && depth < 32; depth++) { + GError *error = NULL; + AtspiRole role = atspi_accessible_get_role(current, &error); + if (error != NULL) { g_error_free(error); break; } + if (role == ATSPI_ROLE_DOCUMENT_WEB || role == ATSPI_ROLE_DOCUMENT_FRAME || + role == ATSPI_ROLE_EMBEDDED || role == ATSPI_ROLE_APPLICATION) break; + if (role == ATSPI_ROLE_FRAME || role == ATSPI_ROLE_WINDOW) { + g_object_unref(current); + return menu; + } + if (menu == NULL) { + if (role == ATSPI_ROLE_MENU) { + if (!inRow) break; // Ordinary menu entries keep their existing events. + AtspiStateSet *states = atspi_accessible_get_state_set(current); + gboolean showing = states != NULL && + atspi_state_set_contains(states, ATSPI_STATE_SHOWING) && + !atspi_state_set_contains(states, ATSPI_STATE_DEFUNCT); + if (states != NULL) g_object_unref(states); + if (!showing) break; + menu = g_object_ref(current); + } else if (role == ATSPI_ROLE_MENU_ITEM || role == ATSPI_ROLE_CHECK_MENU_ITEM || + role == ATSPI_ROLE_RADIO_MENU_ITEM) { + inRow = TRUE; + } + } + AtspiAccessible *parent = atspi_accessible_get_parent(current, &error); + g_object_unref(current); + current = parent; + if (error != NULL) { g_error_free(error); break; } + } + if (current != NULL) g_object_unref(current); + if (menu != NULL) g_object_unref(menu); + return NULL; +} + const gchar *cthulhu_accessible_path(AtspiAccessible *source) { return source != NULL ? ATSPI_OBJECT(source)->path : NULL; } diff --git a/internal/atspi/adapter_native.h b/internal/atspi/adapter_native.h index bdb1630..f68d73c 100644 --- a/internal/atspi/adapter_native.h +++ b/internal/atspi/adapter_native.h @@ -115,6 +115,7 @@ gboolean cthulhu_in_active_document(AtspiAccessible *source); gboolean cthulhu_in_inactive_window(AtspiAccessible *source); gboolean cthulhu_active_menu_window(AtspiAccessible *window); AtspiAccessible *cthulhu_popup_menu_target(AtspiAccessible *window); +AtspiAccessible *cthulhu_inline_menu_container(AtspiAccessible *source); const gchar *cthulhu_accessible_path(AtspiAccessible *source); const gchar *cthulhu_accessible_bus_name(AtspiAccessible *source); AtspiAccessible *cthulhu_window_ancestor(AtspiAccessible *source); diff --git a/internal/atspi/model.go b/internal/atspi/model.go index 277de9c..c2aa424 100644 --- a/internal/atspi/model.go +++ b/internal/atspi/model.go @@ -274,7 +274,7 @@ type Event struct { NavigationEcho bool SnapshotDuration time.Duration Type string - // NativeType retains an AT-SPI text-change origin suffix for debug logs. + // NativeType retains the original AT-SPI type when an event is normalized. // Presentation policies use the canonical Type. NativeType string Source Snapshot diff --git a/internal/input/bindings.go b/internal/input/bindings.go index add3070..e3133ec 100644 --- a/internal/input/bindings.go +++ b/internal/input/bindings.go @@ -52,6 +52,9 @@ func bindingDefaults(layout KeyboardLayout, scope string) CommandBindings { } func CommandLabel(command Command) string { + if binding, ok := LookupDocumentList(command); ok { + return "List " + strings.ToLower(binding.Title) + } switch command { case PresentCellFormula: return "Present the formula for the current cell" @@ -173,6 +176,13 @@ func BindingRows(layout KeyboardLayout, overrides []BindingOverride) []BindingRo for chord, command := range bindingDefaults(layout, scope) { groups[command] = append(groups[command], FormatCommandChord(chord)) } + if scope == "browse" { + for _, binding := range DocumentListBindings() { + if _, exists := groups[binding.Command]; !exists { + groups[binding.Command] = nil + } + } + } if scope == "global" { for _, command := range []Command{PresentCellFormula, SetColumnHeadersRow, ClearColumnHeadersRow, SetRowHeadersColumn, ClearRowHeadersColumn, PresentDate, StickyFocusMode, StickyBrowseMode, PresentPreviousNotification, PresentNextNotification, OCRObject, OCRWindow, OCRDesktop, OCRClipboard} { if _, exists := groups[command]; !exists { diff --git a/internal/input/commands.go b/internal/input/commands.go index 4bffec2..7700268 100644 --- a/internal/input/commands.go +++ b/internal/input/commands.go @@ -435,6 +435,11 @@ func BrowseCommandBindings(layout KeyboardLayout) CommandBindings { bindings[CommandChord{Symbol: binding.symbol}] = binding.next bindings[CommandChord{Symbol: binding.symbol, Modifiers: 1}] = binding.previous } + for _, binding := range DocumentListBindings() { + if binding.Symbol != 0 { + bindings[CommandChord{Symbol: binding.Symbol, Modifiers: 1 | 8}] = binding.Command + } + } headingLevels := []struct { level int next Command diff --git a/internal/input/document_lists.go b/internal/input/document_lists.go new file mode 100644 index 0000000..1d2490b --- /dev/null +++ b/internal/input/document_lists.go @@ -0,0 +1,88 @@ +// Copyright (c) 2026 Storm Dragon +// Authors: Storm Dragon, Angus Kola +// SPDX-License-Identifier: GPL-3.0-or-later + +package input + +const ( + DocumentListBlockQuotes Command = "document-list-block-quotes" + DocumentListButtons Command = "document-list-buttons" + DocumentListCheckBoxes Command = "document-list-check-boxes" + DocumentListComboBoxes Command = "document-list-combo-boxes" + DocumentListEntries Command = "document-list-entries" + DocumentListFormFields Command = "document-list-form-fields" + DocumentListHeadings Command = "document-list-headings" + DocumentListImages Command = "document-list-images" + DocumentListLandmarks Command = "document-list-landmarks" + DocumentListLists Command = "document-list-lists" + DocumentListListItems Command = "document-list-list-items" + DocumentListParagraphs Command = "document-list-paragraphs" + DocumentListRadioButtons Command = "document-list-radio-buttons" + DocumentListTables Command = "document-list-tables" + DocumentListLinks Command = "document-list-links" + DocumentListUnvisitedLinks Command = "document-list-unvisited-links" + DocumentListVisitedLinks Command = "document-list-visited-links" + DocumentListLargeObjects Command = "document-list-large-objects" + DocumentListClickables Command = "document-list-clickables" + DocumentListHeadingsAtLevel1 Command = "document-list-headings-at-level-1" + DocumentListHeadingsAtLevel2 Command = "document-list-headings-at-level-2" + DocumentListHeadingsAtLevel3 Command = "document-list-headings-at-level-3" + DocumentListHeadingsAtLevel4 Command = "document-list-headings-at-level-4" + DocumentListHeadingsAtLevel5 Command = "document-list-headings-at-level-5" + DocumentListHeadingsAtLevel6 Command = "document-list-headings-at-level-6" + DocumentListAnnotations Command = "document-list-annotations" + DocumentListInternalFrames Command = "document-list-internal-frames" + DocumentListMath Command = "document-list-math" +) + +// DocumentListBinding describes browser lists independently of presentation. +// Unbound commands remain available in the shortcut editor, as in Orca. +type DocumentListBinding struct { + Command Command + Symbol uint32 + Title string + Next Command + Level int +} + +func DocumentListBindings() []DocumentListBinding { + return []DocumentListBinding{ + {Command: DocumentListBlockQuotes, Symbol: 'q', Title: "Block quotes", Next: DocumentNextBlockQuote}, + {Command: DocumentListButtons, Symbol: 'b', Title: "Buttons", Next: DocumentNextButton}, + {Command: DocumentListCheckBoxes, Symbol: 'x', Title: "Check boxes", Next: DocumentNextCheckBox}, + {Command: DocumentListComboBoxes, Symbol: 'c', Title: "Combo boxes", Next: DocumentNextComboBox}, + {Command: DocumentListEntries, Symbol: 'e', Title: "Text entries", Next: DocumentNextEntry}, + {Command: DocumentListFormFields, Symbol: 'f', Title: "Form fields", Next: DocumentNextFormField}, + {Command: DocumentListHeadings, Symbol: 'h', Title: "Headings", Next: DocumentNextHeading}, + {Command: DocumentListImages, Symbol: 'g', Title: "Images", Next: DocumentNextImage}, + {Command: DocumentListLandmarks, Symbol: 'm', Title: "Landmarks", Next: DocumentNextLandmark}, + {Command: DocumentListLists, Symbol: 'l', Title: "Lists", Next: DocumentNextList}, + {Command: DocumentListListItems, Symbol: 'i', Title: "List items", Next: DocumentNextListItem}, + {Command: DocumentListParagraphs, Symbol: 'p', Title: "Paragraphs", Next: DocumentNextParagraph}, + {Command: DocumentListRadioButtons, Symbol: 'r', Title: "Radio buttons", Next: DocumentNextRadioButton}, + {Command: DocumentListTables, Symbol: 't', Title: "Tables", Next: DocumentNextTable}, + {Command: DocumentListLinks, Symbol: 'k', Title: "Links", Next: DocumentNextLink}, + {Command: DocumentListUnvisitedLinks, Symbol: 'u', Title: "Unvisited links", Next: DocumentNextUnvisitedLink}, + {Command: DocumentListVisitedLinks, Symbol: 'v', Title: "Visited links", Next: DocumentNextVisitedLink}, + {Command: DocumentListLargeObjects, Symbol: 'o', Title: "Large objects", Next: DocumentNextChunk}, + {Command: DocumentListClickables, Symbol: 'a', Title: "Clickables", Next: DocumentNextClickable}, + {Command: DocumentListHeadingsAtLevel1, Symbol: '1', Title: "Headings at level 1", Next: DocumentNextHeading, Level: 1}, + {Command: DocumentListHeadingsAtLevel2, Symbol: '2', Title: "Headings at level 2", Next: DocumentNextHeading, Level: 2}, + {Command: DocumentListHeadingsAtLevel3, Symbol: '3', Title: "Headings at level 3", Next: DocumentNextHeading, Level: 3}, + {Command: DocumentListHeadingsAtLevel4, Symbol: '4', Title: "Headings at level 4", Next: DocumentNextHeading, Level: 4}, + {Command: DocumentListHeadingsAtLevel5, Symbol: '5', Title: "Headings at level 5", Next: DocumentNextHeading, Level: 5}, + {Command: DocumentListHeadingsAtLevel6, Symbol: '6', Title: "Headings at level 6", Next: DocumentNextHeading, Level: 6}, + {Command: DocumentListAnnotations, Title: "Annotations"}, + {Command: DocumentListInternalFrames, Title: "Internal frames"}, + {Command: DocumentListMath, Title: "Math"}, + } +} + +func LookupDocumentList(command Command) (DocumentListBinding, bool) { + for _, binding := range DocumentListBindings() { + if binding.Command == command { + return binding, true + } + } + return DocumentListBinding{}, false +} diff --git a/internal/input/document_lists_test.go b/internal/input/document_lists_test.go new file mode 100644 index 0000000..a64c556 --- /dev/null +++ b/internal/input/document_lists_test.go @@ -0,0 +1,46 @@ +// Copyright (c) 2026 Storm Dragon +// Authors: Storm Dragon, Angus Kola +// SPDX-License-Identifier: GPL-3.0-or-later + +package input + +import "testing" + +func TestDocumentListShortcutsAndCustomization(t *testing.T) { + for _, layout := range []KeyboardLayout{DesktopKeyboardLayout, LaptopKeyboardLayout} { + bindings := BrowseCommandBindings(layout) + bound := 0 + for _, binding := range DocumentListBindings() { + if binding.Symbol == 0 { + continue + } + bound++ + chord := CommandChord{Symbol: binding.Symbol, Modifiers: 9} + if bindings[chord] != binding.Command || DefaultCommandBindings(layout)[chord] != "" { + t.Fatalf("%s has incorrect scope or shortcut", binding.Command) + } + outside := NewCommandRouter([]uint32{0xff63}, DefaultCommandBindings(layout)) + if consumed, command := outside.Handle(binding.Symbol, 100, 9, true); consumed || command != "" { + t.Fatalf("%s intercepted outside browse mode", binding.Command) + } + router := NewCommandRouter([]uint32{0xff63}, bindings) + if consumed, command := router.Handle(binding.Symbol, 100, 9, true); !consumed || command != binding.Command { + t.Fatalf("%s was not dispatched: %v %s", binding.Command, consumed, command) + } + if consumed, _ := router.Handle(binding.Symbol, 100, 9, false); !consumed { + t.Fatal("lost list key release") + } + } + if bound != 25 { + t.Fatalf("got %d shortcuts, want 19 types and 6 heading levels", bound) + } + overrides := []BindingOverride{{Layout: layout, Scope: "browse", Command: DocumentListMath, Keys: []string{"Alt+Shift+y"}}} + if err := ValidateBindingOverrides(overrides); err != nil { + t.Fatal(err) + } + custom, err := ResolveCommandBindings(layout, true, overrides) + if err != nil || custom[CommandChord{Symbol: 'y', Modifiers: 9}] != DocumentListMath { + t.Fatalf("unbound list customization: %v", err) + } + } +} diff --git a/internal/interaction/session.go b/internal/interaction/session.go index 110ea90..0775730 100644 --- a/internal/interaction/session.go +++ b/internal/interaction/session.go @@ -100,6 +100,20 @@ type Session struct { const maxTextLength = 8192 func NewSession(form Form) (*Session, error) { + return new_session(form, 256) +} + +// NewNavigationSession is for core document lists. External forms retain +// their smaller protocol limit. The selected row follows the reading caret. +func NewNavigationSession(title string, items []Item, selected int) (*Session, error) { + session, err := new_session(Form{Title: title, Elements: []Element{{ID: "items", Label: title, Kind: List, Items: items}}}, 65536) + if err == nil && selected >= 0 && selected < len(items) && !items[selected].Disabled { + session.elements[0].selected = selected + } + return session, err +} + +func new_session(form Form, maxItems int) (*Session, error) { if form.Title == "" || len(form.Title) > 1024 || len(form.Elements) == 0 || len(form.Elements) > 64 { return nil, errors.New("a titled form with 1 to 64 controls is required") } @@ -127,7 +141,7 @@ func NewSession(form Form) (*Session, error) { element.Items = append([]Item(nil), element.Items...) state := elementState{element: element, selected: -1, caret: len([]rune(element.Text)), anchor: len([]rune(element.Text))} itemIDs := map[string]bool{} - if len(element.Items) > 256 { + if len(element.Items) > maxItems { return nil, errors.New("too many list items") } for index, item := range element.Items { diff --git a/internal/interaction/session_test.go b/internal/interaction/session_test.go index 37b2e1f..5a28cc5 100644 --- a/internal/interaction/session_test.go +++ b/internal/interaction/session_test.go @@ -5,10 +5,28 @@ package interaction import ( + "strconv" "strings" "testing" ) +func TestNavigationListsRetainExternalFormLimit(t *testing.T) { + items := make([]Item, 300) + for index := range items { + items[index] = Item{ID: strconv.Itoa(index), Label: "Example item"} + } + if _, err := NewSession(Form{Title: "External form", Elements: []Element{{ID: "items", Label: "Items", Kind: List, Items: items}}}); err == nil { + t.Fatal("external form exceeded its protocol limit") + } + session, err := NewNavigationSession("Links", items, 299) + if err != nil { + t.Fatal(err) + } + if effect := session.Handle(Event{Key: Enter}); effect.Activated != "299" { + t.Fatalf("lost initial selection: %+v", effect) + } +} + func TestListNavigationAndIndependentSessions(t *testing.T) { form := Form{Title: "Actions", Elements: []Element{{ID: "actions", Label: "Actions", Kind: List, Items: []Item{{ID: "click", Label: "Click"}, {ID: "skip", Label: "Unavailable", Disabled: true}, {ID: "press", Label: "Press"}, {ID: "ancestor", Label: "Click ancestor"}}}}} first, err := NewSession(form) diff --git a/internal/platform/xorg/listener_native_test.go b/internal/platform/xorg/listener_native_test.go index 9a44dab..f468c3b 100644 --- a/internal/platform/xorg/listener_native_test.go +++ b/internal/platform/xorg/listener_native_test.go @@ -433,6 +433,12 @@ func TestNativeBrowseKeyTransitions(t *testing.T) { listener.SetBrowseNavigation(false) key("keyup", "h") ready(false) + key("key", "alt+shift+k", "alt+shift+a") + select { + case got := <-listener.Commands(): + t.Fatalf("list shortcut consumed outside browse mode: %s", got.Command) + case <-time.After(100 * time.Millisecond): + } listener.SetBrowseNavigation(true) ready(true) key("key", "Num_Lock") diff --git a/internal/review/document.go b/internal/review/document.go index e2c73ac..fe650c9 100644 --- a/internal/review/document.go +++ b/internal/review/document.go @@ -315,6 +315,9 @@ const ( DocumentVisitedLink DocumentClickable DocumentChunk + DocumentAnnotation + DocumentInternalFrame + DocumentMath ) // Move searches in accessible tree order, wrapping once to the opposite end. @@ -425,7 +428,7 @@ func (cursor *DocumentCursor) Move(ctx context.Context, target DocumentTarget, l } func validDocumentTarget(target DocumentTarget) bool { - return target >= DocumentHeading && target <= DocumentChunk + return target >= DocumentHeading && target <= DocumentMath } func documentTargetMatches(target DocumentTarget, node atspi.DocumentNode, level int) bool { @@ -474,6 +477,12 @@ func documentTargetMatches(target DocumentTarget, node atspi.DocumentNode, level return snapshot.Role == "link" && snapshot.Focusable && snapshot.Visited case DocumentClickable: return snapshot.SupportsClick && !snapshot.Focusable && !documentClickExcludedRole(snapshot) && !documentHasXMLRole(snapshot, "gridcell") + case DocumentAnnotation: + return snapshot.Role == "content deletion" || snapshot.Role == "content insertion" || snapshot.Role == "mark" + case DocumentInternalFrame: + return snapshot.Role == "internal frame" + case DocumentMath: + return snapshot.Role == "math" case DocumentChunk: if documentChunkRole(snapshot) { return snapshot.Role == "heading" || snapshot.Role == "list" || snapshot.Role == "table" || diff --git a/internal/review/document_caret_editable_test.go b/internal/review/document_caret_editable_test.go new file mode 100644 index 0000000..6623b22 --- /dev/null +++ b/internal/review/document_caret_editable_test.go @@ -0,0 +1,109 @@ +// Copyright (c) 2026 Storm Dragon +// Authors: Storm Dragon, Angus Kola +// SPDX-License-Identifier: GPL-3.0-or-later + +package review + +import ( + "context" + "strings" + "testing" + "time" + "unicode/utf8" + + "git.stormux.org/storm/cthulhu/internal/atspi" +) + +func editable_caret_cursor(t *testing.T) (*DocumentCursor, caretTextReader) { + t.Helper() + reader := caretTextReader{ + webID("/before"): "Suggested wording", + webID("/editor"): "\ufffc\ufffc", + webID("/first"): "Review the code.\nReport actionable issues.", + webID("/second"): "Explain the evidence.", + webID("/secret"): "Private fixture text", + webID("/after"): "Following paragraph", + } + node := func(path, role string, editable bool, children ...string) atspi.TreeNode { + snapshot := atspi.NewSnapshot(path, "", role, "", "") + snapshot.BusName = webID(path).BusName + snapshot.Editable = editable + snapshot.CharacterCount = utf8.RuneCountInString(reader[webID(path)]) + result := atspi.TreeNode{Snapshot: snapshot} + for _, child := range children { + result.Children = append(result.Children, webID(child)) + } + return result + } + tree, err := atspi.NewAccessibleTree(webID("/doc"), []atspi.TreeNode{ + node("/doc", "document web", false, "/before", "/editor", "/password", "/after"), + node("/before", "static", false), + node("/editor", "entry", true, "/paragraph", "/second"), + node("/paragraph", "paragraph", true, "/first"), + node("/first", "static", true), + node("/second", "static", true), + node("/password", "password text", true, "/secret"), + // Even a provider which omits editable state on a password child + // must not expose that child's text through browse navigation. + node("/secret", "static", false), + node("/after", "static", false), + }) + if err != nil { + t.Fatal(err) + } + cursor, err := NewDocumentCursor(tree) + if err != nil { + t.Fatal(err) + } + if _, err := cursor.SyncFocus(context.Background(), webID("/before")); err != nil { + t.Fatal(err) + } + return cursor, reader +} + +func TestDocumentCaretReadsEditableDescendants(t *testing.T) { + ctx := context.Background() + cursor, reader := editable_caret_cursor(t) + for _, step := range []struct { + command DocumentCaretCommand + want string + }{ + {CaretNextLine, "Review the code."}, + {CaretNextLine, "Report actionable issues."}, + {CaretNextLine, "Explain the evidence."}, + {CaretNextLine, "Following paragraph"}, + {CaretPreviousLine, "Explain the evidence."}, + {CaretPreviousLine, "Report actionable issues."}, + {CaretLineStart, "R"}, + {CaretNextCharacter, "e"}, + {CaretNextWord, "Report"}, + {CaretPreviousLine, "Review the code."}, + {CaretPreviousLine, "Suggested wording"}, + } { + move, err := cursor.MoveCaret(ctx, reader, step.command) + if err != nil || !move.Moved || strings.TrimSpace(move.Text) != step.want { + t.Fatalf("%s: %q at %s, moved %t, %v; want %q", step.command, move.Text, + move.Position.Snapshot.Path, move.Moved, err, step.want) + } + } +} + +func TestReadAllIncludesEditableDescendants(t *testing.T) { + cursor, reader := editable_caret_cursor(t) + capturer := &readAllCapturer{tree: cursor.tree, caretTextReader: reader} + worker, err := NewWorker(capturer, time.Second) + if err != nil { + t.Fatal(err) + } + worker.SyncFocus(webID("/before")) + result := worker.readAll(context.Background(), cursor) + want := []string{"Suggested wording", "Review the code.", "Report actionable issues.", "Explain the evidence.", "Following paragraph"} + if result.Err != nil || len(result.ReadAllLines) != len(want) { + t.Fatalf("Read All: %d lines, %v; want %d", len(result.ReadAllLines), result.Err, len(want)) + } + for index, line := range result.ReadAllLines { + if got := strings.TrimSpace(line.Contents[0].Text); got != want[index] { + t.Fatalf("Read All line %d: %q; want %q", index, got, want[index]) + } + } +} diff --git a/internal/review/document_caret_nodes.go b/internal/review/document_caret_nodes.go index 5792262..12db2f7 100644 --- a/internal/review/document_caret_nodes.go +++ b/internal/review/document_caret_nodes.go @@ -73,6 +73,10 @@ func caret_node_indices(nodes []atspi.DocumentNode, object atspi.ObjectReference if node.Hidden || skipped[node.Reference.ID()] { continue } + if snapshot.Role == "password text" { + skipChildren(node.Children) + continue + } if !protected[node.Reference.ID()] && decorative_caret_glyph(node) { continue } @@ -83,7 +87,10 @@ func caret_node_indices(nodes []atspi.DocumentNode, object atspi.ObjectReference (node.Snapshot.Role == "entry" || node.Snapshot.Role == "text") && (node.Snapshot.CharacterCount == 0 || len(node.Children) == 0) { indices = append(indices, index) - } else if !node.Snapshot.Editable && len(node.Children) == 0 && node.Snapshot.CharacterCount > 0 { + } else if len(node.Children) == 0 && node.Snapshot.CharacterCount > 0 { + // Rich editors expose their contents as editable text leaves. + // Browse reading must descend into them just like ordinary text; + // the entry's embedded-object placeholders are not its contents. indices = append(indices, index) } } diff --git a/internal/review/document_lists.go b/internal/review/document_lists.go new file mode 100644 index 0000000..beeab88 --- /dev/null +++ b/internal/review/document_lists.go @@ -0,0 +1,251 @@ +// Copyright (c) 2026 Storm Dragon +// Authors: Storm Dragon, Angus Kola +// SPDX-License-Identifier: GPL-3.0-or-later + +package review + +import ( + "context" + "errors" + "strings" + "time" + + "git.stormux.org/storm/cthulhu/internal/atspi" +) + +// DocumentListSnapshot retains exact generation-checked references. Opening a list +// never moves the cursor. Selection cannot recapture or substitute an item. +type DocumentListSnapshot struct { + Entries []DocumentListEntry + Selected int + Target DocumentTarget + Level int + focus atspi.ObjectID + revision uint64 + epoch uint64 + tree *atspi.AccessibleTree +} + +type DocumentListEntry struct { + DocumentPosition + Label string +} + +type documentListReply struct { + list DocumentListSnapshot + err error +} +type documentListRequest struct { + ctx context.Context + target DocumentTarget + level int + result chan documentListReply +} +type documentListSelection struct { + list DocumentListSnapshot + index int +} + +// ListDocument is called by the UI worker, never by the input/event loops. +func (worker *Worker) ListDocument(ctx context.Context, target DocumentTarget, level int) (DocumentListSnapshot, error) { + if err := ctx.Err(); err != nil { + return DocumentListSnapshot{}, err + } + if !validDocumentTarget(target) || level < 0 || level > 6 || target != DocumentHeading && level != 0 { + return DocumentListSnapshot{}, ErrInvalidDirection + } + worker.stateMutex.Lock() + done, started := worker.done, worker.started + worker.stateMutex.Unlock() + if !started { + return DocumentListSnapshot{}, ErrNoCurrent + } + query := &documentListRequest{ctx: ctx, target: target, level: level, result: make(chan documentListReply, 1)} + select { + case worker.requests <- request{documentList: query}: + case <-ctx.Done(): + return DocumentListSnapshot{}, ctx.Err() + case <-done: + return DocumentListSnapshot{}, ErrNoCurrent + } + select { + case reply := <-query.result: + return reply.list, reply.err + case <-ctx.Done(): + return DocumentListSnapshot{}, ctx.Err() + case <-done: + return DocumentListSnapshot{}, ErrNoCurrent + } +} + +func (worker *Worker) SelectDocumentList(list DocumentListSnapshot, index int) error { + if index < 0 || index >= len(list.Entries) || list.tree == nil { + return ErrInvalidDirection + } + select { + case worker.requests <- request{documentEpoch: list.epoch, listSelection: &documentListSelection{list: list, index: index}, queuedAt: time.Now()}: + return nil + default: + return ErrRequestQueueFull + } +} + +func (worker *Worker) listDocument(ctx context.Context, cursor *DocumentCursor, query *documentListRequest) (DocumentListSnapshot, error) { + if err := query.ctx.Err(); err != nil { + return DocumentListSnapshot{}, err + } + worker.stateMutex.Lock() + list := DocumentListSnapshot{Target: query.target, Level: query.level, focus: worker.latestFocus, revision: worker.focusRevision, epoch: worker.documentEpoch.Load()} + worker.stateMutex.Unlock() + if !list.focus.Valid() { + return list, ErrNoCurrent + } + captureContext, cancel := context.WithTimeout(ctx, worker.captureTimeout) + defer cancel() + stop := context.AfterFunc(query.ctx, cancel) + defer stop() + tree, err := worker.capturer.CaptureDocument(captureContext, list.focus) + if err != nil { + return list, err + } + if tree == nil { + return list, errors.New("capture document list: nil tree") + } + ref, err := tree.Reference(list.focus) + if err != nil { + return list, err + } + doc, err := tree.Document(captureContext, ref) + if err != nil { + return list, err + } + current := list.focus + if cursor != nil { + cursor.mu.Lock() + if len(cursor.places) > 0 { + place := cursor.places[len(cursor.places)-1] + if place.document == doc.Context.TopLevel && (place.focus.ID() == list.focus || place.object.ID() == list.focus) { + current = place.object.ID() + } + } + cursor.mu.Unlock() + } + list.tree = tree + // Unnamed clickable sections and containers often expose their readable + // label only in static descendants. Assemble bounded labels once, in + // reverse tree order, instead of rescanning each candidate's subtree. + labels := make(map[atspi.ObjectID]string, len(doc.Nodes)) + for index := len(doc.Nodes) - 1; index >= 0; index-- { + node := doc.Nodes[index] + if node.Hidden { + continue + } + label := node.Snapshot.Name + if node.Snapshot.Role != "password text" { + if label == "" { + label = node.Snapshot.Text + } + if label == "" { + var parts []string + size := 0 + for _, child := range node.Children { + if part := labels[child]; part != "" { + parts = append(parts, part) + size += len(part) + } + if size >= 900 { + break + } + } + label = strings.Join(parts, " ") + } + } + runes := []rune(label) + if len(runes) > 225 { + label = string(runes[:225]) + } + labels[node.Reference.ID()] = label + } + parents := make(map[atspi.ObjectID]atspi.ObjectID, len(doc.Nodes)) + indices := make(map[atspi.ObjectID]int) + for _, node := range doc.Nodes { + parents[node.Reference.ID()] = node.Parent + if node.Hidden || !documentTargetMatches(query.target, node, query.level) { + continue + } + indices[node.Reference.ID()] = len(list.Entries) + list.Entries = append(list.Entries, DocumentListEntry{DocumentPosition: DocumentPosition{Document: doc.Context.TopLevel, Object: node.Reference, Snapshot: node.Snapshot, Origin: StructuralNavigation}, Label: labels[node.Reference.ID()]}) + } + for id := current; id.Valid(); id = parents[id] { + if index, ok := indices[id]; ok { + list.Selected = index + break + } + } + if !worker.currentDocumentList(list) { + return DocumentListSnapshot{}, atspi.ErrStaleObject + } + return list, captureContext.Err() +} + +func (worker *Worker) currentDocumentList(list DocumentListSnapshot) bool { + worker.stateMutex.Lock() + defer worker.stateMutex.Unlock() + return list.focus == worker.latestFocus && list.revision == worker.focusRevision && list.epoch == worker.documentEpoch.Load() +} + +func (worker *Worker) selectDocumentList(ctx context.Context, cursor *DocumentCursor, selection *documentListSelection) (Result, *DocumentCursor) { + list := selection.list + result := Result{DocumentEpoch: list.epoch, DocumentTarget: list.Target, DocumentLevel: list.Level, DocumentFocus: list.focus} + if !worker.currentDocumentList(list) { + result.Err = atspi.ErrStaleObject + return result, cursor + } + position := list.Entries[selection.index] + if _, err := list.tree.Resolve(position.Object); err != nil { + result.Err = err + return result, cursor + } + doc, err := list.tree.Document(ctx, position.Object) + if err != nil { + result.Err = err + return result, cursor + } + if doc.Context.TopLevel != position.Document { + result.Err = atspi.ErrStaleObject + return result, cursor + } + found := false + for _, node := range doc.Nodes { + if node.Reference == position.Object && !node.Hidden && documentTargetMatches(list.Target, node, list.Level) { + found = true + break + } + } + if !found { + result.Err = atspi.ErrStaleObject + return result, cursor + } + if cursor == nil || cursor.tree != list.tree { + cursor, _ = NewDocumentCursor(list.tree) + } + if _, err := cursor.SyncFocus(ctx, list.focus); err != nil { + result.Err = err + return result, cursor + } + if !worker.currentDocumentList(list) { + result.Err = atspi.ErrStaleObject + return result, cursor + } + cursor.mu.Lock() + place := &cursor.places[len(cursor.places)-1] + place.object, place.offset, place.origin = position.Object, 0, StructuralNavigation + place.tableCell = atspi.ObjectID{} + remember_text_parent(doc, place) + result.DocumentPosition = documentPosition(doc, *place) + cursor.mu.Unlock() + result.Moved = true + // Use the exact validated target; retry/recapture belongs to ordinary + // navigation and must not choose a replacement for a menu selection. + return worker.placeDocument(ctx, cursor, result), cursor +} diff --git a/internal/review/document_lists_test.go b/internal/review/document_lists_test.go new file mode 100644 index 0000000..03959b2 --- /dev/null +++ b/internal/review/document_lists_test.go @@ -0,0 +1,143 @@ +// Copyright (c) 2026 Storm Dragon +// Authors: Storm Dragon, Angus Kola +// SPDX-License-Identifier: GPL-3.0-or-later + +package review + +import ( + "context" + "testing" + "time" + + "git.stormux.org/storm/cthulhu/internal/atspi" +) + +func TestDocumentListPreservesCursorAndSelectsExactTarget(t *testing.T) { + tree, cursor := webCursor(t) + ctx := context.Background() + if _, err := cursor.SyncFocus(ctx, webID("/doc")); err != nil { + t.Fatal(err) + } + capturer := &readAllCapturer{tree: tree} + worker, _ := NewWorker(capturer, time.Second) + worker.SyncFocus(webID("/doc")) + before := cursor.diagnostic_state() + list, err := worker.listDocument(ctx, cursor, &documentListRequest{ctx: ctx, target: DocumentHeading}) + if err != nil || len(list.Entries) != 2 { + t.Fatalf("heading list: %+v %v", list, err) + } + if after := cursor.diagnostic_state(); after != before { + t.Fatalf("opening moved cursor: %+v -> %+v", before, after) + } + result, cursor := worker.selectDocumentList(ctx, cursor, &documentListSelection{list: list, index: 1}) + if result.Err != nil || !result.Moved || result.DocumentPosition.Object != list.Entries[1].Object { + t.Fatalf("selection: %+v", result) + } + list, err = worker.listDocument(ctx, cursor, &documentListRequest{ctx: ctx, target: DocumentHeading}) + if err != nil || list.Selected != 1 { + t.Fatalf("did not preserve current row: %+v %v", list, err) + } + filtered, err := worker.listDocument(ctx, cursor, &documentListRequest{ctx: ctx, target: DocumentHeading, level: 2}) + if err != nil || len(filtered.Entries) != 2 || filtered.Entries[0].Snapshot.HeadingLevel != 2 { + t.Fatalf("level filter: %+v %v", filtered, err) + } +} + +func TestDocumentListRejectsChangedTargetsAndOwnership(t *testing.T) { + for _, change := range []string{"target removed", "target reused", "target hidden", "focus changed", "document reloaded"} { + t.Run(change, func(t *testing.T) { + tree, cursor := webCursor(t) + ctx := context.Background() + worker, _ := NewWorker(&readAllCapturer{tree: tree}, time.Second) + worker.SyncFocus(webID("/doc")) + list, err := worker.listDocument(ctx, cursor, &documentListRequest{ctx: ctx, target: DocumentHeading}) + if err != nil { + t.Fatal(err) + } + switch change { + case "target removed": + tree.Invalidate(list.Entries[0].Object.ID()) + case "target reused": + tree.Invalidate(list.Entries[0].Object.ID()) + if err := tree.Replace(webID("/root"), webNodes()); err != nil { + t.Fatal(err) + } + case "target hidden": + nodes := webNodes() + nodes[3].Geometry = atspi.Geometry{VisibilityKnown: true} + if err := tree.Replace(webID("/root"), nodes); err != nil { + t.Fatal(err) + } + case "focus changed": + worker.SyncFocus(webID("/chrome")) + case "document reloaded": + worker.ResetDocument(webID("/doc")) + } + result, _ := worker.selectDocumentList(ctx, cursor, &documentListSelection{list: list, index: 0}) + if result.Err == nil || result.Moved { + t.Fatalf("stale selection succeeded: %+v", result) + } + }) + } +} + +func TestDocumentListDescendantLabelsExcludeHiddenAndPasswordText(t *testing.T) { + node := func(path, role, text string, children ...string) atspi.TreeNode { + snapshot := atspi.Snapshot{BusName: webID(path).BusName, Path: path, Role: role, Text: text} + entry := atspi.TreeNode{Snapshot: snapshot} + for _, child := range children { + entry.Children = append(entry.Children, webID(child)) + } + return entry + } + nodes := []atspi.TreeNode{ + node("/doc", "document web", "", "/clickable"), + node("/clickable", "section", "", "/text", "/hidden", "/password"), + node("/text", "static", "Clickable text"), + node("/hidden", "static", "Hidden content"), + node("/password", "password text", "private fixture password"), + } + nodes[1].Snapshot.SupportsClick = true + nodes[3].Geometry = atspi.Geometry{VisibilityKnown: true} + tree, err := atspi.NewAccessibleTree(webID("/doc"), nodes) + if err != nil { + t.Fatal(err) + } + worker, _ := NewWorker(&readAllCapturer{tree: tree}, time.Second) + worker.SyncFocus(webID("/doc")) + ctx := context.Background() + list, err := worker.listDocument(ctx, nil, &documentListRequest{ctx: ctx, target: DocumentClickable}) + if err != nil || len(list.Entries) != 1 || list.Entries[0].Label != "Clickable text" { + t.Fatalf("unsafe or missing descendant label: %+v %v", list, err) + } +} + +func TestWorkerDocumentListUsesOrderedRequests(t *testing.T) { + tree, _ := webCursor(t) + worker, cancel := startWorker(t, &readAllCapturer{tree: tree}, time.Second) + defer cancel() + worker.SyncFocus(webID("/doc")) + ctx, stop := context.WithTimeout(context.Background(), time.Second) + defer stop() + list, err := worker.ListDocument(ctx, DocumentHeading, 0) + if err != nil { + t.Fatal(err) + } + if err := worker.SelectDocumentList(list, 1); err != nil { + t.Fatal(err) + } + result := waitResult(t, worker) + if result.Err != nil || !result.Moved || result.DocumentPosition.Object != list.Entries[1].Object { + t.Fatalf("queued selection: %+v", result) + } +} + +func TestAdditionalDocumentListRoles(t *testing.T) { + for target, roles := range map[DocumentTarget][]string{DocumentAnnotation: {"content deletion", "content insertion", "mark"}, DocumentInternalFrame: {"internal frame"}, DocumentMath: {"math"}} { + for _, role := range roles { + if !documentTargetMatches(target, atspi.DocumentNode{Snapshot: atspi.Snapshot{Role: role}}, 0) { + t.Fatalf("missing %s", role) + } + } + } +} diff --git a/internal/review/document_visual_line_native_test.go b/internal/review/document_visual_line_native_test.go index 9820c15..f1b1502 100644 --- a/internal/review/document_visual_line_native_test.go +++ b/internal/review/document_visual_line_native_test.go @@ -166,6 +166,16 @@ func TestNativeBrowserFragmentLines(t *testing.T) { if !rankFound { t.Fatal("Read All did not reach member rank") } + var editableLines []string + for _, line := range result.ReadAllLines { + text := strings.TrimSpace(line.Contents[0].Text) + if text == "Review the code." || text == "Report actionable issues." || text == "After the suggestion." { + editableLines = append(editableLines, text) + } + } + if strings.Join(editableLines, "|") != "Review the code.|Report actionable issues.|After the suggestion." { + t.Fatalf("Read All skipped or duplicated embedded editor text: %q", editableLines) + } // Fragmented headings must also hand Down to following content as a unit. if _, err := cursor.MoveCaret(ctx, adapter, CaretDocumentStart); err != nil { t.Fatal(err) @@ -178,4 +188,27 @@ func TestNativeBrowserFragmentLines(t *testing.T) { if err != nil || next.Text != "Code first" { t.Fatalf("Down after split heading: %+v, %v", next, err) } + move, err = cursor.Move(ctx, DocumentHeading, 2, false) + if err != nil || !move.Moved || move.Position.Snapshot.Name != "Embedded suggestion" { + t.Fatalf("editor heading: %q, %v", move.Position.Snapshot.Name, err) + } + for _, step := range []struct { + command DocumentCaretCommand + want string + }{ + {CaretNextLine, ""}, // Stop on the editor control before entering its text. + {CaretNextLine, "Review the code."}, + {CaretNextLine, "Report actionable issues."}, + {CaretNextLine, "After the suggestion."}, + {CaretPreviousLine, "Report actionable issues."}, + {CaretPreviousLine, "Review the code."}, + {CaretNextCharacter, "e"}, + {CaretNextWord, "Review"}, + } { + movement, err := cursor.MoveCaret(ctx, adapter, step.command) + if err != nil || !movement.Moved || strings.TrimSpace(movement.Text) != step.want { + t.Fatalf("embedded editor %s: %q (%s, %s, count %d), moved %t, %v; want %q", step.command, movement.Text, + movement.Position.Snapshot.Role, movement.Position.Snapshot.Path, movement.Position.Snapshot.CharacterCount, movement.Moved, err, step.want) + } + } } diff --git a/internal/review/worker.go b/internal/review/worker.go index 7dc0727..76f8782 100644 --- a/internal/review/worker.go +++ b/internal/review/worker.go @@ -261,6 +261,11 @@ func (worker *Worker) run(ctx context.Context, done chan<- struct{}) { documentEpoch = epoch failedDocumentBefore = time.Time{} } + if request.documentList != nil { + list, err := worker.listDocument(ctx, document, request.documentList) + request.documentList.result <- documentListReply{list, err} + continue + } if request.interest != nil { interest, err := resolve_interest(request.interest.ctx, interestFocus, explicitInterest, cursor, flat, flatTree, document) worker.stateMutex.Lock() @@ -275,18 +280,20 @@ func (worker *Worker) run(ctx context.Context, done chan<- struct{}) { request.interest.result <- interestReply{interest, err} continue } - if (request.brailleRoute != nil || request.nativeCaret || request.readAll != nil || request.line || request.target != 0 || request.caretCommand != 0 || request.activate || request.fragment != "" || request.unit != 0) && request.documentEpoch != documentEpoch { + if (request.brailleRoute != nil || request.nativeCaret || request.readAll != nil || request.line || request.listSelection != nil || request.target != 0 || request.caretCommand != 0 || request.activate || request.fragment != "" || request.unit != 0) && request.documentEpoch != documentEpoch { if request.brailleRoute != nil { request.brailleRoute.done <- atspi.ErrStaleObject } continue } - documentCommand := request.target != 0 || request.caretCommand != 0 || request.activate || request.fragment != "" + documentCommand := request.listSelection != nil || request.target != 0 || request.caretCommand != 0 || request.activate || request.fragment != "" if documentCommand && request.queuedAt.Before(failedDocumentBefore) { continue } var result Result - if request.brailleRoute != nil { + if request.listSelection != nil { + result, document = worker.selectDocumentList(ctx, document, request.listSelection) + } else if request.brailleRoute != nil { result = worker.route_document_braille(ctx, document, request) } else if request.nativeCaret { result, document = worker.syncDocumentCaret(ctx, document, request) @@ -335,7 +342,7 @@ func (worker *Worker) run(ctx context.Context, done chan<- struct{}) { explicitInterest = flatInterest } else if request.direction != 0 { explicitInterest = objectInterest - } else if request.brailleRoute != nil || request.target != 0 || request.caretCommand != 0 || request.nativeCaret || request.fragment != "" || request.readPosition != nil { + } else if request.brailleRoute != nil || request.listSelection != nil || request.target != 0 || request.caretCommand != 0 || request.nativeCaret || request.fragment != "" || request.readPosition != nil { explicitInterest = implicitInterest } worker.stateMutex.Lock() @@ -352,7 +359,7 @@ func (worker *Worker) run(ctx context.Context, done chan<- struct{}) { // can try again; successful navigation keeps its ordered steps. failedDocumentBefore = time.Now() } - if request.brailleRoute != nil || request.target != 0 || request.caretCommand != 0 || request.fragment != "" { + if request.brailleRoute != nil || request.listSelection != nil || request.target != 0 || request.caretCommand != 0 || request.fragment != "" { result = worker.readDocumentBraille(ctx, result) result.Elapsed = time.Since(request.queuedAt) } diff --git a/internal/review/worker_contract.go b/internal/review/worker_contract.go index 86683c3..4e14435 100644 --- a/internal/review/worker_contract.go +++ b/internal/review/worker_contract.go @@ -108,6 +108,8 @@ type Result struct { // activation, fragment, flat unit, then object direction. Flags are deliberately // retained; epoch and timeout retirement depend on their distinct combinations. type request struct { + documentList *documentListRequest + listSelection *documentListSelection brailleRoute *documentBrailleRequest interest *interestRequest nativeCaret bool diff --git a/testdata/parity-ledger.csv b/testdata/parity-ledger.csv index 5fb364b..17bdfe7 100644 --- a/testdata/parity-ledger.csv +++ b/testdata/parity-ledger.csv @@ -58,3 +58,4 @@ libreoffice-calc-basic,Orca src/orca/scripts/apps/soffice/script.py; Orca src/or steam-shader-progress,Steam shader dialog AT-SPI events,web,Report Steam Vulkan shader percentages through shared progress output,,internal/webscripts internal/app,in-progress,go test ./internal/app ./internal/webscripts ./internal/presentation ./internal/sound -count=1; python3 tests/native_progress.py; CTHULHU_NATIVE_BROWSER_TESTS=progress python3 tests/native_browser.py,Private GTK and Brave HTML/ARIA progress checks passed.,,Tone requests verified; live Steam listening pending.,Unrelated Braille ownership covered by unit tests; physical check pending.,English static shader-status name changes only; verified steamwebhelper identity; see PROGRESS_BARS.md. wayland-shared-keyboard-access,src/cthulhu/input_event_manager.py,input,Monitor keys and grab reader shortcuts through the shared accessibility keyboard interface,Storm Dragon,internal/platform/wayland internal/app internal/sessionbus,in-progress,Private dbus-run-session race tests cover wire protocol runtime dispatch configuration rollback secure transitions and cleanup; go test ./... -count=1; python3 tests/check_speech.py,Existing native_x11.py keyboard harness passed; browser tabs and editing passed; browser modes passed on isolated rerun after one failure whose cause was not established.,Shared D-Bus fixture passed without an X display. Live niri and physical key suppression remain unverified.,Pending live listening on Wayland,Pending physical Braille on Wayland,No compositor-specific code. See WAYLAND_SUPPORT.md for supported commands limitations and live acceptance. X11 production listeners are unchanged; the suspension probe now links its existing xkbcommon dependency. web-browse-recovery-activation,Orca src/orca/scripts/web/script_utilities.py; master:src/cthulhu/scripts/web/script.py,web,Recover dynamic browse positions and activate custom clickables silently,,internal/atspi internal/review internal/app,in-progress,go test -race ./... -count=1; python3 tests/native_browser.py; python3 tests/native_browser_navigation.py; python3 tests/native_firefox.py,Private Chromium and Firefox fixtures; real-reader changing-page and activation checks use private Xvfb and D-Bus.,,Recorded SSIP checks verify page-only activation feedback; audible listening remains pending.,Selection-collapse regression and native mode frames covered; physical routing remains pending.,See docs/testing/browser-release-review.md for findings and live acceptance. No full Orca parity claim. +web-structural-lists,Orca src/orca/structural_navigator.py; src/orca/structural_navigator_command_definitions.py,web,List browser elements and move to a selected item,,internal/input internal/review internal/interaction internal/app,in-progress,"go test -race ./... -count=1; CTHULHU_NATIVE_BROWSER_TESTS=lists,keys,document,modes,editing python3 tests/native_browser.py; python3 tests/native_internal_ui.py",Private Brave/Xvfb list workflow covers 302 links hidden content current row selection Escape clickables heading level and empty list.,,Live listening pending.,Shared internal UI frames covered; physical Braille acceptance pending.,25 browse shortcuts use Alt+Shift. All Orca list types are registered including unbound annotations internal frames and math. Exact targets and page ownership are validated before selection. See docs/testing/browser-lists.md for manual review and regression checks. diff --git a/tests/browser-fragments.html b/tests/browser-fragments.html index 4379263..619b760 100644 --- a/tests/browser-fragments.html +++ b/tests/browser-fragments.html @@ -22,5 +22,10 @@

Mixed  text.

★

+

Embedded suggestion

+
+

Review the code.

Report actionable issues.

+
+

After the suggestion.

diff --git a/tests/browser-lists.html b/tests/browser-lists.html new file mode 100644 index 0000000..620dc12 --- /dev/null +++ b/tests/browser-lists.html @@ -0,0 +1,20 @@ + +Cthulhu Browser Probe + +

Browser foundation

+

Second heading

+First link +Second link + +
Custom clickable
+

Link destination

+ + + diff --git a/tests/native_browser.py b/tests/native_browser.py index 299befa..d543bf7 100644 --- a/tests/native_browser.py +++ b/tests/native_browser.py @@ -10,6 +10,8 @@ import time BROWSER_PROOFS = { + "menu": ("./internal/app", "^TestNativeBrowserMenuControls$", 30, 35), + "lists": ("./internal/app", "^TestNativeBrowserLists$", 35, 40, "browser-lists.html"), "pointer": ("./internal/atspi", "^TestNativePointerInput$", 15, 20), "progress": ("./internal/app", "^TestNativeBrowserProgress$", 30, 35, "browser-progress.html"), "fragments": ("./internal/review", "^TestNativeBrowserFragmentLines$", 30, 35, "browser-fragments.html"),