Find works in browsers.
This commit is contained in:
1 parent
1074904c5d
commit
ac67236eca
8 files changed
+307
-7
No files matched your search
@@ -12,6 +12,7 @@ Cthulhu is a Linux screen reader written in Go.
|
|||||||
- [Messaging support](docs/testing/messaging.md): Pidgin message review/copy shortcuts, application boundaries, and verification.
|
- [Messaging support](docs/testing/messaging.md): Pidgin message review/copy shortcuts, application boundaries, and verification.
|
||||||
- [Flatpak foreground accessibility](docs/testing/flatpak-foreground.md): process identity and proxy matching, tests, and live acceptance.
|
- [Flatpak foreground accessibility](docs/testing/flatpak-foreground.md): process identity and proxy matching, tests, and live acceptance.
|
||||||
- [Background event filtering](docs/testing/background-events.md): early foreground checks, notification exceptions, and performance verification.
|
- [Background event filtering](docs/testing/background-events.md): early foreground checks, notification exceptions, and performance verification.
|
||||||
|
- [Browser Find](docs/testing/browser-find.md): native Ctrl+F focus, typing, window ownership and verification.
|
||||||
- [MATE window switcher](docs/testing/window-switcher.md): Marco selection announcements, isolated checks, and live verification.
|
- [MATE window switcher](docs/testing/window-switcher.md): Marco selection announcements, isolated checks, and live verification.
|
||||||
- [MATE desktop](docs/testing/mate-desktop.md): Orca comparison, panel menus and icon labels, regression checks, and live verification.
|
- [MATE desktop](docs/testing/mate-desktop.md): Orca comparison, panel menus and icon labels, regression checks, and live verification.
|
||||||
- [Qt control states](docs/testing/qt-control-states.md): menu checkability, native regressions, and the Qt 5 provider limitation.
|
- [Qt control states](docs/testing/qt-control-states.md): menu checkability, native regressions, and the Qt 5 provider limitation.
|
||||||
|
|||||||
@@ -0,0 +1,52 @@
|
|||||||
|
# Browser Find
|
||||||
|
|
||||||
|
Ctrl+F opens the browser's native Find interface. Cthulhu must announce its
|
||||||
|
entry and release browse-mode keys so typing and navigation reach Find.
|
||||||
|
|
||||||
|
Chromium/Brave exposes Find as an AT-SPI frame nested beneath its main frame.
|
||||||
|
The inner frame has no independent X11 window or ACTIVE state. Taking that
|
||||||
|
inner frame as the window owner caused foreground filtering to discard Find's
|
||||||
|
focus and typing events, leaving browse-mode grabs active. Window ancestry now
|
||||||
|
resolves the enclosing frame, including nested frames in the same provider.
|
||||||
|
Separate application windows still undergo the existing foreground checks.
|
||||||
|
|
||||||
|
## Automated checks
|
||||||
|
|
||||||
|
```sh
|
||||||
|
CTHULHU_NATIVE_BROWSER_TESTS=find,keys,modes,menu,tabs,flat-chrome,document python3 tests/native_browser.py
|
||||||
|
CTHULHU_FIREFOX_FIND=1 python3 tests/native_firefox.py
|
||||||
|
go test ./internal/atspi ./internal/input ./internal/app -count=1
|
||||||
|
go vet ./internal/atspi ./internal/app
|
||||||
|
python3 tests/native_test_session.py env GOCACHE=/tmp/cthulhu-cache CTHULHU_NATIVE_PANEL_APPLET=1 CTHULHU_NATIVE_PANEL_MENU=1 CTHULHU_NATIVE_BACKGROUND_EVENTS=1 go test ./internal/atspi ./internal/app -run '^TestNative(EmbeddedPanelApplet|MatePanelMenu|BackgroundEventFiltering)' -count=1 -p=1 -timeout=90s -v
|
||||||
|
```
|
||||||
|
|
||||||
|
The Find proof uses private Xvfb, D-Bus and browser profiles. It checks Find
|
||||||
|
entry speech, browse-key release, typing a structural-navigation letter,
|
||||||
|
Tab/Shift+Tab, Escape, restored caret/heading commands and reopening Find.
|
||||||
|
It records requests without producing audible speech.
|
||||||
|
|
||||||
|
## Live acceptance
|
||||||
|
|
||||||
|
With a rebuilt reader, check Brave and Firefox separately:
|
||||||
|
|
||||||
|
1. Open a normal webpage, navigate by heading and arrow, and press Ctrl+F.
|
||||||
|
Confirm that Find is announced.
|
||||||
|
2. Type a phrase containing browse keys such as H, F or L. Confirm the phrase
|
||||||
|
reaches Find. Try Enter/Shift+Enter, Tab and Shift+Tab through its controls.
|
||||||
|
3. Press Escape. Confirm reading and heading navigation work, then reopen Find.
|
||||||
|
4. Check address-bar typing, browser menus and switching tabs/windows. Keep a
|
||||||
|
second browser window in the background and confirm its updates stay quiet.
|
||||||
|
5. With Braille, check Find's label, entered text and return to document content.
|
||||||
|
|
||||||
|
Installed-reader, physical-keyboard, audible-speech and physical-Braille
|
||||||
|
acceptance remain separate from these automated checks.
|
||||||
|
|
||||||
|
## Separate Firefox follow-up
|
||||||
|
|
||||||
|
The expanded proof also reproduced a heading-label problem after typing in
|
||||||
|
Find, closing it, pressing Ctrl+Home, then H: Firefox can announce only
|
||||||
|
"heading level 2" for the fixture's "Searchable destination" heading.
|
||||||
|
The same failure was reproduced using the original `adapter_events.c` from
|
||||||
|
commit `1074904` through a Go build overlay. It predates this ancestry fix.
|
||||||
|
The Find proof checks the heading level to verify restored command ownership;
|
||||||
|
it does not establish correct heading text or full Find-result presentation.
|
||||||
@@ -0,0 +1,224 @@
|
|||||||
|
// Copyright (c) 2026 Storm Dragon
|
||||||
|
// Authors: Storm Dragon, Angus Kola
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"log/slog"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.stormux.org/storm/cthulhu/internal/atspi"
|
||||||
|
"git.stormux.org/storm/cthulhu/internal/input"
|
||||||
|
"git.stormux.org/storm/cthulhu/internal/platform/xorg"
|
||||||
|
"git.stormux.org/storm/cthulhu/internal/review"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestNativeBrowserFind(t *testing.T) {
|
||||||
|
if os.Getenv("CTHULHU_NATIVE_BROWSER") != "1" && os.Getenv("CTHULHU_NATIVE_FIREFOX") != "1" {
|
||||||
|
t.Skip("requires private browser session")
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 25*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
adapter := atspi.New()
|
||||||
|
if err := adapter.Start(ctx); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer adapter.Stop(context.Background())
|
||||||
|
listener := xorg.New()
|
||||||
|
if err := listener.Start(ctx); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer listener.Stop(context.Background())
|
||||||
|
commands := xorg.NewCommands([]string{"Insert", "KP_Insert"}, input.DesktopKeyboardLayout)
|
||||||
|
if err := commands.Start(ctx); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer commands.Stop(context.Background())
|
||||||
|
worker, err := review.NewWorker(adapter, 2*time.Second)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := worker.Start(ctx); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer worker.Stop(context.Background())
|
||||||
|
browser := newBrowserModeController(listener, commands, adapter)
|
||||||
|
if err := browser.Start(ctx); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer browser.Stop(context.Background())
|
||||||
|
output := &ocrTestOutput{speech: make(chan speechRequest, 128)}
|
||||||
|
logFile, err := os.CreateTemp("/tmp", "cthulhu-find-*.log")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer logFile.Close()
|
||||||
|
t.Logf("diagnostics: %s", logFile.Name())
|
||||||
|
probe := &menuRecordingAdapter{Adapter: adapter, events: make(chan atspi.Event, 256)}
|
||||||
|
pumpDone := make(chan struct{})
|
||||||
|
go func() {
|
||||||
|
defer close(pumpDone)
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case event := <-adapter.Events():
|
||||||
|
if isFocusGainEvent(event) {
|
||||||
|
t.Logf("native focus %s %q editable=%t", event.Source.Role, event.Source.Name, event.Source.Editable)
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case probe.events <- event:
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
loopDone, commandDone := make(chan struct{}), make(chan struct{})
|
||||||
|
go func() {
|
||||||
|
defer close(loopDone)
|
||||||
|
processEventsWithInput(ctx, slog.New(slog.NewTextHandler(logFile, &slog.HandlerOptions{Level: slog.LevelDebug})), output, probe, nil, worker,
|
||||||
|
20*time.Millisecond, nil, browser, nil, nil)
|
||||||
|
}()
|
||||||
|
go func() {
|
||||||
|
defer close(commandDone)
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case event := <-commands.Commands():
|
||||||
|
browser.observeCommand(event.Command)
|
||||||
|
if command := documentCaretCommand(event.Command); command != 0 {
|
||||||
|
worker.MoveDocumentCaret(command)
|
||||||
|
} else if target, level, backwards := documentReviewCommand(event.Command); target != 0 {
|
||||||
|
worker.MoveDocument(target, level, backwards)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
defer func() { cancel(); <-loopDone; <-commandDone; <-pumpDone }()
|
||||||
|
key := func(chord string) {
|
||||||
|
t.Helper()
|
||||||
|
if out, err := exec.CommandContext(ctx, "xdotool", "key", "--clearmodifiers", chord).CombinedOutput(); err != nil {
|
||||||
|
t.Fatalf("keys: %v %s", err, out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
waitSpeech := func(text, role string) speechRequest {
|
||||||
|
t.Helper()
|
||||||
|
timer := time.NewTimer(3 * time.Second)
|
||||||
|
defer timer.Stop()
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case request := <-output.speech:
|
||||||
|
t.Logf("speech: %s %q (%s)", request.Role, request.Text, request.Cause)
|
||||||
|
if strings.Contains(request.Text, text) && (role == "" || request.Role == role) {
|
||||||
|
return request
|
||||||
|
}
|
||||||
|
case err := <-commands.Failures():
|
||||||
|
t.Fatal(err)
|
||||||
|
case <-timer.C:
|
||||||
|
t.Fatalf("missing speech %q", text)
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal(ctx.Err())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
waitStart := func() {
|
||||||
|
t.Helper()
|
||||||
|
for {
|
||||||
|
if request := waitSpeech("Find fixture start", ""); request.Cause == "document-caret" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// A focusable heading gives both browser families a known browse-mode start.
|
||||||
|
key("ctrl+l")
|
||||||
|
key("Escape")
|
||||||
|
heading := false
|
||||||
|
for attempt := 0; attempt < 25 && !heading; attempt++ {
|
||||||
|
key("Tab")
|
||||||
|
timer := time.NewTimer(200 * time.Millisecond)
|
||||||
|
drain:
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case request := <-output.speech:
|
||||||
|
if request.Role == "heading" && strings.Contains(request.Text, "Find fixture start") {
|
||||||
|
heading = true
|
||||||
|
break drain
|
||||||
|
}
|
||||||
|
case <-timer.C:
|
||||||
|
break drain
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal(ctx.Err())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
timer.Stop()
|
||||||
|
}
|
||||||
|
if !heading {
|
||||||
|
t.Fatal("fixture heading focus missing")
|
||||||
|
}
|
||||||
|
for !commands.BrowseNavigationEnabled() {
|
||||||
|
select {
|
||||||
|
case <-time.After(5 * time.Millisecond):
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal("browse grabs missing")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
key("ctrl+Home")
|
||||||
|
waitStart()
|
||||||
|
// Wait for the native caret/focus echoes of navigation before opening
|
||||||
|
// browser chrome. They must not satisfy the Find field assertion.
|
||||||
|
quiet := time.NewTimer(200 * time.Millisecond)
|
||||||
|
defer quiet.Stop()
|
||||||
|
settle:
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-output.speech:
|
||||||
|
quiet.Reset(200 * time.Millisecond)
|
||||||
|
case <-quiet.C:
|
||||||
|
break settle
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal(ctx.Err())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
key("ctrl+f")
|
||||||
|
waitSpeech("Find", "entry")
|
||||||
|
for commands.BrowseNavigationEnabled() {
|
||||||
|
select {
|
||||||
|
case <-time.After(5 * time.Millisecond):
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal("Find retained browse grabs")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
key("l")
|
||||||
|
waitSpeech("l", "entry")
|
||||||
|
key("Tab")
|
||||||
|
waitSpeech("", "button")
|
||||||
|
key("shift+Tab")
|
||||||
|
waitSpeech("Find", "entry")
|
||||||
|
key("Escape")
|
||||||
|
for !commands.BrowseNavigationEnabled() {
|
||||||
|
select {
|
||||||
|
case <-time.After(5 * time.Millisecond):
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal("Escape did not restore browse grabs")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Firefox can return native focus to the document root after Find,
|
||||||
|
// whereas Chromium restores the previously focused heading.
|
||||||
|
key("ctrl+Home")
|
||||||
|
waitStart()
|
||||||
|
key("h")
|
||||||
|
// Firefox can expose this heading without its text after Find, including
|
||||||
|
// with the original ancestry code. Check restored command ownership here;
|
||||||
|
// heading-label recovery is a separate regression (see browser-find.md).
|
||||||
|
waitSpeech("heading level 2", "heading")
|
||||||
|
key("ctrl+f")
|
||||||
|
waitSpeech("Find", "entry")
|
||||||
|
}
|
||||||
@@ -657,8 +657,9 @@ gboolean cthulhu_may_be_background_notification(AtspiAccessible *source) {
|
|||||||
// its frame; a standalone dialog directly below the application owns itself.
|
// its frame; a standalone dialog directly below the application owns itself.
|
||||||
// GTK plugs also publish a frame, but its parent chain crosses the socket's
|
// GTK plugs also publish a frame, but its parent chain crosses the socket's
|
||||||
// provider before reaching the host window. Keep that host as the owner while
|
// provider before reaching the host window. Keep that host as the owner while
|
||||||
// retaining the source object's provider identity. Same-provider nested frames
|
// retaining the source object's provider identity. Nested frames (such as
|
||||||
// retain their existing nearest-window behavior.
|
// Chromium's Find bar) belong to their outer frame too: they have no independent
|
||||||
|
// X11 foreground window or ACTIVE state.
|
||||||
AtspiAccessible *cthulhu_window_ancestor(AtspiAccessible *source) {
|
AtspiAccessible *cthulhu_window_ancestor(AtspiAccessible *source) {
|
||||||
if (source == NULL) return NULL;
|
if (source == NULL) return NULL;
|
||||||
AtspiAccessible *current = g_object_ref(source);
|
AtspiAccessible *current = g_object_ref(source);
|
||||||
@@ -676,9 +677,7 @@ AtspiAccessible *cthulhu_window_ancestor(AtspiAccessible *source) {
|
|||||||
g_object_unref(current);
|
g_object_unref(current);
|
||||||
return window;
|
return window;
|
||||||
}
|
}
|
||||||
if ((role == ATSPI_ROLE_FRAME || role == ATSPI_ROLE_WINDOW) &&
|
if (role == ATSPI_ROLE_FRAME || role == ATSPI_ROLE_WINDOW) {
|
||||||
(window == NULL || g_strcmp0(cthulhu_accessible_bus_name(window),
|
|
||||||
cthulhu_accessible_bus_name(current)) != 0)) {
|
|
||||||
if (window != NULL) g_object_unref(window);
|
if (window != NULL) g_object_unref(window);
|
||||||
window = g_object_ref(current);
|
window = g_object_ref(current);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ import "strings"
|
|||||||
|
|
||||||
const Name = "Cthulhu"
|
const Name = "Cthulhu"
|
||||||
|
|
||||||
const Version = "2026.11.04"
|
const Version = "2026.11.05"
|
||||||
const Channel = "testing"
|
const Channel = "testing"
|
||||||
|
|
||||||
var Build = Version
|
var Build = Version
|
||||||
|
|||||||
@@ -0,0 +1,8 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<html lang="en"><meta charset="utf-8"><title>Cthulhu Browser Probe</title>
|
||||||
|
<h1 tabindex="0">Find fixture start</h1>
|
||||||
|
<p>First searchable paragraph.</p>
|
||||||
|
<a href="#destination">Searchable link</a>
|
||||||
|
<h2 id="destination">Searchable destination</h2>
|
||||||
|
<label>Answer <input></label>
|
||||||
|
</html>
|
||||||
@@ -12,6 +12,7 @@ from native_test_session import run_private_session
|
|||||||
|
|
||||||
|
|
||||||
BROWSER_PROOFS = {
|
BROWSER_PROOFS = {
|
||||||
|
"find": ("./internal/app", "^TestNativeBrowserFind$", 35, 40, "browser-find.html"),
|
||||||
"actions": ("./internal/app", "^TestNativeBrowserActionOwnership$", 35, 40, "browser-actions.html"),
|
"actions": ("./internal/app", "^TestNativeBrowserActionOwnership$", 35, 40, "browser-actions.html"),
|
||||||
"link-sounds": ("./internal/app", "^TestNativeBrowserCaretLinkSounds$", 40, 45, "browser-link-sounds.html"),
|
"link-sounds": ("./internal/app", "^TestNativeBrowserCaretLinkSounds$", 40, 45, "browser-link-sounds.html"),
|
||||||
"article": ("./internal/atspi", "^TestNativeBrowserArticleContext$", 30, 35, "browser-article.html"),
|
"article": ("./internal/atspi", "^TestNativeBrowserArticleContext$", 30, 35, "browser-article.html"),
|
||||||
|
|||||||
+16
-1
@@ -18,7 +18,10 @@ PAGE = Path(__file__).with_name("firefox-baseline.html")
|
|||||||
|
|
||||||
def run_inside():
|
def run_inside():
|
||||||
linked = os.environ.get("CTHULHU_FIREFOX_LINKED") == "1"
|
linked = os.environ.get("CTHULHU_FIREFOX_LINKED") == "1"
|
||||||
|
find = os.environ.get("CTHULHU_FIREFOX_FIND") == "1"
|
||||||
page = Path(__file__).with_name("browser-linked-headings.html") if linked else PAGE
|
page = Path(__file__).with_name("browser-linked-headings.html") if linked else PAGE
|
||||||
|
if find:
|
||||||
|
page = Path(__file__).with_name("browser-find.html")
|
||||||
browser = os.environ.get("CTHULHU_FIREFOX") or shutil.which("firefox")
|
browser = os.environ.get("CTHULHU_FIREFOX") or shutil.which("firefox")
|
||||||
if not browser:
|
if not browser:
|
||||||
raise RuntimeError("Firefox executable not found")
|
raise RuntimeError("Firefox executable not found")
|
||||||
@@ -40,8 +43,13 @@ def run_inside():
|
|||||||
if process.poll() is not None:
|
if process.poll() is not None:
|
||||||
log.seek(0)
|
log.seek(0)
|
||||||
raise RuntimeError("Firefox exited before the probe: " + log.read().decode(errors="replace"))
|
raise RuntimeError("Firefox exited before the probe: " + log.read().decode(errors="replace"))
|
||||||
|
windowName = "Cthulhu Firefox Baseline"
|
||||||
|
if find:
|
||||||
|
windowName = "Cthulhu Browser Probe"
|
||||||
|
elif linked:
|
||||||
|
windowName = "Cthulhu Linked Headings Probe"
|
||||||
result = subprocess.run(
|
result = subprocess.run(
|
||||||
["xdotool", "search", "--onlyvisible", "--name", "Cthulhu Linked Headings Probe" if linked else "Cthulhu Firefox Baseline"],
|
["xdotool", "search", "--onlyvisible", "--name", windowName],
|
||||||
capture_output=True, text=True,
|
capture_output=True, text=True,
|
||||||
)
|
)
|
||||||
if result.stdout.strip():
|
if result.stdout.strip():
|
||||||
@@ -53,6 +61,13 @@ def run_inside():
|
|||||||
raise RuntimeError("Firefox fixture window missing")
|
raise RuntimeError("Firefox fixture window missing")
|
||||||
environment["CTHULHU_NATIVE_FIREFOX"] = "1"
|
environment["CTHULHU_NATIVE_FIREFOX"] = "1"
|
||||||
environment.setdefault("GOCACHE", "/tmp/cthulhu-cache")
|
environment.setdefault("GOCACHE", "/tmp/cthulhu-cache")
|
||||||
|
if find:
|
||||||
|
subprocess.run(
|
||||||
|
["go", "test", "./internal/app", "-run", "^TestNativeBrowserFind$",
|
||||||
|
"-count=1", "-v", "-timeout=35s"],
|
||||||
|
cwd=ROOT, env=environment, check=True, timeout=40,
|
||||||
|
)
|
||||||
|
return
|
||||||
if linked:
|
if linked:
|
||||||
environment["CTHULHU_NATIVE_LINKED_HEADINGS"] = "1"
|
environment["CTHULHU_NATIVE_LINKED_HEADINGS"] = "1"
|
||||||
subprocess.run(
|
subprocess.run(
|
||||||
|
|||||||
Reference in new issue
Block a user