Added some benchmarks.

2026-10-17 01:22:52 -04:00
1 parent 37fdd4e114
commit a763714fb4
@@ -38,6 +38,8 @@ Expose GUI controls through accessibility APIs. Keep outer preferences combo wid

 Use Go's `testing` package, `*_test.go`, and `TestBehavior` functions. Start with the affected package. Use deterministic synchronization and generic fixtures. Native checks need their relevant harness, such as `python3 tests/native_atspi.py`. No numeric coverage target is configured. Report automated, installed-binary, physical-input, audible-speech, and Braille-display checks separately.

+For performance work, run the relevant `BenchmarkBehavior` functions before and after changes using repeated runs and allocation reporting. Add or expand representative benchmarks when investigating a new bottleneck; keep fixture construction outside timed loops and check that the intended work actually happens. Preserve announcement content, ordering, interruption, focus, and Braille behavior. Benchmark timing alone does not establish a bug or live responsiveness, and noisy timing changes must not become automatic pass/fail thresholds. See [benchmark procedures](docs/testing/benchmarks.md) for commands and measurement limits. Plain `go test` does not run benchmarks.
+
 ## Commit & Pull Request Guidelines

 Use short descriptive subjects consistent with history. Confirm the intended target branch; `legacy` preserves Python. Describe behavior, issues, regressions checked, skipped checks, and remaining live acceptance. Human diff review and testing precede submission under the policy in `CONTRIBUTING.md`.
@@ -0,0 +1,89 @@
+# Performance benchmarks
+
+Benchmarks measure time and Go allocations for repeatable operations. Run the
+relevant cases before and after performance changes, alongside correctness tests.
+Normal `go test` runs do not execute benchmarks; add `-bench` explicitly.
+
+## Browser document benchmarks
+
+Run from the repository root:
+
+```sh
+go test ./internal/review -run '^$' -bench '^BenchmarkDocument(Snapshot|HeadingNavigation|LineAssembly)$' -benchmem -count=10 > /tmp/cthulhu-bench-before.txt
+```
+
+Repeat after the change with the same command and
+`/tmp/cthulhu-bench-after.txt` as the output file. Choose distinct filenames when
+retaining earlier measurements. To check that the benchmarks execute quickly,
+use `-benchtime=1x -count=1`; that is a smoke check, not a useful timing baseline.
+
+The initial cases use 100, 1,000 and 10,000 content objects, plus one section per
+twenty objects and a document root. Fixtures contain headings, links and Unicode
+text, with two objects on each visual row. Each benchmark iteration performs:
+
+- `BenchmarkDocumentSnapshot`: one document snapshot from an existing tree,
+  including traversal and derived metadata.
+- `BenchmarkDocumentHeadingNavigation`: a forward/backward heading round trip
+  on an existing tree. The `adjacent` case visits two headings near the middle;
+  `wrap` moves between the last and first headings. Each iteration returns to
+  the same position, keeping the workload independent of the loop count.
+  Time and allocations are per round trip (two moves), including snapshot
+  construction and navigation bookkeeping.
+- `BenchmarkDocumentLineAssembly`: one line lookup in the middle of a document,
+  combining the two text objects on that row. This includes snapshot construction.
+
+Fixture construction and initial focus synchronization are outside the timed
+loops. The benchmarks fail on errors, missing work or invalid result shapes.
+They use owned tree data and do not start a browser, access the desktop reader,
+query native AT-SPI objects, or produce speech or Braille output. They do not
+measure browser capture, native caret placement, input queues or audible latency.
+Expand the suite when a reproduced bottleneck needs a different workload, such
+as nested documents, tables, long text or event bursts.
+
+Existing benchmarks also cover speech text preparation and Tentacle commands:
+
+```sh
+go test ./internal/speech -run '^$' -bench '^BenchmarkNormalizeSpeakText$' -benchmem -count=10
+go test ./internal/tentacle -run '^$' -bench '^BenchmarkCommandRoundTrip$' -benchmem -count=10
+```
+
+The Tentacle benchmark starts disposable test peers and measures one completed
+command at a time across 1, 8 or 16 processes; it does not measure concurrent
+command saturation.
+
+## Comparing changes
+
+Record the revision, uncommitted changes, Go version, machine and benchmark
+command with results. Compare repeated measurements on the same machine and
+toolchain, under similar load and power settings. Do not run performance
+comparisons with the race detector or coverage instrumentation enabled. Use
+`benchstat` from `golang.org/x/perf/cmd/benchstat`, if installed, to compare files:
+
+```sh
+benchstat /tmp/cthulhu-bench-before.txt /tmp/cthulhu-bench-after.txt
+```
+
+`ns/op` is elapsed time per operation, `B/op` is Go bytes allocated per operation,
+and `allocs/op` counts Go allocations. These are not total process memory or
+native C allocation measurements. A benchmark's `PASS` means it executed without
+failure; it does not mean its performance is acceptable or improved.
+
+Investigate substantial, repeatable regressions before changing code. Profile
+the affected operation when the cause is unclear. Preserve the full workload
+and result checks when optimizing; do not improve scores by dropping required
+content or bypassing validation. Initially keep timings informational rather
+than adding timing gates to tests or push hooks. These commands can be run by
+automation later; no scheduled job or CI timing gate is configured here.
+
+For this benchmark-only change, run the existing affected correctness tests:
+
+```sh
+go test ./internal/review ./internal/atspi -count=1
+```
+
+Future browser performance fixes also need the relevant native browser harness
+and human checks: navigate headings forward and backward through wrapping, read
+lines with arrows, move through links and editable fields, return from browser
+chrome, and verify fresh input interrupts old output. Check speech and physical
+Braille where available. Automated timing does not replace those checks or the
+human review required by `CONTRIBUTING.md`.
@@ -0,0 +1,148 @@
+// Copyright (c) 2026 Storm Dragon
+// Authors: Storm Dragon, Angus Kola
+// SPDX-License-Identifier: GPL-3.0-or-later
+
+package review
+
+import (
+	"context"
+	"fmt"
+	"testing"
+	"unicode/utf8"
+
+	"git.stormux.org/storm/cthulhu/internal/atspi"
+)
+
+// benchmark_document_tree models an already captured page, not native AT-SPI
+// capture. Each section has twenty content objects: a heading, a link, and
+// ordinary text. Pairs of objects share a visual row, including Unicode text.
+func benchmark_document_tree(b *testing.B, contentNodes int) (*atspi.AccessibleTree, []atspi.ObjectID) {
+	b.Helper()
+	rootID := webID("/benchmark/document")
+	root := atspi.NewSnapshot(rootID.Path, "Benchmark page", "document web", "", "")
+	root.BusName = rootID.BusName
+	nodes := []atspi.TreeNode{{Snapshot: root}}
+	contentIDs := make([]atspi.ObjectID, contentNodes)
+	sectionIndex := 0
+	for index := range contentNodes {
+		if index%20 == 0 {
+			sectionID := webID(fmt.Sprintf("/benchmark/section_%d", index/20))
+			section := atspi.NewSnapshot(sectionID.Path, "", "section", "", "")
+			section.BusName = sectionID.BusName
+			nodes[0].Children = append(nodes[0].Children, sectionID)
+			sectionIndex = len(nodes)
+			nodes = append(nodes, atspi.TreeNode{Snapshot: section})
+		}
+		id := webID(fmt.Sprintf("/benchmark/content_%d", index))
+		contentIDs[index] = id
+		role, text := "static", fmt.Sprintf("Café sample %d", index)
+		switch index % 20 {
+		case 0:
+			role, text = "heading", fmt.Sprintf("Heading %d", index/20)
+		case 1:
+			role, text = "link", fmt.Sprintf("Link %d", index/20)
+		}
+		snapshot := atspi.NewSnapshot(id.Path, text, role, "", text)
+		snapshot.BusName = id.BusName
+		snapshot.CharacterCount = utf8.RuneCountInString(text)
+		if role == "heading" {
+			snapshot.HeadingLevel = 2
+		}
+		bounds := atspi.Rectangle{X: int32(index%2) * 200, Y: int32(index/2) * 24, Width: 180, Height: 20}
+		nodes[sectionIndex].Children = append(nodes[sectionIndex].Children, id)
+		nodes = append(nodes, atspi.TreeNode{
+			Snapshot: snapshot,
+			Geometry: atspi.Geometry{Bounds: bounds, BoundsKnown: true, VisibilityKnown: true, Visible: true, Showing: true},
+			Text: atspi.VisibleText{Known: true, Ranges: []atspi.TextRange{{
+				Start: 0, End: snapshot.CharacterCount, Text: text, Bounds: bounds,
+			}}},
+		})
+	}
+	tree, err := atspi.NewAccessibleTree(rootID, nodes)
+	if err != nil {
+		b.Fatal(err)
+	}
+	return tree, contentIDs
+}
+
+func BenchmarkDocumentSnapshot(b *testing.B) {
+	for _, contentNodes := range []int{100, 1000, 10000} {
+		b.Run(fmt.Sprintf("nodes-%d", contentNodes), func(b *testing.B) {
+			tree, _ := benchmark_document_tree(b, contentNodes)
+			root, err := tree.Root()
+			if err != nil {
+				b.Fatal(err)
+			}
+			ctx := context.Background()
+			wantNodes := contentNodes + contentNodes/20 + 1
+			b.ReportAllocs()
+			for b.Loop() {
+				document, err := tree.Document(ctx, root)
+				if err != nil || len(document.Nodes) != wantNodes {
+					b.Fatalf("document snapshot: nodes=%d, error=%v", len(document.Nodes), err)
+				}
+			}
+		})
+	}
+}
+
+func BenchmarkDocumentHeadingNavigation(b *testing.B) {
+	for _, contentNodes := range []int{100, 1000, 10000} {
+		for _, scenario := range []string{"adjacent", "wrap"} {
+			b.Run(fmt.Sprintf("nodes-%d/%s", contentNodes, scenario), func(b *testing.B) {
+				tree, contentIDs := benchmark_document_tree(b, contentNodes)
+				cursor, err := NewDocumentCursor(tree)
+				if err != nil {
+					b.Fatal(err)
+				}
+				ctx := context.Background()
+				startIndex := contentNodes / 40 * 20
+				wantWrapped := scenario == "wrap"
+				if wantWrapped {
+					startIndex = contentNodes - 20
+				}
+				nextIndex := (startIndex + 20) % contentNodes
+				if _, err := cursor.SyncFocus(ctx, contentIDs[startIndex]); err != nil {
+					b.Fatal(err)
+				}
+				b.ReportAllocs()
+				for b.Loop() {
+					// One operation is a round trip. Returning to the same heading
+					// keeps the workload independent of the calibrated loop count.
+					move, err := cursor.Move(ctx, DocumentHeading, 2, false)
+					if err != nil || !move.Moved || move.Position.Object.ID() != contentIDs[nextIndex] || move.Wrapped != wantWrapped {
+						b.Fatalf("forward heading navigation: moved=%v, error=%v", move.Moved, err)
+					}
+					move, err = cursor.Move(ctx, DocumentHeading, 2, true)
+					if err != nil || !move.Moved || move.Position.Object.ID() != contentIDs[startIndex] || move.Wrapped != wantWrapped {
+						b.Fatalf("backward heading navigation: moved=%v, error=%v", move.Moved, err)
+					}
+				}
+			})
+		}
+	}
+}
+
+func BenchmarkDocumentLineAssembly(b *testing.B) {
+	for _, contentNodes := range []int{100, 1000, 10000} {
+		b.Run(fmt.Sprintf("nodes-%d", contentNodes), func(b *testing.B) {
+			tree, contentIDs := benchmark_document_tree(b, contentNodes)
+			middle := contentNodes / 2
+			reference, err := tree.Reference(contentIDs[middle])
+			if err != nil {
+				b.Fatal(err)
+			}
+			ctx := context.Background()
+			b.ReportAllocs()
+			for b.Loop() {
+				line, err := tree.DocumentLineAtOffset(ctx, reference, 1)
+				if err != nil || len(line.Contents) != 2 {
+					b.Fatalf("line assembly: contents=%d, error=%v", len(line.Contents), err)
+				}
+				if line.Contents[0].Reference.ID() != contentIDs[middle] || line.Contents[1].Reference.ID() != contentIDs[middle+1] {
+					b.Fatal("line assembly returned the wrong visual row")
+				}
+			}
+		})
+	}
+}