cthulhu/pyatspi.md

PyAtspi Removal Progress

This file tracks progress on removing the python-atspi (pyatspi) dependency from Cthulhu.

Status: IN PROGRESS - Orca AX modules + focus/input event manager ported (not yet validated)

Problem Discovered

The initial attempt to remove pyatspi broke keyboard navigation:

• Arrow keys in terminals didn't trigger screen reader responses
• Browse mode in web browsers didn't work
• Only Cthulhu modifier keys (like Cthulhu+Q to quit) worked

Root Cause: The simplified removal approach was too aggressive. The legacy keyboard handling code may have been doing something important, or the changes affected how keybindings are registered/matched.

What Was Tried (and reverted)

1. Removed import pyatspi from event_manager.py
2. Removed legacy keystroke listener methods
3. Simplified setKeyHandling() method
4. Changed activate() to call activateNewKeyHandling() directly

What Still Needs Investigation

1. Why did the simplified approach break keybindings?

   • The InputEventManager IS receiving keyboard events (confirmed in logs)
   • Events ARE being processed
   • But handlers aren't being found for navigation keys
   • Cthulhu modifier keys still work

2. What does Orca's migration actually change?

   • Orca took 7 months (July 2023 - Feb 2024) to migrate
   • Need to study their commits more carefully
   • Key commits to review:
     • fc082b1c4 - Use Atspi.event_main/quit instead of pyatspi.Registry.start/stop
     • 58826de39 - Use Atspi.generate_keyboard_event instead of pyatspi's
     • 72492c05a - Use Atspi rather than pyatspi to register for object events

Completed Work (still valid)

1. Test files updated - 20 test files in test/keystrokes/ now use Atspi.Role.* instead of pyatspi.ROLE_*

2. pyproject.toml updated - Removed python-atspi>=2.48 from dependencies (may need to restore)

Next Steps for New Session

1. Restore pyproject.toml if needed (add back python-atspi dependency for now)

2. Study Orca's migration more carefully:

   • Look at the specific changes in each commit
   • Understand what pyatspi APIs were replaced with what Atspi APIs
   • Check if there are subtle differences in how events are registered

3. Take incremental approach:

   • Make one small change at a time
   • Test after each change
   • Don't remove legacy fallback until we understand why it's needed

4. Update call sites to Atspi APIs:

   • Replace queryText/queryComponent/queryTable/etc. with Atspi methods
   • Confirm text attributes and selections behave as expected

5. Validate input events with new InputEventManager:

   • Ensure keybindings are found for navigation keys
   • Verify click-count logic (keyboard + mouse) with new manager
   • Confirm no remaining callers rely on input_event_manager.getManager()

6. Key files to compare between Orca and Cthulhu:

   • event_manager.py - Event registration and handling
   • input_event_manager.py - Keyboard event processing
   • input_event.py - KeyboardEvent class
   • keybindings.py - Keybinding registration and matching

Current State

• event_manager.py: Atspi-only event handling (no pyatspi/legacy path); now uses input_event_manager.get_manager() (unvalidated)
• test files: Updated to use Atspi.Role (this is fine)
• pyproject.toml: python-atspi removed (may need to restore)
• compat layer: Removed; need direct Atspi usage
• AX modules: Replaced/added Orca AX modules (ax_component/ax_document/ax_event_synthesizer/ax_hypertext/ax_object/ax_selection/ax_table/ax_text/ax_utilities/ax_utilities_* etc.) (imports only; unvalidated)
• focus_manager.py: Ported from Orca and synced to cthulhu_state (unvalidated)
• script_manager.py / input_event_manager.py: Added get_manager + snake_case wrappers for Orca-style callers (unvalidated)
• input_event_manager.py: Replaced with Orca version and adapted to Cthulhu input_event API (camelCase methods) + Cthulhu device name (unvalidated)
• input_event.py: InputEvent/MouseButtonEvent setClickCount now accepts optional count (used by input_event_manager) (unvalidated)
• dbus_service.py: Remote controller events now routed via input_event_manager.get_manager() (unvalidated)
• meson.build: Added missing ax_* modules and focus_manager.py to install sources (unvalidated)
• script_manager.py: Lazy import for AXUtilities to break circular import chain (unvalidated)
• focus_manager.py: Lazy import for AXUtilities to break circular import chain (unvalidated)
• script.py: Switched to ax_event_synthesizer.get_synthesizer() (unvalidated)
• script.py: Added locus_of_focus_changed() wrapper calling locusOfFocusChanged() (unvalidated)
• ax_object.py: Added AXObject.get_application() wrapper for Atspi.Accessible.get_application (unvalidated)
• script_manager.py / script.py / input_event.py: Began full snake_case migration (removed script_manager getManager/getActiveScript wrappers, renamed locus_of_focus_changed + get_event_synthesizer, converted InputEvent getters/setters to snake_case, updated call sites) (unvalidated)
• event_manager.py: Replaced AXObject.get_application_toolkit_name() calls with AXUtilities.get_application_toolkit_name() (unvalidated)
• script_manager.py: Replaced AXObject.get_application_toolkit_name() with AXUtilities.get_application_toolkit_name() (unvalidated)
• ax_utilities_relation.py / relation call sites: Migrated relation usage to AXUtilitiesRelation helpers (get_is_* / get_flows_* etc.) and removed AXObject relation calls (unvalidated)
• braille.py: Added BRLAPI priority constants + setBrlapiPriority used by focus_manager (unvalidated)
• debug.py: Added snake_case print_message/print_tokens aliases and lazy AXObject import to avoid cycles (unvalidated)
• debug.py: getAccessibleDetails now uses AXUtilitiesDebugging.object_details_as_string to avoid missing AXObject *_as_string methods (unvalidated)
• speech_generator.py: Replaced queryText usage in text content/selection paths with AXText (unvalidated)
• script_utilities.py: selectedText/substring/getCaretContext/getCharacterAtOffset now use AXText (unvalidated)
• scripts/default.py: getTextLineAtCaret now uses AXText (unvalidated)
• script_utilities.py / WebKitGtk script_utilities.py: Replaced AXObject.get_previous_object/get_next_object with AXUtilities equivalents (unvalidated)
• scripts/default.py: onCaretMoved now uses AXText (unvalidated)
• scripts/terminal/script_utilities.py: Replaced queryText usage with AXText for insertions and caret checks (unvalidated)
• scripts/terminal/script.py: Replaced queryText usage with AXText for caret/echo logic (unvalidated)
• script_utilities.py: updateCachedTextSelection now uses AXText (unvalidated)
• scripts/default.py: sayCharacter now uses AXText (unvalidated)
• scripts/toolkits/Chromium/script.py: Replaced AXObject.get_parent predicate usage with AXObject.find_ancestor_inclusive (unvalidated)
• scripts/web/script_utilities.py: Replaced queryDocument/queryText selection logic with AXDocument/AXText (unvalidated)
• input_event.py / input_event_manager.py: Defer shouldConsume initialization until script/object/window set (unvalidated)
• script_utilities.py: queryNonEmptyText now uses AXText (unvalidated)
• flat_review.py: Replaced queryText/queryComponent usage with AXText/Atspi.Component (unvalidated)
• queryText removal sweep: Replaced remaining queryText usage across core, web, toolkit, app, and plugin codepaths with AXText/Atspi.Text equivalents (unvalidated)
  • core: script_utilities.py (word nav + selection + password/lastContext fixes), default.py (SayAll progress + misspelling checks), speech_generator.py, caret_navigation.py, label_inference.py, structural_navigation.py, spellcheck.py, mouse_review.py, liveregions.py
  • web: scripts/web/script_utilities.py (text-at-offset pipeline + caret context + element line heuristics), scripts/web/script.py, scripts/web/speech_generator.py
  • toolkits: scripts/toolkits/WebKitGtk/script_utilities.py, scripts/toolkits/WebKitGtk/script.py, scripts/toolkits/gtk/script_utilities.py
  • apps: scripts/apps/gnome-shell/script.py, scripts/apps/gnome-shell/script_utilities.py, scripts/apps/gnome-documents/script.py, scripts/apps/Eclipse/script.py, scripts/apps/Thunderbird/script.py, scripts/apps/soffice/script.py, scripts/apps/soffice/script_utilities.py, scripts/apps/soffice/spellcheck.py, scripts/apps/soffice/speech_generator.py
  • plugins: plugins/AIAssistant/plugin.py, plugins/IndentationAudio/plugin.py
• relation helpers: Replaced remaining AXObject.has_relation() uses with AXUtilitiesRelation helpers in web + gtk toolkits (unvalidated)
• web flat review crash: Fixed AXObject.has_relation call in web isErrorMessage() path (unvalidated)

Reference: Orca's Key Commits

fc082b1c4 - 2023-07-27 - Use Atspi.event_main/quit instead of pyatspi.Registry.start/stop
58826de39 - 2023-07-27 - Use Atspi.generate_keyboard_event instead of pyatspi's
6b4a9a23c - 2023-07-27 - Use Atspi's set_cache_mask instead of pyatspi's
72492c05a - 2023-07-27 - Use Atspi rather than pyatspi to register for object events
837c31e9d - 2024-01-19 - Remove obsolete pyatspi-based functions for hypertext/hyperlink
1c496c9ad - 2024-02-19 - CI: Remove pyatspi installation (final removal)