Compare commits
5
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2d9790de88 | ||
|
|
633ff214a4 | ||
|
|
ceb03668b9 | ||
|
|
d7d26c57f4 | ||
|
|
8e792dd4e2 |
@@ -0,0 +1,117 @@
|
|||||||
|
# Guarded Mouse Review Wayland Backport
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Backport Orca's modern mouse review split into Cthulhu in a conservative way. Keep the current X11 `Wnck` plus `mouse:abs` implementation as the existing path. Add a second path that uses `Atspi.Device` pointer monitoring only when the local AT-SPI version is new enough and the compositor/device actually grants `POINTER_MONITOR`. If either condition fails, Cthulhu stays on the current behavior.
|
||||||
|
|
||||||
|
This is explicitly an X-safe design. The X11 path remains first-class and unchanged in behavior. Wayland support is opportunistic and must fail closed.
|
||||||
|
|
||||||
|
## Current State
|
||||||
|
|
||||||
|
- Cthulhu mouse review currently depends on `Wnck` window lookup and `mouse:abs` events in [mouse_review.py](/home/storm/devel/cthulhu/src/cthulhu/mouse_review.py).
|
||||||
|
- Recent local changes already avoid importing `Wnck` on Wayland to suppress the terminal warning, but they do not add functional Wayland mouse review.
|
||||||
|
- Cthulhu already has an `Atspi.Device` for keyboard handling in [input_event_manager.py](/home/storm/devel/cthulhu/src/cthulhu/input_event_manager.py), which is the required foundation for an Orca-style pointer-monitor path.
|
||||||
|
|
||||||
|
## Goals
|
||||||
|
|
||||||
|
- Preserve existing X11 mouse review behavior.
|
||||||
|
- Add a guarded Wayland-capable path modeled on Orca's `Atspi.Device` pointer monitoring.
|
||||||
|
- Avoid startup warnings from `Wnck` on Wayland.
|
||||||
|
- Avoid enabling any new path unless support is positively confirmed at runtime.
|
||||||
|
|
||||||
|
## Non-Goals
|
||||||
|
|
||||||
|
- No full Orca mouse review rewrite beyond the pointer-monitor split.
|
||||||
|
- No removal or semantic change of the current X11 `Wnck` path.
|
||||||
|
- No OCR refactor beyond existing `Wnck` gating already added.
|
||||||
|
- No assumption that Wayland mouse review will work everywhere; unsupported compositors must remain a no-op.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
### Capability Detection
|
||||||
|
|
||||||
|
Mouse review will select its backend at runtime:
|
||||||
|
|
||||||
|
1. If AT-SPI version is at least `2.60`, or the pre-release threshold Orca used (`2.59.90`), Cthulhu may try the new backend.
|
||||||
|
2. The new backend must call `set_capabilities(... | POINTER_MONITOR)` on the existing `Atspi.Device`.
|
||||||
|
3. The new backend is considered available only if `POINTER_MONITOR` is present in the returned capability set.
|
||||||
|
4. If any of those checks fail, Cthulhu uses the existing X11 path.
|
||||||
|
|
||||||
|
This means X11 systems on older stacks continue to use the current implementation. Wayland systems with insufficient AT-SPI support do not partially enable mouse review.
|
||||||
|
|
||||||
|
### InputEventManager Extension
|
||||||
|
|
||||||
|
Extend [input_event_manager.py](/home/storm/devel/cthulhu/src/cthulhu/input_event_manager.py) with narrow pointer-monitor wrappers:
|
||||||
|
|
||||||
|
- `enable_pointer_monitoring() -> bool`
|
||||||
|
- `start_pointer_watcher(callback) -> None`
|
||||||
|
- `stop_pointer_watcher() -> None`
|
||||||
|
|
||||||
|
These wrappers should mirror Orca's error handling:
|
||||||
|
|
||||||
|
- return `False` on missing device or `GLib.GError`
|
||||||
|
- never raise into callers
|
||||||
|
- leave existing keyboard handling unchanged
|
||||||
|
|
||||||
|
No unrelated device-manager refactor is included in this pass.
|
||||||
|
|
||||||
|
### Mouse Review Backend Split
|
||||||
|
|
||||||
|
Update [mouse_review.py](/home/storm/devel/cthulhu/src/cthulhu/mouse_review.py) to maintain two internal paths:
|
||||||
|
|
||||||
|
- Legacy path:
|
||||||
|
- existing `Wnck` window tracking
|
||||||
|
- existing `mouse:abs` listener
|
||||||
|
- existing absolute-coordinate flow
|
||||||
|
- New path:
|
||||||
|
- register a `pointer-moved` watcher on the `Atspi.Device`
|
||||||
|
- use the accessible object and local coordinates delivered by AT-SPI
|
||||||
|
- if the event source is an application, resolve the containing window from the app's accessible children
|
||||||
|
- otherwise use the event object directly
|
||||||
|
|
||||||
|
The common object-presentation logic should stay shared as much as possible. Only the event source and coordinate acquisition differ.
|
||||||
|
|
||||||
|
### Safety Rules
|
||||||
|
|
||||||
|
- Do not change the old X11 path except for harmless factoring needed to share code.
|
||||||
|
- Do not force the new path on X11 systems that are currently using the legacy path successfully.
|
||||||
|
- If pointer monitoring disconnects, errors, or cannot be enabled, mouse review must behave as unavailable rather than falling into a broken mixed state.
|
||||||
|
- Existing `Wnck` suppression on Wayland remains in place.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
### Automated
|
||||||
|
|
||||||
|
Add focused regressions for:
|
||||||
|
|
||||||
|
- version/capability gating chooses the new backend only when allowed
|
||||||
|
- `enable_pointer_monitoring()` returns `False` on capability failure
|
||||||
|
- pointer watcher start/stop are no-ops when there is no device
|
||||||
|
- mouse review activation chooses the legacy backend when AT-SPI support is not available
|
||||||
|
- mouse review activation chooses the pointer-monitor backend when support is available
|
||||||
|
|
||||||
|
Tests should mock AT-SPI rather than requiring a compositor.
|
||||||
|
|
||||||
|
### Manual
|
||||||
|
|
||||||
|
X11:
|
||||||
|
|
||||||
|
- mouse review still enables and behaves as before
|
||||||
|
- no regressions in pointer routing or reviewed object presentation
|
||||||
|
|
||||||
|
Wayland:
|
||||||
|
|
||||||
|
- no `Wnck` warning on startup
|
||||||
|
- mouse review only enables when AT-SPI pointer monitoring is actually available
|
||||||
|
- if supported, pointer hover review works without breaking keyboard handling
|
||||||
|
- if unsupported, failure is clean and explicit rather than noisy or partially broken
|
||||||
|
|
||||||
|
## Risks
|
||||||
|
|
||||||
|
- The local environment here is still on AT-SPI `2.58.4`, so the new backend cannot be exercised live in this workspace.
|
||||||
|
- The guarded design limits that risk by preserving the existing path and enabling the new path only under the same runtime conditions Orca used.
|
||||||
|
- The main regression risk is accidental interaction with Cthulhu's existing keyboard `Atspi.Device`; that is why the change is limited to small wrappers plus mouse-review-specific watcher usage.
|
||||||
|
|
||||||
|
## Recommendation
|
||||||
|
|
||||||
|
Implement the guarded dual-path backport and stop there. Do not port broader Orca mouse review refactors in the same pass.
|
||||||
@@ -44,6 +44,7 @@ from typing import TYPE_CHECKING, Optional, Union, Tuple, List, Dict
|
|||||||
import gi
|
import gi
|
||||||
gi.require_version("Atspi", "2.0")
|
gi.require_version("Atspi", "2.0")
|
||||||
from gi.repository import Atspi
|
from gi.repository import Atspi
|
||||||
|
from gi.repository import GLib
|
||||||
|
|
||||||
from . import debug
|
from . import debug
|
||||||
from . import focus_manager
|
from . import focus_manager
|
||||||
@@ -64,21 +65,36 @@ class InputEventManager:
|
|||||||
self._last_input_event: Optional[input_event.InputEvent] = None
|
self._last_input_event: Optional[input_event.InputEvent] = None
|
||||||
self._last_non_modifier_key_event: Optional[input_event.KeyboardEvent] = None
|
self._last_non_modifier_key_event: Optional[input_event.KeyboardEvent] = None
|
||||||
self._device: Optional[Atspi.Device] = None
|
self._device: Optional[Atspi.Device] = None
|
||||||
|
self._pointer_moved_id: int = 0
|
||||||
self._mapped_keycodes: List[int] = []
|
self._mapped_keycodes: List[int] = []
|
||||||
self._mapped_keysyms: List[int] = []
|
self._mapped_keysyms: List[int] = []
|
||||||
self._grabbed_bindings: Dict[int, keybindings.KeyBinding] = {}
|
self._grabbed_bindings: Dict[int, keybindings.KeyBinding] = {}
|
||||||
self._paused: bool = False
|
self._paused: bool = False
|
||||||
|
|
||||||
|
def activate_device(self) -> Atspi.Device:
|
||||||
|
"""Creates and returns the AT-SPI device used by this manager."""
|
||||||
|
|
||||||
|
if self._device is not None:
|
||||||
|
return self._device
|
||||||
|
|
||||||
|
if Atspi.get_version() >= (2, 55, 90):
|
||||||
|
self._device = Atspi.Device.new_full("org.stormux.Cthulhu")
|
||||||
|
else:
|
||||||
|
self._device = Atspi.Device.new()
|
||||||
|
|
||||||
|
return self._device
|
||||||
|
|
||||||
|
def get_device(self) -> Optional[Atspi.Device]:
|
||||||
|
"""Returns the active AT-SPI device, if any."""
|
||||||
|
|
||||||
|
return self._device
|
||||||
|
|
||||||
def start_key_watcher(self) -> None:
|
def start_key_watcher(self) -> None:
|
||||||
"""Starts the watcher for keyboard input events."""
|
"""Starts the watcher for keyboard input events."""
|
||||||
|
|
||||||
msg = "INPUT EVENT MANAGER: Starting key watcher."
|
msg = "INPUT EVENT MANAGER: Starting key watcher."
|
||||||
debug.print_message(debug.LEVEL_INFO, msg, True)
|
debug.print_message(debug.LEVEL_INFO, msg, True)
|
||||||
if Atspi.get_version() >= (2, 55, 90):
|
self.activate_device().add_key_watcher(self.process_keyboard_event)
|
||||||
self._device = Atspi.Device.new_full("org.stormux.Cthulhu")
|
|
||||||
else:
|
|
||||||
self._device = Atspi.Device.new()
|
|
||||||
self._device.add_key_watcher(self.process_keyboard_event)
|
|
||||||
|
|
||||||
def stop_key_watcher(self) -> None:
|
def stop_key_watcher(self) -> None:
|
||||||
"""Starts the watcher for keyboard input events."""
|
"""Starts the watcher for keyboard input events."""
|
||||||
@@ -87,6 +103,63 @@ class InputEventManager:
|
|||||||
debug.print_message(debug.LEVEL_INFO, msg, True)
|
debug.print_message(debug.LEVEL_INFO, msg, True)
|
||||||
self._device = None
|
self._device = None
|
||||||
|
|
||||||
|
def enable_pointer_monitoring(self) -> bool:
|
||||||
|
"""Enables pointer monitoring on the current device, if possible."""
|
||||||
|
|
||||||
|
device = self.get_device()
|
||||||
|
if device is None:
|
||||||
|
return False
|
||||||
|
|
||||||
|
deviceCapability = getattr(Atspi, "DeviceCapability", None)
|
||||||
|
if deviceCapability is None:
|
||||||
|
return False
|
||||||
|
|
||||||
|
pointerMonitor = getattr(deviceCapability, "POINTER_MONITOR", None)
|
||||||
|
if pointerMonitor is None:
|
||||||
|
return False
|
||||||
|
|
||||||
|
setCapabilities = getattr(device, "set_capabilities", None)
|
||||||
|
if not callable(setCapabilities):
|
||||||
|
return False
|
||||||
|
|
||||||
|
currentCapabilities = 0
|
||||||
|
getCapabilities = getattr(device, "get_capabilities", None)
|
||||||
|
if callable(getCapabilities):
|
||||||
|
currentCapabilities = getCapabilities()
|
||||||
|
|
||||||
|
try:
|
||||||
|
grantedCapabilities = setCapabilities(currentCapabilities | pointerMonitor)
|
||||||
|
except GLib.GError:
|
||||||
|
return False
|
||||||
|
|
||||||
|
if isinstance(grantedCapabilities, bool):
|
||||||
|
return grantedCapabilities
|
||||||
|
|
||||||
|
try:
|
||||||
|
return bool(int(grantedCapabilities) & int(pointerMonitor))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return False
|
||||||
|
|
||||||
|
def start_pointer_watcher(self, callback) -> None:
|
||||||
|
"""Starts the watcher for pointer movement events."""
|
||||||
|
|
||||||
|
device = self.get_device()
|
||||||
|
if device is None:
|
||||||
|
return
|
||||||
|
|
||||||
|
self._pointer_moved_id = device.connect("pointer-moved", callback)
|
||||||
|
|
||||||
|
def stop_pointer_watcher(self) -> None:
|
||||||
|
"""Stops the watcher for pointer movement events."""
|
||||||
|
|
||||||
|
device = self.get_device()
|
||||||
|
if device is None or not self._pointer_moved_id:
|
||||||
|
self._pointer_moved_id = 0
|
||||||
|
return
|
||||||
|
|
||||||
|
device.disconnect(self._pointer_moved_id)
|
||||||
|
self._pointer_moved_id = 0
|
||||||
|
|
||||||
def pause_key_watcher(self, pause: bool = True, reason: str = "") -> None:
|
def pause_key_watcher(self, pause: bool = True, reason: str = "") -> None:
|
||||||
"""Pauses processing of keyboard input events."""
|
"""Pauses processing of keyboard input events."""
|
||||||
|
|
||||||
|
|||||||
@@ -108,6 +108,7 @@ cthulhu_python_sources = files([
|
|||||||
'translation_manager.py',
|
'translation_manager.py',
|
||||||
'tutorialgenerator.py',
|
'tutorialgenerator.py',
|
||||||
'typing_echo_presenter.py',
|
'typing_echo_presenter.py',
|
||||||
|
'wnck_support.py',
|
||||||
'where_am_i_presenter.py',
|
'where_am_i_presenter.py',
|
||||||
])
|
])
|
||||||
|
|
||||||
|
|||||||
+180
-55
@@ -32,30 +32,31 @@ __copyright__ = "Copyright (c) 2008 Eitan Isaacson" \
|
|||||||
"Copyright (c) 2016 Igalia, S.L."
|
"Copyright (c) 2016 Igalia, S.L."
|
||||||
__license__ = "LGPL"
|
__license__ = "LGPL"
|
||||||
|
|
||||||
|
from collections import deque
|
||||||
|
|
||||||
import gi
|
import gi
|
||||||
gi.require_version("Atspi", "2.0")
|
gi.require_version("Atspi", "2.0")
|
||||||
from gi.repository import Atspi
|
from gi.repository import Atspi, GLib
|
||||||
|
|
||||||
import math
|
import math
|
||||||
import time
|
import time
|
||||||
|
|
||||||
from gi.repository import Gdk
|
from gi.repository import Gdk
|
||||||
try:
|
from .wnck_support import load_wnck
|
||||||
gi.require_version("Wnck", "3.0")
|
|
||||||
from gi.repository import Wnck
|
Wnck = load_wnck()
|
||||||
_mouseReviewCapable = True
|
|
||||||
except Exception:
|
|
||||||
_mouseReviewCapable = False
|
|
||||||
|
|
||||||
from . import cmdnames
|
from . import cmdnames
|
||||||
from . import debug
|
from . import debug
|
||||||
from . import keybindings
|
from . import keybindings
|
||||||
from . import input_event
|
from . import input_event
|
||||||
|
from . import input_event_manager
|
||||||
from . import messages
|
from . import messages
|
||||||
from . import cthulhu
|
from . import cthulhu
|
||||||
from . import cthulhu_state
|
from . import cthulhu_state
|
||||||
from . import script_manager
|
from . import script_manager
|
||||||
from . import settings_manager
|
from . import settings_manager
|
||||||
|
from .ax_component import AXComponent
|
||||||
from .ax_object import AXObject
|
from .ax_object import AXObject
|
||||||
from .ax_text import AXText
|
from .ax_text import AXText
|
||||||
from .ax_utilities import AXUtilities
|
from .ax_utilities import AXUtilities
|
||||||
@@ -82,8 +83,9 @@ class _StringContext:
|
|||||||
self._start = start
|
self._start = start
|
||||||
self._end = end
|
self._end = end
|
||||||
self._boundingBox = 0, 0, 0, 0
|
self._boundingBox = 0, 0, 0, 0
|
||||||
if script:
|
if AXObject.supports_text(obj):
|
||||||
self._boundingBox = script.utilities.getTextBoundingBox(obj, start, end)
|
rect = AXText.get_range_rect(obj, start, end)
|
||||||
|
self._boundingBox = rect.x, rect.y, rect.width, rect.height
|
||||||
|
|
||||||
def __eq__(self, other):
|
def __eq__(self, other):
|
||||||
return other is not None \
|
return other is not None \
|
||||||
@@ -183,8 +185,9 @@ class _ItemContext:
|
|||||||
self._string = self._getStringContext()
|
self._string = self._getStringContext()
|
||||||
self._time = time.time()
|
self._time = time.time()
|
||||||
self._boundingBox = 0, 0, 0, 0
|
self._boundingBox = 0, 0, 0, 0
|
||||||
if script:
|
if AXObject.supports_component(obj):
|
||||||
self._boundingBox = script.utilities.getBoundingBox(obj)
|
rect = AXComponent.get_rect(obj)
|
||||||
|
self._boundingBox = rect.x, rect.y, rect.width, rect.height
|
||||||
|
|
||||||
def __eq__(self, other):
|
def __eq__(self, other):
|
||||||
return other is not None \
|
return other is not None \
|
||||||
@@ -237,7 +240,12 @@ class _ItemContext:
|
|||||||
return _StringContext(self._obj, self._script)
|
return _StringContext(self._obj, self._script)
|
||||||
|
|
||||||
string, start, end = self._script.utilities.textAtPoint(
|
string, start, end = self._script.utilities.textAtPoint(
|
||||||
self._obj, self._x, self._y, boundary=self._boundary)
|
self._obj,
|
||||||
|
self._x,
|
||||||
|
self._y,
|
||||||
|
coordType=Atspi.CoordType.WINDOW,
|
||||||
|
boundary=self._boundary,
|
||||||
|
)
|
||||||
if string:
|
if string:
|
||||||
string = self._script.utilities.expandEOCs(self._obj, start, end)
|
string = self._script.utilities.expandEOCs(self._obj, start, end)
|
||||||
|
|
||||||
@@ -351,14 +359,45 @@ class MouseReviewer:
|
|||||||
self._handlerIds = {}
|
self._handlerIds = {}
|
||||||
self._eventListener = Atspi.EventListener.new(self._listener)
|
self._eventListener = Atspi.EventListener.new(self._listener)
|
||||||
self.inMouseEvent = False
|
self.inMouseEvent = False
|
||||||
|
self._eventQueue = deque()
|
||||||
|
self._mouseReviewCapable = False
|
||||||
|
self._useAtspi = False
|
||||||
self._handlers = self._setup_handlers()
|
self._handlers = self._setup_handlers()
|
||||||
self._bindings = self._setup_bindings()
|
self._bindings = self._setup_bindings()
|
||||||
|
|
||||||
if not _mouseReviewCapable:
|
atspiVersion = Atspi.get_version()
|
||||||
msg = "MOUSE REVIEW ERROR: Wnck is not available"
|
capabilityEnum = getattr(Atspi, "DeviceCapability", None)
|
||||||
|
pointerMonitor = getattr(capabilityEnum, "POINTER_MONITOR", 0) if capabilityEnum else 0
|
||||||
|
atspiSupported = pointerMonitor and (
|
||||||
|
atspiVersion[0] > 2
|
||||||
|
or atspiVersion[1] >= 60
|
||||||
|
or (atspiVersion[0] == 2 and atspiVersion[1] == 59 and atspiVersion[2] >= 90)
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
if atspiSupported:
|
||||||
|
manager = input_event_manager.get_manager()
|
||||||
|
manager.activate_device()
|
||||||
|
if manager.enable_pointer_monitoring():
|
||||||
|
self._useAtspi = True
|
||||||
|
self._mouseReviewCapable = True
|
||||||
|
else:
|
||||||
|
self._mouseReviewCapable = Wnck is not None
|
||||||
|
else:
|
||||||
|
self._mouseReviewCapable = Wnck is not None
|
||||||
|
except Exception:
|
||||||
|
self._mouseReviewCapable = False
|
||||||
|
|
||||||
|
if not self._mouseReviewCapable:
|
||||||
|
msg = (
|
||||||
|
"MOUSE REVIEW ERROR: Not supported by AT-SPI device"
|
||||||
|
if atspiSupported
|
||||||
|
else "MOUSE REVIEW ERROR: Wnck or at-spi2-core >= 2.60 required"
|
||||||
|
)
|
||||||
debug.printMessage(debug.LEVEL_INFO, msg, True)
|
debug.printMessage(debug.LEVEL_INFO, msg, True)
|
||||||
return
|
return
|
||||||
|
|
||||||
|
if not self._useAtspi:
|
||||||
display = Gdk.Display.get_default()
|
display = Gdk.Display.get_default()
|
||||||
try:
|
try:
|
||||||
seat = Gdk.Display.get_default_seat(display)
|
seat = Gdk.Display.get_default_seat(display)
|
||||||
@@ -366,15 +405,18 @@ class MouseReviewer:
|
|||||||
except AttributeError:
|
except AttributeError:
|
||||||
msg = "MOUSE REVIEW ERROR: Gtk+ 3.20 is not available"
|
msg = "MOUSE REVIEW ERROR: Gtk+ 3.20 is not available"
|
||||||
debug.printMessage(debug.LEVEL_INFO, msg, True)
|
debug.printMessage(debug.LEVEL_INFO, msg, True)
|
||||||
|
self._mouseReviewCapable = False
|
||||||
return
|
return
|
||||||
except Exception:
|
except Exception:
|
||||||
msg = "MOUSE REVIEW ERROR: Exception getting pointer for default seat."
|
msg = "MOUSE REVIEW ERROR: Exception getting pointer for default seat."
|
||||||
debug.printMessage(debug.LEVEL_INFO, msg, True)
|
debug.printMessage(debug.LEVEL_INFO, msg, True)
|
||||||
|
self._mouseReviewCapable = False
|
||||||
return
|
return
|
||||||
|
|
||||||
if not self._pointer:
|
if not self._pointer:
|
||||||
msg = "MOUSE REVIEW ERROR: No pointer for default seat."
|
msg = "MOUSE REVIEW ERROR: No pointer for default seat."
|
||||||
debug.printMessage(debug.LEVEL_INFO, msg, True)
|
debug.printMessage(debug.LEVEL_INFO, msg, True)
|
||||||
|
self._mouseReviewCapable = False
|
||||||
return
|
return
|
||||||
|
|
||||||
if not self._active:
|
if not self._active:
|
||||||
@@ -421,9 +463,14 @@ class MouseReviewer:
|
|||||||
def activate(self):
|
def activate(self):
|
||||||
"""Activates mouse review."""
|
"""Activates mouse review."""
|
||||||
|
|
||||||
if not _mouseReviewCapable:
|
if not self._mouseReviewCapable:
|
||||||
msg = "MOUSE REVIEW ERROR: Wnck is not available"
|
msg = (
|
||||||
|
"MOUSE REVIEW ERROR: Not supported by AT-SPI device"
|
||||||
|
if self._useAtspi
|
||||||
|
else "MOUSE REVIEW ERROR: Wnck or at-spi2-core >= 2.60 required"
|
||||||
|
)
|
||||||
debug.printMessage(debug.LEVEL_INFO, msg, True)
|
debug.printMessage(debug.LEVEL_INFO, msg, True)
|
||||||
|
self._active = False
|
||||||
return
|
return
|
||||||
|
|
||||||
# Set up the initial object as the one with the focus to avoid
|
# Set up the initial object as the one with the focus to avoid
|
||||||
@@ -437,6 +484,11 @@ class MouseReviewer:
|
|||||||
frame = script.utilities.topLevelObject(obj)
|
frame = script.utilities.topLevelObject(obj)
|
||||||
self._currentMouseOver = _ItemContext(obj=obj, frame=frame, script=script)
|
self._currentMouseOver = _ItemContext(obj=obj, frame=frame, script=script)
|
||||||
|
|
||||||
|
if self._useAtspi:
|
||||||
|
input_event_manager.get_manager().start_pointer_watcher(self._on_pointer_moved)
|
||||||
|
self._active = True
|
||||||
|
return
|
||||||
|
|
||||||
self._eventListener.register("mouse:abs")
|
self._eventListener.register("mouse:abs")
|
||||||
screen = Wnck.Screen.get_default()
|
screen = Wnck.Screen.get_default()
|
||||||
if screen:
|
if screen:
|
||||||
@@ -462,7 +514,15 @@ class MouseReviewer:
|
|||||||
def deactivate(self):
|
def deactivate(self):
|
||||||
"""Deactivates mouse review."""
|
"""Deactivates mouse review."""
|
||||||
|
|
||||||
|
if self._useAtspi:
|
||||||
|
input_event_manager.get_manager().stop_pointer_watcher()
|
||||||
|
else:
|
||||||
|
try:
|
||||||
self._eventListener.deregister("mouse:abs")
|
self._eventListener.deregister("mouse:abs")
|
||||||
|
except GLib.GError as error:
|
||||||
|
msg = f"MOUSE REVIEW: Exception deregistering 'mouse:abs' listener: {error}"
|
||||||
|
debug.printMessage(debug.LEVEL_INFO, msg, True)
|
||||||
|
|
||||||
for key, value in self._handlerIds.items():
|
for key, value in self._handlerIds.items():
|
||||||
value.disconnect(key)
|
value.disconnect(key)
|
||||||
self._handlerIds = {}
|
self._handlerIds = {}
|
||||||
@@ -470,12 +530,13 @@ class MouseReviewer:
|
|||||||
self._windows = []
|
self._windows = []
|
||||||
self._all_windows = []
|
self._all_windows = []
|
||||||
|
|
||||||
|
self._eventQueue.clear()
|
||||||
self._active = False
|
self._active = False
|
||||||
|
|
||||||
def getCurrentItem(self):
|
def getCurrentItem(self):
|
||||||
"""Returns the accessible object being reviewed."""
|
"""Returns the accessible object being reviewed."""
|
||||||
|
|
||||||
if not _mouseReviewCapable:
|
if not self._mouseReviewCapable:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
if not self._active:
|
if not self._active:
|
||||||
@@ -493,7 +554,7 @@ class MouseReviewer:
|
|||||||
def toggle(self, script=None, event=None):
|
def toggle(self, script=None, event=None):
|
||||||
"""Toggle mouse reviewing on or off."""
|
"""Toggle mouse reviewing on or off."""
|
||||||
|
|
||||||
if not _mouseReviewCapable:
|
if not self._mouseReviewCapable:
|
||||||
return
|
return
|
||||||
|
|
||||||
self._active = not self._active
|
self._active = not self._active
|
||||||
@@ -570,52 +631,65 @@ class MouseReviewer:
|
|||||||
|
|
||||||
return [extents.x, extents.y, extents.width, extents.height] == list(bounds)
|
return [extents.x, extents.y, extents.width, extents.height] == list(bounds)
|
||||||
|
|
||||||
def _accessible_window_at_point(self, pX, pY):
|
def _accessible_window_at_point_deprecated(self, pX, pY):
|
||||||
"""Returns the accessible window at the specified coordinates."""
|
"""Returns the accessible window and local coordinates for screen coordinates."""
|
||||||
|
|
||||||
window = None
|
window = None
|
||||||
for w in self._windows:
|
for w in self._windows:
|
||||||
if w.is_minimized():
|
if w.is_minimized():
|
||||||
continue
|
continue
|
||||||
|
|
||||||
x, y, width, height = w.get_geometry()
|
x, y, width, height = w.get_client_window_geometry()
|
||||||
if x <= pX <= x + width and y <= pY <= y + height:
|
if x <= pX <= x + width and y <= pY <= y + height:
|
||||||
window = w
|
window = w
|
||||||
break
|
break
|
||||||
|
|
||||||
if not window:
|
if not window:
|
||||||
return None
|
return None, -1, -1
|
||||||
|
|
||||||
windowApp = window.get_application()
|
windowApp = window.get_application()
|
||||||
if not windowApp:
|
app = AXUtilities.get_application_with_pid(windowApp.get_pid()) if windowApp else None
|
||||||
return None
|
|
||||||
|
|
||||||
app = AXUtilities.get_application_with_pid(windowApp.get_pid())
|
|
||||||
if not app:
|
if not app:
|
||||||
return None
|
return None, -1, -1
|
||||||
|
|
||||||
|
windowX = pX - x
|
||||||
|
windowY = pY - y
|
||||||
|
|
||||||
candidates = [o for o in AXObject.iter_children(
|
candidates = [o for o in AXObject.iter_children(
|
||||||
app, lambda x: self._contains_point(x, pX, pY))]
|
app, lambda obj: self._contains_point(obj, windowX, windowY, Atspi.CoordType.WINDOW))]
|
||||||
if len(candidates) == 1:
|
if len(candidates) == 1:
|
||||||
return candidates[0]
|
return candidates[0], windowX, windowY
|
||||||
|
|
||||||
name = window.get_name()
|
name = window.get_name()
|
||||||
matches = [o for o in candidates if AXObject.get_name(o) == name]
|
matches = [o for o in candidates if AXObject.get_name(o) == name]
|
||||||
if len(matches) == 1:
|
if len(matches) == 1:
|
||||||
return matches[0]
|
return matches[0], windowX, windowY
|
||||||
|
|
||||||
bbox = window.get_client_window_geometry()
|
matches = [o for o in candidates if AXUtilities.is_active(o)]
|
||||||
matches = [o for o in candidates if self._has_bounds(o, bbox)]
|
|
||||||
if len(matches) == 1:
|
if len(matches) == 1:
|
||||||
return matches[0]
|
return matches[0], windowX, windowY
|
||||||
|
|
||||||
return None
|
return None, -1, -1
|
||||||
|
|
||||||
def _on_mouse_moved(self, event):
|
def _accessible_window_at_point(self, app, pX, pY):
|
||||||
"""Callback for mouse:abs events."""
|
"""Returns the accessible window and local coordinates for pointer-moved events."""
|
||||||
|
|
||||||
screen, pX, pY = self._pointer.get_position()
|
def getTuple(obj, x, y):
|
||||||
window = self._accessible_window_at_point(pX, pY)
|
rect = AXComponent.get_rect(obj)
|
||||||
|
return obj, x - rect.x, y - rect.y
|
||||||
|
|
||||||
|
candidates = [o for o in AXObject.iter_children(
|
||||||
|
app, lambda obj: self._contains_point(obj, pX, pY, Atspi.CoordType.WINDOW))]
|
||||||
|
if len(candidates) == 1:
|
||||||
|
return getTuple(candidates[0], pX, pY)
|
||||||
|
|
||||||
|
matches = [o for o in candidates if AXUtilities.is_active(o)]
|
||||||
|
if len(matches) == 1:
|
||||||
|
return getTuple(matches[0], pX, pY)
|
||||||
|
|
||||||
|
return None, -1, -1
|
||||||
|
|
||||||
|
def _mouse_moved_common(self, window, pX, pY):
|
||||||
tokens = [f"MOUSE REVIEW: Window at ({pX}, {pY}) is", window]
|
tokens = [f"MOUSE REVIEW: Window at ({pX}, {pY}) is", window]
|
||||||
debug.printTokens(debug.LEVEL_INFO, tokens, True)
|
debug.printTokens(debug.LEVEL_INFO, tokens, True)
|
||||||
if not window:
|
if not window:
|
||||||
@@ -632,15 +706,15 @@ class MouseReviewer:
|
|||||||
else:
|
else:
|
||||||
menu = AXObject.find_ancestor(cthulhu_state.locusOfFocus, AXUtilities.is_menu)
|
menu = AXObject.find_ancestor(cthulhu_state.locusOfFocus, AXUtilities.is_menu)
|
||||||
|
|
||||||
screen, nowX, nowY = self._pointer.get_position()
|
obj = None
|
||||||
if (pX, pY) != (nowX, nowY):
|
if menu:
|
||||||
msg = f"MOUSE REVIEW: Pointer moved again: ({nowX}, {nowY})"
|
obj = script.utilities.descendantAtPoint(menu, pX, pY, Atspi.CoordType.WINDOW)
|
||||||
debug.printMessage(debug.LEVEL_INFO, msg, True)
|
tokens = ["MOUSE REVIEW: Object in", menu, f"at ({pX}, {pY}) is", obj]
|
||||||
return
|
debug.printTokens(debug.LEVEL_INFO, tokens, True)
|
||||||
|
|
||||||
obj = script.utilities.descendantAtPoint(menu, pX, pY) \
|
if obj is None:
|
||||||
or script.utilities.descendantAtPoint(window, pX, pY)
|
obj = script.utilities.descendantAtPoint(window, pX, pY, Atspi.CoordType.WINDOW)
|
||||||
tokens = [f"MOUSE REVIEW: Object at ({pX}, {pY}) is", obj]
|
tokens = ["MOUSE REVIEW: Object in", window, f"at ({pX}, {pY}) is", obj]
|
||||||
debug.printTokens(debug.LEVEL_INFO, tokens, True)
|
debug.printTokens(debug.LEVEL_INFO, tokens, True)
|
||||||
|
|
||||||
script = self.app.getScriptManager().get_script(AXObject.get_application(window), obj)
|
script = self.app.getScriptManager().get_script(AXObject.get_application(window), obj)
|
||||||
@@ -658,9 +732,8 @@ class MouseReviewer:
|
|||||||
debug.printTokens(debug.LEVEL_INFO, tokens, True)
|
debug.printTokens(debug.LEVEL_INFO, tokens, True)
|
||||||
return
|
return
|
||||||
|
|
||||||
screen, nowX, nowY = self._pointer.get_position()
|
if len(self._eventQueue):
|
||||||
if (pX, pY) != (nowX, nowY):
|
msg = "MOUSE REVIEW: Mouse moved again."
|
||||||
msg = f"MOUSE REVIEW: Pointer moved again: ({nowX}, {nowY})"
|
|
||||||
debug.printMessage(debug.LEVEL_INFO, msg, True)
|
debug.printMessage(debug.LEVEL_INFO, msg, True)
|
||||||
return
|
return
|
||||||
|
|
||||||
@@ -679,22 +752,76 @@ class MouseReviewer:
|
|||||||
if new.present(self._currentMouseOver):
|
if new.present(self._currentMouseOver):
|
||||||
self._currentMouseOver = new
|
self._currentMouseOver = new
|
||||||
|
|
||||||
def _listener(self, event):
|
def _on_mouse_moved_deprecated(self, event):
|
||||||
"""Generic listener, mainly to output debugging info."""
|
"""Callback for mouse:abs events."""
|
||||||
|
|
||||||
|
pX, pY = event.detail1, event.detail2
|
||||||
|
window, windowX, windowY = self._accessible_window_at_point_deprecated(pX, pY)
|
||||||
|
self._mouse_moved_common(window, windowX, windowY)
|
||||||
|
|
||||||
|
def _on_mouse_moved(self, obj, pX, pY):
|
||||||
|
"""Callback for pointer-moved events."""
|
||||||
|
|
||||||
|
if AXObject.get_role(obj) == Atspi.Role.APPLICATION:
|
||||||
|
window, windowX, windowY = self._accessible_window_at_point(obj, pX, pY)
|
||||||
|
self._mouse_moved_common(window, windowX, windowY)
|
||||||
|
return
|
||||||
|
|
||||||
|
self._mouse_moved_common(obj, pX, pY)
|
||||||
|
|
||||||
|
def _process_event_deprecated(self):
|
||||||
|
if not self._eventQueue:
|
||||||
|
return
|
||||||
|
|
||||||
|
event = self._eventQueue.popleft()
|
||||||
|
if len(self._eventQueue):
|
||||||
|
return
|
||||||
|
|
||||||
startTime = time.time()
|
startTime = time.time()
|
||||||
tokens = ["\nvvvvv PROCESS OBJECT EVENT", event.type, "vvvvv"]
|
tokens = ["\nvvvvv PROCESS OBJECT EVENT", event.type, "vvvvv"]
|
||||||
debug.printTokens(debug.LEVEL_INFO, tokens, False)
|
debug.printTokens(debug.LEVEL_INFO, tokens, False)
|
||||||
|
|
||||||
if event.type.startswith("mouse:abs"):
|
|
||||||
self.inMouseEvent = True
|
self.inMouseEvent = True
|
||||||
self._on_mouse_moved(event)
|
self._on_mouse_moved_deprecated(event)
|
||||||
self.inMouseEvent = False
|
self.inMouseEvent = False
|
||||||
|
|
||||||
msg = f"TOTAL PROCESSING TIME: {time.time() - startTime:.4f}\n"
|
msg = f"TOTAL PROCESSING TIME: {time.time() - startTime:.4f}\n"
|
||||||
msg += f"^^^^^ PROCESS OBJECT EVENT {event.type} ^^^^^\n"
|
msg += f"^^^^^ PROCESS OBJECT EVENT {event.type} ^^^^^\n"
|
||||||
debug.printMessage(debug.LEVEL_INFO, msg, False)
|
debug.printMessage(debug.LEVEL_INFO, msg, False)
|
||||||
|
|
||||||
|
def _process_pointer_event(self):
|
||||||
|
if not self._eventQueue:
|
||||||
|
return
|
||||||
|
|
||||||
|
obj, x, y = self._eventQueue.popleft()
|
||||||
|
if len(self._eventQueue):
|
||||||
|
return
|
||||||
|
|
||||||
|
startTime = time.time()
|
||||||
|
tokens = ["\nvvvvv PROCESS POINTER-MOVED EVENT", "vvvvv"]
|
||||||
|
debug.printTokens(debug.LEVEL_INFO, tokens, False)
|
||||||
|
|
||||||
|
self.inMouseEvent = True
|
||||||
|
self._on_mouse_moved(obj, x, y)
|
||||||
|
self.inMouseEvent = False
|
||||||
|
|
||||||
|
msg = f"TOTAL PROCESSING TIME: {time.time() - startTime:.4f}\n"
|
||||||
|
msg += "^^^^^ PROCESS POINTER-MOVED EVENT ^^^^^\n"
|
||||||
|
debug.printMessage(debug.LEVEL_INFO, msg, False)
|
||||||
|
|
||||||
|
def _listener(self, event):
|
||||||
|
"""Generic listener, mainly to output debugging info."""
|
||||||
|
|
||||||
|
if event.type.startswith("mouse:abs"):
|
||||||
|
self._eventQueue.append(event)
|
||||||
|
GLib.timeout_add(50, self._process_event_deprecated)
|
||||||
|
|
||||||
|
def _on_pointer_moved(self, _device, obj, x, y):
|
||||||
|
"""Listener for pointer-moved events from devices."""
|
||||||
|
|
||||||
|
self._eventQueue.append([obj, x, y])
|
||||||
|
GLib.timeout_add(50, self._process_pointer_event)
|
||||||
|
|
||||||
_reviewer = None
|
_reviewer = None
|
||||||
def getReviewer():
|
def getReviewer():
|
||||||
"""Returns the Mouse Reviewer"""
|
"""Returns the Mouse Reviewer"""
|
||||||
@@ -704,5 +831,3 @@ def getReviewer():
|
|||||||
from . import cthulhu
|
from . import cthulhu
|
||||||
_reviewer = MouseReviewer(cthulhu.cthulhuApp)
|
_reviewer = MouseReviewer(cthulhu.cthulhuApp)
|
||||||
return _reviewer
|
return _reviewer
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -27,6 +27,7 @@ from gi.repository import Atspi
|
|||||||
from cthulhu.plugin import Plugin, cthulhu_hookimpl
|
from cthulhu.plugin import Plugin, cthulhu_hookimpl
|
||||||
from cthulhu import debug
|
from cthulhu import debug
|
||||||
from cthulhu import settings_manager
|
from cthulhu import settings_manager
|
||||||
|
from cthulhu.wnck_support import load_wnck
|
||||||
|
|
||||||
# Note: Removed complex beep system - simple announcements work perfectly!
|
# Note: Removed complex beep system - simple announcements work perfectly!
|
||||||
|
|
||||||
@@ -68,17 +69,18 @@ try:
|
|||||||
except ImportError:
|
except ImportError:
|
||||||
WEBCOLORS_AVAILABLE = False
|
WEBCOLORS_AVAILABLE = False
|
||||||
|
|
||||||
# GTK/GDK/Wnck
|
# GTK/GDK
|
||||||
try:
|
try:
|
||||||
import gi
|
|
||||||
gi.require_version("Gtk", "3.0")
|
gi.require_version("Gtk", "3.0")
|
||||||
gi.require_version("Gdk", "3.0")
|
gi.require_version("Gdk", "3.0")
|
||||||
gi.require_version("Wnck", "3.0")
|
from gi.repository import Gtk, Gdk
|
||||||
from gi.repository import Gtk, Gdk, Wnck
|
|
||||||
GTK_AVAILABLE = True
|
GTK_AVAILABLE = True
|
||||||
except ImportError:
|
except Exception:
|
||||||
GTK_AVAILABLE = False
|
GTK_AVAILABLE = False
|
||||||
|
|
||||||
|
Wnck = load_wnck()
|
||||||
|
WNCK_AVAILABLE = Wnck is not None
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
class OCRDesktop(Plugin):
|
class OCRDesktop(Plugin):
|
||||||
@@ -151,7 +153,7 @@ class OCRDesktop(Plugin):
|
|||||||
if not PYTESSERACT_AVAILABLE:
|
if not PYTESSERACT_AVAILABLE:
|
||||||
missing_deps.append("python-pytesseract")
|
missing_deps.append("python-pytesseract")
|
||||||
if not GTK_AVAILABLE:
|
if not GTK_AVAILABLE:
|
||||||
missing_deps.append("GTK3/GDK/Wnck")
|
missing_deps.append("GTK3/GDK")
|
||||||
|
|
||||||
if missing_deps:
|
if missing_deps:
|
||||||
debug.printMessage(debug.LEVEL_INFO,
|
debug.printMessage(debug.LEVEL_INFO,
|
||||||
@@ -360,6 +362,10 @@ class OCRDesktop(Plugin):
|
|||||||
debug.printMessage(debug.LEVEL_INFO, "OCRDesktop: GTK not available for screenshots", True)
|
debug.printMessage(debug.LEVEL_INFO, "OCRDesktop: GTK not available for screenshots", True)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
if not WNCK_AVAILABLE:
|
||||||
|
debug.printMessage(debug.LEVEL_INFO, "OCRDesktop: Wnck not available for active window screenshots", True)
|
||||||
|
return False
|
||||||
|
|
||||||
try:
|
try:
|
||||||
time.sleep(0.3) # Brief delay
|
time.sleep(0.3) # Brief delay
|
||||||
gdkCurrDesktop = Gdk.get_default_root_window()
|
gdkCurrDesktop = Gdk.get_default_root_window()
|
||||||
|
|||||||
@@ -44,6 +44,8 @@ class Utilities(ChromiumUtilities):
|
|||||||
def clearSteamVirtualizedListCaches(self) -> None:
|
def clearSteamVirtualizedListCaches(self) -> None:
|
||||||
self.clearContentCache()
|
self.clearContentCache()
|
||||||
self._steamInferredButtonLabels = {}
|
self._steamInferredButtonLabels = {}
|
||||||
|
self._isUselessImage = {}
|
||||||
|
self._shouldFilter = {}
|
||||||
|
|
||||||
def isSteamVirtualizedList(self, obj) -> bool:
|
def isSteamVirtualizedList(self, obj) -> bool:
|
||||||
if not (obj and self.inDocumentContent(obj)):
|
if not (obj and self.inDocumentContent(obj)):
|
||||||
@@ -76,6 +78,21 @@ class Utilities(ChromiumUtilities):
|
|||||||
cache[obj] = inferredLabel
|
cache[obj] = inferredLabel
|
||||||
return inferredLabel
|
return inferredLabel
|
||||||
|
|
||||||
|
def isUselessImage(self, obj) -> bool:
|
||||||
|
if not (obj and self.inDocumentContent(obj)):
|
||||||
|
return super().isUselessImage(obj)
|
||||||
|
|
||||||
|
cached = self._isUselessImage.get(hash(obj))
|
||||||
|
if cached is not None:
|
||||||
|
return cached
|
||||||
|
|
||||||
|
rv = super().isUselessImage(obj)
|
||||||
|
if not rv:
|
||||||
|
rv = self._isRedundantSteamImage(obj)
|
||||||
|
|
||||||
|
self._isUselessImage[hash(obj)] = rv
|
||||||
|
return rv
|
||||||
|
|
||||||
def _shouldInferSteamButtonLabel(self, obj) -> bool:
|
def _shouldInferSteamButtonLabel(self, obj) -> bool:
|
||||||
if not (obj and self.inDocumentContent(obj)):
|
if not (obj and self.inDocumentContent(obj)):
|
||||||
return False
|
return False
|
||||||
@@ -113,6 +130,34 @@ class Utilities(ChromiumUtilities):
|
|||||||
return "Add Friend"
|
return "Add Friend"
|
||||||
return ""
|
return ""
|
||||||
|
|
||||||
|
def _isRedundantSteamImage(self, obj) -> bool:
|
||||||
|
if not AXUtilities.is_image_or_canvas(obj):
|
||||||
|
return False
|
||||||
|
|
||||||
|
if AXObject.get_name(obj) or AXObject.get_description(obj):
|
||||||
|
return False
|
||||||
|
|
||||||
|
if AXObject.get_child_count(obj):
|
||||||
|
return False
|
||||||
|
|
||||||
|
if AXUtilities.is_focusable(obj):
|
||||||
|
return False
|
||||||
|
|
||||||
|
if not AXObject.has_action(obj, "click-ancestor"):
|
||||||
|
return False
|
||||||
|
|
||||||
|
roleDescription = self._normalizeSteamLabelText(AXObject.get_role_description(obj) or "")
|
||||||
|
if roleDescription and roleDescription.casefold() not in ["unlabeled image", "image"]:
|
||||||
|
return False
|
||||||
|
|
||||||
|
nearbyLabel = self._getSteamNearbyImageLabel(obj)
|
||||||
|
if not self._isUsefulSteamLabel(nearbyLabel):
|
||||||
|
return False
|
||||||
|
|
||||||
|
tokens = ["STEAM: Treating redundant image as useless:", obj, "(label:", nearbyLabel, ")"]
|
||||||
|
debug.printTokens(debug.LEVEL_INFO, tokens, True)
|
||||||
|
return True
|
||||||
|
|
||||||
def _getSteamNearbyButtonLabel(self, obj) -> str:
|
def _getSteamNearbyButtonLabel(self, obj) -> str:
|
||||||
parent = AXObject.get_parent(obj)
|
parent = AXObject.get_parent(obj)
|
||||||
if parent is None:
|
if parent is None:
|
||||||
@@ -132,6 +177,25 @@ class Utilities(ChromiumUtilities):
|
|||||||
|
|
||||||
return self._getSteamLabelFromChildren(grandParent, ignore=parent)
|
return self._getSteamLabelFromChildren(grandParent, ignore=parent)
|
||||||
|
|
||||||
|
def _getSteamNearbyImageLabel(self, obj) -> str:
|
||||||
|
parent = AXObject.get_parent(obj)
|
||||||
|
if parent is None:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
siblingLabel = self._getSteamLabelFromChildren(parent, ignore=obj)
|
||||||
|
if siblingLabel:
|
||||||
|
return siblingLabel
|
||||||
|
|
||||||
|
parentLabel = self._getSteamReadableText(parent)
|
||||||
|
if self._isUsefulSteamLabel(parentLabel):
|
||||||
|
return parentLabel
|
||||||
|
|
||||||
|
grandParent = AXObject.get_parent(parent)
|
||||||
|
if grandParent is None:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
return self._getSteamLabelFromChildren(grandParent, ignore=parent)
|
||||||
|
|
||||||
def _getSteamLabelFromChildren(self, obj, ignore=None) -> str:
|
def _getSteamLabelFromChildren(self, obj, ignore=None) -> str:
|
||||||
for child in AXObject.iter_children(obj):
|
for child in AXObject.iter_children(obj):
|
||||||
if child == ignore:
|
if child == ignore:
|
||||||
|
|||||||
@@ -0,0 +1,40 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
#
|
||||||
|
# Copyright (c) 2026 Stormux
|
||||||
|
#
|
||||||
|
# This library is free software; you can redistribute it and/or
|
||||||
|
# modify it under the terms of the GNU Lesser General Public
|
||||||
|
# License as published by the Free Software Foundation; either
|
||||||
|
# version 2.1 of the License, or (at your option) any later version.
|
||||||
|
|
||||||
|
"""Helpers for loading Wnck only when the session can support it."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib
|
||||||
|
import os
|
||||||
|
from types import ModuleType
|
||||||
|
|
||||||
|
|
||||||
|
def get_session_type() -> str:
|
||||||
|
sessionType = (os.environ.get("XDG_SESSION_TYPE") or "").strip().lower()
|
||||||
|
if sessionType:
|
||||||
|
return sessionType
|
||||||
|
if os.environ.get("WAYLAND_DISPLAY"):
|
||||||
|
return "wayland"
|
||||||
|
if os.environ.get("DISPLAY"):
|
||||||
|
return "x11"
|
||||||
|
return "unknown"
|
||||||
|
|
||||||
|
|
||||||
|
def can_use_wnck(session_type: str | None = None) -> bool:
|
||||||
|
return (session_type or get_session_type()).strip().lower() == "x11"
|
||||||
|
|
||||||
|
|
||||||
|
def load_wnck(session_type: str | None = None) -> ModuleType | None:
|
||||||
|
if not can_use_wnck(session_type):
|
||||||
|
return None
|
||||||
|
|
||||||
|
gi = importlib.import_module("gi")
|
||||||
|
gi.require_version("Wnck", "3.0")
|
||||||
|
return importlib.import_module("gi.repository.Wnck")
|
||||||
@@ -0,0 +1,224 @@
|
|||||||
|
import sys
|
||||||
|
import types
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
|
||||||
|
|
||||||
|
from cthulhu import input_event_manager
|
||||||
|
from cthulhu import mouse_review
|
||||||
|
|
||||||
|
|
||||||
|
class FakeDevice:
|
||||||
|
def __init__(self):
|
||||||
|
self.add_key_watcher_calls = []
|
||||||
|
self.connect_calls = []
|
||||||
|
self.disconnect_calls = []
|
||||||
|
self.set_capabilities_calls = []
|
||||||
|
self.capabilities = 0
|
||||||
|
self.next_handler_id = 17
|
||||||
|
|
||||||
|
def add_key_watcher(self, callback, user_data=None):
|
||||||
|
self.add_key_watcher_calls.append((callback, user_data))
|
||||||
|
|
||||||
|
def connect(self, signalName, callback):
|
||||||
|
self.connect_calls.append((signalName, callback))
|
||||||
|
return self.next_handler_id
|
||||||
|
|
||||||
|
def disconnect(self, handlerId):
|
||||||
|
self.disconnect_calls.append(handlerId)
|
||||||
|
|
||||||
|
def get_capabilities(self):
|
||||||
|
return self.capabilities
|
||||||
|
|
||||||
|
def set_capabilities(self, capabilities):
|
||||||
|
self.set_capabilities_calls.append(capabilities)
|
||||||
|
self.capabilities = capabilities
|
||||||
|
return capabilities
|
||||||
|
|
||||||
|
|
||||||
|
class FakeDeviceFactory:
|
||||||
|
def __init__(self, device):
|
||||||
|
self.device = device
|
||||||
|
self.new_calls = 0
|
||||||
|
self.new_full_calls = 0
|
||||||
|
self.app_ids = []
|
||||||
|
|
||||||
|
def new(self):
|
||||||
|
self.new_calls += 1
|
||||||
|
return self.device
|
||||||
|
|
||||||
|
def new_full(self, appId):
|
||||||
|
self.new_full_calls += 1
|
||||||
|
self.app_ids.append(appId)
|
||||||
|
return self.device
|
||||||
|
|
||||||
|
|
||||||
|
class InputEventManagerPointerMonitorTests(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
self.manager = input_event_manager.InputEventManager()
|
||||||
|
|
||||||
|
def test_activate_device_creates_the_device_only_once(self):
|
||||||
|
device = FakeDevice()
|
||||||
|
deviceFactory = FakeDeviceFactory(device)
|
||||||
|
fakeAtspi = types.SimpleNamespace(
|
||||||
|
get_version=lambda: (2, 58, 4),
|
||||||
|
Device=deviceFactory,
|
||||||
|
)
|
||||||
|
|
||||||
|
with mock.patch.object(input_event_manager, "Atspi", fakeAtspi):
|
||||||
|
firstDevice = self.manager.activate_device()
|
||||||
|
secondDevice = self.manager.activate_device()
|
||||||
|
|
||||||
|
self.assertIs(firstDevice, device)
|
||||||
|
self.assertIs(secondDevice, device)
|
||||||
|
self.assertEqual(deviceFactory.new_full_calls, 1)
|
||||||
|
self.assertEqual(deviceFactory.new_calls, 0)
|
||||||
|
self.assertEqual(deviceFactory.app_ids, ["org.stormux.Cthulhu"])
|
||||||
|
|
||||||
|
def test_enable_pointer_monitoring_returns_false_without_a_device(self):
|
||||||
|
fakeAtspi = types.SimpleNamespace()
|
||||||
|
|
||||||
|
with mock.patch.object(input_event_manager, "Atspi", fakeAtspi):
|
||||||
|
self.assertFalse(self.manager.enable_pointer_monitoring())
|
||||||
|
|
||||||
|
def test_enable_pointer_monitoring_returns_false_when_device_capability_is_missing(self):
|
||||||
|
device = FakeDevice()
|
||||||
|
deviceFactory = FakeDeviceFactory(device)
|
||||||
|
fakeAtspi = types.SimpleNamespace(
|
||||||
|
get_version=lambda: (2, 58, 4),
|
||||||
|
Device=deviceFactory,
|
||||||
|
)
|
||||||
|
|
||||||
|
with mock.patch.object(input_event_manager, "Atspi", fakeAtspi):
|
||||||
|
self.manager.activate_device()
|
||||||
|
self.assertFalse(self.manager.enable_pointer_monitoring())
|
||||||
|
|
||||||
|
self.assertEqual(device.set_capabilities_calls, [])
|
||||||
|
|
||||||
|
def test_enable_pointer_monitoring_returns_true_when_pointer_monitor_is_granted(self):
|
||||||
|
device = FakeDevice()
|
||||||
|
deviceFactory = FakeDeviceFactory(device)
|
||||||
|
fakeDeviceCapability = types.SimpleNamespace(POINTER_MONITOR=8)
|
||||||
|
fakeAtspi = types.SimpleNamespace(
|
||||||
|
get_version=lambda: (2, 58, 4),
|
||||||
|
Device=deviceFactory,
|
||||||
|
DeviceCapability=fakeDeviceCapability,
|
||||||
|
)
|
||||||
|
|
||||||
|
with mock.patch.object(input_event_manager, "Atspi", fakeAtspi):
|
||||||
|
self.manager.activate_device()
|
||||||
|
self.assertTrue(self.manager.enable_pointer_monitoring())
|
||||||
|
|
||||||
|
self.assertEqual(device.set_capabilities_calls, [8])
|
||||||
|
self.assertEqual(device.capabilities, 8)
|
||||||
|
|
||||||
|
def test_start_and_stop_pointer_watcher_connect_and_disconnect_pointer_moved(self):
|
||||||
|
device = FakeDevice()
|
||||||
|
deviceFactory = FakeDeviceFactory(device)
|
||||||
|
fakeAtspi = types.SimpleNamespace(
|
||||||
|
get_version=lambda: (2, 58, 4),
|
||||||
|
Device=deviceFactory,
|
||||||
|
)
|
||||||
|
callback = mock.Mock()
|
||||||
|
|
||||||
|
with mock.patch.object(input_event_manager, "Atspi", fakeAtspi):
|
||||||
|
self.manager.activate_device()
|
||||||
|
self.manager.start_pointer_watcher(callback)
|
||||||
|
self.manager.stop_pointer_watcher()
|
||||||
|
|
||||||
|
self.assertEqual(device.connect_calls, [("pointer-moved", callback)])
|
||||||
|
self.assertEqual(device.disconnect_calls, [17])
|
||||||
|
|
||||||
|
|
||||||
|
class MouseReviewBackendSelectionTests(unittest.TestCase):
|
||||||
|
@staticmethod
|
||||||
|
def _make_app(enabled=False):
|
||||||
|
app = mock.Mock()
|
||||||
|
settingsManager = mock.Mock()
|
||||||
|
settingsManager.getSetting.return_value = enabled
|
||||||
|
app.getSettingsManager.return_value = settingsManager
|
||||||
|
app.getScriptManager.return_value = mock.Mock()
|
||||||
|
return app
|
||||||
|
|
||||||
|
def test_prefers_atspi_backend_when_version_and_capability_are_available(self):
|
||||||
|
listener = mock.Mock()
|
||||||
|
deviceManager = mock.Mock()
|
||||||
|
deviceManager.enable_pointer_monitoring.return_value = True
|
||||||
|
|
||||||
|
with (
|
||||||
|
mock.patch.object(mouse_review.Atspi.EventListener, "new", return_value=listener),
|
||||||
|
mock.patch.object(mouse_review.Atspi, "get_version", return_value=(2, 60, 0)),
|
||||||
|
mock.patch.object(
|
||||||
|
mouse_review.Atspi,
|
||||||
|
"DeviceCapability",
|
||||||
|
new=types.SimpleNamespace(POINTER_MONITOR=8),
|
||||||
|
create=True,
|
||||||
|
),
|
||||||
|
mock.patch.object(mouse_review.input_event_manager, "get_manager", return_value=deviceManager),
|
||||||
|
mock.patch.object(mouse_review, "Wnck", None),
|
||||||
|
):
|
||||||
|
reviewer = mouse_review.MouseReviewer(self._make_app())
|
||||||
|
|
||||||
|
self.assertTrue(reviewer._useAtspi)
|
||||||
|
self.assertTrue(reviewer._mouseReviewCapable)
|
||||||
|
deviceManager.activate_device.assert_called_once_with()
|
||||||
|
deviceManager.enable_pointer_monitoring.assert_called_once_with()
|
||||||
|
|
||||||
|
def test_activate_uses_pointer_watcher_for_atspi_backend(self):
|
||||||
|
listener = mock.Mock()
|
||||||
|
deviceManager = mock.Mock()
|
||||||
|
deviceManager.enable_pointer_monitoring.return_value = True
|
||||||
|
|
||||||
|
with (
|
||||||
|
mock.patch.object(mouse_review.Atspi.EventListener, "new", return_value=listener),
|
||||||
|
mock.patch.object(mouse_review.Atspi, "get_version", return_value=(2, 60, 0)),
|
||||||
|
mock.patch.object(
|
||||||
|
mouse_review.Atspi,
|
||||||
|
"DeviceCapability",
|
||||||
|
new=types.SimpleNamespace(POINTER_MONITOR=8),
|
||||||
|
create=True,
|
||||||
|
),
|
||||||
|
mock.patch.object(mouse_review.input_event_manager, "get_manager", return_value=deviceManager),
|
||||||
|
mock.patch.object(mouse_review.cthulhu_state, "locusOfFocus", None),
|
||||||
|
mock.patch.object(mouse_review, "Wnck", None),
|
||||||
|
):
|
||||||
|
reviewer = mouse_review.MouseReviewer(self._make_app(enabled=False))
|
||||||
|
reviewer.activate()
|
||||||
|
reviewer.deactivate()
|
||||||
|
|
||||||
|
deviceManager.start_pointer_watcher.assert_called_once_with(reviewer._on_pointer_moved)
|
||||||
|
deviceManager.stop_pointer_watcher.assert_called_once_with()
|
||||||
|
listener.register.assert_not_called()
|
||||||
|
|
||||||
|
def test_keeps_legacy_backend_when_atspi_pointer_monitor_is_unavailable(self):
|
||||||
|
listener = mock.Mock()
|
||||||
|
pointer = mock.Mock()
|
||||||
|
seat = mock.Mock()
|
||||||
|
seat.get_pointer.return_value = pointer
|
||||||
|
screen = mock.Mock()
|
||||||
|
screen.get_windows_stacked.return_value = []
|
||||||
|
screen.get_active_workspace.return_value = None
|
||||||
|
fakeWnck = mock.Mock()
|
||||||
|
fakeWnck.Screen.get_default.return_value = screen
|
||||||
|
|
||||||
|
with (
|
||||||
|
mock.patch.object(mouse_review.Atspi.EventListener, "new", return_value=listener),
|
||||||
|
mock.patch.object(mouse_review.Atspi, "get_version", return_value=(2, 58, 4)),
|
||||||
|
mock.patch.object(mouse_review.input_event_manager, "get_manager"),
|
||||||
|
mock.patch.object(mouse_review.cthulhu_state, "locusOfFocus", None),
|
||||||
|
mock.patch.object(mouse_review.Gdk.Display, "get_default", return_value=mock.Mock()),
|
||||||
|
mock.patch.object(mouse_review.Gdk.Display, "get_default_seat", return_value=seat),
|
||||||
|
mock.patch.object(mouse_review, "Wnck", fakeWnck),
|
||||||
|
):
|
||||||
|
reviewer = mouse_review.MouseReviewer(self._make_app(enabled=False))
|
||||||
|
reviewer.activate()
|
||||||
|
|
||||||
|
self.assertFalse(reviewer._useAtspi)
|
||||||
|
self.assertTrue(reviewer._mouseReviewCapable)
|
||||||
|
listener.register.assert_called_once_with("mouse:abs")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -267,5 +267,90 @@ class SteamLabelRecoveryTests(unittest.TestCase):
|
|||||||
self.assertEqual(utilities.displayedLabel(button), "Add Friend")
|
self.assertEqual(utilities.displayedLabel(button), "Add Friend")
|
||||||
|
|
||||||
|
|
||||||
|
class SteamRedundantImageTests(unittest.TestCase):
|
||||||
|
def test_is_useless_image_treats_unlabeled_click_ancestor_image_with_nearby_text_as_useless(self):
|
||||||
|
testScript = mock.Mock(generatorCache={})
|
||||||
|
utilities = steam_script_utilities.Utilities(testScript)
|
||||||
|
image = object()
|
||||||
|
parent = object()
|
||||||
|
entry = object()
|
||||||
|
|
||||||
|
utilities.inDocumentContent = mock.Mock(return_value=True)
|
||||||
|
|
||||||
|
def get_parent(obj):
|
||||||
|
if obj in (image, entry):
|
||||||
|
return parent
|
||||||
|
return None
|
||||||
|
|
||||||
|
def get_name(obj):
|
||||||
|
if obj is entry:
|
||||||
|
return "Search for games or profiles..."
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def get_child_count(obj):
|
||||||
|
if obj is parent:
|
||||||
|
return 2
|
||||||
|
return 0
|
||||||
|
|
||||||
|
def has_action(obj, actionName):
|
||||||
|
return obj is image and actionName == "click-ancestor"
|
||||||
|
|
||||||
|
def iter_children(obj, pred=None):
|
||||||
|
children = [image, entry] if obj is parent else []
|
||||||
|
if pred is not None:
|
||||||
|
children = [child for child in children if pred(child)]
|
||||||
|
return iter(children)
|
||||||
|
|
||||||
|
with (
|
||||||
|
mock.patch.object(steam_script_utilities.ChromiumUtilities, "isUselessImage", return_value=False),
|
||||||
|
mock.patch.object(steam_script_utilities.AXObject, "get_parent", side_effect=get_parent),
|
||||||
|
mock.patch.object(steam_script_utilities.AXObject, "get_name", side_effect=get_name),
|
||||||
|
mock.patch.object(steam_script_utilities.AXObject, "get_description", return_value=""),
|
||||||
|
mock.patch.object(steam_script_utilities.AXObject, "get_child_count", side_effect=get_child_count),
|
||||||
|
mock.patch.object(steam_script_utilities.AXObject, "has_action", side_effect=has_action),
|
||||||
|
mock.patch.object(steam_script_utilities.AXObject, "iter_children", side_effect=iter_children),
|
||||||
|
mock.patch.object(steam_script_utilities.AXObject, "supports_text", return_value=False),
|
||||||
|
mock.patch.object(
|
||||||
|
steam_script_utilities.AXObject,
|
||||||
|
"get_role_description",
|
||||||
|
side_effect=lambda obj: "Unlabeled image" if obj is image else "",
|
||||||
|
),
|
||||||
|
mock.patch.object(
|
||||||
|
steam_script_utilities.AXUtilities,
|
||||||
|
"is_image_or_canvas",
|
||||||
|
side_effect=lambda obj: obj is image,
|
||||||
|
),
|
||||||
|
mock.patch.object(steam_script_utilities.AXUtilities, "is_focusable", return_value=False),
|
||||||
|
mock.patch.object(steam_script_utilities.AXUtilities, "is_button", return_value=False),
|
||||||
|
mock.patch.object(steam_script_utilities.AXUtilities, "is_push_button", return_value=False),
|
||||||
|
):
|
||||||
|
self.assertTrue(utilities.isUselessImage(image))
|
||||||
|
|
||||||
|
def test_is_useless_image_defers_to_generic_logic_without_nearby_text(self):
|
||||||
|
testScript = mock.Mock(generatorCache={})
|
||||||
|
utilities = steam_script_utilities.Utilities(testScript)
|
||||||
|
image = object()
|
||||||
|
|
||||||
|
utilities.inDocumentContent = mock.Mock(return_value=True)
|
||||||
|
|
||||||
|
with (
|
||||||
|
mock.patch.object(steam_script_utilities.ChromiumUtilities, "isUselessImage", return_value=False),
|
||||||
|
mock.patch.object(steam_script_utilities.AXObject, "get_parent", return_value=None),
|
||||||
|
mock.patch.object(steam_script_utilities.AXObject, "get_name", return_value=""),
|
||||||
|
mock.patch.object(steam_script_utilities.AXObject, "get_description", return_value=""),
|
||||||
|
mock.patch.object(steam_script_utilities.AXObject, "get_child_count", return_value=0),
|
||||||
|
mock.patch.object(steam_script_utilities.AXObject, "has_action", return_value=True),
|
||||||
|
mock.patch.object(steam_script_utilities.AXObject, "supports_text", return_value=False),
|
||||||
|
mock.patch.object(steam_script_utilities.AXObject, "get_role_description", return_value=""),
|
||||||
|
mock.patch.object(
|
||||||
|
steam_script_utilities.AXUtilities,
|
||||||
|
"is_image_or_canvas",
|
||||||
|
side_effect=lambda obj: obj is image,
|
||||||
|
),
|
||||||
|
mock.patch.object(steam_script_utilities.AXUtilities, "is_focusable", return_value=False),
|
||||||
|
):
|
||||||
|
self.assertFalse(utilities.isUselessImage(image))
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|||||||
@@ -0,0 +1,66 @@
|
|||||||
|
import sys
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
import gi
|
||||||
|
|
||||||
|
gi.require_version("Gdk", "3.0")
|
||||||
|
gi.require_version("Gtk", "3.0")
|
||||||
|
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
|
||||||
|
|
||||||
|
from cthulhu import wnck_support
|
||||||
|
from cthulhu.plugins.OCR import plugin as ocr_plugin
|
||||||
|
|
||||||
|
|
||||||
|
class WnckSupportTests(unittest.TestCase):
|
||||||
|
def test_load_wnck_skips_import_when_session_is_wayland(self):
|
||||||
|
with mock.patch.object(
|
||||||
|
wnck_support.importlib,
|
||||||
|
"import_module",
|
||||||
|
side_effect=AssertionError("import_module should not be called"),
|
||||||
|
):
|
||||||
|
self.assertIsNone(wnck_support.load_wnck(session_type="wayland"))
|
||||||
|
|
||||||
|
def test_load_wnck_imports_wnck_when_session_is_x11(self):
|
||||||
|
fakeGi = mock.Mock()
|
||||||
|
fakeWnck = object()
|
||||||
|
|
||||||
|
def importModule(name):
|
||||||
|
if name == "gi":
|
||||||
|
return fakeGi
|
||||||
|
if name == "gi.repository.Wnck":
|
||||||
|
return fakeWnck
|
||||||
|
raise AssertionError(f"Unexpected import: {name}")
|
||||||
|
|
||||||
|
with mock.patch.object(wnck_support.importlib, "import_module", side_effect=importModule):
|
||||||
|
self.assertIs(wnck_support.load_wnck(session_type="x11"), fakeWnck)
|
||||||
|
|
||||||
|
fakeGi.require_version.assert_called_once_with("Wnck", "3.0")
|
||||||
|
|
||||||
|
|
||||||
|
class OCRWnckHandlingTests(unittest.TestCase):
|
||||||
|
def test_check_dependencies_does_not_fail_when_wnck_is_unavailable(self):
|
||||||
|
testPlugin = ocr_plugin.OCRDesktop.__new__(ocr_plugin.OCRDesktop)
|
||||||
|
|
||||||
|
with (
|
||||||
|
mock.patch.object(ocr_plugin, "PIL_AVAILABLE", True),
|
||||||
|
mock.patch.object(ocr_plugin, "PYTESSERACT_AVAILABLE", True),
|
||||||
|
mock.patch.object(ocr_plugin, "GTK_AVAILABLE", True),
|
||||||
|
mock.patch.object(ocr_plugin, "WNCK_AVAILABLE", False),
|
||||||
|
):
|
||||||
|
self.assertTrue(testPlugin._checkDependencies())
|
||||||
|
|
||||||
|
def test_screen_shot_window_returns_false_when_wnck_is_unavailable(self):
|
||||||
|
testPlugin = ocr_plugin.OCRDesktop.__new__(ocr_plugin.OCRDesktop)
|
||||||
|
|
||||||
|
with (
|
||||||
|
mock.patch.object(ocr_plugin, "GTK_AVAILABLE", True),
|
||||||
|
mock.patch.object(ocr_plugin, "WNCK_AVAILABLE", False),
|
||||||
|
):
|
||||||
|
self.assertFalse(testPlugin._screenShotWindow())
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
Reference in New Issue
Block a user