Fixed missing headers in po/ab.po. Updated the CLAUDE.md file. Now it's time to release.
This commit is contained in:
906
CLAUDE.md
906
CLAUDE.md
@@ -4,13 +4,18 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
||||
|
||||
## Project Overview
|
||||
|
||||
Cthulhu is a fork of the Orca screen reader, providing access to the graphical desktop via speech and/or braille. It's designed as a supplemental screen reader for advanced users, particularly useful for older Qt applications and specific window managers like i3.
|
||||
Cthulhu is a fork of the Orca screen reader, designed as a desktop-agnostic screen reader for Linux. It works with applications supporting AT-SPI (Assistive Technology Service Provider Interface) including GTK+, Qt, Java Swing, LibreOffice, Gecko, and WebKitGtk.
|
||||
|
||||
## Build System and Commands
|
||||
**Key differentiators from Orca:**
|
||||
- Desktop-agnostic: Works on KDE, XFCE, i3, Sway, and other environments (not just GNOME)
|
||||
- Extensible plugin architecture using pluggy framework
|
||||
- D-Bus remote control interface for external automation
|
||||
- Enhanced support for tiling window managers
|
||||
|
||||
## Build Commands
|
||||
|
||||
### Local Development Build (Recommended)
|
||||
```bash
|
||||
# Build and install locally to ~/.local (no system overwrite)
|
||||
# Local development build (installs to ~/.local)
|
||||
./build-local.sh
|
||||
|
||||
# Test local installation
|
||||
@@ -21,825 +26,166 @@ Cthulhu is a fork of the Orca screen reader, providing access to the graphical d
|
||||
|
||||
# Clean build artifacts and local installation
|
||||
./clean-local.sh
|
||||
```
|
||||
./clean-local.sh --build-only # Clean only build artifacts
|
||||
./clean-local.sh --install-only # Remove only local installation
|
||||
|
||||
### System Build (Meson)
|
||||
```bash
|
||||
# Configure and build for system installation
|
||||
meson setup _build --prefix=/usr
|
||||
# Manual Meson build
|
||||
meson setup _build --prefix=$HOME/.local
|
||||
meson compile -C _build
|
||||
sudo meson install -C _build
|
||||
meson install -C _build
|
||||
```
|
||||
|
||||
### Alternative Build (Python packaging)
|
||||
```bash
|
||||
# Using pip/hatchling
|
||||
pip install -e .
|
||||
```
|
||||
## Testing
|
||||
|
||||
### Testing
|
||||
```bash
|
||||
# Run all regression tests
|
||||
# Run all tests
|
||||
test/harness/runall.sh
|
||||
|
||||
# Run single test
|
||||
test/harness/runone.sh <test.py> <app-name>
|
||||
|
||||
# Run specific app tests
|
||||
# App-specific tests
|
||||
test/harness/runall.sh -a /path/to/app/tests
|
||||
|
||||
# Coverage analysis
|
||||
test/harness/runall.sh -c
|
||||
|
||||
# Profile mode
|
||||
test/harness/runall.sh -p
|
||||
```
|
||||
|
||||
## Core Architecture
|
||||
## Architecture Overview
|
||||
|
||||
### Main Components
|
||||
- **Event System**: `event_manager.py`, `signal_manager.py` - Central event handling
|
||||
- **Accessibility Layer**: `ax_object.py`, `ax_utilities.py` - AT-SPI object management
|
||||
- **Output Systems**: `speech.py`, `braille.py` - Speech and braille generation
|
||||
- **Navigation**: `structural_navigation.py`, `flat_review.py` - Document and screen navigation
|
||||
- **Plugin System**: `plugin_system_manager.py` - Extensible plugin architecture using pluggy
|
||||
### Core Components (`src/cthulhu/`)
|
||||
|
||||
### Application Scripts
|
||||
Scripts in `src/cthulhu/scripts/` provide application-specific behavior:
|
||||
- `scripts/apps/` - Specific applications (Firefox, Thunderbird, LibreOffice)
|
||||
- `scripts/toolkits/` - Toolkit support (GTK, Qt, Gecko, WebKit)
|
||||
- `scripts/web/` - Web browsing enhancements
|
||||
**Event & Input Processing:**
|
||||
- `event_manager.py` - Central AT-SPI event handling and routing
|
||||
- `input_event_manager.py` - Keyboard/mouse input processing
|
||||
- `input_event.py` - Input event data structures
|
||||
- `focus_manager.py` - Focus tracking across applications
|
||||
|
||||
### Plugin Development
|
||||
System plugins in `src/cthulhu/plugins/`, user plugins in `~/.local/share/cthulhu/plugins/`:
|
||||
**Accessibility Layer (ax_* files):**
|
||||
- `ax_object.py` - Core accessible object abstraction
|
||||
- `ax_utilities.py` - Common accessibility utilities
|
||||
- `ax_utilities_role.py` - Role-based accessibility helpers
|
||||
- `ax_utilities_collection.py` - Collection traversal utilities
|
||||
- `ax_text.py`, `ax_table.py`, `ax_component.py` - Specialized interfaces
|
||||
|
||||
**Output Systems:**
|
||||
- `speech.py`, `speechdispatcherfactory.py` - Speech synthesis
|
||||
- `braille.py`, `braille_generator.py` - Braille display support
|
||||
- `speech_generator.py` - Generate speech from accessible objects
|
||||
- `sound.py`, `sound_generator.py` - Audio cue generation
|
||||
|
||||
**Script System:**
|
||||
- `script_manager.py` - Loads/manages application-specific scripts
|
||||
- `script.py` - Base script class
|
||||
- `scripts/default.py` - Default behavior for all applications
|
||||
- `scripts/apps/` - Application-specific customizations (Firefox, LibreOffice, etc.)
|
||||
- `scripts/toolkits/` - Toolkit-specific scripts (GTK, Qt, WebKitGtk)
|
||||
- `scripts/web/` - Web content handling
|
||||
|
||||
**Plugin System:**
|
||||
- `plugin.py` - Base plugin class with `cthulhu_hookimpl` decorator
|
||||
- `plugin_system_manager.py` - Plugin discovery and lifecycle
|
||||
- `plugins/` - Built-in plugins (OCR, Clipboard, SpeechHistory, etc.)
|
||||
|
||||
**D-Bus Remote Control:**
|
||||
- `dbus_service.py` - D-Bus service implementation
|
||||
- Service: `org.stormux.Cthulhu.Service`
|
||||
- Path: `/org/stormux/Cthulhu/Service`
|
||||
|
||||
**Settings & State:**
|
||||
- `settings_manager.py` - User preferences persistence
|
||||
- `settings.py` - Default settings values
|
||||
- `cthulhu_state.py` - Runtime state (active script, focus, etc.)
|
||||
|
||||
### Key Design Patterns
|
||||
|
||||
**Script Inheritance:** Application scripts extend `default.Script`, overriding methods for custom behavior. The script manager selects scripts based on the focused application.
|
||||
|
||||
**Event Flow:** AT-SPI events → `event_manager` → active script's event handler → generators → output systems (speech/braille)
|
||||
|
||||
**Plugin Hooks:** Plugins implement hooks via `@cthulhu_hookimpl` decorator. Use `registerGestureByString()` for keybindings.
|
||||
|
||||
## Adding New Modules
|
||||
|
||||
When adding new Python modules under `src/cthulhu/`:
|
||||
|
||||
1. **Update `src/cthulhu/meson.build`** - Add the `.py` file to `cthulhu_python_sources`
|
||||
|
||||
2. **For new script modules** (e.g., `scripts/newmodule/`):
|
||||
- Create the directory with `__init__.py` and `script.py`
|
||||
- Add `meson.build` in the new directory
|
||||
- Add subdirectory to `src/cthulhu/scripts/meson.build`
|
||||
|
||||
3. **For new plugins**:
|
||||
- Create directory under `src/cthulhu/plugins/` with:
|
||||
- `__init__.py`, `plugin.py`, `plugin.info`, `meson.build`
|
||||
- Add subdirectory to `src/cthulhu/plugins/meson.build`
|
||||
|
||||
## Plugin Structure
|
||||
|
||||
#### Basic Plugin Structure
|
||||
```
|
||||
src/cthulhu/plugins/MyPlugin/
|
||||
├── __init__.py # Package import: from .plugin import MyPlugin
|
||||
├── plugin.py # Main implementation
|
||||
├── plugin.info # Metadata
|
||||
└── meson.build # Meson install integration
|
||||
├── __init__.py # from .plugin import *
|
||||
├── plugin.py # Main implementation extending Plugin
|
||||
├── plugin.info # Metadata (name, version, description, etc.)
|
||||
└── meson.build # install_data for .py and .info files
|
||||
```
|
||||
|
||||
#### Minimal Plugin Template
|
||||
Example plugin.py:
|
||||
```python
|
||||
from cthulhu.plugin import Plugin, cthulhu_hookimpl
|
||||
|
||||
class MyPlugin(Plugin):
|
||||
def __init__(self, *args, **kwargs):
|
||||
super().__init__(*args, **kwargs)
|
||||
self._enabled = True
|
||||
|
||||
@cthulhu_hookimpl
|
||||
def activate(self, plugin=None):
|
||||
if plugin is not None and plugin is not self:
|
||||
return
|
||||
# Plugin activation logic
|
||||
return True
|
||||
|
||||
@cthulhu_hookimpl
|
||||
def deactivate(self, plugin=None):
|
||||
if plugin is not None and plugin is not self:
|
||||
return
|
||||
# Cleanup logic
|
||||
self.registerGestureByString(self.myHandler, "description", "kb:cthulhu+key")
|
||||
|
||||
def myHandler(self, script=None, inputEvent=None):
|
||||
self.app.getDynamicApiManager().getAPI('CthulhuState').activeScript.presentMessage('Message')
|
||||
return True
|
||||
```
|
||||
|
||||
#### Plugin Integration Patterns
|
||||
## D-Bus Testing
|
||||
|
||||
**1. Keybinding Registration:**
|
||||
```python
|
||||
def _register_keybinding(self):
|
||||
if not self.app:
|
||||
return
|
||||
api_helper = self.app.getAPIHelper()
|
||||
self._kb_binding = api_helper.registerGestureByString(
|
||||
"Cthulhu+key", self._handler_method, "Description"
|
||||
)
|
||||
```
|
||||
|
||||
**2. Accessing Cthulhu APIs:**
|
||||
```python
|
||||
# Get current active script
|
||||
state = self.app.getDynamicApiManager().getAPI('CthulhuState')
|
||||
active_script = state.activeScript
|
||||
|
||||
# Access speech/messages
|
||||
speech = self.app.getDynamicApiManager().getAPI('Speech')
|
||||
messages = self.app.getDynamicApiManager().getAPI('Messages')
|
||||
```
|
||||
|
||||
**3. Sound Generation:**
|
||||
```python
|
||||
from cthulhu import sound
|
||||
from cthulhu.sound_generator import Tone
|
||||
|
||||
# Initialize player
|
||||
self._player = sound.getPlayer()
|
||||
|
||||
# Create and play tone
|
||||
tone = Tone(duration=0.15, frequency=400, volumeMultiplier=0.7)
|
||||
self._player.play(tone, interrupt=False)
|
||||
```
|
||||
|
||||
**4. Script Method Monkey-Patching:**
|
||||
```python
|
||||
def _monkey_patch_script_methods(self):
|
||||
state = self.app.getDynamicApiManager().getAPI('CthulhuState')
|
||||
if state and state.activeScript:
|
||||
script = state.activeScript
|
||||
self._original_method = script.methodName
|
||||
|
||||
def wrapped_method(*args, **kwargs):
|
||||
result = self._original_method(*args, **kwargs)
|
||||
# Add custom logic here
|
||||
return result
|
||||
|
||||
script.methodName = wrapped_method
|
||||
```
|
||||
|
||||
#### Plugin Files
|
||||
|
||||
**plugin.info format:**
|
||||
```ini
|
||||
[Core]
|
||||
Name = PluginName
|
||||
Module = PluginName
|
||||
|
||||
[Documentation]
|
||||
Description = Plugin description
|
||||
Author = Author Name
|
||||
Version = 1.0.0
|
||||
Website = https://example.com
|
||||
```
|
||||
|
||||
**meson.build template:**
|
||||
```meson
|
||||
plugin_python_sources = files([
|
||||
'__init__.py',
|
||||
'plugin.py',
|
||||
])
|
||||
|
||||
python3.install_sources(
|
||||
plugin_python_sources,
|
||||
subdir: 'cthulhu/plugins/PluginName'
|
||||
)
|
||||
|
||||
install_data(
|
||||
'plugin.info',
|
||||
install_dir: python3.get_install_dir() / 'cthulhu' / 'plugins' / 'PluginName'
|
||||
)
|
||||
```
|
||||
|
||||
**Build Integration:**
|
||||
Add plugin to `src/cthulhu/plugins/meson.build`:
|
||||
|
||||
```meson
|
||||
subdir('PluginName')
|
||||
```
|
||||
|
||||
#### Advanced Plugin Features
|
||||
|
||||
**Event System Integration:**
|
||||
- Register with Dynamic API: `api_manager.registerAPI('MyPlugin', self)`
|
||||
- Hook into script changes via monkey-patching
|
||||
- Access event manager for custom event handling
|
||||
|
||||
**Text Analysis:**
|
||||
- Use `script.getTextLineAtCaret(obj)` for current line
|
||||
- Access text attributes via `script.utilities.getTextAttributes()`
|
||||
- Leverage existing utilities like `indentationDescription()`
|
||||
|
||||
**Settings Integration:**
|
||||
- Access settings manager: `settings_manager.getManager()`
|
||||
- Create plugin-specific settings with prefixes
|
||||
- Integrate with preferences GUI if needed
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### Contributing
|
||||
- Repository: https://git.stormux.org/storm/cthulhu
|
||||
- Create branch for changes, merge to master when ready
|
||||
- Email patches to storm_dragon@stormux.org if no direct access
|
||||
- Community: IRC #stormux on irc.stormux.org
|
||||
|
||||
### Key Dependencies
|
||||
- Python 3.9+, pygobject-3.0, pluggy, gtk+-3.0
|
||||
- AT-SPI2, ATK for accessibility
|
||||
- Optional: BrlTTY/BrlAPI (braille), Speech Dispatcher, liblouis, GStreamer
|
||||
|
||||
### Version Information
|
||||
Current version and codename in `src/cthulhu/cthulhuVersion.py`
|
||||
|
||||
### Self-voicing Feature
|
||||
Direct speech output via Unix socket:
|
||||
```bash
|
||||
echo "Hello world." | socat - UNIX-CLIENT:/tmp/cthulhu.sock
|
||||
echo "<!#APPEND#!>Hello world." | socat - UNIX-CLIENT:/tmp/cthulhu.sock # No interrupt
|
||||
echo "Hello world.<#APPEND#>" | socat - UNIX-CLIENT:/tmp/cthulhu.sock # Persistent braille
|
||||
```
|
||||
|
||||
## Key Directories
|
||||
|
||||
- `src/cthulhu/` - Core source code
|
||||
- `test/` - Regression test framework with keystroke-based testing
|
||||
- `po/` - Internationalization files (extensive i18n support)
|
||||
- `help/` - Multi-language user documentation
|
||||
- `ci/` - Continuous integration scripts
|
||||
|
||||
## Testing Notes
|
||||
|
||||
The test system uses keystroke recording/playback with speech and braille output comparison. Tests are organized by application and toolkit. The testing framework automatically handles application launching and result comparison for regression testing.
|
||||
|
||||
---
|
||||
|
||||
## ORCA vs CTHULHU COMPARISON & INTEGRATION CONSIDERATIONS
|
||||
|
||||
### **Fork History & Strategic Decisions**
|
||||
- **Cthulhu Origin**: Forked from Orca 45 (GNOME-45 release) - `git log` shows initial commit `a523205`
|
||||
- **Strategic Goals**:
|
||||
- Supplemental screen reader for advanced users
|
||||
- Better support for older Qt applications
|
||||
- Enhanced window manager support (i3, etc.)
|
||||
- Plugin system with pluggy framework
|
||||
- Community-driven development
|
||||
|
||||
### **Major Architectural Differences**
|
||||
|
||||
#### **Build Systems**
|
||||
- **Cthulhu**: Meson/Ninja (primary build system)
|
||||
- **Orca**: Meson/Ninja
|
||||
|
||||
#### **Plugin Architecture**
|
||||
- **Cthulhu**: Extensive pluggy-based plugin system with 9 core plugins
|
||||
- **Orca**: No comparable plugin system - features are integrated directly into core
|
||||
- **Strategic Value**: Plugin system is Cthulhu's key differentiator - maintain this advantage
|
||||
|
||||
#### **New Orca Features (v49.alpha)**
|
||||
1. **D-Bus Remote Controller** (`dbus_service.py`)
|
||||
- Service: `org.gnome.Orca.Service`
|
||||
- Remote command execution, module access
|
||||
- Requires: `dasbus` library
|
||||
- **Integration Value**: HIGH - would enable external control of Cthulhu
|
||||
|
||||
2. **Spiel TTS Support** (`spiel.py`)
|
||||
- Alternative to speech-dispatcher
|
||||
- Multi-synthesizer support (eSpeak, Piper)
|
||||
- Flatpak-based providers
|
||||
- **Integration Value**: MEDIUM - broader TTS options
|
||||
|
||||
3. **Enhanced Module Architecture**
|
||||
- `focus_manager.py`, `input_event_manager.py`
|
||||
- Better separation of concerns
|
||||
- **Integration Value**: LOW - architectural improvement but not user-facing
|
||||
|
||||
### **Current Bug Investigation: Plugin Keybindings**
|
||||
[Previous debugging section remains valid - plugin keybinding integration is working but needs refinement]
|
||||
|
||||
### **Integration Recommendations**
|
||||
|
||||
#### **HIGH Priority - D-Bus Remote Controller**
|
||||
- **Benefit**: External application control, automation, integration with other tools
|
||||
- **Risk**: LOW - self-contained feature, minimal core impact
|
||||
- **Files to Port**: `dbus_service.py`, related D-Bus infrastructure
|
||||
- **Dependencies**: Add `dasbus` to Cthulhu's requirements
|
||||
|
||||
#### **MEDIUM Priority - Build System Migration**
|
||||
- **Benefit**: Faster builds, better dependency management, alignment with GNOME ecosystem
|
||||
- **Risk**: MEDIUM - significant build system changes, potential disruption
|
||||
- **Approach**: Gradual migration, maintain autotools compatibility initially
|
||||
|
||||
#### **LOW Priority - Spiel Integration**
|
||||
- **Benefit**: More TTS options, potentially better voice quality
|
||||
- **Risk**: MEDIUM - additional complexity, Flatpak dependencies
|
||||
- **Approach**: Optional feature, don't replace speech-dispatcher by default
|
||||
|
||||
### **Files Requiring Attention for Integration**
|
||||
|
||||
#### **D-Bus Remote Controller Integration**:
|
||||
```bash
|
||||
# Core files to review and potentially port:
|
||||
src/orca/dbus_service.py # Main D-Bus service implementation
|
||||
README-REMOTE-CONTROLLER.md # API documentation and examples
|
||||
```
|
||||
|
||||
#### **Build System Files**:
|
||||
```bash
|
||||
# Modern build configuration to consider:
|
||||
meson.build # Root build configuration
|
||||
meson_options.txt # Build options
|
||||
subprojects/spiel.wrap # Subproject integration
|
||||
```
|
||||
|
||||
### **Strategic Questions for Decision**
|
||||
|
||||
1. **Build System**: Should Cthulhu migrate to Meson for better GNOME ecosystem alignment?
|
||||
2. **D-Bus Interface**: High value feature - should this be priority #1 for integration?
|
||||
3. **Plugin System**: How to maintain Cthulhu's plugin advantage while integrating Orca improvements?
|
||||
4. **Version Strategy**: Selective feature backporting vs. major version sync?
|
||||
|
||||
## AI Assistant Integration
|
||||
|
||||
### **NEW FEATURE**: AI-Powered Accessibility Assistant
|
||||
Cthulhu now includes an optional AI assistant plugin for enhanced accessibility support:
|
||||
|
||||
- **Vision Analysis**: Screenshots + AT-SPI data for understanding unlabeled UI elements
|
||||
- **Safe Actions**: Confirmed element clicking and navigation assistance
|
||||
- **Multi-Provider Support**: Claude, ChatGPT, Gemini, and Ollama backends
|
||||
- **Privacy-First**: Disabled by default, requires explicit opt-in and API key configuration
|
||||
|
||||
### AI Assistant Configuration
|
||||
```bash
|
||||
# Access via Cthulhu Preferences
|
||||
~/.local/bin/cthulhu -s # Opens preferences dialog
|
||||
# Navigate to "AI Assistant" tab
|
||||
# 1. Check "Enable AI Assistant"
|
||||
# 2. Select provider (Claude, ChatGPT, Gemini, Ollama)
|
||||
# 3. Set API key file path
|
||||
# 4. Configure safety and quality settings
|
||||
```
|
||||
|
||||
### AI Assistant Keybindings
|
||||
- **Cthulhu+Control+Shift+Q**: Ask questions about current screen
|
||||
- **Cthulhu+Control+Shift+D**: Describe current screen
|
||||
- **Cthulhu+Control+Shift+A**: Request actions (click, type, copy)
|
||||
|
||||
### AI Provider Setup
|
||||
|
||||
#### 1. Claude (Anthropic) - **Recommended**
|
||||
```bash
|
||||
# Get API key from: https://console.anthropic.com/
|
||||
# 1. Sign up/login → "Get API Keys" → Create new key
|
||||
# 2. Copy the key (starts with "sk-ant-...")
|
||||
# 3. Save to file:
|
||||
mkdir -p ~/.config/cthulhu
|
||||
echo "sk-ant-your-actual-key-here" > ~/.config/cthulhu/claude-api-key
|
||||
chmod 600 ~/.config/cthulhu/claude-api-key
|
||||
|
||||
# Pricing: ~$3 per million input tokens, ~$15 per million output tokens
|
||||
# Best vision capabilities and safety for accessibility use
|
||||
```
|
||||
|
||||
#### 2. ChatGPT (OpenAI)
|
||||
```bash
|
||||
# Get API key from: https://platform.openai.com/api-keys
|
||||
# 1. Sign up/login → "Create new secret key"
|
||||
# 2. Copy immediately (can't view again, starts with "sk-...")
|
||||
# 3. Save to file:
|
||||
mkdir -p ~/.config/cthulhu
|
||||
echo "sk-your-actual-openai-key" > ~/.config/cthulhu/openai-api-key
|
||||
chmod 600 ~/.config/cthulhu/openai-api-key
|
||||
|
||||
# Pricing: ~$2.50 per million input tokens, ~$10 per million output tokens
|
||||
# Good vision capabilities, widely supported
|
||||
```
|
||||
|
||||
#### 3. Gemini (Google)
|
||||
```bash
|
||||
# Get API key from: https://aistudio.google.com/app/apikey
|
||||
# 1. Sign up/login → "Create API key"
|
||||
# 2. Copy the generated key
|
||||
# 3. Save to file:
|
||||
mkdir -p ~/.config/cthulhu
|
||||
echo "your-actual-gemini-key" > ~/.config/cthulhu/gemini-api-key
|
||||
chmod 600 ~/.config/cthulhu/gemini-api-key
|
||||
|
||||
# Pricing: Free tier (15 requests/min), then ~$1.25 per million tokens
|
||||
# Good for testing, has generous free allowance
|
||||
```
|
||||
|
||||
#### 4. Ollama (Local) - **Privacy-Focused**
|
||||
```bash
|
||||
# Install Ollama (no API key needed!)
|
||||
sudo pacman -S ollama # Arch Linux
|
||||
# OR: curl -fsSL https://ollama.ai/install.sh | sh
|
||||
|
||||
# Start service
|
||||
systemctl --user enable ollama
|
||||
systemctl --user start ollama
|
||||
|
||||
# Download vision-capable model (required for AI assistant)
|
||||
ollama pull llama3.2-vision # 7.9GB download
|
||||
# OR smaller model: ollama pull moondream # 1.7GB
|
||||
|
||||
# Verify installation
|
||||
ollama list # Should show downloaded models
|
||||
|
||||
# No API key needed - runs entirely offline!
|
||||
# Free to use, privacy-focused, but slower than cloud providers
|
||||
```
|
||||
|
||||
### AI Assistant Usage Patterns
|
||||
- **Information Queries**: "What does this unlabeled button do?"
|
||||
- **Navigation Help**: "Where is the login form?"
|
||||
- **Action Assistance**: "Click the submit button", "Type hello world and press enter"
|
||||
- **Layout Understanding**: "Describe the main sections of this page"
|
||||
- **Text Operations**: "Copy this text to clipboard", "Enter my username in the field"
|
||||
|
||||
### Safety Framework
|
||||
- **Confirmation Required**: All actions require user approval by default
|
||||
- **Action Descriptions**: Clear explanation of what will happen before execution
|
||||
- **Safe Defaults**: Conservative timeouts and quality settings
|
||||
- **Privacy Protection**: API keys stored securely, no data logging
|
||||
- **Action Types**: Click, Type, Copy operations via PyAutoGUI (Wayland/X11 compatible)
|
||||
|
||||
### Troubleshooting AI Assistant Setup
|
||||
|
||||
#### Common Issues
|
||||
```bash
|
||||
# Check if AI settings loaded correctly
|
||||
~/.local/bin/cthulhu -s # Open preferences, check AI Assistant tab
|
||||
|
||||
# Verify API key file permissions and format
|
||||
ls -la ~/.config/cthulhu/*-api-key # Should show 600 permissions
|
||||
cat ~/.config/cthulhu/claude-api-key # Should contain only the API key
|
||||
|
||||
# Test Ollama connection
|
||||
curl http://localhost:11434/api/version # Should return Ollama version
|
||||
ollama ps # Should show running models
|
||||
|
||||
# Check dependencies
|
||||
python3 -c "import requests, PIL, pyautogui; print('Dependencies OK')"
|
||||
|
||||
# Test screenshot capability (requires X11/Wayland)
|
||||
python3 -c "
|
||||
from gi.repository import Gdk
|
||||
window = Gdk.get_default_root_window()
|
||||
print('Screenshot capability available')
|
||||
"
|
||||
```
|
||||
|
||||
#### Required Permissions
|
||||
- **File Access**: API key files in `~/.config/cthulhu/`
|
||||
- **Screen Access**: Screenshot capture (automatic on most setups)
|
||||
- **Network Access**: HTTP requests to AI providers (except Ollama)
|
||||
- **AT-SPI Access**: Accessibility tree traversal (enabled by default)
|
||||
- **Input Synthesis**: PyAutoGUI for action execution (click, type, copy)
|
||||
|
||||
## Cthulhu Plugin System - Developer Reference
|
||||
|
||||
### **Plugin Architecture Overview**
|
||||
|
||||
Cthulhu uses a **pluggy-based plugin system** with the following components:
|
||||
|
||||
1. **Plugin Manager**: `src/cthulhu/plugin_system_manager.py` - Central plugin loading/management
|
||||
2. **Base Plugin Class**: `src/cthulhu/plugin.py` - Provides common functionality
|
||||
3. **Hook System**: Uses `@cthulhu_hookimpl` decorators for lifecycle management
|
||||
4. **Plugin Discovery**: Automatic scanning of `src/cthulhu/plugins/` and `~/.local/share/cthulhu/plugins/`
|
||||
|
||||
### **Plugin Directory Structure**
|
||||
|
||||
Every plugin must follow this exact structure:
|
||||
```
|
||||
src/cthulhu/plugins/YourPlugin/
|
||||
├── __init__.py # Import: from .plugin import YourPlugin
|
||||
├── plugin.py # Main plugin class
|
||||
├── plugin.info # Metadata (name, version, description)
|
||||
└── meson.build # Meson install integration
|
||||
```
|
||||
|
||||
### **Essential Plugin Files**
|
||||
|
||||
#### **`__init__.py`** - Package Import
|
||||
```python
|
||||
from .plugin import YourPlugin
|
||||
```
|
||||
|
||||
#### **`plugin.info`** - Metadata
|
||||
```ini
|
||||
name = Your Plugin Name
|
||||
version = 1.0.0
|
||||
description = What your plugin does
|
||||
authors = Your Name <email@example.com>
|
||||
website = https://example.com
|
||||
copyright = Copyright 2025
|
||||
builtin = false
|
||||
hidden = false
|
||||
```
|
||||
|
||||
#### **`meson.build`** - Build Integration
|
||||
```meson
|
||||
yourplugin_python_sources = files([
|
||||
'__init__.py',
|
||||
'plugin.py',
|
||||
])
|
||||
|
||||
python3.install_sources(
|
||||
yourplugin_python_sources,
|
||||
subdir: 'cthulhu/plugins/YourPlugin'
|
||||
)
|
||||
|
||||
install_data(
|
||||
'plugin.info',
|
||||
install_dir: python3.get_install_dir() / 'cthulhu' / 'plugins' / 'YourPlugin'
|
||||
)
|
||||
```
|
||||
|
||||
### **Plugin Class Template**
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
import logging
|
||||
from cthulhu.plugin import Plugin, cthulhu_hookimpl
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
class YourPlugin(Plugin):
|
||||
"""Your plugin description."""
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
"""Initialize the plugin."""
|
||||
super().__init__(*args, **kwargs)
|
||||
logger.info("YourPlugin initialized")
|
||||
|
||||
# Keybinding storage - use individual variables, NOT dictionaries
|
||||
self._kb_binding = None
|
||||
|
||||
@cthulhu_hookimpl
|
||||
def activate(self, plugin=None):
|
||||
"""Activate the plugin."""
|
||||
if plugin is not None and plugin is not self:
|
||||
return
|
||||
|
||||
try:
|
||||
logger.info("=== YourPlugin activation starting ===")
|
||||
|
||||
# Register keybindings
|
||||
self._register_keybinding()
|
||||
|
||||
logger.info("YourPlugin activated successfully")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error activating YourPlugin: {e}")
|
||||
return False
|
||||
|
||||
@cthulhu_hookimpl
|
||||
def deactivate(self, plugin=None):
|
||||
"""Deactivate the plugin."""
|
||||
if plugin is not None and plugin is not self:
|
||||
return
|
||||
|
||||
logger.info("Deactivating YourPlugin")
|
||||
self._kb_binding = None
|
||||
return True
|
||||
|
||||
def _register_keybinding(self):
|
||||
"""Register plugin keybindings."""
|
||||
try:
|
||||
# CRITICAL: Use this exact parameter order!
|
||||
self._kb_binding = self.registerGestureByString(
|
||||
self._your_handler_method, # Handler method (first)
|
||||
"Description of action", # Description (second)
|
||||
'kb:cthulhu+your+keys' # Gesture string (third)
|
||||
)
|
||||
|
||||
if self._kb_binding:
|
||||
logger.info(f"Registered keybinding: {gesture_string}")
|
||||
else:
|
||||
logger.error(f"Failed to register keybinding")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error registering keybinding: {e}")
|
||||
|
||||
def _your_handler_method(self, script=None, inputEvent=None):
|
||||
"""Handle the keybinding activation."""
|
||||
try:
|
||||
logger.info("Keybinding triggered")
|
||||
|
||||
# Your plugin logic here
|
||||
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"Error in handler: {e}")
|
||||
return False
|
||||
```
|
||||
|
||||
### **🚨 CRITICAL Keybinding Patterns**
|
||||
|
||||
#### **✅ CORRECT Pattern (What Works)**
|
||||
```python
|
||||
# Individual binding storage (NOT dictionaries)
|
||||
self._kb_binding = None
|
||||
self._kb_binding_action1 = None
|
||||
self._kb_binding_action2 = None
|
||||
|
||||
# Correct registerGestureByString parameter order
|
||||
self._kb_binding = self.registerGestureByString(
|
||||
self._handler_method, # 1st: Handler method
|
||||
"Action description", # 2nd: Description
|
||||
'kb:cthulhu+your+keys' # 3rd: Gesture string
|
||||
)
|
||||
```
|
||||
|
||||
#### **❌ INCORRECT Patterns (What Fails)**
|
||||
```python
|
||||
# DON'T use dictionaries for keybinding storage
|
||||
self._kb_bindings = {} # ❌ WRONG
|
||||
self._kb_bindings['action'] = self.registerGestureByString(...) # ❌ WRONG
|
||||
|
||||
# DON'T use wrong parameter order
|
||||
self.registerGestureByString(
|
||||
'kb:cthulhu+keys', # ❌ WRONG ORDER
|
||||
"Description",
|
||||
self._handler_method
|
||||
)
|
||||
|
||||
# DON'T use description as handler parameter
|
||||
self.registerGestureByString(
|
||||
self._handler_method,
|
||||
'kb:cthulhu+keys', # ❌ WRONG ORDER
|
||||
"Description"
|
||||
)
|
||||
```
|
||||
|
||||
### **Plugin Registration & Activation**
|
||||
|
||||
#### **Add to Build System**
|
||||
1. **Add to `src/cthulhu/plugins/meson.build`**:
|
||||
```meson
|
||||
subdir('YourPlugin')
|
||||
```
|
||||
|
||||
#### **Add to Default Active Plugins**
|
||||
In `src/cthulhu/settings.py`:
|
||||
```python
|
||||
activePlugins = ['YourPlugin', 'DisplayVersion', 'PluginManager', ...]
|
||||
```
|
||||
|
||||
### **Plugin Lifecycle Events**
|
||||
|
||||
1. **`__init__`**: Plugin instance created
|
||||
2. **`activate`**: Plugin enabled (register keybindings, connect events)
|
||||
3. **`deactivate`**: Plugin disabled (cleanup, disconnect)
|
||||
|
||||
**Note**: `activate()` may be called multiple times for different script contexts.
|
||||
|
||||
### **Common Plugin Patterns**
|
||||
|
||||
#### **Settings Integration**
|
||||
```python
|
||||
from cthulhu import settings_manager
|
||||
|
||||
class YourPlugin(Plugin):
|
||||
def __init__(self, *args, **kwargs):
|
||||
super().__init__(*args, **kwargs)
|
||||
self._settings_manager = settings_manager.getManager()
|
||||
|
||||
def activate(self, plugin=None):
|
||||
# Check if plugin should be active
|
||||
enabled = self._settings_manager.getSetting('yourPluginEnabled')
|
||||
if not enabled:
|
||||
return
|
||||
```
|
||||
|
||||
#### **Message Presentation**
|
||||
```python
|
||||
def _present_message(self, message):
|
||||
"""Present a message to the user via speech."""
|
||||
try:
|
||||
if self.app:
|
||||
state = self.app.getDynamicApiManager().getAPI('CthulhuState')
|
||||
if state and state.activeScript:
|
||||
state.activeScript.presentMessage(message, resetStyles=False)
|
||||
except Exception as e:
|
||||
logger.error(f"Error presenting message: {e}")
|
||||
```
|
||||
|
||||
#### **Sound Generation**
|
||||
```python
|
||||
from cthulhu import sound
|
||||
from cthulhu.sound_generator import Tone
|
||||
|
||||
def _play_sound(self):
|
||||
player = sound.getPlayer()
|
||||
tone = Tone(duration=0.15, frequency=400, volumeMultiplier=0.7)
|
||||
player.play(tone, interrupt=False)
|
||||
```
|
||||
|
||||
### **Debugging Plugin Issues**
|
||||
|
||||
#### **Common Debug Techniques**
|
||||
1. **Add debug output to both logger and print**:
|
||||
```python
|
||||
logger.info("Plugin message")
|
||||
print("DEBUG: Plugin message") # Shows in terminal
|
||||
```
|
||||
|
||||
2. **Check plugin loading**:
|
||||
```python
|
||||
# In __init__
|
||||
with open('/tmp/your_plugin_debug.log', 'a') as f:
|
||||
f.write("Plugin loaded\n")
|
||||
```
|
||||
|
||||
3. **Verify keybinding registration**:
|
||||
```python
|
||||
if self._kb_binding:
|
||||
print(f"DEBUG: Keybinding registered: {self._kb_binding}")
|
||||
else:
|
||||
print("DEBUG: Keybinding registration FAILED")
|
||||
```
|
||||
|
||||
#### **Common Issues & Solutions**
|
||||
|
||||
| Issue | Symptom | Solution |
|
||||
|-------|---------|----------|
|
||||
| Plugin not loading | No __init__ debug output | Check `activePlugins` list |
|
||||
| Keybindings not working | "stored for later registration" | Use correct parameter order |
|
||||
| Import errors | Plugin fails to activate | Check module imports and dependencies |
|
||||
| Settings not loading | Default values used | Verify settings key names |
|
||||
|
||||
### **Working Plugin Examples**
|
||||
- **`DisplayVersion`**: Simple keybinding + message
|
||||
- **`PluginManager`**: GUI dialog + settings management
|
||||
- **`IndentationAudio`**: Event listening + sound generation
|
||||
- **`AIAssistant`**: Complex settings + multi-keybinding + external APIs
|
||||
|
||||
## D-Bus Remote Controller Integration
|
||||
|
||||
### **EXISTING FEATURE**: D-Bus Service for Remote Control
|
||||
Cthulhu includes a D-Bus service (ported from Orca v49.alpha) for external control and automation:
|
||||
|
||||
- **Service Name**: `org.stormux.Cthulhu.Service`
|
||||
- **Object Path**: `/org/stormux/Cthulhu/Service`
|
||||
- **Dependency**: `dasbus` library (automatically detected)
|
||||
|
||||
### Testing D-Bus Functionality
|
||||
```bash
|
||||
# Start Cthulhu with D-Bus service
|
||||
~/.local/bin/cthulhu
|
||||
|
||||
# Test service availability
|
||||
# Check service availability
|
||||
busctl --user list | grep Cthulhu
|
||||
|
||||
# Get Cthulhu version via D-Bus
|
||||
busctl --user call org.stormux.Cthulhu.Service /org/stormux/Cthulhu/Service org.stormux.Cthulhu.Service GetVersion
|
||||
# Get version
|
||||
busctl --user call org.stormux.Cthulhu.Service /org/stormux/Cthulhu/Service \
|
||||
org.stormux.Cthulhu.Service GetVersion
|
||||
|
||||
# Present message to user via D-Bus
|
||||
busctl --user call org.stormux.Cthulhu.Service /org/stormux/Cthulhu/Service org.stormux.Cthulhu.Service PresentMessage s "Hello from D-Bus"
|
||||
# Present message
|
||||
busctl --user call org.stormux.Cthulhu.Service /org/stormux/Cthulhu/Service \
|
||||
org.stormux.Cthulhu.Service PresentMessage s "Hello"
|
||||
|
||||
# List available modules and commands
|
||||
busctl --user call org.stormux.Cthulhu.Service /org/stormux/Cthulhu/Service org.stormux.Cthulhu.Service ListModules
|
||||
# Introspect available methods
|
||||
busctl --user introspect org.stormux.Cthulhu.Service /org/stormux/Cthulhu/Service
|
||||
```
|
||||
|
||||
### Integration Status
|
||||
- ✅ **Core D-Bus service**: Fully ported and integrated
|
||||
- ✅ **Service lifecycle**: Automatic start/shutdown with Cthulhu
|
||||
- ✅ **Message presentation**: `PresentMessage()` method working - **FULLY FUNCTIONAL**
|
||||
- ✅ **Version info**: `GetVersion()` method working - **FULLY FUNCTIONAL**
|
||||
- ✅ **Circular import issues**: All resolved - Cthulhu imports work correctly
|
||||
- ✅ **GDK version conflicts**: Fixed with proper gi.require_version calls
|
||||
- ✅ **Presenter singleton patterns**: Fixed with lazy initialization
|
||||
- ✅ **Settings manager backend**: Fixed with proper activation sequence
|
||||
- ✅ **ATSPI registry startup**: Fixed with deferred D-Bus service initialization
|
||||
- ✅ **API naming conventions**: All Orca→Cthulhu API differences resolved
|
||||
- 🔄 **Module registration**: Ready for individual managers to register D-Bus commands
|
||||
- 🔄 **Plugin integration**: Plugins can expose D-Bus commands using decorators
|
||||
## Key Keybindings
|
||||
|
||||
### **✅ COMPLETED - Enhanced D-Bus Remote Controller with Speech and Key Echo Controls**
|
||||
The D-Bus Remote Controller from Orca v49.alpha has been successfully re-ported and enhanced with comprehensive speech and typing echo controls.
|
||||
- `Cthulhu+Space` - Open preferences
|
||||
- `Cthulhu+H` - Enter learn mode
|
||||
- `Cthulhu+Ctrl+Alt+Shift+Q` - Toggle sleep mode for current application
|
||||
- `Cthulhu+Q` - Quit Cthulhu
|
||||
|
||||
**Latest Enhancement (2025)**:
|
||||
- **SpeechManager Module**: Complete D-Bus control over speech settings (muting, verbosity, punctuation, capitalization, number pronunciation)
|
||||
- **TypingEchoManager Module**: Granular key echo controls (character/word/sentence echo, per-key-type settings)
|
||||
- **No systemd dependency**: Direct session bus registration without service files
|
||||
- **Real-time effect**: All settings take effect immediately
|
||||
## Self-Voicing Interface
|
||||
|
||||
**Files Created/Modified for Enhanced D-Bus Integration**:
|
||||
- `src/cthulhu/speech_and_verbosity_manager.py` - Enhanced with D-Bus getters/setters for all speech settings
|
||||
- `src/cthulhu/typing_echo_presenter.py` (NEW FILE) - Complete typing echo system with D-Bus controls
|
||||
- `src/cthulhu/cthulhu.py` - D-Bus service registration for speech and typing echo managers
|
||||
- `src/cthulhu/meson.build` - Added typing_echo_presenter.py to build
|
||||
- `README-REMOTE-CONTROLLER.md` - Updated with comprehensive speech and key echo examples
|
||||
|
||||
**Available D-Bus Modules**:
|
||||
- **SpeechManager**: Speech muting, verbosity, punctuation, capitalization, number styles, indentation speech
|
||||
- **TypingEchoManager**: Master key echo, character/word/sentence echo, per-key-type controls (alphabetic, numeric, punctuation, space, modifier, function, action, navigation, diacritical keys)
|
||||
- **DefaultScript**: Core Cthulhu commands
|
||||
|
||||
**D-Bus Interface Design**:
|
||||
- Service: `org.stormux.Cthulhu.Service`
|
||||
- Module paths: `/org/stormux/Cthulhu/Service/ModuleName`
|
||||
- Generic interface: `org.stormux.Cthulhu.Module`
|
||||
- Methods: `ExecuteRuntimeGetter`, `ExecuteRuntimeSetter`, `ExecuteCommand`
|
||||
|
||||
### Bug Fixes Applied
|
||||
- Fixed circular imports in presenter modules (learn_mode_presenter, notification_presenter, etc.)
|
||||
- Added lazy imports for BrailleEvent to break cthulhu.py ↔ input_event.py cycle
|
||||
- Fixed GDK version conflicts by adding gi.require_version("Gdk", "3.0") to input_event.py
|
||||
- Changed all presenter singleton patterns to lazy initialization to prevent import-time issues
|
||||
- Added settings manager activation before loadUserSettings() to fix backend initialization
|
||||
|
||||
### **Commands for Analysis**
|
||||
External applications can speak through Cthulhu via Unix socket:
|
||||
```bash
|
||||
# Compare specific features between projects
|
||||
diff -r /home/storm/devel/cthulhu/src/cthulhu /home/storm/devel/orca/src/orca
|
||||
|
||||
# Test Orca's new features
|
||||
cd /home/storm/devel/orca && meson setup _build && meson compile -C _build
|
||||
|
||||
# Test D-Bus interface
|
||||
# (requires running Orca instance with D-Bus support)
|
||||
echo "Hello world" | socat - UNIX-CLIENT:/tmp/cthulhu.sock
|
||||
echo "<!#APPEND#!>Without interrupt" | socat - UNIX-CLIENT:/tmp/cthulhu.sock
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
**Core:** Python 3.9+, pygobject-3.0, gtk+-3.0, AT-SPI2, ATK
|
||||
**Speech:** python-speechd, gstreamer-1.0
|
||||
**Plugins:** pluggy, python-dasbus (optional, for D-Bus)
|
||||
**Braille:** BrlTTY, BrlAPI, liblouis (optional)
|
||||
|
||||
## Repository
|
||||
|
||||
- **Main repo:** https://git.stormux.org/storm/cthulhu
|
||||
- **Community:** IRC #stormux on irc.stormux.org
|
||||
- **Email patches:** storm_dragon@stormux.org
|
||||
|
||||
2
po/ab.po
2
po/ab.po
@@ -1,7 +1,9 @@
|
||||
msgid ""
|
||||
msgstr ""
|
||||
"Project-Id-Version: cthulhu 48.0\n"
|
||||
"Report-Msgid-Bugs-To: \n"
|
||||
"POT-Creation-Date: 2026-01-02 10:46-0500\n"
|
||||
"PO-Revision-Date: 2026-01-02 10:46-0500\n"
|
||||
"Last-Translator: Нанба Наала <naala-nanba@rambler.ru>\n"
|
||||
"Language-Team: Abkhazian <daniel.abzakh@gmail.com>\n"
|
||||
"Language: ab\n"
|
||||
|
||||
Reference in New Issue
Block a user