Skip to main content

​
AI Companion System

The companion system transforms your agent into a virtual companion with personality, emotional awareness, and a gamified growth system.

​
Features

15 Species + 9 Hats

Choose from 15 companion species and 9 hats, each rendered as custom SVG icons with rarity-based glow effects.

Agent Appearance Sync

Companion automatically changes species and hat when you switch agents — robot for Developer, owl for Researcher, fox for Writer. Custom agents with emoji avatars are mapped automatically.

Hermes 7-State Petdex Engine

Sprite overlay uses seven Petdex-aligned states (idle/run/review/jump/wave/failed/waiting). HITL blocked signals (approval, clarify, desktop/browser takeover) drive waiting — not false waving. SSE workflow events use transient/sticky/release modes with heartbeat auto-recovery.

Desktop Pet Overlay

On Tauri Desktop, the sprite renders embedded in the main window by default. Use Pop Out to Desktop (context menu) or Shift+click to open a transparent always-on-top puppet window (/pet-overlay) that stays visible while the app is minimized. The popped-out pet supports drag, status speech bubbles, a mini composer (single-click), double-click to toggle the main window, Shift+click to pop back in, alpha click-through, and a mail icon when a turn completes while you are away. On WebUI/SaaS, the sprite renders as an in-page draggable overlay (no OS pop-out).

Growth & Evolution

5-tier rarity system (Common → Legendary) with stat evolution, XP progression, daily snack interactions, and birthday detection.

HITL Waiting UX

When the agent needs your approval or clarification, the desktop pet shows the waiting animation (Hermes-aligned). Routine errors use brief review/failed transients — not stuck distress loops.

Community Petdex Gallery

Browse and install hundreds of community-created SpriteSheets from petdex.dev via a built-in gallery with Canvas thumbnails, lazy loading, search, and 1-click install. No manual URL entry required.

Dynamic Sprite Compatibility

The resolvePetSheetRow() engine uses alias-based mapping to automatically adapt Codex 9-row, Legacy 8-row, and custom spritesheet formats. Any community sprite “just works” without manual configuration.

Cross-Device Config Sync

Companion preferences including selected sprite are persisted server-side via the /companion/config API. Switch devices or browsers without losing your personalization.

Session /pet Command

Type /pet in chat to open the Pet Palette instantly. Submit interception handles the slash locally with zero LLM tokens — no agent round-trip required.

Installed Pet Management

Installed pets appear as chips above the gallery. Use the chip menu to remove a pet with confirmation. Uninstalling the active sprite clears selection and syncs server config automatically.

Offline Fail-Open

When the petdex.dev catalog is unavailable, already-installed pets remain usable. Switch sprites without waiting for manifest recovery.

Atlas Quality Pre-check

On install, spritesheets are validated against Codex 8×9 and Legacy 8×8 grids. Invalid dimensions are rejected before installation, preventing broken or pixelated sprites from entering your collection.

GUI Health Check Doctor

One-click diagnostics covering 6 checks: feature gate, configuration, disk presence, atlas format, SHA integrity, and local serve. Results are bilingual (EN/ZH) and accessible from the Gallery or automatically triggered when a sprite fails to load.

Zero-Config Theme Sync

Companion accent colors (rarity glow, ring shadow, status bubble borders) automatically follow your workspace Theme Profile via CSS variable token derivation. Change your theme in Appearance settings — the companion updates instantly, no separate configuration needed. The Tauri pop-out overlay (/pet-overlay) stays lightweight: it skips wallpaper/ArtLayer loading while still syncing accent tokens cross-window.

​
Configuration

  1. Navigate to Settings > Companion — the companion system is enabled by default
  2. Select a species and hat from 15 + 9 options
  3. Switch to the Gallery tab to browse community SpriteSheets from petdex.dev — search, preview, and install with one click
  4. Manage installed pets from the chip row — switch active sprite or remove with confirmation
  5. Type /pet in chat anytime to open the Pet Palette without consuming LLM tokens
  6. Your selected sprite and configuration are automatically synced across devices via the server API
  7. The companion automatically syncs with your active agent’s avatar

​
How It Works

The companion system operates on two visual layers:
  • SVG Layer (default): Lightweight icon-based rendering embedded near the chat input. Automatically reflects the active agent’s identity through species and hat changes.
  • Sprite Layer (optional): Full Canvas 2D animation engine supporting Codex 8×9 standard SpriteSheets (1536×1872px). On Desktop Tauri, renders embedded in the main window or in a popped-out transparent OS window (PSUA — main window owns state, puppet mirrors via events). On WebUI, renders as a draggable in-page overlay. resolvePetSheetRow() automatically detects Codex (9-row) vs Legacy (8-row) layouts and maps seven Petdex states (Hermes-aligned): idle, run, review, jump, wave, failed, and waiting. Status speech bubbles appear only on the popped-out desktop window.
The sprite state machine combines SSE workflow events with GUI blocked-on-user signals:
  • Workflow SSE (via stepKeyToPetEvent): planning → review (sticky), execution → run (sticky), failures → failed (transient), milestones → jump/review (transient)
  • HITL blocked (via deriveBlockedOnUser from approval/clarify/desktop/browser stores): waiting — overrides run/wave while you must act; maps to Codex row 6 (legacy sheets fall back to idle)
  • Stream liveness: busy → review; turn end → wave (unless blocked)
All state is managed via Zustand with localStorage persistence. Companion preferences sync to the server via the /companion/config API.

​
Verified Quality

The companion system is backed by 291 tests all passed (2026-08-02 verified):
  • Backend pytest: 57/57 passed — pet_store (7) + companion API (10) + feature gate (23) + companion config (17)
  • Frontend vitest: 234/234 passed — companion service (28: petInstall/petSpritesheet/formatLabel/doctorI18n/slashCommand/companionTheme) + components (147: Gallery/Sprite/Icons/snack/generator/manifest/assets) + sprite (65: Engine/StateMachine/stateMapping/surfaceBridge/statusBubble/blockedOnUser/awayCompletion)
Key coverage:
  • PetStateMachine: 100% line coverage
  • CompanionSprite: 91.83% line coverage
  • petSurfaceBridge: 100% line coverage
  • companionGenerator: 99.13% line coverage
  • Atlas quality pre-check: Codex/Legacy/non-standard/invalid all validated
  • GUI Doctor: 6-item diagnostic chain with bilingual i18n
#12 Companion ↔ Theme Profile SSOT (2026-08-02):
  • Gate script scripts/dev/e2e-companion-theme-ssot-gate.sh: 15/15 vitest (companionTheme incl. inline style mutation + marketing-paths + ThemeProfileProvider pet-overlay)
  • Chrome MCP smoke: /settings/preferences theme swatches + Companion settings「follow workspace theme」read-only link
  • Optional: COMPANION_THEME_SSOT_CHROME=1 runs pytest chrome_e2e against live stack
  • Honest boundary: Canvas sprite pixels are not theme-tinted (accent UI only)