Voice Interaction
Communicate with your agent using voice on supported channels.Supported Engines
Text-to-Speech (TTS)
7 TTS engines available for agent voice output: OpenAI, ElevenLabs, Fish Audio, MiniMax, free Edge, and the local Piper and Voicebox. Two TTS paths (by design):- Settings > Voice — channel replies, WebRTC, and IM voice bubbles (7 engines, streaming, smart summary)
- Agent config TTS — chat command “read this out loud” produces an Audio artifact via the
tts_generatetool (OpenAI/ElevenLabs + Gateway billing)
llms/tts unit tests (98% coverage).
Speech-to-Text (STT)
5 STT engines for voice input recognition: Local Whisper (free, privacy-first), OpenAI Whisper, Groq, Deepgram, and xAI Grok STT. Custom OpenAI-compatible endpoints are also supported via the Base URL setting.Video Audio Transcription
Video messages (including Telegram circular video notes) are automatically transcribed using the same STT pipeline. The agent understands what is said in video messages without any extra configuration.Audio File Upload
Upload audio files directly in the WebUI chat input. 8 formats supported: mp3, wav, ogg, flac, m4a, aac, wma, opus. Three upload methods:- Click the attach button to select files
- Drag and drop audio files into the chat area
- Paste (some formats)
Voice Modes
Myrm Agent supports 4 voice interaction modes, selectable in Settings > Voice:Native Function Calling (Realtime & Gemini Live)
In Realtime and Gemini Live modes, tool declarations are dynamically injected from the selected Agent Profile. The AI can call tools (web search, memory, file ops, code execution, browser, kanban, background tasks) directly during the voice conversation — without server-side Agent relay. This enables sub-300ms tool-augmented voice interactions.Voice and text are one conversation
Voice is another way into the same agent, not a separate assistant with a weaker brain:- Same session, same permissions: a voice session is bound to a chat, so memory, skills, and tool permissions are exactly the ones that chat already has — talking instead of typing never quietly grants extra reach.
- Long turns keep going after you stop talking: in Agent Bridge mode, a voice turn that runs past ~15 seconds is handed to a background task instead of being cut off, and the result comes back to the same chat.
- Results come back to you: the finished work lands in the originating chat, and your next voice session announces the summary — no polling, no “is it done yet?”.
Voice Background Tasks
During a voice conversation, you can say things like “research XXX for me” and the AI will automatically dispatch it as a background task:- Instant dispatch: A single voice command creates a background research task without interrupting the conversation
- Panel visibility: Dispatched tasks appear in the Activity Panel in real time, with one-click cancel or steer
- Durable persistence: Tasks are persisted via the Kanban system with restart recovery and zombie detection
- Completion notification: Results are automatically pushed to the originating chat when the task finishes
Background Task Completion Announcement
When a voice-dispatched background task finishes, your agent proactively speaks up during your next voice session — across all four voice modes symmetrically (Audio Only / Agent Bridge / OpenAI Realtime / Gemini Live):- Speak, walk away, hear the result: Dispatch a long task by voice, leave the computer, and the agent announces the result summary when you’re back in a voice session. No need to poll the panel or ask “is it done yet?”
- Never interrupts current speech: If the agent is mid-sentence when the task completes, the announcement queues safely in the TTS pipeline and plays the moment it finishes speaking — queue-safe insertion, no abrupt cut-offs
- Announcement is part of the conversation: The announcement is spoken as a natural continuation of the chat, and the chat timeline refreshes in real time so you can immediately follow up with details and the agent answers with full context
- No duplicate announcements across windows: With multiple windows or devices open, only the active voice session announces — session-scoped deduplication, exactly once
Full-Duplex & Barge-in Awareness
Myrm Agent supports full-duplex voice conversations — you can interrupt the Agent at any time while it’s speaking (Barge-in). When you do:- TTS playback stops immediately
- The AI model is automatically notified that its previous reply was interrupted — it won’t repeat what it already said, and will naturally address your new input
Voice Coding & Written Diff Approval Pipeline
When away from your desk, voice-driven coding is a transformative workflow. Myrm strictly implements the golden interaction rule: “Voice starts the work, a written diff approves it”:- Voice Initiates the Task: While driving, walking, or away from your computer, dictate repository changes via phone call, PWA, or voice stream;
- Sandboxed Offline Execution: The agent pulls the repository inside an isolated sandbox, implements changes, and runs unit test assertions;
- Written Code Diff Push: Before executing irreversible actions (
git push), Myrm captures an incrementalFILE_DIFFshadow snapshot and pushes an interactive, syntax-highlighted diff card directly to your mobile messaging channel (SMS, Telegram, WeChat) — never reading out long, confusing code over audio; - Verified Approval & Physical PR Delivery: Tap “Approve” on your phone to trigger the remote push, with the task remaining connected until the tangible GitHub PR link actually lands;
- 10-Minute Rollback Safety Net: If you spot an error after approval, a 10-minute TTL undo card allows instant, one-click workspace rollback.
Desktop Pet Companion & Voice Orb Integration
In the Tauri desktop client, voice interaction is seamlessly unified with the floating Desktop Pet Companion:- Floating Pet with Halo Ripple: The pet companion features a dynamic acoustic halo wave that pulses with speaking volume, replacing heavy full-screen voice overlays.
- Independent Alpha Mouse Click-Through: The transparent pet window stays on top with pixel-level alpha sampling, ensuring zero mouse interception on underlying IDEs or browsers.
- Global PTT with Screen Vision Context: Holding the global push-to-talk hotkey captures the current active window screenshot, OCR text, and title, streaming multimodal context directly into the voice session without switching windows.
- Quick Replay (F13 DoD): Double-clicking the pet companion instantly replays the last spoken agent response or interrupts active playback.
- 100ms Smart Throttling: Audio volume levels and state transitions are throttled to 10fps over Tauri IPC, reducing CPU overhead by 83% for all-day background copilot presence.
Setup
- Navigate to Settings > Voice
- Select your preferred TTS and STT engines
- Configure language and voice preferences
- For custom STT endpoints, set the Base URL field (available for OpenAI and xAI providers)
- For Realtime mode, configure an OpenAI API key in Settings > Providers
- For Gemini Live mode, configure a Google API key in Settings > Providers