Skip to content

feat(voice): add OpenAI text-to-speech - #233

Draft
johnmatthewtennant wants to merge 12 commits into
mainfrom
openai-tts-base
Draft

feat(voice): add OpenAI text-to-speech#233
johnmatthewtennant wants to merge 12 commits into
mainfrom
openai-tts-base

Conversation

@johnmatthewtennant

@johnmatthewtennant johnmatthewtennant commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds OpenAI as an independently selectable speech output backend for Voice Conversation while preserving Pocket as the default.

  • Streams 24 kHz PCM from /v1/audio/speech using gpt-4o-mini-tts so playback begins before synthesis completes.
  • Resolves OpenAI credentials and compatible endpoint configuration using Goose precedence, including environment credentials, secure-store and file fallback, custom paths, and query parameters.
  • Keeps provider readiness synchronized across windows without reading Keychain secrets during passive status checks.
  • Bounds request startup and cancellation, binds playback to its owning window, and suppresses microphone capture through device-output latency and later streamed audio.
  • Preserves assistant source-text delivery mapping, tool-boundary flushes, interruption cancellation, stale-response suppression, completion tracking, and server-side playback speed.

Reviewer-reproducible examples

  1. Configure OpenAI in Berd and select OpenAI text-to-speech under Settings > Voice.
  2. Start Voice Conversation and receive a multi-sentence assistant response. Speech should begin before the response finishes synthesizing.
  3. Interrupt playback, including near a tool boundary. Cancelled audio must not resume, and subsequent speech must continue from the unspoken source-text suffix.
  4. Allow a response to pause between streamed chunks. Capture suppression must remain correct when audio resumes and through final device drain.
  5. Change playback speed and start another response. The synthesized voice should use the selected speed without local playback-rate processing.
  6. Configure an environment credential or compatible OpenAI base URL. Voice readiness and requests should follow the same effective Goose configuration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant