Skip to content

feat: add audio transcription tool - #1538

Merged
avoidwork merged 4 commits into
mainfrom
feat/audio-transcription-tool
Oct 25, 2026
Merged

avoidwork merged 4 commits into
mainfrom
feat/audio-transcription-tool

Conversation

@avoidwork

@avoidwork avoidwork commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

Description

Add a transcribeAudio tool that converts audio files (voice memos, meeting recordings, podcasts) to text, closing the input-side gap in madz's media toolset. The tool uses a hybrid backend: the OpenAI audio/transcriptions endpoint when an API key is present, and a local Whisper pipeline via @xenova/transformers otherwise. The path is validated against the sandbox allowlist and size-capped via checkFileLimit.

Type of Change

  • New feature (non-breaking change which adds functionality)

Testing

  • Unit tests (tests/unit/tools/audio.test.js): 12 tests covering schema validation (missing path, invalid path, unsupported extension, empty file, file outside sandbox allowlist, very large file), the API path (success + failure), and the local Whisper path (success, language/prompt passthrough, failure).
  • Integration tests (tests/integration/tools/audio.test.js): 2 tests covering a WAV fixture via the local backend and the API backend.
  • Full suite: 4349 tests pass. Lint: 0 warnings, 0 errors.

Coverage

  • Line coverage maintained

Checklist

  • npm run lint passes
  • Tests pass with maintained line coverage
  • No forbidden patterns used
  • Conventional Commit style applied

Closes #1516

Add OpenSpec proposal, design, spec, and tasks for the transcribeAudio tool.
Closes #1516
@avoidwork avoidwork self-assigned this Oct 25, 2026
Add transcribeAudio tool with hybrid OpenAI/local Whisper backend,
sandbox path validation, and registration. Includes unit and integration tests.
Archive the OpenSpec change and sync the audio-transcription spec.
@avoidwork

Copy link
Copy Markdown
Owner Author

Implementation Audit: audio-transcription-tool

Goal Fulfillment

  • Goal 1 (add transcribeAudio tool): Fulfilled. src/tools/audio/index.js exposes transcribeAudio (LangChain tool) and transcribeAudioImpl.
  • Goal 1.1 (input path/language/prompt): Fulfilled. Zod schema requires path, accepts optional language and prompt.
  • Goal 1.2 (output JSON): Fulfilled. Returns { ok: true, text, model, backend } or { ok: false, error }.
  • Goal 1.3 (hybrid backend): Fulfilled. API path via OpenAI audio/transcriptions when key present; local Whisper via @xenova/transformers otherwise.
  • Goal 1.4 (sandbox validation): Fulfilled. Uses validatePath + checkFileLimit from src/tools/common.js.
  • Goal 1.5 (registration): Fulfilled. Registered in TOOL_PERMISSIONS, TOOL_CLASSIFICATIONS, TOOLS, and buildToolConfig.

Spec Compliance

  • Transcribe audio file to text: Met. Both API and local paths return { ok: true, text, model, backend }.
  • Validate path against sandbox allowlist: Met. validatePath rejects out-of-scope paths.
  • Enforce audio file size limit: Met. checkFileLimit enforces the size cap.
  • Validate tool input schema: Met. Zod schema requires non-empty path.
  • Handle unsupported audio formats: Met. AUDIO_EXTENSIONS set rejects unsupported extensions.
  • Use async file system operations: Met. Uses node:fs/promises (readFile, stat).

Task Completion

  • All 11 tasks in tasks.md are marked [x].
  • Tasks 1.1-1.5 (implementation), 2.1-2.4 (registration), 3.1-3.2 (tests) all complete.

Quality Check

  • Tests: 12 unit + 2 integration tests pass. Full suite: 4349 tests pass.
  • Lint: 0 warnings, 0 errors.
  • Coverage: Audio tool at 86.49% line coverage (up from 77.51%). Overall coverage maintained.
  • No forbidden patterns: No console.log, no sync I/O in async, no silent catch, no eval, no wildcard exports.
  • DRY: Reuses validatePath/checkFileLimit from common.js; mirrors embedder.js WASM shim pattern.
  • Dependency Inversion: transcribeLocal accepts an injectable loadPipeline for testability.

Verdict

All goals fulfilled, spec-compliant, tasks complete, no quality issues. No errors found.

@avoidwork
avoidwork enabled auto-merge (squash) October 25, 2026 23:33
@avoidwork
avoidwork merged commit 972010f into main Oct 25, 2026
2 checks passed
@avoidwork
avoidwork deleted the feat/audio-transcription-tool branch October 25, 2026 23:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add audio transcription tool

1 participant