fix: Slow output stage after "saving output to ..." when using --dialize (-di)

[mitslabo]

Slow output stage after "saving output to ..." when using --diarize (-di)

Description

When running whisper-cli with --diarize (-di) on long audio files with many
segments, the transcription (encoder/decoder) itself completes quickly, but the
final output-writing stage — the part that prints saving output to '<file>' —
takes a noticeably long time, sometimes much longer than the transcription itself.

Root cause

In examples/cli/cli.cpp, all of the output-writing functions
(output_txt, output_vtt, output_srt, output_csv, output_json,
output_wts, output_lrc, output_score) take the stereo PCM buffer
by value:

static void output_srt(struct whisper_context * ctx, std::ofstream & fout,
                        const whisper_params & params,
                        std::vector<std::vector<float>> pcmf32s) { // <-- copy

pcmf32s holds the entire decoded stereo audio (two float channels spanning
the whole input duration), so every call to one of these functions copies the
complete audio buffer once.

More importantly, estimate_diarization_speaker() also takes pcmf32s by value:

static std::string estimate_diarization_speaker(
    std::vector<std::vector<float>> pcmf32s, // <-- copy
    int64_t t0, int64_t t1, bool id_only = false) {

and it is called once per segment whenever params.diarize is true, from
inside the per-segment loop of output_txt, output_vtt, output_srt,
output_csv, output_json, and output_wts (and from the live
whisper_print_segment_callback during transcription).

This means the entire stereo audio buffer gets copied O(n_segments) times per
output format. For a long recording with diarization enabled and hundreds of
segments, this results in copying gigabytes of audio data, which is exactly what
shows up as a slow, CPU/memory-bound stall right after the
saving output to '<file>' message and before the program actually exits.

The slowdown scales with:

• audio length (bigger pcmf32s)
• number of segments (more calls to estimate_diarization_speaker)
• number of output formats requested at once (-otxt -ovtt -osrt -ocsv ... each
  re-copies pcmf32s again)

Steps to reproduce

./build/bin/whisper-cli \
  -m models/ggml-base.en.bin \
  -f <a long, e.g. 30-60+ minute, stereo wav file> \
  -di \
  -otxt -osrt -ovtt -ocsv

Observe that after saving output to ... is printed for each format, there is a
multi-second (or longer, depending on length/segment count) pause before the
program returns, disproportionate to the file size/model used.

Proposed fix

Change pcmf32s parameters from by-value to const std::vector<std::vector<float>> &
in estimate_diarization_speaker() and in every output_* function
(output_txt, output_vtt, output_srt, output_csv, output_json,
output_wts, output_lrc, output_score). These parameters are only ever read,
never mutated, so switching to a const reference is behavior-preserving and
removes all of the redundant copying.

Patch attached: fix-diarization-pcmf32s-copy.patch

Environment

• whisper.cpp: master (examples/cli/cli.cpp)
• Affects any usage of -di/--diarize with one or more -o* output flags

[Copilot]

Pull request overview

This PR addresses a performance bottleneck in whisper-cli when running with diarization enabled on long audio: the output-writing stage was slow due to repeated copying of the full stereo PCM buffer.

Changes:

• Switched estimate_diarization_speaker() to take the stereo PCM buffer as const std::vector<std::vector<float>> & instead of by value.
• Switched all output_* functions that receive pcmf32s to also take it by const &, eliminating redundant full-buffer copies during output generation.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.