Voice stack field guide15 September 2026

Voice is not one product.

A decision guide to VoiceBox, GPT‑Realtime‑2, ElevenLabs and Wispr Flow — and where each belongs in Koinaku, FlavourMind and your wider project stack.

VoiceBoxlocal voice I/O

GPT‑Realtime‑2live reasoning

ElevenLabsvoice production

Wispr Flowpersonal capture

Swipe or use ↓

TL;DRPlain English first

The 30-second answer

Choose by job, not by brand.

These tools overlap, but their centres of gravity are different. Treating them as direct substitutes creates the wrong architecture.

GPT‑Realtime‑2Live product brain

Best when a user should talk naturally, interrupt, see or hear context, and trigger tools during the conversation.4

ElevenLabsProduction voice layer

Best for polished voice generation, cloning, dubbing and managed voice agents across web, mobile and telephony.6

VoiceBoxLocal voice studio

Best for private, low-cost prototyping: local dictation, cloning, TTS and MCP/REST output on your own machine.1

Wispr FlowPersonal input tool

Best for getting your spoken thoughts into any app quickly. It is a productivity product, not your application’s embedded voice runtime.10

Mental modelDifferent layers

Why the comparison feels confusing

They meet at “voice”. They diverge everywhere else.

One is a local studio, one is a realtime reasoning model, one is a cloud voice platform, and one is an end-user dictation app.

Placement is analytical: it reflects product centre of gravity, not an objective benchmark.

VoiceBoxlocal studio + API

GPT‑Realtime‑2live agent reasoning

ElevenLabsvoice platform + agents

Wispr Flowdictation + notes

Interactive chooserSelect a primary job

Start here

What are you actually trying to build?

Recommended route

GPT‑Realtime‑2

Use it as the conversation engine when turn-taking, reasoning and tool calls matter more than voice cloning.

GPT‑Realtime‑2WebRTCyour tools

My recommendationFor Vivid’s portfolio

One coherent stack

Use all four — but never in the same role.

The highest-leverage setup is a layered portfolio: prototype locally, reason in realtime, ship polished voices, and capture your own thoughts with a personal dictation tool.

CaptureWispr Flow

Your cross-app thinking and coding input. Keep it outside the product runtime.

PrototypeVoiceBox

Local voices, private experiments, MCP agent notifications, placeholder character audio.

InteractGPT‑Realtime‑2

The low-latency conversational brain that sees context and calls product tools.

PublishElevenLabs

Brand voice, high-fidelity characters, dubbing, telephony and production governance.

This is a recommendation based on your project needs, not a vendor claim.

Section breakDeep dive

IHow the stack really differs

Capability scorecardVendor-documented features

A fair comparison

Separate capability from convenience.

“Yes” means the capability is part of the documented product. It does not imply equal quality, latency or maturity.

Capability	VoiceBox	GPT‑Realtime‑2	ElevenLabs	Wispr Flow
Primary form	Desktop studio + local API	Cloud model/API	Cloud platform + APIs	Desktop/mobile app
Speech input	Yes — Whisper dictation1	Yes — realtime audio4	Yes — Scribe + agents6	Yes — its core job10
Speech output	7 local TTS engines1	Native audio output4	Core platform capability6	No — text output product
Voice cloning	Local zero-shot cloning1	Not the core proposition	Instant + professional7	No
Tool calling	MCP exposes voice tools	Native function calling4	Agent tools + MCP8	MCP exports meeting notes
Runs fully local	Yes1	No	Enterprise private deployment9	No — cloud transcription11
Best fit	Local prototyping	Interactive agents	Production voice	Personal productivity

Product deep diveVoiceBox

Local · open source · voice I/O

VoiceBox

A workshop bench for voice.

What it is: A free, open-source local voice studio with cloning, TTS, Whisper dictation, REST and MCP.1
Strength: Privacy and iteration economics.Models, voice data and captures are documented as staying on your machine.
Developer fit: REST endpoints for generation/transcription plus MCP tools for agent speech.2
Trade-off: You own setup, model downloads, hardware limits and quality variation across engines.
Important gap: Streaming transcription and end-to-end speech LLMs are on the roadmap, not documented as shipped.3

Use it for: KoH character prototyping, private FlavourMind narration experiments, and spoken Hermes Agent notifications.

Product deep diveGPT‑Realtime‑2

Realtime · multimodal · tool-using

GPT‑Realtime‑2

The conversational brain.

What it is: OpenAI’s most capable realtime voice model, with configurable reasoning and tool use.4
Modalities: Text and audio input/output; image input; no video input.4
Transport: Realtime endpoint with WebRTC, WebSocket and SIP guidance in the platform docs.
Context: 128,000-token context window; 32,000 max output tokens.4
Trade-off: Cloud dependency, token-metered audio, orchestration work and higher reasoning effort can increase latency and output usage.

Use it for: Koinaku coaching conversations, a FlavourMind research guide, or Otto’s hands-busy cooking dialogue.

Product deep diveElevenLabs

Voice craft · agents · distribution

ElevenLabs

The production soundstage.

What it is: A broad AI voice platform spanning TTS, STT, cloning, agents and generative audio.6
Strength: Expressive output, mature voice workflows, SDKs, telephony, dubbing and agent operations.
Realtime: Flash v2.5 is documented at roughly 75 ms model latency; Scribe v2 Realtime at roughly 150 ms, excluding app/network latency.6
Agents: Visual workflow builder, tools, knowledge base, tests, evals, analytics and web/mobile/telephony deployment.8
Trade-off: Credits, subscription tiers, vendor-hosted voice assets and feature complexity. Voice clones cannot be exported as standalone models.7

Use it for: production KoH characters, Koinaku’s branded guide, multilingual public demos and polished FlavourMind explainers.

Product deep diveWispr Flow

Dictation · notes · personal workflow

Wispr Flow

The fastest front door.

What it is: A voice-to-text app across Mac, Windows, iOS and Android, plus a meeting notetaker.10
Strength: Low-friction dictation across apps, personal dictionary, formatting, snippets and 100+ languages.
AI bridge: Meeting notes can be used in Claude, ChatGPT and other AI tools through MCP.10
Privacy fact: Transcription always occurs in the cloud; data sharing, storage, context awareness and dictionary monitoring have separate controls.11
Boundary: The published product is for human dictation and notes. Treating it as an embeddable product voice API would be an inference, not a documented capability.

Use it for: dictating PRDs, research notes, prompts and code comments — not as Koinaku’s or FlavourMind’s runtime.

EconomicsSnapshot: 15 Sep 2026

Cost is shaped differently

Subscription, tokens, credits or your own hardware.

There is no single “cost per minute” that remains fair across all four. Compare the billing model to the workload.

VoiceBox: no vendor usage fee; you pay in setup, compute and maintenance.

GPT‑Realtime‑2: audio tokens are listed at $32 input / $64 output per 1M; text at $4 / $24.4

ElevenLabs: free tier plus subscriptions from $6/month on the published page; products draw from shared credits.12

Wispr Flow: free tier; Pro is shown as $15 monthly or $12 on annual billing per user.10

Dollar symbols reproduced as published. Taxes, region and negotiated terms may differ.

ElevenLabsmanaged breadth

GPT‑RT‑2token metered

Wisprseat subscription

VoiceBoxlocal compute

Privacy & deploymentArchitecture, not a badge

Three trust models

Where audio travels matters more than the logo.

Local by design

VoiceBox

Vendor documentation says models, voice data and captures stay on-device.

Strongest default for sensitive prototypes
No external inference dependency
You own endpoint security and backups

Source [1]

Cloud with controls

OpenAI + Wispr

OpenAI API data is not used for training unless opted in; default abuse logs may retain content up to 30 days, with approved ZDR controls.5

Wispr says transcription always occurs in the cloud and offers separate sharing/storage controls.11

Design implication: classify data before routing it.

Cloud → private enterprise

ElevenLabs

Standard data is documented as hosted in the US; Enterprise residency exists in the EU, India and Singapore.13

Private deployments can run in an organisation’s cloud or hardware so content stays within its network.9

Design implication: excellent option when procurement can fund the control plane.

Architecture patternsCompose the layers

Pattern A — live product conversation

Audio in. Reason. Act. Voice out.

User audioBrowser or app microphone

Realtime sessionTurn detection, context, instructions

Domain toolsKoinaku plan, pantry, research corpus

Voice choiceNative output or ElevenLabs

TelemetryLatency, task success, safety

Use native GPT audio when

Turn-taking and reasoning are the experience. It reduces orchestration and preserves conversational timing.

Add ElevenLabs when

A distinctive branded or character voice is strategically important enough to justify a second realtime hop.

Interactive project labYour portfolio

Choose a project

See the smallest sensible voice stack.

A bilingual financial coach that can act.

GPT‑Realtime‑2→Koinaku tools→native voice

Start with one spoken coaching flow in Indonesian and English. Let the model call read-only budget and learning tools; keep transactions out of scope until the evals are strong.

First experiment

A five-minute “explain my spending pattern” conversation with interruption handling, citations to the user’s own data and a post-call summary.

Project recipeKoinaku

Recommended stack

Realtime first. Branded voice later.

The value is trustworthy dialogue, not a celebrity voice.

Use GPT‑Realtime‑2 for

Conversational financial education, code-switching, interruptions, visual context and calls to read-only Koinaku tools.

Add ElevenLabs only when

A distinctive “Koinaku guide” voice improves trust in user tests and the extra latency/cost is acceptable.

Use Wispr Flow outside the app

Dictate PRDs, user-test notes and Indonesian terminology into your design and coding tools.

Safety boundary: the voice agent should explain and rehearse decisions, not silently execute financial actions.

Project recipeFlavourMind

Recommended stack

Separate private capture from public explanation.

Research voice data has a different risk profile from a narrated demo.

Private working mode

Use VoiceBox for sensitive local dictation or private synthetic narration experiments. Keep raw research audio and transcripts local by default.

Interactive exploration

Use GPT‑Realtime‑2 for a conversational guide over a deliberately scoped, provenance-aware research corpus.

Public communication

Use ElevenLabs for polished multilingual explainers once the script is approved and contains no sensitive research material.

Governance boundary: route data by sensitivity before choosing the voice vendor.

Project recipesKoH · Hermes · Otto

Three more fits

Different prototypes deserve different defaults.

Kingdom of Hype

Prototype local. Perform in production.

VoiceBox for cheap NPC line exploration; GPT‑Realtime‑2 for live character intent and game-state tools; ElevenLabs for hero characters and trailers.

Hermes Agent

Let local agents speak.

VoiceBox’s MCP tool is the cleanest fit for spoken completion alerts and per-agent voices on the older MacBook.2

Otto Sous‑Chef

Optimise for hands-busy dialogue.

GPT‑Realtime‑2 for interruptions and appliance/recipe tools; use native audio until a branded voice proves product value.

Section breakImplementation guide

IIBuild the smallest useful slice

How to implementLive conversation

Path A

Ship one safe realtime conversation.

Define one job

Example: “Explain my spending pattern”, not “be my financial adviser”.

Create an ephemeral session

Mint client credentials on your server; never expose the standard API key in the browser.

Connect audio

Use WebRTC in the client and keep privileged tool execution server-side.

Add two read-only tools

Retrieve a budget summary and a cited learning explanation.

Log the right signals

First-audio latency, interruptions, tool success and unsupported claims.

// Browser-side sketch — keep the real key server-side
const pc = new RTCPeerConnection();
const stream = await navigator.mediaDevices
  .getUserMedia({ audio: true });

stream.getTracks().forEach(track =>
  pc.addTrack(track, stream)
);

const offer = await pc.createOffer();
await pc.setLocalDescription(offer);

// Your server exchanges the SDP with OpenAI
const answer = await fetch("/api/realtime/session", {
  method: "POST",
  body: offer.sdp
}).then(r => r.json());

await pc.setRemoteDescription({
  type: "answer", sdp: answer.sdp
});

Done means

A user can interrupt naturally, the agent calls only approved tools, and every financial claim is grounded or explicitly uncertain.

OpenAI WebRTC guide

How to implementLocal voice layer

Path B

Add local voice to Hermes or KoH.

Install the desktop app

Choose the build for Apple Silicon, Intel or Windows; Docker is also documented.

Start with a preset

Validate the workflow before creating any cloned voice.

Create one consented profile

Keep the reference recording and provenance with the project.

Connect MCP

Bind a recognisable voice per agent so spoken output is attributable.

Prototype the loop

Agent completes a task, calls voicebox.speak, and the system shows the speaking state.

// MCP client configuration
{
  "mcpServers": {
    "voicebox": {
      "url": "http://127.0.0.1:17493/mcp",
      "headers": {
        "X-Voicebox-Client-Id": "hermes-agent"
      }
    }
  }
}

// Agent tool call
await voicebox.speak({
  text: "Prototype complete.",
  profile: "Guide"
});

Done means

Hermes announces one completion locally, with no cloud voice dependency and a visible speaking indicator.

VoiceBox MCP documentation

How to implementProduction voice

Path C

Promote a proven voice to production.

Prove the character in text

Voice quality cannot rescue unclear dialogue or weak interaction design.

Choose TTS or ElevenAgents

Use TTS for controlled lines; an agent when it must listen, reason and act.

Get explicit voice consent

Record who owns the source audio, allowed uses and deletion path.

Evaluate two voices

Blind-test comprehension, trust, character fit and pronunciation.

Add operational controls

Version prompts, monitor conversations and set spend/concurrency limits.

# Minimal Python TTS sketch
from elevenlabs.client import ElevenLabs

client = ElevenLabs(api_key=os.environ[
    "ELEVENLABS_API_KEY"
])

audio = client.text_to_speech.convert(
    voice_id=VOICE_ID,
    model_id="eleven_flash_v2_5",
    text="Welcome back. Shall we continue?"
)

with open("guide.mp3", "wb") as f:
    for chunk in audio:
        f.write(chunk)

Done means

The voice wins a blinded user test and the production pipeline records consent, model, voice ID, script and generation version.

ElevenLabs API reference

How to implementPersonal capture

Path D

Make dictation part of your daily harness.

Wispr Flow belongs in your creation workflow, not inside the customer-facing architecture.

Create project dictionaries

Add Koinaku, FlavourMind, Gaggiuino and recurring Indonesian terms.

Define two snippets

One for structured bug reports and one for experiment notes.

Set privacy controls

Review model-improvement sharing, cloud storage, context awareness and auto-dictionary behaviour.

Use it in Cursor and docs

Dictate intent and acceptance criteria before asking an agent to code.

Review weekly

Track whether dictation reduces friction without making briefs less precise.

Done means

You can dictate a structured experiment brief in under two minutes and the domain terms arrive correctly.

Wispr Flow help centre

EvaluationThe missing implementation step

Before scaling

Measure the conversation, not just the voice.

A beautiful voice can still be slow, ungrounded, interruptible in the wrong way or unsafe.

Latency

Time to first audible response; interruption recovery; tool round-trip.

Understanding

Word error rate on names, Indonesian terms, accents and noisy rooms.

Task success

Correct tool, correct arguments, grounded answer, clear next action.

Safety

No unauthorised action; disclosure; uncertainty; voice consent and provenance.

Golden set

30–50 short conversations spanning clean speech, interruptions, code-switching, jargon, ambiguous intent and refusal cases.

Ship gate

Define thresholds per project. A Koinaku mistake and a KoH character mismatch do not carry the same risk.

Decision checklistUse before procurement

Eight questions

The vendor choice falls out of the answers.

Is this for me or my customer?Wispr ↔ API

Must it reason live?GPT‑RT‑2 / Agents

Must audio stay local?VoiceBox

Is voice identity strategic?ElevenLabs

Does it call tools?Realtime / Agents

Is telephony required?ElevenLabs / SIP

What is the sensitive-data class?Route before inference

What failure is unacceptable?Design the eval first

Final recommendationStart small, preserve options

If I were sequencing this for you

Build Koinaku’s live voice slice first.

It demonstrates the most valuable capability — reasoning through voice with tools — while creating an evaluation harness you can reuse for FlavourMind, Otto and KoH.