benchmark
verified July 2026
How the API behaves under load.
Single-stream latency is the easy number. This page is the harder one: the same harness, the same rules, pointed at one GPU carrying sixteen simultaneous conversations — and what the API does past the point where the numbers stop.
01
The setup
- Same discipline as every figure on this site: production API, server time to first audio, call-shaped load, percentile pairs.
- Sixteen concurrent streams on one GPU, each with a distinct cloned voice — the shape of sixteen real callers, not one prompt replayed sixteen times.
- Zero failed requests across the run.
02
The figures, drawn
Fig. — First audio, single stream vs 16 callers on one GPU
Single stream · p50
107 ms
Single stream · p95
108 ms
16 callers · p50
430 ms
16 callers · p95
813 ms
one second
Read the spread, not just the median: the p95 is held inside a second and stated rather than hidden. Load costs latency here; it does not cost errors.
03
Past capacity
Beyond what the fleet can serve, the API refuses fast with a retryable 503 at_capacity instead of letting every stream degrade. A full lane never quietly slows the others, and a refused request is one your orchestrator can retry or spill — dead air is not on the menu. The full three-tier posture behind that behavior is on the reliability sheet.
04
What is not on this page
Fleet-scale figures. The numbers above are one GPU because that is what we can rerun on demand; a multi-GPU load figure joins this page when it reproduces the same way. Meanwhile the bench reruns live on any demo call, with your script.
See also
Related sheets.
benchmark
5 rules
How we measure TTS latency
The methodology behind every figure on this site: server-time TTFA on the production API under call-shaped load, percentile pairs, published conditions.
capability
3 tiers
Serving that degrades before it fails
Three serving tiers behind one endpoint — primary GPU fleet, warm spill, independent fallback — and a fast retryable 503 past absolute capacity.
glossary
1 line
Concurrent lines
A line is one simultaneous conversation. Why concurrency — not minutes — is the real capacity unit for voice agents, and how metered plans cap it quietly.
The harness reruns on demand, on the production API, with your script in it.
Watch us rerun this on a call