glossary
verified July 2026
Warm spill
Warm spill is serving capacity that is already running — model loaded, answering health checks — but takes traffic only when the primary fleet is full. Warm is the load-bearing word.
01
Warm vs cold
A cold spare must boot, load model weights, and warm its caches before serving its first request — an interval during which traffic fails. A warm tier holds the model in memory the whole time, and costs money while idle; that standing cost is what buys overflow that degrades instead of drops. The caller pays some latency; the caller never pays availability.
02
Where warm spill ends
Spill is not infinite. Past absolute capacity the API refuses fast with a retryable 503 at_capacity rather than queueing a live call into silence — a busy signal your orchestrator can act on, instead of dead air your caller cannot. Where this tier sits in our own serving — between the primary fleet and an independent fallback — is drawn on the reliability sheet.
See also
Related sheets.
capability
3 tiers
Serving that degrades before it fails
Three serving tiers behind one endpoint — primary GPU fleet, warm spill, independent fallback — and a fast retryable 503 past absolute capacity.
glossary
1 line
Concurrent lines
A line is one simultaneous conversation. Why concurrency — not minutes — is the real capacity unit for voice agents, and how metered plans cap it quietly.
benchmark
813 ms
How the API behaves under load
What one GPU does under 16 concurrent callers — 430 ms median, 813 ms p95, zero failed requests — and how the API refuses traffic past absolute capacity.
Every term on this page is measurable on a live call — watch the readout while your own script synthesizes.
See the measured figures live