AI Voice Agent Development

What this service covers

Almost every voice agent that fails in production fails on the same thing, and it is not the language model. It is the gap between the caller finishing a sentence and the agent starting to answer. The rough thresholds the industry works to: under about 0.8 seconds a conversation feels alive, past roughly 1.5 seconds people start talking over the agent, and past two seconds they assume the line dropped. Every architectural decision in a voice pipeline is really a decision about where that budget gets spent.

I build the whole pipeline: turn detection that knows the difference between a pause for breath and the end of a thought, streaming speech recognition, a language model call that starts generating before transcription is complete, and speech synthesis that begins speaking the first sentence while the rest is still being written. Then the parts that are unglamorous and decide whether the product survives: barge-in so a caller can interrupt mid-sentence, recovery when a model call stalls, and conversation state that stays coherent across a long session without resending the whole history on every turn.

I spent a year on exactly this for a consumer voice AI companion that is live on Google Play: audio streaming in both directions over WebSockets, a pipelined speech-to-text, model and text-to-speech chain, and interruption handling as a first-class feature rather than a patch.

I will also tell you when you do not need me. If you want an appointment-booking agent for one clinic, Vapi or Retell will do it faster and cheaper than a custom build. Custom starts paying for itself when you need latency those platforms cannot reach, per-minute costs at volume that they cannot match, model or voice choices they do not offer, or data handling that a shared platform cannot satisfy. If you are already on one of them, see migrating off Vapi or Retell; if you want the numbers first, read what a voice agent really costs.

What you get

• A working voice agent, deployed, with its latency budget measured turn by turn so you can see where the time goes
• Turn detection and barge-in tuned on real recordings, not on defaults
• Streaming STT to LLM to TTS pipeline, tuned against the latency target we agree up front
• Graceful degradation when a model call is slow, rate-limited or fails outright
• Function calling and tool use, so the agent can actually do things rather than only talk
• Load and concurrency behaviour tested before launch, not discovered after