Real-Time AI Voice Agent: One Year in Production, Live on Google Play

The product is a 3D AI companion that users talk to face to face rather than type at. That framing sets the engineering problem: everything is judged on whether the conversation feels alive, and the thing that breaks the illusion fastest is the pause between a user finishing a sentence and the companion starting to answer.

I spent a year on the conversational AI layer. The backend runs on WebSockets rather than request and response, so audio streams continuously in both directions instead of waiting for complete turns. Speech recognition, the language model call and speech synthesis are pipelined, so generation starts before the user has finished speaking and the first audio returns while the rest of the response is still being produced.

The harder work was not the happy path but everything around it: handling interruptions when a user talks over the companion, recovering when a model call is slow or fails, and keeping conversational context coherent across a long session without resending the entire history on every turn.

The client is a funded startup whose product is live on Google Play. Under NDA I can describe the engineering but not name the company or share its metrics.