AI Call Centre Technology Explained: What Is Actually Under the Hood

Research date: 23 September 2026. Last updated: 23 September 2026.

An AI call centre is four pieces of software in a row: something that carries the phone call, something that turns speech into text, something that decides what to say, and something that turns text back into speech. The part everyone talks about — the language model — is the part that has been solved. The part that decides whether a call feels natural is knowing when the caller has finished speaking, and that is still genuinely hard.

This page explains what each layer does, what it costs you in milliseconds, and which of the terms vendors use describe real engineering differences rather than positioning.

Brilo AI publishes this page and sells a product built on the stack described in it. Where Brilo's own implementation is relevant it is named as such, and what it does not do is stated.

How we compiled this

This is documented research, not a benchmark. Every figure below was read from the vendor's or project's own technical documentation on 23 September 2026 and is cited where it appears. Nothing here is a measurement taken on a live call, and published latency figures are laboratory figures — most exclude network time, which is stated where the vendor states it.

The four layers

Telephony. The call arrives over the public phone network and has to be handed to software. In practice that is SIP trunking or a forwarded number, plus a rented phone number. This layer is old, boring and completely solved, and it is also the reason an AI agent can sit in front of your existing phone system instead of replacing it.

Speech to text (ASR). Streaming audio in, text out, continuously, while the caller is still talking. Not the batch transcription that produces a tidy document afterwards — this has to emit partial results in real time so the next layer can start thinking.

The language model. Takes the text, the conversation so far and its instructions, and decides what to say and what to do. This is also the layer that calls out to your systems.

Text to speech (TTS). Turns the reply back into audio, streamed so the first syllable plays before the last one has been generated.

Around all four sits an orchestration layer that holds the conversation state, decides when it is the agent's turn to speak, handles interruptions, and manages the handoff to a human. Vapi's documentation describes its agents as combining exactly these three core sequential components — transcriber, model, voice — with "dozens of providers" available at each stage, including OpenAI, Anthropic, Google, Gladia, Deepgram and ElevenLabs. Source: docs.vapi.ai, read 23 September 2026.

That modularity matters commercially: most platforms are assembling the same third-party components, so the stack is rarely where a vendor differentiates. The orchestration is.

The latency budget

Every layer adds delay, and they add up in series. A caller notices a pause of roughly a second, and starts talking over the agent shortly after that.

Published figures give a sense of the budget. ElevenLabs quotes ~75ms for Flash v2.5, its fastest synthesis model, and ~280ms for v3 Conversational, its more expressive one — both marked "excluding application & network latency." Source: elevenlabs.io/docs/models, read 23 September 2026. Vapi advertises "sub-600ms response times with natural turn-taking" for the whole round trip. Source: docs.vapi.ai, read 23 September 2026.

Read those two together and the shape of the problem is clear. If the whole budget is 600ms and synthesis alone takes 75ms in the lab before any network time, the remaining layers — recognition, the model's own thinking, and the decision that it is even the agent's turn — are sharing a few hundred milliseconds.

The practical consequence for a buyer: the most expressive voice is not free. Choosing v3 Conversational over Flash spends roughly 200ms of a budget measured in hundreds. That is a real trade-off between sounding good and responding quickly, and it is a configuration decision someone has to make deliberately rather than discover.

Turn-taking is the hard part

Everything above is a solved engineering problem you can buy. Knowing when the caller has finished their sentence is not.

The naive approach is voice activity detection: watch for silence, and when there has been enough of it, assume the caller is done. Deepgram's endpointing works this way, and its documentation is unusually candid about the trade-off. The endpointing parameter is "enabled by default and set to 10 milliseconds", and adjusting it balances two failure modes: lower it and transcripts finalise faster but risk cutting the caller off mid-thought; raise it — their example uses a value of 500 — and utterances come through complete but the agent feels slow. Source: developers.deepgram.com/docs/endpointing, read 23 September 2026.

There is no silence threshold that gets this right, because humans pause mid-sentence. "My account number is..." followed by two seconds of the caller finding their card is not the end of a turn. A pure VAD system interrupts them. Everyone has been interrupted by one.

The current answer is to add meaning to the signal. LiveKit ships a turn detector plugin it describes as "a custom, open-weights language model that adds conversational context as an additional signal to voice activity detection", and it is the default in an agent session; the documentation recommends VAD-only "when you need minimal latency or support for a spoken language the turn detector model doesn't support." Source: docs.livekit.io, read 23 September 2026. In other words, a small model reads what has been said so far and judges whether it sounds finished.

The same documentation describes a second trick worth knowing about, because it explains a cost line. Preemptive generation speculatively starts the model and synthesis calls before end-of-turn is confirmed — it "can reduce response latency but can incur extra compute costs if the user interrupts." Source: docs.livekit.io, read 23 September 2026. The agent starts composing its answer on a guess, and throws the work away when the guess is wrong. Speed bought with wasted tokens.

What to take from this section: if you are evaluating platforms, turn-taking is the thing to test, and it is the thing a scripted demo hides. Ask the agent a question and then pause in the middle of your answer. See what it does.

What the labels actually mean

Five terms get used for what is largely one product. Here is which distinctions are real.

Conversational AI is the oldest of them and the broadest. It predates large language models and originally described intent-classification systems: the software matched what you said to one of a fixed list of intents and followed a designed flow. Much software still sold as conversational AI works this way. The distinction that matters is not the label but whether the system follows a flow somebody drew, or generates its response.

Generative AI, in this context, means the reply is composed by a language model rather than retrieved from a script. That is a genuine architectural difference and it is the change that made the current generation of voice agents possible. It is also the source of their main weakness: a generated answer can be wrong in ways a scripted one cannot.

Agentic AI is the newest and the most abused. Used precisely it means the system can take actions in other systems and chain several steps toward a goal, rather than only producing text. Used loosely — which is most of the time — it means nothing at all. The test is simple: ask what the agent can do other than talk, and to which systems it writes.

Chatbots and voice bots differ in more than channel. A chatbot can take a second to reply and nobody minds; everything in the latency section above exists because voice cannot. Text also lets the user re-read, scroll back and click a link. Voice is linear and unforgiving: the caller hears the answer once. Copy that works in chat frequently fails read aloud.

IVR is the press-one-for-sales menu. It routes and it does not understand. It is included here because a good deal of software marketed as AI is an IVR with speech recognition bolted on — you speak instead of pressing, but you are still navigating somebody's menu tree.

Where it stops being a demo: tool calling

An agent that only talks is a voicemail with better manners. The layer that makes it useful is function or tool calling: the model decides to invoke an API, the orchestration layer runs it, and the result comes back into the conversation. Vapi describes tool integration as connecting "to your APIs, databases, and existing systems." Source: docs.vapi.ai, read 23 September 2026.

This is where deployments actually succeed or fail, and it is worth being precise about the levels:

Reading is easy. Looking up an order status or opening hours needs one endpoint and breaks nothing.

Writing is where the work is. Booking an appointment means the agent holds a slot, handles the case where it was taken two seconds ago, and leaves your calendar correct if the call drops halfway through. That is ordinary integration engineering, and the AI does not make it shorter.

Writing to something without an API is where projects stall. A lot of the systems these agents need to reach — older practice management software, in-house scheduling tools — have no usable interface. No amount of model quality solves that.

The published price of the platform assumes the integration already exists. Whenever a deployment costs far more than the quote, this is usually why.

The three shapes you can buy

The same underlying stack is sold at three levels, and the difference is how much of the assembly you do.

Infrastructure. You get the components and the plumbing, and you build the agent. Amazon Connect is the clearest example: no seat licence and $0.038 per voice minute plus telephony, but the whole call flow is a build. Maximum control, maximum engineering. Source: aws.amazon.com/connect/pricing, read 22 September 2026.

Platform. You get the pipeline pre-assembled and choose the parts. Vapi and Retell AI sit here — you configure transcriber, model and voice, write the prompt, wire up the tools. Priced per minute, usually with the model and voice passed through at cost on top of a hosting rate.

Product. You get a configured agent and change settings, not components. Less control, and nobody has to understand endpointing.

The right level is a question about your engineering capacity, not your call volume. A team with no developer buying at the infrastructure level has not bought a cheaper agent; they have bought an unfunded project.

What this technology still cannot do

Hear reliably through noise and strong accents. Recognition quality degrades on exactly the calls that matter most, and vendors' language lists say which languages are supported, not how well.

Know when it is wrong. A generated answer arrives with the same confident delivery whether it is right or invented. This is why the escalation path matters more than the containment rate.

Handle a caller who is upset. Distress is the clearest signal to route to a person, and a system that cannot detect it will keep trying.

Fix a broken process. If callers ring three times because your booking confirmation never arrives, an AI agent answers all three calls faster. The underlying problem is untouched.

Brilo's stack, and what it does not do

Brilo is a voice agent built on the pipeline described here, sold at the product level: you configure an agent rather than assemble components. Published pricing is a free tier at 10 minutes a month, Pro at $149 a month for 600 minutes with overage at $0.16, and Growth at $499 for 2,500 minutes at $0.14, with a 10% annual discount. Source: brilo.ai/pricing, read 22 September 2026.

What it will not do: it is a voice agent, not a contact centre platform — no omnichannel routing, no workforce management, no queue reporting for a floor of human agents. If you are replacing Genesys, this is the wrong shape of product. It also cannot reach a system that has no API, which is the constraint that decides most deployments regardless of vendor.