Voice AI platform
AI Caller
A multi-tenant AI receptionist that answers the phone, the browser and web chat — with real barge-in, booking into live calendars, and the language model running on infrastructure you control.
In production · multi-tenant · 2026
3
channels, one pipeline
4
interface languages
20 ms
voice detection frame
0
cloud LLM APIs required
01 The problem
- Calls arriving after hours and at peak go to voicemail, and the caller rings the next business within minutes.
- Generic phone bots read a script. They cannot answer a question about a specific business because they do not know anything about it.
- Most voice AI talks over the caller, or cannot be interrupted — which is immediately obvious and immediately annoying.
02 The approach
- One pipeline serving three channels, so a tenant configures their agent once and it behaves identically on the phone, in the browser and in chat.
- A retrieval layer over the tenant's own website and documents, so answers come from their material rather than from the model's imagination.
- Real barge-in: voice activity detection on every 20 ms frame, with playback cut and the in-flight model and speech synthesis cancelled the moment the caller speaks.
03 Architecture
Per-turn pipeline — identical on every channel
audio → STT browser SR, or server-side Whisper
→ language script + romanised marker detection
→ context retrieval · booking state · scoped search
→ LLM local model, streamed deltas
→ TTS sentence-assembled, barge-in cancellable
→ caller04 What it does
Three channels, one agent
Phone over Twilio or Plivo, a shareable browser call link, an embeddable widget, and web chat — all running the same pipeline and the same knowledge base.
Genuine interruption handling
When the caller speaks over the agent, playback stops and the in-flight generation is cancelled. Only the words actually spoken are committed to conversation history — so the agent never responds to a sentence the caller never heard.
Trained on the business, not the internet
A per-tenant knowledge base ingests the business's own website and documents. Falls back to site-restricted search before general search, never the other way round.
Books real appointments
A booking state machine runs inside the conversation, so the call ends with an appointment in the calendar rather than a message for someone to action later.
Four languages, including RTL
English, Arabic, Hindi and Spanish across the dashboard, with right-to-left Arabic properly mirrored rather than bolted on.
Multi-tenant from day one
Tenants, users, agents, numbers, call logs with transcripts, usage metering and billing hooks — not a single-customer prototype with a tenant column added later.
Built with
- Python 3.12
- FastAPI
- SQLAlchemy
- PostgreSQL
- Ollama
- Twilio
- Plivo
- faster-whisper
- Kokoro TTS
- Silero VAD
- WebSockets
- Jinja2