Tel-Agent sits between a SIP phone line and an AI agent, so you can decide who gets through, which model answers, and where every conversation lives.
It answers calls and messages in real time, but keeps the important decisions in your hands: routing, models, recordings, tools and credentials.
Run it on your own machine or LAN. Recordings and transcripts stay in the installation you control.
STT, LLM and TTS are separate provider contracts, so the dashboard isn’t welded to one model vendor.
Phone, web chat, SMS, email and social channels can reach the same conversation and memory layer.
Rather than becoming a CRM or a workflow platform, it exposes webhooks and an HTTP tool so the agent can act on other systems.
What the repository is actually building, where its boundaries sit, and why the hard part is not “just connect a phone”.
The design decision that keeps scope finite: a channel is where the person talks; an integration is what the agent acts on.
That “no” list is useful product strategy: reach everything else with webhooks or the generic HTTP tool.
When the caller interrupts, queued audio must be cancelled immediately. Otherwise the agent talks over the person and the whole product feels broken.
The repository’s target is under 800 ms from the end of caller speech to the first audio response.
Partial results matter because endpointing and turn-taking determine how “alive” the call feels.
The first chunk should speak while later text is still being generated.
The repo names Ollama, Whisper and Piper as a later local-model path, with a GPU caveat for natural conversation.
Hand a conversation to a human.
Capture a structured message.
Close politely and cleanly.
Notify Telegram or email.
Escape hatch for systems outside the core.
Ground the agent in uploaded content.
Read availability with a safety boundary.
Expose a controlled tool surface to compatible clients.
Calendar rule: propose and confirm, or write to a review calendar. A wrong booking destroys trust faster than a missed call.
Streaming, interruption, storage and a transcript are the product proof in a cheap environment.
Web chat, WhatsApp, Telegram, Messenger, Instagram, Discord, Slack, email and SMS are wired in the current roadmap snapshot.
Phone transport is proven ahead of its milestone; the repository still treats SIP, latency and barge-in as the hard case.
“Every interface built before phone must still be shaped for streaming.”
Text-channel comfort is not evidence that a call will feel natural. The latency budget is written down early, even when it is measured later.
Ports bind to loopback. Reach the installation through a private network, VPN or TLS reverse proxy.
API keys are encrypted at rest and never returned in full after saving.
Recording announcements are on by default; human takeover does not make the legal obligation disappear.
Monitor SIP registration, provider reachability, database state and per-call telemetry.
The repo’s answer is an explicit health endpoint, immediate alerting and an unmistakable dashboard status.
A deterministic path from clone to first answered conversation, followed by the SIP step that turns a demo into a phone service.
The fastest proof is a running dashboard and a streamed conversation. Don’t begin by designing the whole phone system.
Open localhost:38471. First visit creates the administrator; there are no default credentials.
Inspect the generated API at localhost:38472/docs.
Conversations live in the tel-agent-data volume by default.
Buy or use a number from Twilio, Telnyx or an equivalent SIP provider.
Create an inbound SIP trunk and dispatch rule. Keep the E.164 number and credentials handy.
Call from a mobile. Confirm the provider sees the call reach the trunk before debugging your own code.
Test forwarding too. Make sure routing rules see the number you think they will see.
Python 3.11+ · ffmpeg · outbound internet · Deepgram key · cloud LLM key · ElevenLabs key · G.711 codec on the trunk
Provider console shows the inbound call at the trunk.
It stops ringing. Silence is acceptable at this stage.
A hardcoded greeting proves the audio path.
The caller’s words appear as text in the terminal.
Only after this full loop works should you add routing, dashboards, second providers or MCP.
Track STT final, LLM first token, TTS first chunk, interruption count and language accuracy.
Answer opening hours, collect the reason for calling, search a knowledge base, then hand off when a person should take over.
Take messages, check availability, notify Telegram or email, and keep the transcript searchable across channels.
Reuse the same agent, tools and memory for web chat, SMS, email, social messaging and phone.
My read: Tel-Agent is most interesting as an agent-controlled communications layer, not as a replacement for every business system around it.
The deck reflects the public README and repository documents available on 20 September 2026. Status and milestones can change.