Blog · November 5, 2026

Can a voice AI running on your own Mac handle a phone call?

We ran local voice setups on a laptop through the same phone-call test as the cloud models. One was as fast as the cloud but never used its tools. Two small models got close enough to be useful. And the slowest setup was slow because of its engine, not its model.

Why try local at all

When your agent calls you through agentring, a realtime voice model does the talking. In our first test, Phonic, Grok Voice and Gemini Live all started speaking in about 1.2–1.6 seconds. They all run in someone else's cloud, though, and they bill per minute or need an API key.

A local model would mean no key, no per-minute bill and no call audio leaving your machine. The question is whether it can still hold up the two things that make a call feel like a call:

• Starting to talk quickly after you stop. Past about a second and a half, you notice the pause.
• Using tools properly: checking the build, opening an issue, asking your agent. A voice that sounds great but makes things up is worse than useless on a call about your production servers.

The setup

Everything ran on one Mac with an M1 Pro chip and 32 GB of memory, at the end of October 2026.

• Hugging Face speech-to-speech 1.0.0 as the server. It speaks the OpenAI Realtime protocol, so agentring connects to it with the same class it uses for OpenAI, just with a different URL. It runs a chain of three models: speech-to-text, a language model and text-to-speech.
• Speech-to-text: Parakeet TDT 0.6B v3. Text-to-speech: Qwen3-TTS 1.7B.
• Language models: HF's default for Macs (Qwen3-4B-Instruct-2507, 4-bit, on Apple's MLX), the same Qwen3-4B on llama.cpp's llama-server, and Ministral 3 3B on llama-server. Both llama.cpp models were 4-bit (Q4_K_M) with thinking off.
• Gemma 4 E4B, which understands audio directly, so it skips the speech-to-text step. It ran on llama-server with its audio encoder.

The test is the one from the first post: the same system prompt, the same tools and the same four Spanish prompts streamed in real time, three sessions each. The prompts are a plain chat, a quick tool (what time is it), two tools in a row (check the build, then open an issue) and a tool that takes 3 seconds to answer (ask the agent). First audio is the time from the end of the user's speech to the first audio back. Right tools counts the 9 tool turns where the model called what we expected. Grok Voice is in the chart as the cloud reference.

Results

* Fast because in 2 of the 3 sessions it answered without calling the tool.

Gemma 4 E4B: as fast as the cloud, but it won't use its tools

Gemma was the surprise. It started talking in 1.3 seconds, level with Grok, and then got every tool turn wrong. Asked for the time, it made one up ("a quarter past three"). Asked to check the build, it said "let me check the build" and never did.

To find out whether that was the model or our setup, we sent it the same prompt and tools straight through llama-server:

At this size, Gemma only uses tools reliably when it's allowed to think, and thinking costs 2 to 9 seconds before the first word. With a very short prompt it did call the clock tool. With a realistic phone-assistant prompt it stopped. For a voice that has to look things up, that rules it out for now.

The slow setup was slow because of its engine

HF speech-to-speech with its Mac defaults took 5.4 seconds just to answer a chat. Its logs showed where the time went: speech-to-text took 80 ms, text-to-speech 0.24 s, and the language model 3.3 s. With the slow tool, the model passed a malformed argument, the server dropped the call and it never spoke.

So we took the language model out and served it with llama-server instead. Before doing the full voice runs, we checked five small open-weight models on text alone (the four prompts, three runs each, first response only, thinking off):

The same Qwen3-4B that took 3.3 s inside HF's default setup starts answering in 0.39 s on llama-server. The model was never the bottleneck. With Ministral 3 3B or Qwen3-4B behind it, the full voice chain answers a chat in about 2.4 s and a quick tool in about 1.7 s.

What still gets in the way

• Silence during slow tools. Ministral called the 3-second tool correctly but said nothing until it returned. The cloud models say "let me check" first. A short filler line before slow tools would fix most of this.
• Mishearing. Parakeet heard a la gente ("to the people") instead of al agente ("to the agent"). That's one wrong syllable, and on a call it changes what the voice does.
• One GPU for everything. The language model and the text-to-speech model share the Mac's GPU, so the first audio from text-to-speech ranged from 0.3 to 2 seconds depending on what else was running.
• One call at a time, using about 6–7 GB of memory.

So, can it?

Almost. In late 2026, a 3–4B model on a laptop can hold a phone call and use tools correctly most of the time. It takes about 2 seconds to reply where the cloud takes about 1.3, and about 5 when it has to chain two tools. The setup that worked best was:

• HF speech-to-speech as the server, with Parakeet for speech-to-text and Qwen3-TTS for the voice;
• Ministral 3 3B or Qwen3-4B-Instruct-2507, 4-bit, on llama-server with thinking off.

For calls about things that matter, like a production outage or a security alert, we'd still use a cloud model: Grok Voice for tool-heavy calls, or Gemini Live to start for free. Local is a good choice if privacy or cost matters more to you than the last second.

To try it with self-hosted agentring, start the local server and pick it as the voice: agentring call "…" --ai --provider local. agentring voice bench --provider local runs this same test on your own machine (see the voice docs for the full setup).

Caveats

This is a small test: four prompts in Spanish, three sessions per setup, spoken by the macOS synthetic voice, on one Mac. The HF server crashed once during the Ministral run, so we reran its slow-tool prompt on its own (3/3). Local models and runtimes change every few weeks, and a faster Mac or a dedicated GPU would move every number here. Treat the results as a snapshot and run your own prompts if the choice matters to you.