STT, LLM and TTS
from one command, on your GPU.

A barge-in watcher runs alongside the pipeline: talk over the reply and generation and playback stop. The server decides that, because it hears clean audio — echo cancellation happens on the device, in the browser or in the terminal client, so it works on laptop speakers without headphones.

$ frun bench --audio order --callers 1,4,8,12,16 --target-ms 1500
   1 caller(s):  median   824 ms   worst   832 ms
  12 caller(s):  median  1440 ms   worst  3636 ms
Speech is where callers wait: stt went from 220 to 4553 ms from 1 to 16 caller(s),
while the LLM held (llm first token 49 ms).
Up to 12 caller(s) stayed under 1500 ms on this setup.Copy

Tested on an RTX 3090 with vLLM and SGLang. Model server docs →

agent = Agent(
    prompt="You are the order line for ShopKart.",
    greeting="Hi, this is ShopKart. Which order can I help you with?",
    llm=LLM("vllm:hf:Qwen/Qwen2.5-7B-Instruct-AWQ"),   # or "sglang:hf:..."
)Copy

Measured on an RTX 4090: about 0.8–0.9 s from the end of a turn to the first audio, the LLM's first token in 23–55 ms. vLLM and SGLang docs →

from fusion_runtime import Agent, LLM, tool

@tool
async def order_status(order_id: str) -> dict:
    """Look up where an order is and when it will arrive.

    Args:
        order_id: The order number, as the caller reads it out.
    """
    return await orders.lookup(order_id)

agent = Agent(prompt="...", llm=LLM("vllm:hf:Qwen/Qwen2.5-7B-Instruct-AWQ"), tools=[order_status])Copy

Tools need a model server that can call them: vLLM, SGLang, llama-server --jinja or a hosted API. The in-process llama.cpp model can't yet, and frun up says so at startup. Tools docs →

Interruptions fired on six of six attempts, and correctly ignored 192 ms of speech over the agent. The point isn't that a GPU beats a laptop — it's that a 7 B model answers as fast as a 0.5 B did on an 8 GB MacBook Air, because the pause you configure and the first sentence of speech dominate, not the model. Every figure comes from the runtime's own per-turn telemetry, so you can reproduce them on your own hardware.

Several callers at once

Same 3090, real WebSocket sessions, three turns each, 26 September 2026. The server's own response time, turn end to first audio:

In-process, one llama.cpp context decodes one reply at a time, so four callers is the ceiling. vLLM and SGLang batch every caller together: the model's first token stayed between 36 and 72 ms from one caller to sixteen. What grows now is speech — Kokoro's first audio reaches 1.5–2 s at sixteen callers — so batching speech is what comes next. With every caller asking about an order, vLLM made the lookup in 122 of 123 turns; SGLang in 109, skipping only questions it had already looked up. Qwen2.5-7B-Instruct-AWQ, 60% of the card for the model.

<script src="https://your-server/fusion-runtime.js"></script>
<button id="talk"></button>
<script>FusionRuntime.attach({ button: "#talk" });</script>Copy

The browser's own echo canceller handles the speaker, the microphone is resampled off the main thread, and replies are scheduled slightly ahead of real time so network jitter doesn't leave gaps. The page never holds an API key — your backend mints it a token that works once. Microphones need https://, so a deployment needs TLS and wss://. Browser client docs →