We introduce Haru-4B, a full-duplex continuous spoken dialogue foundation model capable of listening and speaking simultaneously. (View Highlight)
Haru-4B focuses on natural human-AI interaction, learning conversational behaviors such as backchannels, interruptions, and turn-taking from large-scale real-world speech, rather than relying on synthetic conversations or explicitly scripted turn-taking behavior. Our goal is to make interacting with AI feel more like a natural human conversation, unlike interacting with agentic assistants. (View Highlight)
We are releasing Haru-4B as a pretrained base model, providing a foundation for further fine-tuning and research on real-time spoken interaction. The model weights are available on Hugging Face, and the code is available on GitHub. (View Highlight)
Haru-4B at a Glance
• Ultra-Low-Latency Interaction: About 65 ms end-to-end latency on a MacBook.
• 7.5 Hz Native Full-Duplex Speech-to-Speech: Simultaneous processing of incoming and outgoing speech every 133 ms through synchronized audio-text streams.
• Single-Step Continuous Audio Generation: A lightweight MLP head with one-step MeanFlow sampling, eliminating the need for an additional acoustic Transformer.
• 40 Million Hours of Real Speech: Large-scale multilingual pretraining without incorporating LLM- or TTS-generated turn-based synthetic dialogues.
• Multilingual Speech Capabilities: Training data spanning over 160 languages, with a primary focus on nine languages: English, Spanish, Portuguese, German, French, Chinese, Japanese, Russian, and Korean.
• Long-Form Full-Duplex Training: Trained on full-duplex conversations up to 60 minutes long, enabling the model to learn extended conversational dynamics and long-range interaction patterns.
• Controllable Character and Voice: Supports character customization through text prompts and reference audio, enabling control over persona, speaking style, and voice characteristics.
• Open Pretrained Base Model: We are releasing model weights (Hugging Face), code (GitHub), and demos to support further researc (View Highlight)
Haru-4B is built on Qwen3.5-4B 1 and designed to process incoming and outgoing speech simultaneously within a unified language model. Its architecture combines low-rate continuous audio representations, synchronized multi-stream processing, and efficient single-step generation. (View Highlight)
Native full-duplex multi-stream modeling. Haru processes three synchronized streams: incoming user audio, outgoing model audio, and model text 23. Their embeddings are combined through element-wise addition and processed by a shared language model. This enables simultaneous listening and speaking 4 without interleaving the three streams token by token 56. Compared with interleaving, this reduces the context length to one third while keeping the text stream uninterrupted. (View Highlight)
Continuous speech tokens at 7.5 Hz. Most spoken language models rely on discrete audio tokens, often produced through Residual Vector Quantization (RVQ) 78, where each audio frame is represented by multiple discrete codebook indices. In contrast, Haru uses 64-dimensional continuous acoustic latents at just 7.5 Hz 9, representing approximately 133 ms of speech per token. By avoiding discrete codebooks and operating at a low token rate, Haru reduces the number of sequential LLM modeling steps required for training and inference. (View Highlight)
Single-step continuous speech generation. Haru generates continuous acoustic tokens directly from language-model hidden states using a lightweight Multi-Layer Perceptron (MLP) within a next-latent diffusion framework 101112. We primarily train the generation head with Flow Matching 13, followed by MeanFlow distillation 1415 for training and inference efficiency. At inference time, each acoustic token is generated in a single sampling step, without iterative flow sampling or an additional acoustic Transformer. (View Highlight)
Chunk-level text alignment. Instead of aligning text and audio at the individual token level, Haru represents text in chunks 3, followed by padding positions and a dedicated end-of-padding token for predicting speech initiation. This accommodates the different temporal resolutions of text and audio while maintaining synchronization across streams and mitigating catastrophic forgetting of pretrained language-model knowledge. (View Highlight)
We believe large-scale real-world speech is essential for learning the dynamics of natural interaction. Unlike synthetic conversations, real recordings contain spontaneous interruptions, hesitations, backchannels, overlapping speech, and irregular turn boundaries. (View Highlight)
Haru-4B is pretrained using over 40 million hours of speech, covering more than 160 languages, with nine primary languages. To learn natural human conversational patterns, we intentionally exclude LLM-generated dialogues and TTS-synthesized speech from our training mixture. (View Highlight)
Beyond separated speech. Separated speech makes full-duplex dialogue pretraining scalable, but separation errors and noise can remain, especially in overlapping speech. To further improve natural full-duplex interaction, we are fine-tuning Haru on native full-duplex two-channel conversational recordings. We are also developing full-duplex dialogue and speech generation models to create natural synthetic full-duplex data that enhances response quality, conversational diversity, and character-specific dialogue capabilities. (View Highlight)
Scaling up to 100B+ parameters. Although Haru-4B demonstrates efficient full-duplex speech modeling, its relatively small language-model backbone limits its response quality. We are already pretraining larger full-duplex models with over 100 billion parameters, aiming to combine natural conversational timing with stronger language understanding and knowledge. (View Highlight)
Toward omni-modal full-duplex interaction. Human interaction involves more than speech. Visual context, facial expressions, and nonverbal cues all contribute to natural conversation. We are extending our architecture to take real-time video input, moving toward omni-modal full-duplex dialogue models that can see, listen, and speak continuously. (View Highlight)