Abstract
MOSS-VL is an open vision-language model family enabling real-time interaction by attending to vision via gated cross-attention during generation, using a synthesized interaction corpus and staged curriculum to achieve strong streaming performance with reduced time-to-first-token latency.
We present MOSS-VL, an open vision-language model family that treats real-time interaction -- perceiving while it speaks -- as a first-class capability. It is co-designed across the stack: the language decoder attends to vision only through gated cross-attention, so the model can naturally see incoming frames while generating; a synthesized interaction corpus supervises when to speak, when to stay silent, and when to revise; and a staged curriculum concentrates all real-time-specific training in one light final stage over a strong offline foundation. Offline, MOSS-VL-Instruct is competitive at comparable scale and leads temporal-reasoning video sets. Across four streaming benchmarks, MOSS-VL-Realtime posts the best average on three (second on the fourth) among open-source streaming models, sweeping the three subsets that squarely test proactive behavior -- 66.0 vs. 37.5 for the best baseline on OmniMMI Proactive Alerting. With 11.3B parameters but visual tokens outside the decoded sequence, MOSS-VL widens its time-to-first-token advantage over same-backbone Qwen3-VL-8B from 2.8x to 5.1x as visual context grows. We release all five checkpoints, the training curriculum, and the real-time inference code at https://github.com/OpenMOSS/MOSS-VL.
Community
MOSS-VL treats real-time interaction — perceiving while it speaks — as a first-class capability: vision enters the decoder only through gated cross-attention, so the model keeps watching incoming frames while it generates. Among open-source streaming models it posts the best average on 3 of 4 streaming benchmarks and sweeps all three proactive subsets (66.0 vs. 37.5 on OmniMMI Proactive Alerting), while widening its TTFT advantage over same-backbone Qwen3-VL-8B from 2.8× to 5.1× as visual context grows. All five checkpoints, the training curriculum, and the real-time inference code are released.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model (2026)
- StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMs (2026)
- TuringViT: Making SOTA Vision Transformers Accessible to All (2026)
- Qwen-MusicAVQA-7B: A Multimodal Model for Music Audio-Visual QA (2026)
- ReToken: One Token to Improve Vision-Language Models for Visual Retrieval (2026)
- video-SALMONN-R3: Learning to ReWatch, ReAsk, and ReAnswer for Efficient Video Understanding (2026)
- Think at 5 Hz, Act at 20 Hz: Asynchronous Fast-Slow Vision-Language-Action Inference for Closed-Loop Driving (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2608.15045 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 7
OpenMOSS-Team/MOSS-VL-Instruct-0708
Datasets citing this paper 0
No dataset linking this paper