Topic
Multimodal
Interfaces that combine text, images, speech or browser actions.
Follow this topic
On this site · ElsewhereTopic graph
16 nodes · 43 links- Entries in this topic
- Neighbours outside the topic
- Builds on · Successor · Extends
- Prerequisite · Uses
- Explains · Supports
Skills and knowledge entries are written in Chinese, as are signals from Chinese sources.
Published signals
17 signalsWelcome Muse Spark Open: open-weight frontier multimodal mixture-of-experts model from Meta
Welcome Muse Spark Open: open-weight frontier multimodal mixture-of-experts model from Meta
何恺明团队 VISTA：让模型回看原始画面，打满 ARC-AGI-3
MIT 何恺明团队的 VISTA 是一套视觉原生的智能体外壳：环境返回的每一帧都原样存档，模型推理时可以按需调回、放大、并排对比，基础模型不用重新训练。配上 Claude Opus 5.0，ARC-AGI-3 的 25 个公开游戏全部通关，相对人类动作效率得满分 100，比人类首次游玩的基线少用 57.4% 的动作。
EmbeddingGemma 2: an open, lightweight multimodal embedding model
EmbeddingGemma 2: an open, lightweight multimodal embedding model
Holo4: powering generalist computer-use agents
H company released Holo4, a family of open-weight agentic models for computer use across GUIs, code, MCP and APIs: Holo4-27B (dense), Holo4-35B-A3B (MoE) and Holotron4 Nano (30B-A3B). Holo4-27B scores 61.7% on OSWorld 2.0.
Accelerating vision-language models with LFM2.5-VL-DSpark
Accelerating vision-language models with LFM2.5-VL-DSpark
Introducing Gemini 3.8 Live with Live Avatar
Introducing Gemini 3.8 Live with Live Avatar
Muse Glimmer 开源：30B 压进消费级显卡，报错不停下
Meta 发布 Muse Glimmer：300 亿参数、Apache 2.0 开放权重，专为常驻本地的智能体工作流设计。4 位动态量化把 55GB 以上的显存需求压到 17 至 20GB，配合 DFlash 推测解码在 M5 Max 与 RTX 5090 上把吞吐提到最多 3.1 倍。真正值得注意的不是尺寸，而是它把「工具调用失败之后怎么办」写进了训练目标：API 或终端命令报错时，模型诊断故障并另找路径，而不是终止执行。
Qwen 3.8 27B is excellent, but it defaults to wildly overthinking things
Hands-on evaluation of Qwen 3.8 27B on a 128GB M5 Max and an NVIDIA DGX Spark: the shipped xhigh reasoning default burns tokens on trivial prompts, vision bounding boxes are strong, and throughput of 15–30 tokens per second is the real catch.
Qwen3.8-27B 开源：消费级显卡跑得动，尺子却是自己命名的
270 亿参数、原生多模态、262K 上下文可外推到 100 万，Apache 2.0，量化后一张 24GB 消费级显卡装得下——这一代 Qwen 把「本地跑得动的智能体」这条线又往前推了一格。榜单确实好看，但领先幅度最大的那一项跑在一把以厂商自己命名的基准上，而所有对比都由厂商自己完成。
Putting sign language AI into users' hands
Google DeepMind introduced SL2T, a massively multilingual sign-language-to-text model now shipping in Gboard dictation and Live Transcribe on Pixel 11, starting with ASL to English.
AMIE, our research medical AI system, demonstrates real-time clinical video consultation capabilities in a first-of-its-kind study
Google Research and DeepMind advanced AMIE to real-time clinical video consultations using a multi-agent architecture on Gemini and Project Astra, evaluated favorably against primary care physicians in a randomized simulated study.
Meta is back with Muse Glimmer: local, agentic, multimodal, and open source
Meta released Muse Glimmer, a 30B multimodal model distilled from Muse under Apache 2.0, designed for local agentic use, with day-0 support across transformers, llama.cpp and vLLM.
Gemini Robotics 2 brings whole body intelligence to robots
Gemini Robotics 2 is a family of three models — a vision-language-action model for whole-body humanoid control, an embodied reasoning model (ER 2) that orchestrates tools and tracks task progress, and an on-device VLA that adapts to a new robot body in hours.
Fine-tune video and image models at scale with NVIDIA NeMo Automodel and Diffusers
NVIDIA and Hugging Face ship a NeMo Automodel integration that brings production-grade distributed diffusion training to any Diffusers-format model on the Hub — no checkpoint conversion, no model rewrites, scaling from one GPU to hundreds, Apache 2.0.
Welcome Inkling by Thinking Machines
Thinking Machines releases Inkling, an open-weight ~1T-parameter (41B active) Mixture-of-Experts multimodal model with a 1M context window, trained on 45T tokens of text, images, audio and video, with day-one support across Transformers, vLLM, SGLang, llama.cpp and Unsloth.
Introducing Real World VoiceEQ: Measuring the human quality of voice AI
Hume AI launches Real World VoiceEQ, a benchmark evaluating the human quality of voice interaction across 40+ models, 15+ dimensions and 60+ metrics, grounded in over 1 million human ratings.
Introducing GPT-Live
A new generation of voice models for natural human-AI interaction, now powering ChatGPT Voice.