Topic
On-device AI
Capable models running locally on laptops, phones or edge devices.
Follow this topic
On this site · ElsewhereTopic graph
16 nodes · 58 links- Entries in this topic
- Neighbours outside the topic
- Builds on · Successor · Extends
- Prerequisite · Uses
- Explains · Supports
- Related
Skills and knowledge entries are written in Chinese, as are signals from Chinese sources.
Published signals
22 signalsAnyLanguageModel 1.0: One Year of Local Models on Apple Platforms
AnyLanguageModel 1.0: One Year of Local Models on Apple Platforms
MPS in PyTorch 2.15
MPS in PyTorch 2.15
EmbeddingGemma 2: an open, lightweight multimodal embedding model
EmbeddingGemma 2: an open, lightweight multimodal embedding model
Ollama v0.35.1: Clef decision models
Clef decision models Ollama now supports Clef and Clef Flash, Cloudflare's new open-source decision models, through /v1/systemone. Clef (27B) and Clef Flash (9B) are multimodal: requests can now include images alongside the text state, shared by all questions…
ADK Kotlin 1.0：智能体逻辑不必再绕回 Python
Google 发布 Kotlin 版智能体开发套件 1.0，功能上与 Python 版和 Java 版对齐，可用于 Android、JVM 和服务端。几处设计值得留意：工具的描述在编译期由注解生成，不靠运行时反射；敏感工具可以声明为执行前必须人工确认；技能存放在 SKILL.md 里按需加载。端侧推理支持 LiteRT-LM 与仍在测试阶段的 ML Kit。
Colibrì：25GB 内存跑 744B 模型，权重按需从硬盘读
开源推理引擎 Colibrì 把显存、内存和固态硬盘当成同一套存储层级：744B 的 GLM-5.2 每个 token 只激活约 40B 参数，于是常驻内存的只有约 9.9GB 的稠密部分，19,456 个路由专家留在硬盘上按需读取。它不保证速度，只保证不悄悄改动模型：25GB 的开发机冷缓存时每秒 0.05 到 0.1 个 token，128GB 的纯 CPU 台式机约 1.8 个。
Accelerating vision-language models with LFM2.5-VL-DSpark
Accelerating vision-language models with LFM2.5-VL-DSpark
模型之间只传 17 比特：一座桥补上一半差距
大模型生成一个 token 时内部构建约两兆字节的状态，最后只吐出 17 个比特——而所有多模型协作系统（子智能体、模型委员会、路由调度）都建立在这 17 个比特之上。Mostik 让 753B 的 GLM-5.2 把隐藏状态通过一座小桥直接交给 4B 的 Qwen-3.5，全程不产生文本、两边权重完全冻结、桥是唯一被训练的部分。结果：4B 补上与 753B 约 50% 的差距、自身准确率 +25%、难题子集上 2 倍，而大模型侧推理成本降到约二十分之一。技术细节尚未公开，团队以比赛未结束为由拒绝披露。
FreeToken：消费级机器不是缩水的服务器，是异构资源
伯克利与 MIT 开源 FreeToken，一个面向消费级硬件的 MoE 推理引擎，共同作者包括 Databricks 联合创始人 Matei Zaharia 与 Ion Stoica，以及 Song Han、Kurt Keutzer。它的前提是换一个看法：个人电脑不是资源受限的数据中心节点，而是可弹性调度的异构资源——缓存未命中时不让 GPU 停下来等，而是按实时互连吞吐在 CPU 与 GPU 之间分配 token 计算。论文报告在 8GB 显存的 RTX 4060 笔记本上跑 Qwen3.6-35B 约每秒 39 个 token，同批 MoE 模型上解码快 3 到 4 倍、预填充快 6 到 30 倍。而社区正在争论的，恰好是这个基线是不是经过手工调优。
Muse Glimmer 开源：30B 压进消费级显卡，报错不停下
Meta 发布 Muse Glimmer：300 亿参数、Apache 2.0 开放权重，专为常驻本地的智能体工作流设计。4 位动态量化把 55GB 以上的显存需求压到 17 至 20GB，配合 DFlash 推测解码在 M5 Max 与 RTX 5090 上把吞吐提到最多 3.1 倍。真正值得注意的不是尺寸，而是它把「工具调用失败之后怎么办」写进了训练目标：API 或终端命令报错时，模型诊断故障并另找路径，而不是终止执行。
Qwen 3.8 27B is excellent, but it defaults to wildly overthinking things
Hands-on evaluation of Qwen 3.8 27B on a 128GB M5 Max and an NVIDIA DGX Spark: the shipped xhigh reasoning default burns tokens on trivial prompts, vision bounding boxes are strong, and throughput of 15–30 tokens per second is the real catch.
Qwen3.8-27B 开源：消费级显卡跑得动，尺子却是自己命名的
270 亿参数、原生多模态、262K 上下文可外推到 100 万，Apache 2.0，量化后一张 24GB 消费级显卡装得下——这一代 Qwen 把「本地跑得动的智能体」这条线又往前推了一格。榜单确实好看，但领先幅度最大的那一项跑在一把以厂商自己命名的基准上，而所有对比都由厂商自己完成。
State of Open Models: Summer 2026 Observations
Hugging Face's mid-2026 ecosystem report on the Hub: model size distribution by lab, licensing, derivative counts, download concentration, and the first measurement of agent traffic.
Meta 重回开放权重：一封长文、一个独立董事会，和一组本地部署的数字
Muse Glimmer 以 Apache 2.0 开放权重的同时，Meta 宣布重回开放路线：扎克伯格发表长文反对能力集中，公司设立独立董事会逐项审核每次模型发布，并披露了让本地部署成立的量化与推测解码数字。Muse Spark 1.2 的权重据称随后开放。
Putting sign language AI into users' hands
Google DeepMind introduced SL2T, a massively multilingual sign-language-to-text model now shipping in Gboard dictation and Live Transcribe on Pixel 11, starting with ASL to English.
Meta is back with Muse Glimmer: local, agentic, multimodal, and open source
Meta released Muse Glimmer, a 30B multimodal model distilled from Muse under Apache 2.0, designed for local agentic use, with day-0 support across transformers, llama.cpp and vLLM.
Deploy local agents everywhere with LFM2.5-2.6B
LFM2.5-2.6B is built to power capable agents entirely on-device. It supports tool calling and multi-step workflows while staying small and fast enough for everyday hardware, from laptops to phones.
LFM2.5-Encoders for Fast Long-Context Inference on CPU
Two new encoder models, LFM2.5-Encoder-230M and 350M: they match larger encoders on GLUE, SuperGLUE and multilingual tasks, handle 8,192-token context, and run about 3.7x faster than ModernBERT-base on CPU at long context.
Gemini Robotics 2 brings whole body intelligence to robots
Gemini Robotics 2 is a family of three models — a vision-language-action model for whole-body humanoid control, an embodied reasoning model (ER 2) that orchestrates tools and tracks task progress, and an on-device VLA that adapts to a new robot body in hours.
Ollama v0.32.4
Ollama v0.32.4 adds Laguna support on Apple GPUs via the MLX engine, quantizes speculative-decoding draft output heads at the requested type, and fixes Qwen3 MoE decoding for differently-quantized experts.
Ollama v0.32.0
Ollama v0.32.0 turns the bare ollama command into an interactive agent for chatting, coding, web search, and delegating work — the local runtime is repositioning itself as an agent workbench entry point.
Ollama v0.31.2
Ollama, the local LLM runtime for running open models on personal and edge hardware, shipped stable release v0.31.2.