v0.32.0
v0.32.0 Highlights This release features 814 commits from 363 contributors (121 new)! Structured decisions and generic parsers: the new POST /v1/systemone endpoint answers choice, noul and score questions over a structured state (#59299), is served by default…
Marked important by the editors and published within 30 days.
On this page13 sections
The editors' notes below — why it matters, the context, the learning path — are written in Chinese.
Release line
Release line
#This is the latest release on the line; the earlier ones it follows are below.
- 2026-06-30vLLM v0.24.0
- 2026-07-11vLLM v0.25.0
- 2026-07-25vLLM v0.26.0
- 2026-08-10vLLM v0.27.0
- 2026-09-22v0.30.0
- 2026-10-18v0.32.0CurrentLatest
v0.30.0 之后站内收录的下一个 vLLM 大版本（v0.31 未收录）
Record
The signal
#v0.32.0 Highlights This release features 814 commits from 363 contributors (121 new)! Structured decisions and generic parsers: the new POST /v1/systemone endpoint answers choice, noul and score questions over a structured state (#59299), is served by default (#60677, #60696), and also runs on Cloudflare/clef-omni (#61064), LiquidAI/d1-3B (#60676) and DiffusionGemma (#60466). The new hf tool and reasoning parser (--tool-call-parser hf, --reasoning-parser hf) reads the checkpoint's response_template, so models that ship it need no model-specific parser (#58604, #58602), in both the Python and Rust frontends (#59005). Low-precision KV cache everywhere: NVFP4 KV cache now runs on SM8x and SM12x (#46963), SM90 (#60620) and with XQA decode on SM12x (#60452), and on MiniMax-M3 sparse attention on SM100 (#59300); GLM-5.x defaults to an FP8 KV cache on SM100, halving KV memory per rank with 2.3-5.5% more throughput at a 7-11% TTFT cost in the PR's benchmark (#60140, #60281); fp8_ds_mla on SM90 gives GLM-5.3-Flash up to 31% more output throughput (#59246); ROCm gains the 4-bit ultraquant_4bit KV cache with about 2x the density of FP8 (#57057). Model Runner V2: a GPU n-gram speculator with variable-length drafts (#40704), Elastic EP (#53934), pipeline parallelism with prefill context parallelism for MLA models (#59139), a stock torch.compile mode (#59079), and MRV2 is now the default on ROCm with the model-specific MRV1 exceptions removed (#56360). Faster startup and serving: vllm download-kernels installs FlashInfer's precompiled kernels (#58765); vllm preload --preload-autotune ke…
Why now
Why it matters
#vLLM 是主流的开源推理引擎，这次更新同时改了解析接口和 KV 缓存精度默认值，直接关系到部署时的显存占用与首 token 延迟。GLM-5.x 在 SM100 上默认改用 FP8 KV 缓存，意味着升级后行为会变化，不实测就可能遇到 TTFT 回退。
Background
Technical context
#KV 缓存保存生成新 token 时回看上下文所需的中间结果，压缩它的精度能在同样显存下容纳更长上下文或更多并发，但会带来精度损失。工具调用与推理解析器负责把模型输出拆成结构化调用或思维内容；vLLM 此前多按模型写专用解析器，这次改为读取 checkpoint 自带的模板。MRV2 是 vLLM 新一代模型运行器，影响批处理与并行调度路径。
Audience
Who should care
#Next
Learning path
#- 先读 KV 缓存与长上下文推理的说明，弄清为什么压缩 KV 精度能省显存、代价落在哪里。
- 对照发布说明中的 PR 编号，逐个确认自己使用的硬件与模型是否在受影响范围内。
- 在自己的负载上对比升级前后的 TTFT、输出吞吐和显存占用，重点验证 GLM-5.x 的 FP8 KV 默认值。
- 如果模型自带 response_template，试跑 hf 解析器并与现有专用解析器的输出比对。
Network
Where it sits in the network
#How this signal connects to neighbouring technologies, skills and background knowledge. Follow any link to keep exploring.
Further thought
Follow-up questions
#- GLM-5.x 默认 FP8 KV 缓存后，在我们的平均与长上下文负载上 TTFT 实际增加多少？
- 通用 hf 解析器对我们现有模型的工具调用成功率与专用解析器相比是否有差异？
- ROCm 默认切到 MRV2 后，原有模型的行为和性能有没有回归？
Publication record
Featured in
#The briefings that carried this signal, newest first; the last row is the weekly review of its week. The tag is the section it ran in.
技术简报 - 2026-10-20
Immediate attentionWeekly review · Oct 12 – Oct 18
Immediate attention
Sources