Agent Lightning v1.0: A 3,500-Line Lightweight Agentic RL Framework for Training Agents with Real Harnesses
Training AI agents with reinforcement learning can be challenging because their tools, context, and decision-making are managed by complex frameworks. Agent Lightning connects existing agents to RL training, making it easier to improve them without rebuilding…
Marked important, but published more than 30 days ago, so it moved down to this band.
On this page12 sections
The editors' notes below — why it matters, the context, the learning path — are written in Chinese.
Record
The signal
#At a glance Harnessed Agentic RL: Microsoft Research Asia introduces a training paradigm in which the same agent harness used in deployment participates directly in reinforcement learning, removing the need to reimplement the agent inside the training framework. Lightweight by design: Agent Lightning v1.0 delivers a complete agent RL control plane in roughly 3,500 lines of code. Native Kubernetes support: agents run as standard Kubernetes jobs on self-managed clusters, cloud Kubernetes, or local infrastructure, with no dependency on paid commercial sandbox services. Data-efficient training recipe: an end-to-end coding agent pipeline raised Qwen3.5-9B from 41.8% to 56.4% Pass@1 on SWE-bench Verified, a 14.6 percentage point gain, using only about 6,000 training samples based on open sourced dataset. AI agents have evolved from single models to complex full-stack systems built from models, tools, and execution environments. Their capabilities increasingly depend on the agent harness that coordinates them from outside the model. Reinforcement learning (RL) is an approach where AI systems learn through trial and error, guided by rewards and penalties for their actions. RL can make those agents better, but most agent RL systems require developers to reimplement the agent inside the training framework. That is costly, and it means the agent being trained is not quite the agent that gets deployed. To address this, researchers at Microsoft Research Asia have introduced the Harnessed Agentic RL training paradigm and open-sourced a fully rebuilt Agent Lightning v1.0 (opens in new ta…
Why now
Why it matters
#多数智能体强化学习要求把线上 harness 在训练框架里重写一遍，既费工又会让训练对象与部署对象产生偏差。Agent Lightning v1.0 用 LLM 代理把真实 harness 直接接进 RL，并用约 3500 行代码、Kubernetes 原生支持和约 6000 条样本给出了可复现的端到端例子。对已有智能体产品的团队，这意味着训练管线可以复用现有执行栈，而不必另起一套。
Background
Technical context
#Harnessed Agentic RL 的核心变化是交互循环的归属：传统 agentic RL 由训练框架掌管环境与智能体循环，而这里由 harness 掌管，训练系统只观察模型请求与响应。因此样本边界不再天然对齐 rollout，需要在 rollout 级做优势计算与损失归一化。LLM 代理则负责把每次模型调用与所属 rollout 关联，并记录训练所需的 log probability。Collocated Async RL 让 rollout 与更新分时共用 GPU，以缓解同步等待造成的空转。
Audience
Who should care
#Next
Learning path
#- 先读 Agent Lightning v1.0 技术报告中的 Harnessed Agentic RL 定义，弄清训练框架与 harness 各自负责哪一段循环。
- 对照 verl 等传统 agentic RL 系统的 ReAct 式轨迹假设，理解为什么真实 harness 会让一次 rollout 拆成多个样本。
- 在小型 harness 上把模型端点指向 Agent Lightning 代理，跑通一次 rollout 记录，确认 prompt、response 和 log probability 能被正确归集。
- 用 rollout 级与样本级两种方式分别计算优势和归一化，比较验证奖励与策略熵，验证原文结论是否在自己的任务上成立。
Network
Where it sits in the network
#How this signal connects to neighbouring technologies, skills and background knowledge. Follow any link to keep exploring.
Further thought
Follow-up questions
#- 把现有 harness 的模型端点切到 Agent Lightning 代理后，哪些调用不会被记录或归集会出错？
- rollout 级优势与归一化在非编码类智能体任务上是否同样带来稳定收益？
- Collocated Async RL 在 rollout 时长分布更极端时，2 倍加速还能保留多少？
Publication record
Featured in
#The briefings that carried this signal, newest first; the last row is the weekly review of its week. The tag is the section it ran in.
技术简报 - 2026-10-20
Immediate attentionWeekly review · Oct 5 – Oct 11
Worth tracking
Sources