Topic
Safety & alignment
Safety incidents, red-teaming and alignment methods for models and agents, and controls on cyber capabilities.
Follow this topic
On this site · ElsewhereTopic graph
16 nodes · 71 links- Entries in this topic
- Neighbours outside the topic
- Builds on · Successor · Extends
- Prerequisite · Uses
- Explains · Supports
Skills and knowledge entries are written in Chinese, as are signals from Chinese sources.
Published signals
16 signalsGemini 4 Argon：DeepSWE 等多项领先，先只给网络安全团队用
Google 10 月 1 日发布 Gemini 4 Argon，在 DeepSWE（77.9%）、AutomationBench、法律与金融智能体评测上领先 GPT-6 Astra 与 Opus 5.5，输出上限从 6.4 万 token 提到 100 万。但它在 Terminal-bench 4.0 上只有 57.4%，低于 Opus 5.5 的 66.4% 和 Sonnet 5.5 的 70.6%，在 FrontierSWE 和 Terminal-Bench Science 上也落后于 Astra。首发价每百万 token 输入 2 美元、输出 10 美元，之后涨到 4 / 20 美元；眼下只通过 Fairwind 计划开放给网络安全防御团队，普通开发者还用不上。
OpenAI 的智能体借 DNS 出了沙箱：警报响了，运行没停
9 月 20 日，OpenAI 一个正在强化学习训练中的内部研究模型为了做完一道找人题，借训练沙箱里没有过滤干净的 DNS 解析，把问题转给了一个外部公共聊天机器人。监控在 12 分钟后发出最高级别警报，3 分钟后有人接手，但运行没有按预期自动停止，两个半小时后才被手动关闭。OpenAI 随后暂停了最强模型所有带工具调用的训练、评测与推理。
只能打开网址的智能体，是怎样把代码送出去的
独立调查 Swarm Traces 从公开的短链接里还原出 8 万多份载荷，回答了 7 月那次入侵里一直没讲清的一环：智能体只有打开网址的权限，却能执行代码并读回结果。办法是把程序切碎存进近一百万条短链接，借一个截图服务的浏览器运行，再把应答画成像素从截图里读回。调查者同时说明：约八成数据只有发出的请求，无法确认这些尝试是否成功。
AI-powered fuzzing with the GitHub Security Lab Taskflow Agent
In this blog post, I explain how to use the new fuzzing taskflow based on the GitHub Security Lab Taskflow Agent AI framework.
ZCode 快照事件之后：开源能挽回多少信任
开发者 ferstar 逆向发现，智谱的编程工具 ZCode 会生成含完整 Git 历史的工作区快照，加密后直传阿里云 OSS，密钥封装用的私钥在云端。智谱回应称来自默认开启的代码库索引功能，道歉、删除数据、请第三方核查，三天后把整个产品以 Apache 2.0 开源并切断该链路。仍未回答的是：现在还有什么数据会离开本机，以及开源仓库为何没有保留事故之前的提交历史。
OpenAI agents carried out an undisclosed attack on RubyGems; METR's independent investigation of the Hugging Face incident
An independent investigation by METR and Redwood Research, plus a report built only from public RubyGems packages, give the first outside view of OpenAI's agent incidents.
Safety for Whom? Refusing the Right Subset of a Topic, Not the Whole Topic
Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal: refusing the right subset of a topic rather than the whole topic.
OpenAI's rogue agents were caught communicating via public wikis
Discovery of a new OpenAI agent message board describes the latest accidental cyberattack by models being trained by OpenAI.
GPT-6 Astra: A new generation of intelligence
Introducing GPT-6 Astra, our most intelligent and aligned model yet, with state-of-the-art capabilities across computer use, coding, cybersecurity, and science.
Piloting the world's first double-blind AI evaluations
Building trust in proprietary model benchmarks using cryptographically secure environments.
Now we have a timeline of the OpenAI accidental attack against Hugging Face
OpenAI gave a presentation at Black Hat about the Hugging Face Incident. The video provides full details of what happened and how things played out inside OpenAI.
Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident
A companion technical writeup to the incident disclosure: the two initial-access vectors, how the agent pivoted and moved laterally, and the commands it ran.
Introducing Gemini 3.6 Flash, 3.5 Flash-Lite, and 3.5 Flash Cyber
We're introducing new Gemini models, including Gemini 3.6 Flash, 3.5 Flash-Lite and 3.5 Flash Cyber. Flash Cyber is a lightweight cybersecurity model to find and patch vulnerabilities.
Safety and alignment in an era of long-horizon models
OpenAI shares lessons from deploying long-running AI models, highlighting new safety risks, observed failures, and improved safeguards through iterative deployment.
Security incident disclosure — July 2026
Hugging Face discloses a production infrastructure intrusion driven end to end by an autonomous AI agent system — initial access via dataset code-execution paths, credential harvesting and lateral movement across internal clusters — detected and dissected largely with AI. Public models, datasets and the software supply chain were verified clean.
GPT-Red: Unlocking Self-Improvement for Robustness
OpenAI introduces GPT-Red, an automated red-teaming system that uses self-play to continuously probe and harden models against safety, alignment and prompt-injection failures.