写作的艺术:装备精心设计的 skill,写作智能体能骗过 AI 检测The Art of Writing: Agents with Well-Curated Skills Can Cheat AI Detectors
我们从人类撰写的规范论文中提炼写法,做成了一个学术写作 skill。装备该 skill 的智能体能写出流畅有逻辑的文字,且能降低 AI 检测率,甚至能一次写完一篇会议论文。我们开源如下:
We distilled the writing practices of human-written papers that follow academic conventions into an academic-writing skill. An agent equipped with the skill can write fluent, logical prose with a lower AI detection rate and can even write a complete conference paper in one run. We open-source the skill at the link below.
我们开源了一个符合学术规范的写作 skill,装备该 skill 的智能体能极大提高通过 AI 检测器 Pangram 的概率。
We open-source a writing skill that follows academic conventions: agents equipped with it are far more likely to pass the AI detector Pangram.
Skill 生成的文本降低了 Pangram 检测的概率。我们模拟了从草稿到成文的过程:在四篇论文上,Opus 5.5 不使用 skill 时平均 94% 的文字被判为 AI,使用 skill 时只有 29%,与人类写作持平。Text generated with the skill is less likely to be detected by Pangram. We simulate the writing process from a draft to a finished text. On four papers, Pangram labels 94% of Opus 5.5’s text as AI without the skill, compared with 29% with the skill, the same as the human-written text.
使用 skill 生成的文本符合学术规范。它按人类规范论文的写法组织和用词,skill 检查器找出的机器草稿口癖从 75 处降到 6 处。The text generated with the skill follows academic conventions. It is organized and phrased like human-written papers that follow academic conventions, and the skill’s checker finds 6 habits of machine-written drafts instead of 75.
Skill 还支持 one-shot 生成一篇符合学术规范的会议论文。在高级用法中,智能体只凭一段提示词,约 82 分钟就独立写成一篇 ICML 2027 投稿,实验也是它自己跑的。The skill also supports one-shot generation of a conference paper that follows academic conventions. In Advanced Usage, an agent given one prompt runs the experiments and writes an ICML 2027 submission alone in about 82 minutes.
Skill 能节省撰写时间,但代替不了人对逻辑的思考。如果内容本身就是 AI 生成的,用 AI 改也纠正不过来:人类写作本身就被判为 AI 的两篇论文,以及只给题目写成的论文,使用 skill 后仍有 95% 以上被判为 AI。The skill saves drafting time but cannot replace human thinking about the logic. If the content is AI-generated to begin with, revising it with AI cannot correct it. On two papers whose human-written text is itself labeled AI, and on papers written from a topic alone, Pangram still labels at least 95% of the text generated with the skill as AI.
适用范围:能通过检测的,一般是基于人写草稿的较短文本;直接由 AI 生成或篇幅较长的文章,被判为 AI 的比例仍然较高。
Scope: The texts that pass the detector are mostly short ones written from a human draft. Text generated directly by AI, or a long paper, is still likely to be labeled AI.
直接让大模型写出的论文往往很糟糕:说法空泛,还堆满 “delve”“underscore” 这类花哨的词Kobak et al., Science Advances 2025. 查看参考文献。这样的文字正大量涌入学术圈:Pangram 估计,ICLR 2026 有 21% 的审稿意见完全由 AI 生成,9% 的投稿里 AI 生成的文字超过一半Emi, Pangram Blog 2025. 查看参考文献。
Papers written directly by an LLM tend to read badly: they make generic claims and lean on ornate words such as “delve” and “underscore”Kobak et al., Science Advances 2025. View reference. Such text now floods academia. For example, Pangram estimates that 21% of the ICLR 2026 reviews were fully AI-generated and that 9% of the submissions contained more than half AI-generated textEmi, Pangram Blog 2025. View reference.
AI 检测器正是为此而出现的。比如 Pangram 在人写文档和前沿模型的输出上训练分类器,它的技术报告称准确率达到 99%Emi and Spero, arXiv 2024. 查看参考文献。
AI detectors emerged to counter this. For example, Pangram trains a classifier on human documents and frontier-model outputs, and reports 99% accuracy in its technical reportEmi and Spero, arXiv 2024. View reference.
然而,AI 是提高效率的工具,我们不应片面地抛弃 AI 生成的内容。如果 AI 写的文字能和人写的一样流畅易懂,就应该用它来加速研究,让研究者把时间花在推敲逻辑上,而不是撰写上。为此,我们做了一个写论文的 skill,它从人类撰写的规范论文中提炼写法,让编程智能体照规范论文的方式起草论文(第 2 节)。我们首先探索:
However, as AI is a tool for working faster, we should not abandon AI-generated content wholesale. If AI-generated text could read as fluently as human writing, we should use it to speed up research, letting researchers spend their time on the logic rather than on drafting. To this end, we build a paper-writing skill that distills the writing of human-written papers that follow academic conventions, so that a coding agent drafts papers the same way (Section 2). We first explore:
精心整理的 skill,能否让智能体写出流畅易懂的文字,甚至骗过 AI 检测器?Can a well-curated skill make an agent write fluent, readable prose, and even cheat AI detectors?
02 · THE SKILLSkill 不仅润色文字,还对全文排布进行宏观约束The skill not only polishes the wording but also constrains the layout of the whole paper
这个 skill 是 Claude Code 这类编程智能体写论文前先读的一个文件夹。它把人类规范论文的写法蒸馏成流程、规则、短语和检查工具,既约束全文排布,比如各节骨架、页数预算和引言的论证链,也约束每一句的写法,比如句子规则、短语库和口癖表。
The skill is a folder that a coding agent such as Claude Code reads before writing a paper. It distills the writing of human-written papers that follow academic conventions into a workflow, rules, phrases and checking tools. These constrain both the layout of the whole paper, from the section skeleton to the argument chain of the introduction, and the wording of each sentence, from the phrase bank to the habits to avoid.
Tools such as Humanizer only polish the wording sentence by sentence, and paper-writing skills such as ARIS focus on planning the writing pipeline. In contrast, our skill constrains a paper systematically, from its layout to each sentence.
Open the folder and hover over or tap a card to see what each part does; the Appendix has more details.
SKILL.md写作流程Workflow
8 个步骤,从会议要求写到最后自查eight steps, from the call for papers to a final self-review
主文件把写一篇论文拆成 8 步:确认会议并抓取当年的征稿要求、列提纲、分配页数、起草、做图表、写附录和声明、编译核对页数,最后逐条自查。每一步都写明该读哪份参考文档。The main file splits a paper into eight steps: confirm the venue and fetch this year’s call for papers, outline, budget the pages, draft, make the figures and tables, write the appendix and statements, compile and check the pages, and self-review. Each step names the reference file to read.
12 份参考文档:结构、引言、实验、图表12 reference files on structure, introductions, experiments and figures
12 份参考文档,写到对应步骤时才读:论文类型、结构和页数蓝图、引言、贡献表述、句子与衔接、实验、数学、图表、LaTeX 源码、术语、会议规则和示例。Twelve reference files, each read at the step that needs it: paper types, structure and page blueprints, introductions, contribution statements, sentences and cohesion, experiments, math, figures and tables, LaTeX source, terms, venue rules and examples.
动笔前先写 8–10 条一句话论断8–10 one-line claims planned before any prose
动笔前,先把引言写成 8–10 条一句话的论断:已有工作共同的设计、它没达到的标准、两个缺陷,以及否定这个共同设计的核心想法。每一环都成立了才开始起草,段落和段首句都由这条链决定。Before any prose, the agent writes the introduction as 8–10 one-line claims: the design all prior work shares, the standard it misses, two drawbacks and a key idea that negates the shared design. It drafts only when every link holds, and the paragraphs and their openers follow from the chain.
共同设计 → 没达到的标准 → 两个缺陷 → 核心想法shared design → missed standard → two drawbacks → key idea
prose.md句子规则Sentence rules
15 项句子检查,句长中位数 20 词15 sentence checks, with a median of 20 words
15 项句子检查,都在人类规范论文上测量过:句长中位数 20 词,主语前最多 10 个词,一句只有一个主语,重点放在主句末尾。衔接规则要求几乎每一句都接住上一句。Fifteen sentence checks, each measured on human-written papers: a median of 20 words, at most ten words before the subject, one subject per sentence and the point at the end of the main clause. Cohesion rules make nearly every sentence link back to the previous one.
27 种机器草稿的口癖,每种附改法27 habits of machine-written drafts, each with a fix
把一份 2.4 万词的机器草稿和人类规范论文逐一比对,找出机器反复出现的 27 种习惯,每种都给出论文里的写法。检查器会逐行标出草稿里的每一处。Comparing a 24,000-word machine-written draft with human-written papers turned up 27 habits that machines repeat, each with the form the papers use instead. The checker flags every occurrence in a draft, line by line.
1,272 条原文短语,按写作步骤分成 22 区1,272 attested phrases in 22 sections by writing step
从人类规范论文中摘出的 1,272 条原文表达,按写作步骤分成 22 区,从背景、已有工作的不足一直到结论,外加连接词和词类。每条标了状态,智能体写每一句前先查对应的分区。1,272 expressions taken from human-written papers, sorted into 22 sections by writing step, from the background and the gap in prior work to the conclusion, plus connectives and word classes. Each has a tier, and the agent checks the matching section before each sentence.
短语库记下每个词在人类规范论文里出现的次数,智能体据此挑这些论文真正会用的词。它们很少用的词,比如 “Furthermore” 和 “Notably”,被列为禁用。The phrase bank records how often each word appears in human-written papers, so the agent picks the words those papers actually use. Words they rarely use, such as “Furthermore” and “Notably”, are banned.
specifically 48 · however 43 · moreover 34 · each 94 · every 4specifically 48 · however 43 · moreover 34 · each 94 · every 4
tools/ps.py短语查询Phrase lookup
查任意短语在人类规范论文中出现几次how often any phrase occurs in human-written papers
查任意五词以内的短语在人类规范论文里出现几次,背后是随 skill 发布的词序列计数索引,不含任何原句。拿不准一个说法时,智能体先查它。Reports how often any phrase of up to five words occurs in human-written papers, from a shipped index of word-sequence counts that holds no sentences. When unsure of a wording, the agent looks it up first.
ps.py “to this end” → 多数论文都用 · 常用;“delve into” → 已弃用ps.py “to this end” → most papers · core; “delve into” → retired
scripts/check_paper.py草稿检查器Draft checker
报告字数、页数、会议声明和每处口癖word budget, page limit, required statements and each habit
读整篇 LaTeX 草稿,报告每章字数和预算、正文结束在第几页、会议要求的声明和匿名、术语和衔接、句子规则,以及 27 种口癖的每一处匹配和行号。Reads the whole LaTeX draft and reports the words per chapter against the budget, the page where the main text ends, the venue’s required statements and anonymity, terms and cohesion, the sentence rules, and each match of the 27 habits with its line number.
ICLR、NeurIPS、ICML 的页数、匿名要求和必需声明page limits, anonymity and required statements for ICLR, NeurIPS and ICML
ICLR、NeurIPS、ICML 的页数上限、匿名要求、必需声明和引用格式,外加一份单栏与双栏之间改投的核对清单。规则每年都变,所以第一步就要抓取当年的征稿要求核对。Page limits, anonymity, required statements and citation commands for ICLR, NeurIPS and ICML, plus a checklist for moving a paper between one and two columns. Rules change every year, so the first step fetches the current call for papers.
论文的配色、绘图风格和框图画法the papers’ palette, plot style and diagram recipe
规范论文常用的配色、绘图样式和框图画法:自家方法用饱和色,基线用浅色;框图用 TikZ 样式包画,一张图只有一个实心的焦点模块,渲染脚本会检查宽度、字体和配色。The palette, plot style and diagram recipe of well-formed papers: saturated colors for the method and tints for baselines; diagrams drawn in TikZ with one filled focal module, and a render script that checks the width, fonts and colors.
三个会议的 LaTeX 模板和表格样式LaTeX templates and table styles for the three venues
三个会议的 LaTeX 骨架和分章文件、共用的导言区、8 种表格模板(主结果、消融、超参数等)、绘图样式和框图起步文件。实验还没跑完时,表格先用占位格。LaTeX skeletons with chapter files for the three venues, a shared preamble, eight table templates (main results, ablations, sweeps and more), the plot style and a diagram starter. Tables can hold placeholder cells until the experiments finish.
03 · RESULTSSkill 生成的文本降低了 Pangram检测的概率Text generated with the skill is less likely to be detected by Pangram
这个 skill 最常见的用法是:思路和实验结果都有了,想快速写成一篇规范的论文。我们模拟这一写作过程,对每篇论文比较三份文本:人类写作,以及 Opus 5.5 不使用和使用 skill 生成的文本。本节报告四篇论文;人类写作本身就被判为 AI 的两篇放在第 4 节。
The skill is most often used when a researcher has the idea and the results and wants to write them up quickly as a paper. We simulate this writing process and compare three texts per paper: the human-written text and the texts that Opus 5.5 generates without and with the skill. This section reports four papers, and Section 4 covers the two whose human-written text is itself labeled AI.
Simulated scenario. We choose six method papers newly posted to arXivThey cover conversion-rate prediction in recommendation18, the calibration of conditional generative models19, bilevel optimization20, imitation learning21, federated DDoS detection22 and online conformal risk control23. and condense each into a content brief of about 300 words that lists the facts without the original sentences. Claude Opus 5.5 writes the abstract and introduction twice from each brief, with and without the skill, each time in a fresh session without web access. Pangram then scans both generated texts and the human-written text. The briefs, generated texts and Pangram screenshots are at the end of this section, and the setup and prompts are in the Appendix.
四篇论文上,使用 skill 后AI 比例都大幅下降On four papers, the AI share drops sharply with the skill
Pangram 4.1 判为 AI 生成或 AI 辅助的文字占比,四篇模拟论文生成。Share of each text that Pangram 4.1 labels AI-generated or AI-assisted, over four simulated paper generations.
人类写作Human-writtenOpus 5.5 生成(不使用 skill)Generated by Opus 5.5 (without the skill)Opus 5.5 生成(使用 skill)Generated by Opus 5.5 (with the skill)
As shown in Figure 1, the skill reduces the share that Pangram labels AI from 94% to 29% on average, the same as the human-written text. The skill was distilled from none of these papers. Therefore, the drop does not come from imitating how they are written. The Appendix analyzes which aspects of the style the skill changes.
四篇论文的生成案例Generated texts of the four papers
原论文:Paper: Pavel Dolin, Xun Zou, Jiatian Wu, and Deguang Kong. Serving-Aligned Entire-Space Conversion Modeling via Click-Through Rate Knowledge Distillation. arXiv 2026. 我们逆向生成一个用于写论文的摘要和引言的 prompt 如下(使用 skill 和不使用 skill 的测试共享同一个生成 prompt)。We reverse-engineer the following prompt for writing the paper’s abstract and introduction; the runs with and without the skill share this generation prompt.
Content brief:
- Area: recommender systems; post-click conversion rate (pCVR) prediction for conversion-oriented ranking.
- Context: ranking may use the predicted click-through conversion rate (pCTCVR), the product of a predicted click-through rate (pCTR) and pCVR from separately trained models. Clicks are fast and dense, conversions slow and sparse, which motivates separate training schedules and independently developed pCTR models. Many pCVR models build on the Entire Space Multi-Task Model (ESMM), which trains a pCVR tower together with an auxiliary pCTR tower using pCTR and pCTCVR losses, but deploys only the pCVR tower.
- Problem: at serving time the pCVR tower is paired with a separately trained production pCTR model. When its predictions differ from those of the auxiliary tower, the pCTCVR used at serving differs from the one seen in training: a training-serving mismatch.
- Constraint: production pCTR models are large and expensive to train; retraining them, or training them jointly with the pCVR model, is impractical, especially when their predictions are shared across ranking stages.
- Method: fix the mismatch from the pCVR side with knowledge distillation (KD). ESMM's auxiliary pCTR tower is supervised with logged scores of the frozen serving pCTR model, mixed with the hard click labels; variants are named KD-alpha, where alpha weights the hard-label loss. Neither ESMM's architecture nor the serving pCTR model changes.
- Evidence: on the Ali-CCP dataset, KD-0.3, the best variant on serving pCTCVR, improves serving pCTCVR and pCVR ROC-AUC by 2.1% and 1.6%, and PR-AUC by 9.9% and 8.8%, over ESMM. Under negative downsampling that keeps 5% of non-clicks, it improves mean AUC by 1.2% and 0.9% and PR-AUC by 8.5% and 4.5% over downsampled ESMM. Code is released.
Conversion-oriented ranking in recommender systems often scores items by the predicted click-through conversion rate (pCTCVR). This is the product of a predicted click-through rate (pCTR) and a post-click conversion rate (pCVR), each produced by a separately trained model. Many pCVR models build on the Entire Space Multi-Task Model (ESMM). ESMM trains a pCVR tower together with an auxiliary pCTR tower, but only the pCVR tower is deployed. At serving time this tower is paired with an independently developed production pCTR model. Whenever that model disagrees with the auxiliary tower, the pCTCVR used at serving differs from the one optimized during training, which creates a training-serving mismatch. Production pCTR models are large and expensive, and they are often shared across ranking stages, so retraining them or training them jointly with the pCVR model is impractical. We therefore address the mismatch from the pCVR side through knowledge distillation (KD). We supervise ESMM's auxiliary pCTR tower with logged scores of the frozen serving pCTR model, mixed with hard click labels. Neither ESMM's architecture nor the serving pCTR model changes. On the Ali-CCP dataset, our best variant improves serving pCTCVR and pCVR ROC-AUC over ESMM by 2.1% and 1.6%, and PR-AUC by 9.9% and 8.8%. Under negative downsampling that keeps 5% of non-clicks, it improves mean AUC by 1.2% and 0.9% and PR-AUC by 8.5% and 4.5% over downsampled ESMM. Code is released.
Introduction
Many recommender systems are built to drive conversions such as purchases, sign-ups or installs, and not just clicks. In these systems, items are commonly ranked by the predicted click-through conversion rate (pCTCVR), the probability that a user both clicks an item and then converts. pCTCVR factorizes naturally into a predicted click-through rate (pCTR) and a predicted post-click conversion rate (pCVR). Production systems usually obtain the two factors from separately trained models. The split is practical. Clicks are fast and dense, while conversions are slow and sparse, so the two signals call for different training schedules. As a result, pCTR models are often developed independently of pCVR models.
Many pCVR models build on the Entire Space Multi-Task Model (ESMM). ESMM trains a pCVR tower together with an auxiliary pCTR tower. It supervises the pCTR tower with click labels and the product of the two towers with click-and-conversion labels. This lets the pCVR tower learn over the entire impression space and not only over clicked samples. Only the pCVR tower is deployed, however. The auxiliary pCTR tower exists to shape training and is discarded at serving time.
This deployment pattern hides a problem. At serving time, the pCVR tower is not multiplied with the auxiliary pCTR tower it was trained with. It is multiplied with a separately trained production pCTR model. The two pCTR estimates can differ, since they come from different architectures, features, data and training schedules. When they differ, the pCTCVR that ranks items at serving time is not the one ESMM optimized during training. The pCVR tower has learned to complement one pCTR model and is then paired with another. We call this a training-serving mismatch. It weakens ESMM's main advantage: the pCTCVR objective that shapes the pCVR tower no longer matches the quantity actually used for ranking.
A direct fix would be to align the two pCTR models from the pCTR side. One option is to deploy the auxiliary tower instead of the production model. Another is to train the production pCTR model jointly with the pCVR model. Neither is realistic in practice. Production pCTR models are large and expensive to train, and their predictions are often shared across several ranking stages. Retraining them, or tying their training to a downstream pCVR model, would be costly and would risk regressions elsewhere in the system. Any practical solution must therefore treat the serving pCTR model as fixed.
We address the mismatch from the pCVR side using knowledge distillation (KD). Our key observation is that the auxiliary pCTR tower in ESMM does not have to be an independent click predictor. It only has to stand in, during training, for the pCTR model the pCVR tower will be paired with at serving time. We therefore supervise the auxiliary tower with logged scores of the frozen serving pCTR model, mixed with the hard click labels. We denote the resulting variants KD-alpha, where alpha weights the hard-label loss. The auxiliary tower then tracks the serving pCTR model, so the pCTCVR seen during training more closely resembles the pCTCVR used at serving time. The method changes neither ESMM's architecture nor the serving pCTR model. It only needs logged pCTR scores, which production systems typically already record.
We evaluate the approach on the Ali-CCP dataset, measuring performance in the setting that matters for deployment, where the pCVR model is paired with the serving pCTR model. KD-0.3 is the best variant on serving pCTCVR. Compared with ESMM, it improves serving pCTCVR and pCVR ROC-AUC by 2.1% and 1.6%, and PR-AUC by 9.9% and 8.8%. The gains hold under negative downsampling that keeps only 5% of non-clicks, a common practice for reducing training cost. In that setting, KD-0.3 improves mean AUC by 1.2% and 0.9% and PR-AUC by 8.5% and 4.5% over downsampled ESMM.
Our contributions are as follows:
We identify a training-serving mismatch in ESMM-style pCVR models. The auxiliary pCTR tower used during training differs from the separately trained pCTR model used at serving time, so the serving pCTCVR differs from the training objective.
We propose a simple remedy based on knowledge distillation. It supervises the auxiliary pCTR tower with logged scores of the frozen serving pCTR model mixed with hard click labels, and it requires no change to ESMM's architecture or to the serving pCTR model.
We show on Ali-CCP that this approach consistently improves serving pCTCVR and pCVR quality over ESMM, both with and without negative downsampling, and we release our code.
Conversion-oriented ranking often scores items by the predicted click-through conversion rate (pCTCVR), i.e., the product of a predicted click-through rate (pCTR) and a predicted post-click conversion rate (pCVR) from separately trained models. Many pCVR models build on the Entire Space Multi-Task Model (ESMM). ESMM trains a pCVR tower together with an auxiliary pCTR tower under pCTR and pCTCVR losses, but deploys only the pCVR tower. However, at serving time the pCVR tower is paired with a separately trained production pCTR model. When the production model's predictions differ from those of the auxiliary tower, the pCTCVR used at serving differs from the one seen in training (termed training-serving mismatch). Removing this mismatch from the pCTR side is impractical. Retraining the production pCTR model, or training it jointly with the pCVR model, is costly because such models are large and expensive to train, especially when their predictions are shared across ranking stages. To address this mismatch, we fix it from the pCVR side with knowledge distillation (KD), which makes pCVR training consistent and practical. Specifically, we supervise the auxiliary pCTR tower with logged scores of the frozen serving pCTR model, mixed with the hard click labels, while the ESMM architecture and the serving pCTR model remain unchanged. Experiments on a public dataset show that the best KD variant improves both ROC-AUC and PR-AUC of serving pCTCVR and pCVR over ESMM, including under negative downsampling of non-clicks. Our code is released.
Introduction
Conversion-oriented ranking in recommender systems often orders items by the predicted click-through conversion rate (pCTCVR), i.e., the product of a predicted click-through rate (pCTR) and a predicted post-click conversion rate (pCVR). In practice, the two factors come from separately trained models. Clicks are fast and dense, while conversions are slow and sparse, which motivates separate training schedules and pCTR models developed independently of the pCVR model. As a result, the pCTCVR that decides the ranking multiplies the outputs of two models that are never trained together.
Many pCVR models build on the Entire Space Multi-Task Model (ESMM). Specifically, ESMM trains a pCVR tower together with an auxiliary pCTR tower: a pCTR loss supervises the auxiliary tower with click labels, and a pCTCVR loss supervises the product of the two towers. After training, only the pCVR tower is deployed. This multi-task design has made ESMM a widely adopted basis for pCVR prediction. Whether through the pCTR loss or the pCTCVR loss, the auxiliary pCTR tower learns from labels alone, independently of the pCTR model used at serving.
Intuitively, effective pCVR training for pCTCVR ranking should be consistent (i.e., the pCTCVR it optimizes matches the pCTCVR used at serving) and practical (i.e., it leaves the production pCTR model untouched). ESMM-style training is practical by design, because it does not touch the production pCTR model. However, this design is not consistent. As illustrated in Figure 1, at serving time the pCVR tower is paired with the separately trained production pCTR model rather than with the auxiliary pCTR tower. When the predictions of the two pCTR models differ, the pCTCVR used at serving differs from the one seen in training, leading to a training-serving mismatch. As a result, the pCVR tower is optimized for a pCTCVR that differs from the one that ranks items.
This mismatch could be removed from the pCTR side, by retraining the production pCTR model or by training it jointly with the pCVR model. However, such remedies gain consistency at the cost of practicality. Production pCTR models are large and expensive to train, which makes retraining them, or training them jointly with the pCVR model, impractical. This cost is amplified when their predictions are shared across ranking stages, because a change to the pCTR model then affects each stage that uses its predictions. This gap calls for a remedy on the pCVR side that is both consistent and practical.
To this end, we fix the training-serving mismatch from the pCVR side with knowledge distillation (KD). The key idea is to let the auxiliary pCTR tower learn from the pCTR model used at serving, rather than from labels alone. Specifically, we supervise the auxiliary tower with the logged scores of the frozen serving pCTR model (i.e., the pCTR predictions recorded at serving), thereby reducing the gap between the pCTCVR seen in training and the one used at serving. To keep the click labels as the ground truth of the pCTR task, we mix the distillation loss with the hard-label loss and name each variant KD-α, where α weights the hard-label loss. The ESMM architecture and the serving pCTR model remain unchanged: the logged scores serve as fixed targets, and only the pCVR tower is deployed, as in ESMM. The distillation targets align training with serving, while the frozen logged scores leave the production pCTR model untouched; together, they make pCVR training consistent and practical.
We conduct experiments on the Ali-CCP dataset to examine the effectiveness of KD (cf. Section 4). First, KD-0.3, the best variant on serving pCTCVR, improves the ROC-AUC of serving pCTCVR and pCVR by 2.1% and 1.6%, respectively, and their PR-AUC by 9.9% and 8.8%, compared with ESMM. Second, under negative downsampling that keeps 5% of non-clicks, KD-0.3 improves mean AUC by 1.2% and 0.9% and PR-AUC by 8.5% and 4.5% compared with downsampled ESMM. Third, these gains come without retraining the production pCTR model or changing the ESMM architecture, because KD only changes the supervision of the auxiliary pCTR tower. We highlight that KD makes pCVR training consistent and practical, offering a pCVR-side remedy for the training-serving mismatch that leaves the serving pCTR model untouched.
原论文:Paper: Bowen Wang and Osvaldo Simeone. Capacity-Constrained Delayed Feedback for Online Conformal Risk Control. arXiv 2026. 我们逆向生成一个用于写论文的摘要和引言的 prompt 如下(使用 skill 和不使用 skill 的测试共享同一个生成 prompt)。We reverse-engineer the following prompt for writing the paper’s abstract and introduction; the runs with and without the skill share this generation prompt.
Content brief:
- Area: reliable machine learning; online conformal risk control (OCRC), which turns model outputs into prediction sets with long-term risk guarantees.
- Context: conformal prediction converts the outputs of a pre-trained model into prediction sets that contain the true label with a chosen probability; conformal risk control (CRC) extends this from miscoverage to general losses. Online versions (OCP, OCRC) update the sets from feedback on past losses, so the long-term average risk approaches a target level for any, even adversarial, data sequence.
- Problem 1: these guarantees assume that the loss of each round is revealed by the end of that round. In practice feedback can be delayed: medical outcomes are known only after follow-up examinations, and the correctness of answers in open-domain question answering only after slow human annotation.
- Problem 2: tracking every prediction until its feedback arrives costs monitoring and storage resources, which are limited (patient follow-up capacity, annotation budgets, finite memory in wireless systems, ad conversions that arrive up to a month later). Some feedback is therefore lost.
- Goal: jointly design the prediction sets and a feedback-tracking policy (which rounds to track and for how long) so that the long-term risk still approaches the target level.
- Method name: capacity-aware OCRC (CA-OCRC). A randomized tracking policy samples a proxy delay for each round, which decides whether and for how long its feedback is tracked. The prediction sets are updated with inverse probability weighting, which compensates for feedback lost to delays or to the capacity limit.
- Theory: risk bounds for arbitrary and for stochastic delays, which reveal a trade-off between receiving more feedback and saturating the tracking capacity. Optimizing the proxy-delay distribution yields scaling laws for the tracking capacity that guarantee the same long-term risk as OCRC with unlimited feedback.
- Evidence: experiments on medical imaging, natural language processing, computer vision and wireless communications; CA-OCRC is more effective than existing OCRC mechanisms.
Online conformal risk control (OCRC) turns the outputs of a pre-trained model into prediction sets. It updates these sets from feedback on past losses so that the long-term average risk approaches a target level for any data sequence, including adversarial ones. This guarantee assumes that the loss of each round is revealed by the end of that round. In practice, feedback is often delayed: medical outcomes become known only after follow-up examinations, and the correctness of answers in open-domain question answering only after slow human annotation. Tracking every prediction until its feedback arrives also consumes monitoring and storage resources, which are limited. As a result, some feedback is lost. We study how to design the prediction sets jointly with a feedback-tracking policy that decides which rounds to track and for how long. We propose capacity-aware OCRC (CA-OCRC). In CA-OCRC, a randomized tracking policy samples a proxy delay for each round, and this delay determines whether and for how long the feedback of that round is tracked. The prediction sets are updated with inverse probability weighting, which compensates for feedback lost to delays or to the capacity limit. We derive risk bounds for arbitrary and for stochastic delays. These bounds reveal a trade-off between receiving more feedback and saturating the tracking capacity. By optimizing the proxy-delay distribution, we obtain scaling laws for the tracking capacity under which CA-OCRC matches the long-term risk of OCRC with unlimited feedback. We run experiments on medical imaging, natural language processing, computer vision and wireless communications, and CA-OCRC is more effective than existing OCRC mechanisms in all of them.
Introduction
Modern machine learning models are increasingly deployed in settings where errors are costly, such as clinical decision support, question answering and wireless network management. In these settings, a point prediction is often not enough: decision makers also need a calibrated measure of uncertainty. Conformal prediction meets this need by converting the outputs of any pre-trained model into prediction sets that contain the true label with a user-chosen probability. Conformal risk control (CRC) extends this idea from miscoverage to general loss functions. This lets practitioners bound, for example, the fraction of missed findings in a medical image or the error rate of a set of candidate answers.
Online variants of these methods, namely online conformal prediction (OCP) and online conformal risk control (OCRC), remove the need for exchangeable data. After each round, they use feedback on the loss just incurred to adjust the size of the next prediction set. If the sets have been too risky, they grow, and if they have been too conservative, they shrink. This simple feedback loop ensures that the long-term average risk approaches a target level for any data sequence, even an adversarial one. The guarantee is therefore attractive in non-stationary environments where distribution shift is the norm rather than the exception.
These guarantees, however, rest on an assumption that rarely holds in practice: the loss of each round is revealed by the end of that round. In many applications feedback is delayed. In medicine, whether a diagnosis was correct is known only after follow-up examinations, which may take place weeks or months later. In open-domain question answering, whether a set of answers contains a correct one is determined only after slow human annotation. In online advertising, conversions may arrive up to a month after an ad is shown. When feedback arrives late, an online method must update its sets based on stale and incomplete information, and existing guarantees no longer apply directly.
Delay is only part of the difficulty. To use delayed feedback at all, the system must keep track of every outstanding prediction until its outcome is observed. This tracking consumes monitoring and storage resources, and these resources are limited. Hospitals have finite capacity for patient follow-up, annotation campaigns run on fixed budgets, and wireless systems operate with finite memory. Under such constraints it is impossible to track every round until its feedback arrives, so some feedback is inevitably lost. Simply dropping rounds once the capacity is reached, or abandoning feedback after a fixed horizon, biases the information available to the online update and can undermine the risk guarantee.
In this paper we argue that prediction sets and feedback tracking should be designed together. The central question we address is the following: given a limited tracking capacity and possibly long feedback delays, how should a system decide which rounds to track and for how long, and how should it update its prediction sets, so that the long-term risk still approaches the target level?
We answer this question with capacity-aware OCRC (CA-OCRC). CA-OCRC uses a randomized tracking policy that samples a proxy delay for each round. The proxy delay determines whether the feedback of that round is tracked and, if so, for how long the system waits for it. Because the tracking probabilities are known by design, the prediction sets can be updated with inverse probability weighting. This compensates for feedback that is lost either because it arrives too late or because the tracking capacity is saturated. The randomization thus turns an unavoidable loss of information into a controlled sampling process whose effect on the risk can be corrected and analyzed.
Our contributions are as follows:
We formulate the problem of online conformal risk control under delayed feedback and limited tracking capacity, and we propose CA-OCRC, which jointly designs a randomized feedback-tracking policy and an inverse-probability-weighted update of the prediction sets.
We derive long-term risk bounds for CA-OCRC under both arbitrary and stochastic delays. The bounds reveal a fundamental trade-off: tracking rounds for longer yields more feedback but risks saturating the tracking capacity.
By optimizing the distribution of proxy delays, we derive scaling laws for the tracking capacity under which CA-OCRC guarantees the same long-term risk as OCRC with unlimited feedback.
We evaluate CA-OCRC on tasks from medical imaging, natural language processing, computer vision and wireless communications, and show that it is more effective than existing OCRC mechanisms.
Online conformal risk control (OCRC) turns the outputs of a pre-trained model into prediction sets whose long-term average risk approaches a target level for any, even adversarial, data sequence. To achieve this, OCRC updates the sets with the loss of each round, which is assumed to be revealed by the end of that round. However, this assumption hardly holds in practice: (i) feedback can be delayed (e.g., medical outcomes are known only after follow-up examinations), and (ii) tracking each prediction until its feedback arrives consumes limited monitoring and storage resources, causing some feedback to be lost. To address both issues, we propose capacity-aware OCRC (CA-OCRC), a delay-tolerant and capacity-aware method that jointly designs the prediction sets and a feedback-tracking policy. Specifically, a randomized tracking policy samples a proxy delay for each round, which decides whether and for how long its feedback is tracked. The prediction sets are then updated with inverse probability weighting, which compensates for the feedback lost to delays or to the capacity limit. We derive risk bounds for arbitrary and for stochastic delays, which reveal a trade-off between receiving more feedback and saturating the tracking capacity. By optimizing the proxy-delay distribution, we further obtain scaling laws for the tracking capacity that guarantee the same long-term risk as OCRC with unlimited feedback. Experiments on medical imaging, natural language processing, computer vision and wireless communications show that CA-OCRC is generally more effective than existing OCRC mechanisms.
Introduction
Conformal prediction converts the outputs of a pre-trained model into prediction sets that contain the true label with a chosen probability. Conformal risk control (CRC) extends this guarantee from miscoverage to general losses. Building on these tools, online conformal prediction (OCP) and online conformal risk control (OCRC) update the sets from feedback on past losses. As a result, the long-term average risk approaches a target level for any, even adversarial, data sequence. Since this guarantee requires no assumption on how the data are generated, OCRC is attractive for reliable machine learning in deployed systems.
Concretely, OCRC maintains a threshold that sets the size of each prediction set. Specifically, at each round, it outputs a set for the new input, observes the loss of this set by the end of the round, and adjusts the threshold according to the gap between this loss and the target level. In practice, this loop relies on two conditions: the loss of each round is revealed before the next round begins, and the loss of each round enters the update. In other words, OCP and OCRC share one design: the sets are updated once per round with the full and immediate feedback of that round. Intuitively, effective online risk control should be delay-tolerant (i.e., its long-term risk approaches the target level even when losses arrive many rounds later) and capacity-aware (i.e., it tracks no more predictions than the available monitoring resources allow).
However, both properties are hard to meet under this design, mainly for two reasons. (i) Delayed feedback: since each update relies on the loss of the latest round, it has no loss to use when this loss arrives later. Such delays are common in practice: medical outcomes are known only after follow-up examinations, and the correctness of answers in open-domain question answering is confirmed only after slow human annotation. As a result, the long-term risk guarantee of OCRC no longer applies, and the method is not delay-tolerant. (ii) Limited tracking capacity: waiting for each delayed loss requires tracking each prediction until its feedback arrives, which consumes monitoring and storage resources. These resources are limited, as with patient follow-up capacity, annotation budgets, finite memory in wireless systems, and ad conversions that arrive up to a month after the ad is shown. Consequently, OCRC is not capacity-aware: once the tracking capacity is exhausted, some feedback is lost, and an update that treats the remaining losses as complete may drift from the target level. These limitations motivate the need for a joint design of the prediction sets and a feedback-tracking policy (i.e., which rounds to track and for how long) that makes OCRC both delay-tolerant and capacity-aware.
To this end, we propose capacity-aware OCRC (CA-OCRC), a delay-tolerant and capacity-aware extension of OCRC. Rather than assuming that the loss of each round arrives in full and immediately, the key idea is to update the sets only with the losses that do arrive, and to decide for each round whether and for how long its loss is tracked. However, the received losses alone do not reflect the risk of all rounds, and reweighting them requires each loss to have a known, nonzero probability of being received. Therefore, we randomize the tracking: for each round, a tracking policy samples a proxy delay from a chosen distribution, which decides whether and for how long the feedback of this round is tracked. The prediction sets are then updated with inverse probability weighting, which compensates for the feedback lost to delays or to the capacity limit. The weighted update maintains the long-term risk near the target level despite delayed and missing losses, while the tracking policy limits the tracked predictions to the available capacity; together, they make OCRC delay-tolerant and capacity-aware.
We then analyze CA-OCRC in theory and evaluate it in experiments. First, we derive risk bounds for arbitrary and for stochastic delays, which reveal a trade-off between receiving more feedback and saturating the tracking capacity. Second, by optimizing the proxy-delay distribution under this trade-off, we obtain scaling laws for the tracking capacity that guarantee the same long-term risk as OCRC with unlimited feedback. Third, experiments on medical imaging, natural language processing, computer vision and wireless communications show that CA-OCRC is generally more effective than existing OCRC mechanisms. We highlight that CA-OCRC retains the long-term risk guarantee of OCRC while tracking only as much feedback as the available resources allow.
原论文:Paper: Benjamin Remy, Raphael Rossellini, Chihway Chang, and Rebecca Willett. Training Calibrated Conditional Generative Models with a Sample-Only Coverage Loss. arXiv 2026. 我们逆向生成一个用于写论文的摘要和引言的 prompt 如下(使用 skill 和不使用 skill 的测试共享同一个生成 prompt)。We reverse-engineer the following prompt for writing the paper’s abstract and introduction; the runs with and without the skill share this generation prompt.
Content brief:
- Area: uncertainty quantification for conditional generative models that sample posteriors in scientific inverse problems (medical imaging, astrophysics, engineering).
- Context: normalizing flows, GANs, diffusion and flow models can sample the posterior over signals given indirect and noisy measurements, but they are often overconfident (mode collapse in GANs, overconfident flows, diffusion models that under-represent posterior uncertainty), which is a critical failure where accurate uncertainty is required.
- Goal: valid conditional coverage, which is equivalent to the learned posterior equaling the true posterior; it cannot be checked directly, since a dataset holds one ground-truth signal per measurement.
- Problem 1: coverage objectives add a differentiable coverage term computed on highest-probability-density regions. They need the model's density, which restricts them to models with tractable likelihoods such as flows, and they enforce only marginal coverage, which does not identify the posterior (the prior also has valid marginal coverage).
- Problem 2: strictly proper scoring rules such as CRPS and the energy score need only samples and are minimized at the posterior, but they do not target coverage and can leave models poorly calibrated in practice (finite samples, misspecification, the geometry of the loss).
- Idea: for each measurement, pick a reference point and ask how much of the model's probability mass lies closer to it than the true signal does; for a correct model this matches the observed frequency at every probability level. Departures are measured with a Cramer-von Mises discrepancy, building on TARP (Tests of Accuracy with Random Points). The discrepancy is computed separately for many measurement-dependent reference rules and then averaged, rather than pooling their coverage, where over- and under-coverage at different measurements can cancel.
- Method name: TARP-CvM, a differentiable loss estimated from samples alone; it applies to any conditional generative model without densities or architectural changes.
- Theory: under a sufficient condition on the distribution of reference rules, the loss is a strict divergence: it vanishes if and only if the model equals the true posterior.
- Evidence: on simulation-based inference benchmarks it matches a density-based coverage loss without evaluating the density; on high-dimensional imaging inverse problems it gives better-calibrated posteriors than scoring-rule regularizers, and it is the only tested regularizer that brings the posterior within the calibration test's null band on all three problems, with little change in the quality of point estimates.
Conditional generative models such as normalizing flows, GANs, and diffusion and flow models are increasingly used to sample posteriors in scientific inverse problems, where a signal has to be recovered from indirect and noisy measurements. These models are often overconfident, and in medical imaging, astrophysics, and engineering that is a critical failure. The property we want is valid conditional coverage, which is equivalent to the learned posterior equaling the true posterior. It cannot be checked directly, because a dataset holds only one ground-truth signal per measurement. Existing coverage objectives need the model's density and enforce only marginal coverage, which does not identify the posterior. Strictly proper scoring rules need only samples, but they do not target coverage and can leave models poorly calibrated in practice. We introduce TARP-CvM, a differentiable calibration loss built on Tests of Accuracy with Random Points. For each measurement we choose a reference point and ask how much of the model's probability mass lies closer to it than the true signal does. Departures from the frequencies expected under a correct model are measured with a Cramer-von Mises discrepancy. The discrepancy is computed separately for many measurement-dependent reference rules and then averaged, so over- and under-coverage at different measurements cannot cancel. The loss is estimated from samples alone and applies to any conditional generative model. Under a sufficient condition on the distribution of reference rules, it vanishes if and only if the model equals the true posterior. On simulation-based inference benchmarks, TARP-CvM matches a density-based coverage loss without evaluating the density. On high-dimensional imaging inverse problems, it is the only tested regularizer that brings the posterior within the calibration test's null band on all three problems, with little change in point-estimate quality.
Introduction
Many problems in science and engineering are inverse problems: recover an unknown signal from measurements that are indirect, incomplete, and corrupted by noise. Examples include reconstructing an image from undersampled medical scans and inferring physical parameters from astrophysical observations. In these settings a single reconstruction is rarely enough. Many signals can be consistent with the same measurement, and decisions downstream depend on knowing which ones are plausible. The natural object to work with is therefore the posterior distribution over signals given the measurement. Conditional generative models have become the main tool for sampling it, including normalizing flows, generative adversarial networks, and diffusion and flow models.
These models can produce realistic samples and still misrepresent uncertainty. GANs are prone to mode collapse, normalizing flows are often overconfident, and diffusion models can under-represent posterior uncertainty. In each case the sampled posterior is narrower than the true one, and credible regions miss the true signal more often than their nominal level implies. In medical imaging, astrophysics, and engineering, uncertainty estimates are often the point of the analysis, so this overconfidence is a critical failure.
The property we want is valid conditional coverage: for every measurement and every probability level, the model's credible regions should contain the true signal with exactly that probability. Conditional coverage is equivalent to the learned posterior equaling the true posterior. It cannot be checked directly, because a dataset holds only one ground-truth signal per measurement, and a single draw says almost nothing about calibration at that measurement. Training procedures therefore rely on proxies, and the two main families each have a significant limitation.
The first family adds a differentiable coverage term to the training objective. This term is computed on highest-probability-density regions. Constructing those regions requires the model's density, which restricts the approach to models with tractable likelihoods such as normalizing flows and excludes GANs and most diffusion samplers. These objectives also enforce only marginal coverage, i.e. coverage averaged over measurements. Marginal coverage does not identify the posterior: the prior, which ignores the measurement entirely, also has valid marginal coverage. The second family regularizes with strictly proper scoring rules such as the continuous ranked probability score and the energy score. These need only samples and are minimized at the true posterior. However, they do not target coverage directly. Finite samples, model misspecification, and the geometry of the loss can leave models trained with them poorly calibrated in practice.
We propose TARP-CvM, a calibration loss that keeps the strengths of both families. It needs only samples, like scoring rules, and it targets coverage directly, like coverage objectives, while identifying the full posterior. We build on Tests of Accuracy with Random Points (TARP). For each measurement we pick a reference point and ask what fraction of the model's probability mass lies closer to this point than the true signal does. If the model equals the true posterior, the distribution of this fraction matches the observed frequency at every probability level. We measure departures with a Cramer-von Mises discrepancy. One reference rule on its own can hide errors: pooling coverage across rules or measurements allows over-coverage at some measurements to cancel under-coverage at others. We therefore compute the discrepancy separately for many measurement-dependent reference rules and average the results. The resulting loss is differentiable and estimated from model samples alone. It applies to any conditional generative model without density evaluations or changes to the architecture.
We make the following contributions:
We introduce TARP-CvM, a sample-based, differentiable calibration loss for conditional generative models that requires neither tractable densities nor architectural changes.
We show that, under a sufficient condition on the distribution of reference rules, TARP-CvM is a strict divergence: it vanishes if and only if the model equals the true posterior. Coverage objectives that enforce only marginal coverage lack this guarantee.
On simulation-based inference benchmarks, we show that TARP-CvM matches a density-based coverage loss without ever evaluating the density.
On high-dimensional imaging inverse problems, we show that TARP-CvM gives better-calibrated posteriors than scoring-rule regularizers. It is the only tested regularizer that brings the posterior within the calibration test's null band on all three problems, with little change in the quality of point estimates.
Together, these results show that training conditional generative models toward valid conditional coverage does not require densities or sacrifice reconstruction quality. Calibrated posterior sampling therefore becomes available to the full range of modern generative models used in scientific inverse problems.
Conditional generative models such as normalizing flows, generative adversarial networks (GANs) and diffusion models can sample posteriors in scientific inverse problems, but they are often overconfident, a critical failure when accurate uncertainty is required. Calibrating them is hard: valid conditional coverage is equivalent to the learned posterior equaling the true posterior, and it cannot be checked directly with one ground-truth signal per measurement. Existing regularizers have two drawbacks: (i) coverage objectives require the model's density and enforce only marginal coverage; and (ii) strictly proper scoring rules require only samples but do not target coverage. To resolve both, we propose TARP-CvM, a sample-based and conditional calibration loss that builds on Tests of Accuracy with Random Points (TARP) and requires no density or architectural change. Specifically, for each measurement, it computes the share of model mass that lies closer to a reference point than the true signal does; for a correct model, these shares match their observed frequency at each probability level. TARP-CvM measures departures from this match with a Cramer-von Mises discrepancy, computed separately for many measurement-dependent reference rules and then averaged, rather than pooling their coverage, which lets over- and under-coverage cancel. We show that, under a sufficient condition on the distribution of reference rules, the loss vanishes if and only if the learned posterior equals the true posterior. Experiments show that TARP-CvM matches a density-based coverage loss on simulation-based inference benchmarks without the density, and gives better-calibrated posteriors than scoring-rule regularizers on imaging inverse problems, with little change in point estimates.
Introduction
Scientific inverse problems in medical imaging, astrophysics and engineering ask for a signal that is observed only through indirect and noisy measurements. Conditional generative models, including normalizing flows, generative adversarial networks (GANs), diffusion and flow models, have emerged as an effective way to sample the posterior over signals given such measurements. However, these models are often overconfident: GANs suffer from mode collapse, flows tend to report overly narrow posteriors, and diffusion models under-represent posterior uncertainty. As a result, the sampled posterior can look precise while excluding the true signal, which is a critical failure in applications where accurate uncertainty is required.
The goal of calibration is valid conditional coverage (i.e., for each measurement, a region holding a given share of the posterior mass contains the true signal with that probability). This property is equivalent to the learned posterior equaling the true posterior, but it cannot be checked directly, since a dataset holds only one ground-truth signal per measurement. To target coverage during training, coverage objectives add a differentiable coverage term to the loss. Specifically, they compute the highest-probability-density (HPD) region of the model that reaches the true signal of each training pair, and penalize the gap between this coverage, averaged over the dataset, and the nominal level. These objectives share the same design: coverage is measured on density regions of the model and pooled over all measurements.
Intuitively, an effective calibration loss should be sample-based (i.e., computable from model samples alone) and conditional (i.e., zero only when coverage holds at each measurement, rather than on average). However, this shared design violates both properties:
Density dependence. Coverage objectives define HPD regions through the model's density, which requires a tractable likelihood. As a result, they are restricted to models such as normalizing flows, leaving GANs, diffusion and flow models, which provide samples but no density, without a coverage term.
Marginal coverage only. Coverage objectives pool coverage over all measurements, where over-coverage at some measurements can cancel under-coverage at others. As a result, the coverage term enforces only marginal coverage, which does not identify the posterior: the prior itself has valid marginal coverage.
A sample-based alternative is to train with strictly proper scoring rules, such as the continuous ranked probability score (CRPS) and the energy score, which require only samples and are minimized at the true posterior. However, these rules do not target coverage, and finite samples, misspecification and the geometry of the loss can leave the trained models poorly calibrated in practice. Therefore, we keep coverage in the loss and change how it is measured. To this end, we propose TARP-CvM, a sample-based and conditional calibration loss that builds on Tests of Accuracy with Random Points (TARP).
The key idea is to measure coverage on regions defined by distances between model samples and a reference point, and to keep one discrepancy per reference rule rather than pooling all rules into one coverage statistic. Specifically, for each measurement, a reference rule picks a reference point, and we compute the fraction of the model's probability mass that lies closer to this point than the true signal does. For a correct model, this fraction matches the observed frequency at each probability level, and we measure departures from this match with a Cramer-von Mises discrepancy. We compute the discrepancy separately for many measurement-dependent reference rules and then average it, because pooling their coverage would let over- and under-coverage at different measurements cancel. Estimated from samples alone, the resulting loss is differentiable and applies to any conditional generative model without densities or architectural changes. Moreover, we show that, under a sufficient condition on the distribution of reference rules, TARP-CvM is a strict divergence: it vanishes if and only if the learned posterior equals the true posterior. The distance-based regions remove the density dependence, while the per-rule discrepancies exclude marginal solutions such as the prior; together, they make calibration training both sample-based and conditional.
We then evaluate TARP-CvM on simulation-based inference benchmarks and high-dimensional imaging inverse problems. First, TARP-CvM matches a density-based coverage loss on the simulation-based inference benchmarks without evaluating the density. Second, on the imaging inverse problems, TARP-CvM gives better-calibrated posteriors than scoring-rule regularizers, and it is the only tested regularizer that brings the posterior within the null band of the calibration test (i.e., the range expected under calibration) on all three problems. Third, these gains come with little change in the quality of point estimates. We highlight that TARP-CvM makes calibration training sample-based and conditional, opening it to generative models that provide samples but no density.
Content brief:
- Area: imitation learning (IL) from offline demonstrations; continuous control.
- Context: IL learns skills directly from demonstrations and works well with large, high-quality data. Real datasets come from many human or algorithmic demonstrators whose skill levels differ and are unknown (e.g., driving data from many electric-vehicle drivers), so trajectories are of mixed quality; treating every demonstrator as an expert can make agents unreliable or unsafe.
- Problem: existing methods assume (i) homogeneous demonstrators, with optimal data dominating the dataset, (ii) labels that mark which demonstrations are optimal, or (iii) a known expertise level for each demonstrator. These requirements are unrealistic for uncontrolled, large-scale sources. The setting is hard because neither expertise nor optimality is observed, and all demonstrators may be poor.
- Formulation: each demonstrator is a mixture of a latent optimal policy and suboptimal behavior; its expertise is the probability of choosing actions aligned with the optimal policy, varies across demonstrators and is never observed (a single expert query resolves one remaining ambiguity).
- Method name: LEOS-IL (Latent Expertise and Optimality Scoring based Imitation Learning), in two stages. Offline, an EM-style alternating self-training jointly estimates each demonstrator's expertise and a scoring model that gives every state-action pair an optimality score. Online, the scoring model serves as a surrogate reward for policy optimization, and the agent's own rollouts iteratively refine it to counter distribution shift.
- Theory: guarantees for identifying the optimality score and for recovering the conditional optimal occupancy and policy under explicit sufficient conditions.
- Evidence: continuous-control benchmarks in a wide-range expertise setting and in a low-quality setting where all demonstrators are highly suboptimal; the method outperforms the baselines and is comparable to methods trained only on the purely optimal demonstrations.
Imitation learning (IL) lets agents acquire skills directly from demonstrations and works well when large, high-quality datasets are available. Real datasets, however, are usually pooled from many human or algorithmic demonstrators whose skill levels differ and are unknown, so the trajectories are of mixed quality. An agent that treats every demonstrator as an expert can become unreliable or unsafe. Existing approaches assume that demonstrators are homogeneous and that optimal data dominates, or they require labels marking optimal demonstrations or a known expertise level for each demonstrator. None of these assumptions holds for uncontrolled, large-scale data sources. We model each demonstrator as a mixture of a latent optimal policy and suboptimal behavior. A demonstrator's expertise is the probability that it chooses actions aligned with the optimal policy; expertise varies across demonstrators and is never observed. Within this formulation we propose LEOS-IL (Latent Expertise and Optimality Scoring based Imitation Learning), which has two stages. Offline, an EM-style alternating self-training procedure jointly estimates each demonstrator's expertise and a scoring model that assigns an optimality score to every state-action pair. Online, the scoring model serves as a surrogate reward for policy optimization, and the agent's own rollouts iteratively refine it to counter distribution shift. Under explicit sufficient conditions, we prove that the optimality score is identifiable and that the conditional optimal occupancy and policy can be recovered. On continuous-control benchmarks with wide-ranging expertise, and in a low-quality setting where every demonstrator is highly suboptimal, LEOS-IL outperforms baselines and performs comparably to methods trained only on purely optimal demonstrations.
Introduction
Imitation learning (IL) offers a direct path to acquiring skills: rather than hand-designing a reward function, an agent learns the desired behavior from demonstrations. When demonstrations are plentiful and of high quality, this approach is very effective, and it has become a natural choice for continuous control problems where specifying a reward is difficult. Large datasets, however, are rarely curated by a single expert. They are typically gathered from many human or algorithmic demonstrators whose skill varies widely and is not recorded. Driving data collected from a large fleet of electric-vehicle drivers is a typical example: some drivers are careful and efficient, others are not, and the dataset does not say which is which. The resulting trajectories are of mixed quality, and an agent that imitates all demonstrators as if they were experts inherits their mistakes. The agent can then become unreliable or even unsafe.
Prior work on imperfect demonstrations makes progress on this problem, but only under assumptions that rarely hold when data comes from uncontrolled, large-scale sources. Existing methods typically require one of the following:
the demonstrators are homogeneous, and optimal data dominates the dataset;
labels mark which demonstrations are optimal;
the expertise level of each demonstrator is known.
Each of these assumptions removes the core difficulty instead of addressing it. In the setting we consider, neither the expertise of a demonstrator nor the optimality of an individual action is observed. Moreover, nothing guarantees that a good demonstrator exists at all: every demonstrator may be poor. The learner must therefore work out what optimal behavior looks like and how far each demonstrator departs from it, using unlabeled and possibly low-quality data alone.
We approach this problem by modeling where suboptimality comes from. Each demonstrator is treated as a mixture of a shared latent optimal policy and its own suboptimal behavior. A demonstrator's expertise is the probability that it chooses an action aligned with the optimal policy. Expertise varies across demonstrators and is never observed. This formulation turns the vague notion of mixed-quality data into a latent-variable problem with clear structure: demonstrations from different demonstrators share a common optimal component but mix it in different proportions. The variation in mixing proportions is what makes the optimal component recoverable. One ambiguity remains even then, and a single expert query is enough to resolve it.
Building on this formulation, we propose LEOS-IL (Latent Expertise and Optimality Scoring based Imitation Learning), which has two stages. In the offline stage, an EM-style alternating self-training procedure jointly estimates each demonstrator's expertise and a scoring model that assigns an optimality score to every state-action pair. The two estimates reinforce each other: better expertise estimates sharpen the optimality scores, and better scores in turn refine the expertise estimates. In the online stage, the scoring model serves as a surrogate reward for policy optimization. As the agent improves, it visits states that are poorly covered by the demonstrations. To counter this distribution shift, we use the agent's own rollouts to iteratively refine the scoring model, so the reward remains informative where the policy actually operates.
We support the method with theory. Under explicit sufficient conditions, we prove that the optimality score is identifiable from mixed-quality demonstrations with unobserved expertise. We further show that the conditional optimal occupancy and the corresponding optimal policy can be recovered. These results clarify when learning from unlabeled demonstrators of heterogeneous and unknown skill is possible in principle, and they ground the design choices of LEOS-IL.
We evaluate LEOS-IL on continuous-control benchmarks in two regimes. In the first, demonstrator expertise spans a wide range. In the second, all demonstrators are highly suboptimal. In both regimes LEOS-IL outperforms the baselines, and its performance is comparable to that of methods trained only on purely optimal demonstrations, even though LEOS-IL never observes which demonstrations are optimal.
Our contributions are as follows:
We formulate imitation learning from heterogeneous demonstrators with unknown expertise, modeling each demonstrator as a mixture of a latent optimal policy and suboptimal behavior, without requiring dominant optimal data, optimality labels, or known expertise levels.
We propose LEOS-IL, which jointly estimates demonstrator expertise and state-action optimality scores offline through EM-style self-training, then uses the scores as a surrogate reward that the agent's rollouts refine online.
We provide guarantees for identifying the optimality score and for recovering the conditional optimal occupancy and policy under explicit sufficient conditions.
We show empirically that LEOS-IL outperforms baselines on continuous-control tasks in both wide-range and low-quality expertise settings, and matches the performance of methods trained only on purely optimal demonstrations.
While imitation learning (IL) works well with large, high-quality demonstrations, real datasets come from many demonstrators whose skill levels differ and are unknown. Treating them all as experts can make agents unreliable or unsafe. Existing methods assume that optimal data dominate, that labels mark the optimal demonstrations, or that each demonstrator's expertise is known. However, these assumptions face two drawbacks: (i) such labels and expertise levels are rarely available for uncontrolled, large-scale sources; and (ii) optimal data no longer dominate when all demonstrators are poor, leading the agent to imitate suboptimal behavior. To address these limitations, we propose Latent Expertise and Optimality Scoring based Imitation Learning (LEOS-IL), a scalable and robust method that treats demonstration quality as latent rather than observed. Specifically, LEOS-IL models each demonstrator as mixing a latent optimal policy with suboptimal behavior, where its expertise is the probability of choosing actions aligned with that policy. Offline, EM-style alternating self-training jointly estimates each demonstrator's expertise and a scoring model assigning an optimality score to each state-action pair. Online, this scoring model serves as a surrogate reward for policy optimization, and the agent's own rollouts iteratively refine it to counter distribution shift. Under explicit sufficient conditions, we provide guarantees for identifying the optimality score and recovering the conditional optimal occupancy and policy. Experiments on continuous-control benchmarks, where demonstrator expertise varies widely or all demonstrators are highly suboptimal, show that LEOS-IL generally outperforms the baselines and is comparable to methods trained only on purely optimal demonstrations.
Introduction
Imitation learning (IL) learns skills directly from demonstrations, and it works well when large, high-quality datasets are available. While important, collecting such data at scale remains challenging: real datasets come from many human or algorithmic demonstrators whose skill levels differ and are unknown (e.g., driving data collected from many electric-vehicle drivers). As a result, the trajectories in these datasets are of mixed quality. An agent that treats each demonstrator as an expert then imitates poor behavior along with good behavior, which can make it unreliable or even unsafe.
To learn from such mixed-quality demonstrations, existing methods supply the quality of the data in one of three ways. Specifically, one line of work assumes homogeneous demonstrators, with optimal data dominating the dataset, and lets the agent follow the dominant behavior of the data. A second line relies on labels that mark which demonstrations are optimal. A third line assumes that the expertise level of each demonstrator is known in advance. These methods show potential in reducing the influence of suboptimal demonstrations when the required information is available. Whether the quality comes from the composition of the dataset, from optimality labels or from known expertise levels, these methods share one design: the quality of each demonstration is treated as observed.
Intuitively, effective IL from mixed-quality demonstrations should be scalable (i.e., applicable to uncontrolled, large-scale sources without quality annotations) and robust (i.e., able to recover optimal behavior even when all demonstrators are poor). However, treating quality as observed can violate both properties. On the one hand, optimality labels and expertise levels must be supplied in advance for each demonstration or demonstrator, a requirement that is infeasible for uncontrolled, large-scale sources and thus limits scalability. On the other hand, the dominance assumption ties the learned policy to the dominant behavior of the data. When all demonstrators are poor, this dominant behavior is itself suboptimal, leading the agent to imitate it and limiting robustness. Achieving both properties is hard, since expertise and optimality are both unobserved and all demonstrators may be poor.
To this end, we propose Latent Expertise and Optimality Scoring based Imitation Learning (LEOS-IL), a scalable and robust IL method for mixed-quality demonstrations. The key idea is to treat the quality of each demonstration as latent rather than observed. To achieve this, LEOS-IL (i) infers the expertise of each demonstrator and the optimality of each state-action pair from the demonstrations alone, and (ii) optimizes the policy toward the inferred optimality rather than toward the dominant behavior of the data. Specifically, we model each demonstrator k as the mixture π_k = β_k π* + (1 − β_k) μ_k, where π* is the latent optimal policy, μ_k is the suboptimal behavior of demonstrator k, and the expertise β_k is the probability of choosing actions aligned with π*. This expertise varies across demonstrators and is never observed. In the offline stage, an EM-style alternating self-training jointly estimates β_k for each demonstrator and a scoring model that gives each state-action pair an optimality score. We show that, under explicit sufficient conditions, this optimality score is identifiable and the conditional optimal occupancy and policy can be recovered, with a single expert query resolving the one remaining ambiguity.
In the online stage, the scoring model serves as a surrogate reward for policy optimization. However, this model is learned on the states that the demonstrations visit, whereas the policy may visit new states as it improves (i.e., distribution shift). Therefore, we let the agent's own rollouts iteratively refine the scoring model during policy optimization. The offline stage removes the need for optimality labels and expertise levels, while the online stage frees the policy from the dominant behavior of the data; together, they make IL from mixed-quality demonstrations scalable and robust.
We then run experiments on continuous-control benchmarks in two settings: a wide-range expertise setting, where the expertise of the demonstrators varies widely, and a low-quality setting, where all demonstrators are highly suboptimal. First, LEOS-IL generally outperforms the baselines in the wide-range expertise setting. Second, LEOS-IL keeps this advantage in the low-quality setting, where optimal data no longer dominate. Third, LEOS-IL achieves results comparable to those of methods trained only on the purely optimal demonstrations. We highlight that LEOS-IL makes IL from mixed-quality demonstrations scalable and robust, requiring no optimality labels or expertise levels beyond a single expert query.
04 · FAILURE CASESSkill 替代不了人的思考The skill does not replace human thinking
Skill 能改变文字怎么写,却改变不了内容写什么:如果内容本身就是 AI 生成的,用 AI 改也纠正不过来。下面两种场景里,使用 skill 生成的文本仍被判为 AI:人类写作本身就被判为 AI 的两篇论文,以及只给题目、从零写成的论文。
The skill changes how a text is written rather than what it says. If the content is AI-generated to begin with, revising it with AI cannot correct it. In the two settings below, Pangram still labels the text generated with the skill as AI: two papers whose human-written text is itself labeled AI, and papers written from a topic alone.
在这些场景里,使用 skill 生成的文本仍被判为 AIIn these settings, the text generated with the skill is still labeled AI
Pangram 4.1 判为 AI 生成或 AI 辅助的文字占比:人类写作被判为 AI 的两篇论文,以及两篇只给题目的论文。Share of each text that Pangram 4.1 labels AI-generated or AI-assisted, for two papers whose human-written text is labeled AI and two papers written from a topic alone.
人类写作Human-writtenOpus 5.5 生成(不使用 skill)Generated by Opus 5.5 (without the skill)Opus 5.5 生成(使用 skill)Generated by Opus 5.5 (with the skill)
Pangram labels 87% and 99% of the human-written text of ANE (bilevel optimization) and CCA (federated DDoS detection) as AI. As shown in Figure 2, the text generated without the skill is labeled 100% AI on these papers, and the skill only reduces the share to 95% and 97%. The skill changes the style, reducing the habits from 19 and 15 to 1 and 2, but it does not change the content. We cannot verify how these two papers were written, and on such papers the researcher must still read and revise the generated text.
只给题目时,用不用 skill 都是 100% AIFrom a topic alone, both generated texts are 100% AI
We also give the agent only a topic, benchmark contamination in LLM evaluation or the scaling of test-time compute, and let it write a position paper from scratch. As shown in Figure 2, Pangram labels the abstracts and introductions of all four papers as 100% AI, with or without the skill. In these papers, the claims, arguments and examples all come from the model. With such AI-generated content, the skill changes only the wording.
Therefore, the skill can turn what a researcher has thought through into fluent prose and save drafting time, but it does not replace the researcher. A person must still decide what the paper says and how it argues, and must read and revise the generated text.
两篇论文和两个题目的提示词与生成文本Prompts and generated texts of the two papers and the two topics
原论文:Paper: Jason Bohne, Paweł Polak, Gary Kazantsev, and David Rosenberg. Your Hypergradient is Skewed: Antithetic Neumann Estimation for Bilevel Optimization. arXiv 2026. 我们逆向生成一个用于写论文的摘要和引言的 prompt 如下(使用 skill 和不使用 skill 的测试共享同一个生成 prompt)。We reverse-engineer the following prompt for writing the paper’s abstract and introduction; the runs with and without the skill share this generation prompt.
Content brief:
- Area: stochastic bilevel optimization, used for hyperparameter tuning, data reweighting, robust learning, reinforcement learning and imbalanced classification.
- Context: gradient-based bilevel methods need the hypergradient, which involves the inverse of the inner Hessian. The standard scalable approach approximates it with a truncated Neumann series, a product of stochastic Hessian factors. Existing stochastic and variance-reduced bilevel methods judge hypergradient estimators only by their bias and variance.
- Problem: the product structure of the Neumann series adds a third source of error, skewness. Unlike ordinary stochastic gradients, whose asymmetry is averaged away by mini-batches, asymmetric noise in each Hessian factor enters the odd-order terms of the product; under weak contraction the third-moment error grows with the truncation depth, the inner condition number and the noise asymmetry.
- When it matters: corrupted labels, outliers, and heterogeneous or class-imbalanced data, which make the Hessian noise asymmetric. Variance-reduced methods reduce variance across iterations but do not act on the product inside a single estimate.
- Method name: the Antithetic Neumann Estimator (ANE). It pairs each stochastic Hessian sample with its mirror image around a reference Hessian and averages the two resulting inverse-Hessian estimates. It changes only the Hessian-vector step and combines with existing methods: ANE-SGD, and ANE-VR, which adds recursive variance reduction.
- Theory: with an exact reference, the antithetic product keeps only even-order noise terms and cancels all odd-order ones, also for non-commuting matrices; with an approximate reference, the remaining odd-order error is bounded linearly in the reference error. An iterate-level convergence bound shows that the inverse-Hessian third-moment term shrinks.
- Evidence: robust regression, data cleaning, offline reinforcement learning and imbalanced classification, 20 seeds each. ANE-based methods improve on stochastic and variance-reduced baselines on every task, lowering loss by 4-25% relative to the best baseline; the largest gain is a 25% lower test loss on robust regression. Synthetic controls show no gain when the noise is symmetric.
Stochastic bilevel optimization underlies hyperparameter tuning, data reweighting, robust learning, reinforcement learning and imbalanced classification. Gradient-based bilevel methods need the hypergradient, which involves the inverse of the inner Hessian. The standard scalable approach approximates this inverse with a truncated Neumann series, which is a product of stochastic Hessian factors. Existing stochastic and variance-reduced methods judge such estimators only by their bias and variance. We show that the product structure adds a third source of error: skewness. Asymmetric noise in each Hessian factor enters the odd-order terms of the product. Under weak contraction, the resulting third-moment error grows with the truncation depth, the inner condition number and the degree of noise asymmetry. Corrupted labels, outliers and heterogeneous or class-imbalanced data all produce this kind of asymmetry. We propose the Antithetic Neumann Estimator (ANE), which pairs each stochastic Hessian sample with its mirror image around a reference Hessian and averages the two resulting inverse-Hessian estimates. With an exact reference, the antithetic product cancels every odd-order noise term, even when the matrices do not commute. With an approximate reference, the remaining odd-order error is bounded linearly in the reference error. ANE changes only the Hessian-vector step, so it plugs into existing methods. We study two instances: ANE-SGD and ANE-VR, which adds recursive variance reduction. We evaluate on robust regression, data cleaning, offline reinforcement learning and imbalanced classification, with 20 seeds per task. ANE-based methods lower loss by 4-25% relative to the best baseline on every task. Synthetic controls with symmetric noise show no gain, which supports skewness as the mechanism behind the improvement.
Introduction
Many learning problems have a nested structure. An outer objective depends on the solution of an inner optimization problem. Hyperparameter tuning is one example: validation loss is minimized over hyperparameters that shape a training problem. Other examples include data reweighting, robust learning, reinforcement learning and imbalanced classification, where outer variables such as sample weights, policies or class-dependent parameters steer an inner learner. Stochastic bilevel optimization gives a common framework for these problems. Gradient-based methods have become the standard way to solve them at scale.
The main computational step in these methods is the hypergradient, the gradient of the outer objective with respect to the outer variables. It involves the inverse of the inner Hessian. Forming or inverting that matrix is impractical in high dimensions. The usual scalable alternative approximates the inverse with a truncated Neumann series, which only needs stochastic Hessian-vector products. Each term of the series is a product of independently sampled Hessian factors. Many stochastic bilevel methods are built on this estimator, including variance-reduced ones. Their analyses judge it by two quantities: its bias, which comes from truncation and sampling, and its variance.
We argue that bias and variance leave out an important part of the error. The product structure of the Neumann series adds a third source, skewness. For ordinary stochastic gradients, asymmetry in the noise distribution is largely averaged away by mini-batching. The Neumann estimator behaves differently. When the noise in each Hessian factor is asymmetric, it enters the odd-order terms of the product and compounds across factors. Under weak contraction, the third-moment error grows with the truncation depth, the inner condition number and the degree of noise asymmetry. In short, the regimes that need long Neumann series are the same regimes in which skewness grows.
This is a practical concern, because the settings in which bilevel methods are most often used tend to make Hessian noise asymmetric. Corrupted labels, outliers, heterogeneous data and class imbalance all produce mini-batch Hessians whose deviations from the mean are lopsided rather than symmetric. Variance reduction does not fix this. Variance-reduced methods lower the variance of estimates across iterations, but they do not act on the product of factors inside a single estimate, which is where the skewness arises. An estimator can therefore have low variance and still carry substantial odd-order error.
We propose the Antithetic Neumann Estimator (ANE) to address this error directly. For each stochastic Hessian sample, ANE builds its mirror image around a reference Hessian. It then computes the inverse-Hessian estimate from both the original and the mirrored samples and averages the two. Because the mirrored deviations have the opposite sign, the odd-order contributions of the two estimates cancel when averaged, while the even-order contributions remain. ANE changes only the Hessian-vector step of the hypergradient computation, so it can be combined with existing bilevel methods. We study two instances. ANE-SGD uses the estimator inside a stochastic bilevel method. ANE-VR adds recursive variance reduction, so that variance and skewness are reduced together.
We support the method with theory at three levels. With an exact reference Hessian, the antithetic product keeps only even-order noise terms and cancels all odd-order ones. This holds even though the Hessian factors do not commute, so the cancellation is not limited to a scalar or diagonal case. With an approximate reference, as used in practice, the remaining odd-order error is bounded linearly in the reference error, so ANE degrades gracefully as the reference becomes less accurate. At the iterate level, a convergence bound shows that the third-moment term contributed by the inverse-Hessian estimate shrinks under ANE.
We test ANE on four tasks: robust regression, data cleaning, offline reinforcement learning and imbalanced classification, with 20 seeds for each. ANE-based methods outperform stochastic and variance-reduced baselines on every task. They lower loss by 4-25% relative to the best baseline, and the largest gain is a 25% lower test loss on robust regression. Synthetic controls show no gain when the noise is symmetric. This is consistent with our analysis and indicates that the improvement comes from cancelling skewness rather than from some other effect.
Our contributions are as follows:
We identify skewness as a third source of error in Neumann-series hypergradient estimators, separate from bias and variance. We show that under weak contraction it grows with truncation depth, condition number and noise asymmetry.
We propose the Antithetic Neumann Estimator, a drop-in replacement for the Hessian-vector step, and two methods built on it: ANE-SGD and ANE-VR.
We prove that ANE cancels all odd-order noise terms with an exact reference, including for non-commuting matrices. We also bound the remaining odd-order error linearly in the reference error and show that the inverse-Hessian third-moment term shrinks in the convergence bound.
We show consistent gains over stochastic and variance-reduced baselines on four bilevel tasks. Synthetic controls tie the gains to asymmetric noise.
Stochastic bilevel optimization underlies hyperparameter tuning, data reweighting, robust learning and reinforcement learning, and its hypergradient involves the inverse of the inner Hessian. Existing stochastic and variance-reduced methods approximate this inverse with a truncated Neumann series (i.e., a product of stochastic Hessian factors) and judge the resulting estimator only by its bias and variance. However, this product adds a third source of error, skewness (i.e., the third moment of the estimation error), because asymmetric noise in each Hessian factor enters the odd-order terms of the product. This error increases with the truncation depth, the inner condition number and the noise asymmetry. It also persists under variance reduction, which acts across iterations rather than inside each estimate. To address this, we propose the Antithetic Neumann Estimator (ANE), which makes hypergradient estimates symmetric and low-variance. Specifically, ANE pairs each stochastic Hessian sample with its mirror image around a reference Hessian and averages the two resulting inverse-Hessian estimates. Because it changes only the Hessian-vector step, ANE combines with plain stochastic updates (ANE-SGD) and with recursive variance reduction (ANE-VR). With an exact reference, the antithetic product cancels all odd-order noise terms, even for non-commuting matrices. With an approximate reference, the remaining odd-order error is bounded linearly in the reference error. Experiments on robust regression, data cleaning, offline reinforcement learning and imbalanced classification show that ANE-based methods outperform stochastic and variance-reduced baselines on each task, while synthetic controls show no gain under symmetric noise.
Introduction
Stochastic bilevel optimization, in which an outer objective depends on the solution of an inner problem, underlies hyperparameter tuning, data reweighting, robust learning, reinforcement learning and imbalanced classification. Gradient-based bilevel methods update the outer variables with the hypergradient, which involves the inverse of the inner Hessian. To keep this step scalable, the standard approach approximates the inverse with a truncated Neumann series. Specifically, each inverse-Hessian estimate sums products of the form (I - ηH_1)(I - ηH_2)...(I - ηH_k) up to a truncation depth K, where η is a step size and each stochastic Hessian factor H_j is drawn independently from a mini-batch. Stochastic bilevel methods use these factors directly. Variance-reduced methods also apply recursive corrections across iterations. Both families have shown promise in reducing hypergradient error, and both judge an estimator by its bias and variance.
Whether stochastic or variance-reduced, these methods share the same design: they build each inverse-Hessian estimate as a product of independently drawn Hessian factors, and they control its error only through its first two moments. Intuitively, an effective hypergradient estimator should be symmetric (i.e., its noise should not favor one direction within a single estimate) and low-variance (i.e., its estimates should vary little across iterations). Variance-reduced methods target the second property. However, the product structure adds a third source of error, skewness (i.e., the third moment of the estimation error), and existing methods leave it uncontrolled for two reasons.
On the one hand, asymmetric noise survives mini-batching. For ordinary stochastic gradients, mini-batches average such asymmetry away. In contrast, within the Neumann product the asymmetric noise of each independently drawn factor enters the odd-order terms, thus accumulating across the factors. Under weak contraction (i.e., when each factor reduces vector norms only slightly), the resulting third-moment error increases with the truncation depth, the inner condition number and the noise asymmetry. On the other hand, variance reduction does not reach the product. Recursive corrections reduce the variance across iterations, but they leave the product inside each estimate unchanged. As a result, a variance-reduced estimate can be low-variance and still skewed. This problem is amplified when corrupted labels, outliers, or heterogeneous and class-imbalanced data make the Hessian noise asymmetric.
To address these issues, we propose the Antithetic Neumann Estimator (ANE), which applies antithetic sampling (i.e., pairing each random draw with a mirrored copy so that their deviations offset) to the Neumann series to make hypergradient estimates symmetric and low-variance. The key idea is to draw the Hessian factors in mirrored pairs rather than independently, so that each inverse-Hessian estimate controls its third moment in addition to its first two. Specifically, ANE pairs each stochastic Hessian sample H with its mirror image 2R - H around a reference Hessian R, and averages the two resulting inverse-Hessian estimates. The odd-order noise terms then appear with opposite signs in the two products. However, the expected Hessian is unknown in practice, and the mirror is only as accurate as its reference. We therefore analyze both cases. With an exact reference, the antithetic product keeps only even-order noise terms and cancels all odd-order ones, even for non-commuting matrices. With an approximate reference, the remaining odd-order error is bounded linearly in the reference error. An iterate-level convergence bound also shows that the inverse-Hessian third-moment term decreases under ANE. Because ANE changes only the Hessian-vector step, it combines with existing methods: ANE-SGD uses plain stochastic updates, and ANE-VR adds recursive variance reduction. The antithetic pairing cancels odd-order terms inside each estimate, which mitigates skewness. The unchanged outer update keeps the variance reduction across iterations. Together, they make hypergradient estimates symmetric and low-variance.
We then run experiments on robust regression, data cleaning, offline reinforcement learning and imbalanced classification, with 20 seeds per task. First, ANE-based methods outperform stochastic and variance-reduced baselines on all four tasks, reducing the loss by 4-25% compared to the best baseline. Second, the largest gain, a 25% lower test loss, appears on robust regression, where outliers are likely to make the Hessian noise strongly asymmetric. Third, synthetic controls show no gain when the noise is symmetric, which suggests that the improvement comes from cancelling odd-order terms rather than from averaging two estimates. We highlight that ANE requires no change beyond the Hessian-vector step, making hypergradient estimates symmetric and low-variance within existing bilevel methods.
Content brief:
- Area: federated learning for network intrusion (DDoS) detection across distributed sensors.
- Context: privacy rules and bandwidth prevent pooling raw traffic records, so detection runs at each organization's own vantage point (edge routers, gateways, service endpoints), and no sensor sees the full traffic mix. Federated learning trains a shared model without sharing raw data. For binary attack-versus-benign detection, federated averaging (FedAvg) is close to centralized training, because every client holds both classes.
- Problem 1: multiclass detection produces class-concentrated non-IID data: an attack class such as LDAP or MSSQL amplification is held almost entirely by one client. FedAvg then collapses that class in a specific way: the output-layer neuron of the class receives an active suppression signal from the many clients that never saw it (cross-entropy pushes down the logits of absent classes), not mere dilution. On CIC-DDoS2019 the concentrated LDAP class falls to zero F1 on every seed.
- Problem 2: the closest prior remedy keeps a small globally shared dataset, which reintroduces the data sharing that federated learning exists to avoid; other federated methods recover the class only unreliably.
- Observation: the failure is confined to the output layer, the only parameter block that belongs to a single class; the hidden layers are unaffected.
- Method name: Class-Competence Aggregation (CCA). Each client reports its per-class validation F1, and the server uses these values as per-class weights to aggregate the output layer only; the hidden layers keep standard size-weighted FedAvg. It needs no server-held data and adds one F1 value per class per round.
- Evidence: on CIC-DDoS2019 with a CNN-LSTM, CCA alone brings the LDAP class back from collapse, and the full method closes most of the gap between federated and centralized macro F1. Compared with FedAvg, FedProx, FedNova and FedDWC under identical partitions and seeds, only CCA recovers the concentrated class substantially and reliably (FedNova and FedDWC recover it on some seeds only). A smaller MLP shows the same qualitative recovery, results hold over three seeds and on a second dataset, CICIDS2017, and a single unseeded ablation run was found to misattribute credit, so all reported results are seeded.
Federated learning lets distributed network sensors train a shared intrusion detector without pooling raw traffic. That matters because privacy rules and bandwidth limits keep traffic records at each organization's vantage point. For binary attack-versus-benign detection, federated averaging (FedAvg) comes close to centralized training. Multiclass DDoS detection is harder. Attack classes such as LDAP or MSSQL amplification are often held almost entirely by a single client. We show that FedAvg then collapses the concentrated class, and the cause is active suppression rather than dilution. The output neuron for that class is pushed down by the cross-entropy gradients of the many clients that never observe it. On CIC-DDoS2019, the concentrated LDAP class drops to zero F1 on every seed. The failure is confined to the output layer, the only parameter block that belongs to a single class. Building on this, we propose Class-Competence Aggregation (CCA). Each client reports its per-class validation F1, and the server uses these scores as per-class weights when it aggregates the output layer. The hidden layers keep standard size-weighted FedAvg. CCA needs no server-held data and adds only one F1 value per class per round. With a CNN-LSTM on CIC-DDoS2019, CCA alone brings LDAP back from collapse, and the full method closes most of the gap between federated and centralized macro F1. We compare against FedAvg, FedProx, FedNova and FedDWC under identical partitions and seeds. Only CCA recovers the concentrated class substantially and reliably. The findings hold for a smaller MLP, over three seeds, and on CICIDS2017.
Introduction
Distributed denial-of-service (DDoS) attacks are detected at many vantage points: edge routers, gateways, and service endpoints run by different organizations. Each sensor sees only part of the traffic mix. A single detector trained on the union of all traffic would be ideal, but privacy regulation and bandwidth constraints usually rule out pooling raw traffic records. Federated learning addresses this. Clients train locally and share only model updates, and a server aggregates the updates into a shared model. For binary detection, which separates attack from benign traffic, this works well. Every client holds examples of both classes, and federated averaging (FedAvg) performs close to centralized training.
Operators often need to know which attack is happening, not just whether one is. Amplification attacks that abuse LDAP, MSSQL, or other protocols call for different mitigations. Multiclass detection, however, produces a severe form of non-IID data. In binary detection the clients differ only in their class proportions. In multiclass detection whole attack classes are concentrated at one client. LDAP amplification traffic, for example, may be observed almost entirely by a single sensor, while every other client has no examples of it.
Under this class-concentrated partition, FedAvg does more than underperform: it erases the concentrated class. On CIC-DDoS2019 the LDAP class falls to zero F1 on every seed we ran. The intuitive explanation is dilution: the one client that knows the class is outvoted in the average. We argue that the actual mechanism is stronger. Cross-entropy training pushes down the logits of classes absent from a client's data. Each client that never sees LDAP therefore sends an active suppression signal to the LDAP output neuron. These signals are averaged together with the single informative update from the client that holds the class. The many push that neuron's logit down and the one pushes it up, so the class collapses.
Existing remedies handle this setting poorly. The closest prior approach keeps a small globally shared dataset to rebalance training, but this reintroduces the data sharing that federated learning is meant to avoid. Methods designed for general heterogeneity, such as FedProx, FedNova, and FedDWC, recover the concentrated class only unreliably. In our experiments FedNova and FedDWC recover it on some seeds but not others, which is not acceptable for a security system.
Our starting point is an observation about where the failure occurs. The collapse is confined to the output layer, which is the only parameter block that belongs to a single class. The hidden layers learn representations shared across classes and are unaffected. So there is no need to redesign the whole aggregation scheme; a targeted fix is enough. When the server averages the output neuron for a class, it should weight each client by how well that client actually recognizes the class. It should not weight clients by dataset size, because that lets the many clients that have never seen the class dominate.
We call this method Class-Competence Aggregation (CCA). Each round, every client reports its per-class validation F1. The server uses these values as per-class weights to aggregate the output layer, and it aggregates the hidden layers with standard size-weighted FedAvg. Clients that never see LDAP contribute little to the LDAP neuron, while the client that holds the class dominates it. CCA needs no server-held data, leaves local training unchanged, and adds only one scalar per class per round to the communication.
We evaluate CCA on CIC-DDoS2019 with a CNN-LSTM detector. CCA alone brings the LDAP class back from collapse, and the full method closes most of the gap between federated and centralized macro F1. We compare against FedAvg, FedProx, FedNova, and FedDWC under identical partitions and seeds. Only CCA recovers the concentrated class both substantially and reliably. A smaller MLP shows the same qualitative recovery. The results hold over three seeds and on a second dataset, CICIDS2017. We also report a methodological lesson. A single unseeded ablation run misattributed credit among components of the method, so every result we report is seeded.
Our contributions are as follows:
We identify class-concentrated non-IID data as the characteristic failure mode of federated multiclass intrusion detection. We show that FedAvg collapses concentrated classes through active suppression of their output neurons, not mere dilution.
We localize the failure to the output layer and propose Class-Competence Aggregation. It weights output-layer aggregation by per-class validation F1, needs no shared data, and adds negligible communication.
We show on CIC-DDoS2019 and CICIDS2017, across two architectures and three seeds, that CCA reliably recovers collapsed classes and closes most of the federated-to-centralized macro F1 gap, where FedAvg, FedProx, FedNova, and FedDWC do not.
Federated learning (FL) lets distributed network sensors train a shared intrusion detector without pooling raw traffic. For binary attack-versus-benign detection, federated averaging (FedAvg) comes close to centralized training. However, multiclass detection produces class-concentrated data, in which an attack class is held almost entirely by one client. In this setting, current practice faces two drawbacks: (i) clients that never saw a class push down its logit under cross-entropy, actively suppressing its output neuron until the class collapses; and (ii) the closest remedy restores the class only through a globally shared dataset, reintroducing the data sharing that FL is meant to avoid. We observe that this failure is limited to the output layer, the only parameter block that belongs to a single class. Based on this observation, we propose Class-Competence Aggregation (CCA), a class-aware and data-free aggregation rule for multiclass FL. Specifically, each client reports its per-class validation F1, and the server uses these values as per-class weights to aggregate the output layer, while the hidden layers keep size-weighted FedAvg. CCA requires no server-held data and only adds one F1 value per class per round. Experiments across datasets and model architectures show that CCA brings the concentrated class back from collapse and closes most of the gap to centralized training, whereas other federated methods recover the class only unreliably.
Introduction
Detecting distributed denial-of-service (DDoS) attacks increasingly relies on sensors that many organizations place at their own vantage points (e.g., edge routers, gateways and service endpoints). Privacy rules and bandwidth limits prevent these organizations from pooling raw traffic records. As a result, each sensor learns only from what its own vantage point observes, and no sensor sees the full traffic mix. Federated learning (FL) fits this setting, since it trains a shared model across sensors without moving raw data. For binary detection (i.e., attack versus benign), federated averaging (FedAvg) comes close to centralized training, because each client holds both classes.
Moving from binary to multiclass detection changes this picture. Specifically, multiclass traffic yields class-concentrated non-IID data: an attack class such as LDAP or MSSQL amplification is held almost entirely by one client. In each round, FedAvg averages the local updates of all clients, from the hidden layers to the output neuron of each class, weighting each client by its data size. Each client, in turn, trains with cross-entropy over all classes, including the classes it has never observed. Most aggregation rules, FedAvg included, share the same design: one weight per client applies to all parameter blocks, regardless of which classes the client can recognize. Intuitively, effective aggregation for multiclass detection should be class-aware (i.e., the output neuron of each class is shaped mainly by the clients that can recognize that class) and data-free (i.e., the server holds no traffic records of its own).
However, current practice falls short on both properties. (i) Size-weighted averaging suppresses concentrated classes. Since cross-entropy pushes down the logits of absent classes, each client that never saw a class sends an update that actively lowers the output neuron of that class. With one weight per client, the many clients without the class outweigh the single client that holds it, suppressing the class rather than merely diluting it. As illustrated in Figure 1, the concentrated class then collapses: on CIC-DDoS2019, the LDAP class falls to zero F1 on all seeds. (ii) Existing remedies give up either data-freeness or reliability. The closest prior remedy keeps a small globally shared dataset, which reintroduces the data sharing that FL exists to avoid. Other federated methods for non-IID data remain data-free, yet they recover the concentrated class only unreliably. These limitations call for an aggregation rule that is both class-aware and data-free.
Moreover, the collapse is limited to the output layer, the only parameter block that belongs to a single class, while the hidden layers remain unaffected. Based on this observation, we propose Class-Competence Aggregation (CCA), a class-aware and data-free aggregation rule for multiclass FL. The key idea is to give each client one weight per class for the output layer, set by its competence on that class, rather than one weight per client for all parameter blocks. Specifically, each client reports its per-class validation F1, and the server uses these values as per-class weights to aggregate the output neuron of each class. The hidden layers keep standard size-weighted FedAvg, since the collapse does not reach them. In this way, a client that never saw a class receives little weight on the output neuron of that class, and its suppression signal no longer outweighs the client that holds the class. Together, the per-class weights keep aggregation class-aware, and the client-side F1 reports keep it data-free, adding only one value per class per round with no server-held data.
We evaluate CCA on CIC-DDoS2019 with a CNN-LSTM detector. First, CCA alone brings the LDAP class back from collapse, and the full method closes most of the gap between federated and centralized macro F1 (cf. Section 4.1). Second, compared with FedAvg, FedProx, FedNova and FedDWC under identical partitions and seeds, only CCA recovers the concentrated class substantially and reliably, while FedNova and FedDWC recover it on some seeds only (cf. Section 4.2). Third, a smaller multilayer perceptron (MLP) shows the same qualitative recovery, and the results hold over three seeds and on a second dataset, CICIDS2017 (cf. Section 4.3). All reported results are seeded, because a single unseeded ablation run was found to misattribute credit. We highlight that CCA makes multiclass federated detection class-aware and data-free without sharing any traffic records with the server.
写作模型只拿到提示词里的一句题目,要写一篇 2,000 到 4,000 词的立场论文(position paper),没有内容提要。使用 skill 的提示词在开头多了阅读 skill 文件的步骤,其余相同。下面是两篇论文的摘要和引言,也就是送去 Pangram 检测的部分;两篇全文都在 3,000 词以上。The writer receives only the one-sentence topic in the prompt and is asked for a position paper of 2,000 to 4,000 words, with no content brief. The prompt with the skill adds steps to read the skill’s files first and is otherwise the same. Below are the abstract and introduction of both papers, which is the part scanned by Pangram. Both full papers run past 3,000 words.
不使用 skill 的提示词Prompt without the skill: the topic is its last line
You are writing an ML position paper.
Write an ML position paper of 2,000-4,000 words (aim for about 3,000) on the topic below, with these sections: Abstract, Introduction, Background, two or three main sections that develop the argument, Limitations, and Conclusion.
Write it the way you normally would for a strong ICLR / NeurIPS / ICML submission.
Output format (the same for every writer in this study; it is not a style instruction):
- Plain text only: no LaTeX commands (write i.e. and e.g., never \ie or \eg). The only markdown allowed is one `## ` line per section heading (start with `## Abstract`). If you use a list, put each item on its own line starting with `* `.
- No citations, reference markers or URLs. You may name widely known benchmarks, models or methods that you are sure exist.
- Do not invent numbers or experimental results.
- Reply with the paper only, starting with `## Abstract`: no title line, no preamble, no notes, no closing remark.
Topic: Benchmark contamination in large language models: why scores on public benchmarks can overstate what a model can do when test items appear in its training data, and how an evaluation can guard against it.
Public benchmarks are still the main way the machine learning community compares large language models. Their value depends on an assumption that is rarely stated and is becoming hard to defend: that the test items were not seen during training. When a model is pretrained on trillions of tokens scraped from the open web, and benchmark items are published, discussed, translated and paraphrased on that same web, the assumption fails by default. We argue that benchmark contamination should be treated as the expected condition of public evaluation rather than as an occasional accident. A score on a contaminated benchmark measures a mixture of capability and recall, and the evaluator usually cannot tell how much of each. We describe the ways contamination inflates scores, including verbatim memorization, near-duplicate exposure, answer-format leakage and indirect exposure through synthetic data, and we explain why common detection methods give only weak guarantees. We then argue for an evaluation practice organized around three commitments. First, contamination claims should be framed as bounds with stated assumptions, not as binary declarations of cleanliness. Second, held-out, dynamic and perturbation-based evaluation should be standard rather than optional. Third, reporting norms should make the provenance of evaluation data and the evaluator's access to training data explicit. We close by discussing the costs and limits of these proposals, including the fact that no protocol can fully certify a closed model.
Introduction
A benchmark score is a claim about generalization. When we say a model reaches some accuracy on a reading comprehension or mathematical reasoning benchmark, we mean, usually without saying it, that the model would do about as well on new problems drawn from the same distribution. This claim holds only if the benchmark items work as a sample from that distribution that the model has not seen. Classical machine learning enforced this through the train/test split. The split was a property of the experimental design: the researcher controlled both sides of it and could check that they did not overlap.
Large language models broke this arrangement. A model developer does not build a pretraining corpus by choosing examples for a task. They build it by gathering as much text as possible from sources such as web crawls, code repositories, books, forums and, more and more, model-generated data. The benchmark creator, for their part, publishes test sets openly so that results can be reproduced. These two practices interact in an obvious way. Test items released on the web become candidates for inclusion in the next crawl. Even when the original files are filtered out, the items come back in blog posts, solution repositories, tutorial notebooks, discussion threads, translations and study guides. Separating training data from test data is no longer something the experimenter controls. It depends on the state of the internet.
The consequence is that a large share of reported benchmark results rests on an unverified premise. Some high scores may reflect genuine capability. Others may partly reflect that the model has seen the question, the answer, or a close variant. The difficulty is not only that contamination happens. It is that from the outside, a contaminated success and a genuine success look the same in a single number.
This paper takes a position on how the community should respond. Our central claim is this:
Contamination is the default state of public benchmarks for web-scale models. Evaluation practice should therefore move from asking "was this benchmark contaminated?" to asking "what evidence bounds how much of this score could be explained by exposure?" It should also adopt held-out and dynamic evaluation as the primary basis for capability claims.
We develop this claim in three steps. We first describe the mechanisms by which contamination inflates scores and why these mechanisms are more varied than verbatim memorization. We then examine the detection methods currently used and argue that each gives at most one-sided evidence. Finally, we propose concrete evaluation practices and reporting norms. We also take seriously the arguments against our position, including the view that contamination is overstated, that memorized knowledge is still capability, and that the costs of private evaluation are too high.
Public benchmarks largely decide which large language models (LLMs) are reported as capable, yet reading a benchmark score as a measure of capability is hard: test sets are posted on the open web, and the corpora that train LLMs are crawled from that same web and are rarely disclosed. When test items appear in training data (i.e., benchmark contamination), a model can answer them by recall, and its score overstates what it can do on new inputs. This paper takes the position that the problem lies less in individual models than in one design shared by current evaluation: a single fixed test set, published once, scores each model trained after its release, and the claim that the model has not seen the set relies on the developer's own statement. However, this design suffers from two key drawbacks: (i) fixed public items are exposed to later training corpora; and (ii) this exposure cannot be checked by anyone without access to those corpora. To address both, we propose an unseen and verifiable evaluation. Specifically, it scores each model on items released after the model's training cutoff or held in rotating private splits, and it attaches to each score a contamination report (i.e., the training cutoff, the overlap audit and paired scores on public and fresh items). Together, these two practices let a benchmark score be read as evidence of capability rather than of exposure.
1 Introduction
Public benchmarks such as MMLU, GSM8K and HumanEval have become the main basis for comparing large language models (LLMs), and a gain of a few points often decides which model is adopted, cited or deployed. Such decisions are sound only if a score measures how a model handles inputs it has not seen before. While important, securing this premise remains challenging: (a) benchmark test sets are posted on the open web, often together with their answers, and (b) the corpora that train LLMs are crawled from that same web at a scale that no one inspects item by item. As a result, the gap between what a score reports and what a model can do has become a central question for the credibility of LLM evaluation.
The prevailing practice is simple and has served the field well. A benchmark team collects a fixed set of test items, publishes them with their reference answers to allow reproduction, and freezes the set to keep scores comparable over years. Model developers then train on web-scale corpora collected up to a cutoff date, evaluate on the frozen set, and report the score, often with a statement that the benchmark was excluded from training. Whether a benchmark tests knowledge, mathematical reasoning or code, these evaluations share one design: a single fixed test set, published once, scores each model trained after its release, and the claim that the model has not seen the set relies on the developer's own statement.
Intuitively, an effective evaluation should be unseen (i.e., its test items are absent from the training data of each model it scores) and verifiable (i.e., parties other than the model developer can check that absence). However, the shared design meets neither property, for two reasons. First, because the test set is published with its answers and frozen for years, its items are copied into forums, tutorials and solution repositories, and later crawls collect these copies into training corpora. A model may then reproduce the answer to a GSM8K problem it has read together with a worked solution, and its score mixes recall with reasoning, violating the unseen property. Second, because the claim of exclusion relies on the developer's statement, and the training corpora of most frontier models are not released, outsiders cannot inspect the overlap. Consequently, detection must work from model outputs alone, which paraphrased or partial exposure easily evades, leaving the verifiable property unmet.
This paper takes the position that benchmark contamination should be treated as a property of the evaluation design rather than as a defect of individual models. The key idea is to replace a single fixed test set, published once and trusted on the developer's statement, with (i) test items that are renewed or withheld to ensure that each model meets items it cannot have seen, and (ii) contamination reports that publish the evidence others require to check this claim. However, the most direct way to keep items unseen, a private test set, removes exactly the evidence that outsiders require, since only the benchmark owner can inspect the items. We therefore pair time separation and rotating private splits with reports that state the training cutoff, the overlap audit and paired scores on public and fresh items. Time separation keeps test items unseen, mitigating their exposure to later corpora, while contamination reports make that separation checkable; together, they yield an unseen and verifiable evaluation.
We develop this position in three steps. First, we trace how exposure inflates scores, and why the inflation is uneven across models and therefore distorts rankings as well as absolute numbers (cf. Section 3). Second, we examine why current detection methods cannot make exposure verifiable without access to training data (cf. Section 4). Third, we describe practices that make an evaluation unseen and verifiable, together with the trade-off they must resolve (cf. Section 5). We highlight that an unseen and verifiable evaluation requires no change to how models are trained, only to how benchmarks are released and how scores are reported.
写作模型只拿到提示词里的一句题目,要写一篇 2,000 到 4,000 词的立场论文(position paper),没有内容提要。使用 skill 的提示词在开头多了阅读 skill 文件的步骤,其余相同。下面是两篇论文的摘要和引言,也就是送去 Pangram 检测的部分;两篇全文都在 3,000 词以上。The writer receives only the one-sentence topic in the prompt and is asked for a position paper of 2,000 to 4,000 words, with no content brief. The prompt with the skill adds steps to read the skill’s files first and is otherwise the same. Below are the abstract and introduction of both papers, which is the part scanned by Pangram. Both full papers run past 3,000 words.
不使用 skill 的提示词Prompt without the skill: the topic is its last line
You are writing an ML position paper.
Write an ML position paper of 2,000-4,000 words (aim for about 3,000) on the topic below, with these sections: Abstract, Introduction, Background, two or three main sections that develop the argument, Limitations, and Conclusion.
Write it the way you normally would for a strong ICLR / NeurIPS / ICML submission.
Output format (the same for every writer in this study; it is not a style instruction):
- Plain text only: no LaTeX commands (write i.e. and e.g., never \ie or \eg). The only markdown allowed is one `## ` line per section heading (start with `## Abstract`). If you use a list, put each item on its own line starting with `* `.
- No citations, reference markers or URLs. You may name widely known benchmarks, models or methods that you are sure exist.
- Do not invent numbers or experimental results.
- Reply with the paper only, starting with `## Abstract`: no title line, no preamble, no notes, no closing remark.
Topic: Scaling test-time compute for reasoning: when letting a language model think longer at inference time improves accuracy, and when the extra computation stops helping.
Allocating more computation at inference time has become a central lever for improving the reasoning performance of large language models. Longer chains of thought, repeated sampling with verification, tree search, and iterative self-refinement can all yield accuracy gains that rival or exceed those from scaling parameters or pretraining data. This position paper argues that the field has adopted an overly monolithic picture of test-time compute. In that picture, "thinking longer" is a single dial with a smooth, monotone return curve. We contend that test-time compute helps only when three conditions hold together: the model can already generate a correct solution with non-negligible probability, some signal exists to select or steer toward it, and the problem's difficulty matches the structure of the extra computation. When any of these fails, more compute plateaus, wastes resources, or reduces accuracy through overthinking, error accumulation, and selection against correct answers. We break test-time scaling into a coverage component and a selection component. We describe the regimes in which each one saturates, and we argue that evaluation practice should report full compute-accuracy curves under matched budgets rather than single operating points. We end with recommendations for adaptive compute allocation and for benchmarks that can tell real reasoning gains apart from artifacts of sampling.
Introduction
For most of the history of neural language modeling, compute was something spent before deployment. Parameters, tokens, and training FLOPs were scaled, and inference was treated as a fixed, cheap forward pass per token. That assumption has broken down. Chain-of-thought prompting showed that letting a model write intermediate steps can sharply improve performance on arithmetic and symbolic tasks. Self-consistency showed that sampling many reasoning paths and taking a majority vote improves on a single greedy decode. More recently, models trained with reinforcement learning to produce long internal deliberations have shown large gains on competition mathematics, programming contests, and scientific question answering. These gains are often shown as accuracy rising with the number of reasoning tokens.
These results have produced a slogan: inference-time compute is a new scaling axis. The slogan is partly right. It has also encouraged a habit of reporting best-case curves, treating token count as a stand-in for reasoning effort, and assuming that a model which thinks longer is thinking better. Practitioners now see a growing set of counterexamples. Models spend thousands of tokens on trivial questions. Accuracy falls when reasoning traces are forced to be longer. Majority voting converges confidently on wrong answers. Reward-model reranking improves at first and then gets worse as the candidate pool grows.
Our position is that test-time compute is not one resource with one scaling law. It is a family of mechanisms. Each one turns computation into accuracy only under specific conditions, and each has its own failure mode. Concretely, we argue:
The returns to test-time compute are bounded by coverage, meaning the probability that the model's generative distribution places mass on a correct solution. No amount of selection can recover answers the model never produces.
Beyond coverage, the returns are governed by the quality of the selection or steering signal, and an imperfect verifier turns more samples into more chances to be fooled.
Sequential "thinking longer" and parallel "thinking wider" are not interchangeable. Which one helps depends on whether a problem's difficulty comes from search breadth or from depth of dependent computation.
Current evaluation practice routinely hides these distinctions. We need matched-budget comparisons, full scaling curves, and per-difficulty breakdowns.
Seeing test-time compute this way matters for several reasons. Inference costs are now a significant share of the total cost of deploying reasoning systems. Latency limits what users will accept. Misreading where gains come from can also lead the field to overinvest in inference tricks that only cover gaps in training, or to underinvest in verifiers that are the real bottleneck.
Letting a large language model (LLM) think longer at inference time has become a main way to improve its reasoning accuracy, either by sampling more reasoning chains in parallel or by extending one chain in sequence. However, the prevailing practice fixes one compute budget for all problems and judges it by average accuracy. This practice suffers from two drawbacks: (i) the budget ignores how likely the model is to reach a correct answer, wasting compute on problems already solved or out of reach; and (ii) the rule that picks the final answer cannot reliably separate correct chains from incorrect ones, leading accuracy to plateau or decline at large budgets. This paper takes the position that test-time scaling should be coverage-aware and verifier-bounded. Specifically, we decompose accuracy at any budget into coverage (i.e. the probability that at least one reasoning path reaches the correct answer) and selection accuracy (i.e. the probability that the correct answer is chosen when such a path exists). Under this decomposition, additional computation helps only while it still increases coverage on problems within the model's reach, and only while the selection rule can turn that coverage into a correct final answer. Based on this view, we call for allocating compute per problem from early signals, limiting the budget by the measured accuracy of the selection rule, and reporting both quantities instead of one averaged curve.
1 Introduction
Chain-of-thought reasoning lets large language models (LLMs) solve multi-step problems in mathematics, programming and science, and the amount of computation spent at inference time strongly affects whether the final answer is correct. Recent reasoning models such as OpenAI o1 and DeepSeek-R1 write long chains of thought before they answer. Simple methods such as self-consistency (i.e. majority voting over sampled chains) and best-of-N sampling also improve accuracy without any additional training. As a result, test-time compute has emerged as a second axis of scaling, in addition to model size and training data. While important, deciding how much compute to spend remains hard: the gain from additional tokens differs widely across problems, and the same budget that helps on one problem may be wasted or even harmful on another.
The prevailing practice treats test-time compute as a single budget B. In parallel scaling, the model samples B independent reasoning chains for a problem, and a fixed rule (e.g. majority voting or a reward model) selects one final answer. In sequential scaling, the model writes one chain of up to B tokens, sometimes extended by prompts that ask it to keep checking its work, and the last answer in the chain is taken as final. In both cases, the same B is applied to each problem in a benchmark, and the method is evaluated by plotting average accuracy as a function of B. On many reasoning benchmarks, such curves increase steadily over a wide range of budgets, which has made larger budgets a widely adopted default. Whether compute is spent in parallel or in sequence, these practices share one design: the budget is fixed before the problem is seen, and its value is judged only through the average accuracy it produces.
Intuitively, effective test-time scaling should be coverage-aware (i.e. it spends compute where the model can still turn a wrong answer into a correct one) and verifier-bounded (i.e. it spends no more than the answer-selection rule, or verifier, can exploit). However, a budget fixed in advance violates both properties. On the one hand, applying the same B to each problem ignores how likely the model is to reach the correct answer, because a problem solved in one attempt and a problem beyond the model's reach receive identical compute. As a result, much of the budget is spent on problems where it cannot change the outcome, and the average curve hides this waste behind the subset of problems that still benefit. On the other hand, a fixed selection rule ignores the gap between producing a correct chain and choosing it. Once the model produces a correct chain with high probability, further gains depend entirely on selection, and majority voting or an imperfect reward model may prefer a frequent but wrong answer, leading to accuracy that plateaus or declines as B increases.
This paper takes the position that the budget should be decided per problem and after observation, rather than fixed in advance and judged by one average. Specifically, we decompose accuracy at budget B as acc(B) = cov(B) · sel(B). Here, cov(B) is the coverage (i.e. the probability that at least one reasoning path reaches the correct answer), and sel(B) is the selection accuracy (i.e. the probability that the selected answer is correct, given that a correct path exists). However, neither factor is observed at test time, because the correct answer is unknown. Therefore, we propose two principles that rely only on signals available during inference or on held-out problems. The first principle, coverage-aware allocation, judges from the agreement among a few early samples whether a problem is already solved, within reach or out of reach, and spends additional compute only on problems within reach. The second principle, verifier-bounded scaling, limits the budget to the range in which the selection rule can still distinguish correct candidates from incorrect ones. Together, the two principles make test-time scaling coverage-aware and verifier-bounded.
We develop this position in three steps. First, we explain when longer thinking increases coverage and when coverage stops responding to additional compute (cf. Section 3). Second, we examine how the selection rule limits the useful budget in both parallel and sequential scaling (cf. Section 4). Third, we discuss two alternative views, namely that cheap compute makes large budgets worthwhile and that training will make allocation rules unnecessary (cf. Section 5). We highlight that a coverage-aware and verifier-bounded view turns test-time compute from one global setting into a per-problem decision, without requiring any change to the model itself.
05 · ADVANCED USAGE装备 skill 的智能体一次写完一篇 ICML 论文With the skill, an agent writes a complete ICML paper in one run
Skill 还能带着智能体把一个想法写成排好版的投稿。我们只给装备 skill 的 Claude Opus 5.5 一段提示词(见附录),让它在一台只有 CPU 的 Mac 上独立完成:选定想法、写代码、下载数据、训练两个网络、跑实验、画图、上网核对参考文献,最后用 ICML 2027 模板排版。整个过程一次跑完,共 209 轮、约 82 分钟(实验约 37 分钟),API 费用 19.91 美元,没有人工干预。
The skill can also take an agent from an idea to a typeset submission. We give Claude Opus 5.5 with the skill a single prompt (see Appendix) and let it work alone in Claude Code on a CPU-only Mac. It chooses an idea, writes the code, downloads the data, trains two networks, runs the experiments, draws the figures, checks each reference on the web and typesets the paper in the ICML 2027 template. The single run takes 209 turns and about 82 minutes, 37 of them for the experiments, and costs $19.91 in API fees, without human intervention.
图 3 · Figure 3 · Claude Opus 5.5 生成,未经人工修改。页脚的 “Under review” 是 ICML 模板的默认文字,论文并未投稿。Generated by Claude Opus 5.5 without manual edits. The “Under review” footer is the default text of the ICML template, and the paper was not submitted.
The experiments were actually run. Each number we spot-checked matches the raw outputs, the agent fixed the method and settings before it saw the first result, and the paper reports where the results miss its expectations. However, no human has reviewed the paper, and its experiments are small. It shows that an agent with the skill can produce a complete manuscript with traceable numbers from an idea, while whether the research holds up is still for a researcher to judge.
风险提示:对于一次完整的生成,尽管写出来像是人类写的,但因为篇幅过长,仍然会被判为 AI。
Risk: for a complete generation, although the writing reads as human, it is still labeled AI because of its length.
06 · CONCLUSION结论Conclusion
我们模拟了从草稿到成文的写作过程。在四篇论文上,从人类规范论文提炼的写作 skill 让生成文本被 Pangram 判为 AI 的比例降到与人类写作相当。不过,能通过检测的,一般是基于人写草稿的较短文本;直接由 AI 生成或篇幅较长的文章,被判为 AI 的比例仍然较高。因此,论文写什么、怎么论证,仍要由研究者来想。
We simulate the writing process from a draft to a finished text. On four papers, a writing skill distilled from human-written papers that follow academic conventions reduces the AI share of the generated text to that of the human-written text. However, the texts that pass the detector are mostly short ones written from a human draft, while text generated directly by AI, or a long paper, is still likely to be labeled AI. Therefore, the researcher must still work out what the paper says and how it argues.
This post has two limitations. On the one hand, the results are examples rather than rates, since they come from one writer model, six papers, two topics and a single scan of each text. On the other hand, whether a skill distilled from other well-formed papers works as well remains unexplored.
07 · RELATED WORK相关工作Related work
主流的 AI 文本检测有两类。第一类看文字对语言模型有多好猜:GLTRGehrmann et al., ACL 2019. 查看参考文献 按模型对每个词的排名上色,DetectGPTMitchell et al., ICML 2023. 查看参考文献 比较文字与其扰动版本的似然,Fast-DetectGPTBao et al., ICLR 2024. 查看参考文献 用可解析计算的条件概率曲率代替扰动,BinocularsHans et al., ICML 2024. 查看参考文献 用两个模型的交叉困惑度归一化困惑度。第二类在人写和 AI 文本上训练分类器:RADARHu et al., NeurIPS 2023. 查看参考文献 与一个改写器对抗训练,GhostbusterVerma et al., NAACL 2024. 查看参考文献 组合几个弱模型的词元概率,PangramEmi and Spero, arXiv 2024. 查看参考文献 用难负样本压低误报。分类器只和它的训练数据一样新。RAIDDugan et al., ACL 2024. 查看参考文献 等基准在不同领域、模型和攻击下比较这些方法。
Mainstream AI-text detectors come in two families. The first asks how predictable a text is to a language model. For example, GLTRGehrmann et al., ACL 2019. View reference colors each word by its rank among a model’s guesses, and DetectGPTMitchell et al., ICML 2023. View reference compares a passage’s likelihood with that of perturbed copies. Fast-DetectGPTBao et al., ICLR 2024. View reference replaces the perturbations with an analytic conditional curvature, and BinocularsHans et al., ICML 2024. View reference normalizes perplexity by the cross-perplexity of two models. The second family trains a classifier on human and AI text. RADARHu et al., NeurIPS 2023. View reference trains it jointly with a paraphraser that tries to evade it, GhostbusterVerma et al., NAACL 2024. View reference feeds it token probabilities from weaker models, and PangramEmi and Spero, arXiv 2024. View reference mines hard negatives to keep false positives low. Such a classifier is only as current as its training data. Benchmarks such as RAIDDugan et al., ACL 2024. View reference compare these methods across domains, generators and attacks.
此外,水印Kirchenbauer et al., ICML 2023. 查看参考文献(如已部署的 SynthID-TextDathathri et al., Nature 2024. 查看参考文献)让生成模型偏好一组秘密词元,持有密钥者可以事后检验,但它需要模型提供方主动加入,本文的文字都不带水印。这些方法都假设 AI 文本在统计上不同于人写文本;本文检验的是,当大模型按人类规范论文的写法写作时,这一假设是否还成立。
Moreover, a watermarkKirchenbauer et al., ICML 2023. View reference, as deployed in SynthID-TextDathathri et al., Nature 2024. View reference, makes the generator prefer a secret subset of tokens that a key holder can test for later. It requires the model provider to embed it, and no text in this post contains one. All of these methods assume that AI text is statistically different from human text. In contrast, this post tests whether that assumption holds when an LLM writes in the style of human-written papers that follow academic conventions.
与本文最接近的是规避检测的研究。DIPPERKrishna et al., NeurIPS 2023. 查看参考文献 改写 AI 文本,把 DetectGPT 的检测准确率从 70.3% 降到 4.6%;Sadasivan 等人Sadasivan et al., TMLR 2024. 查看参考文献表明改写能攻破现有检测器;Liang 等人Liang et al., Patterns 2023. 查看参考文献发现,让 ChatGPT 用更文学化的语言改写,检测率会接近于零。这些攻击都在成文之后改写;skill 则从不针对检测器,而是在写作时就遵循规范论文的写法。
Studies on evading detectors are the most related to ours. DIPPERKrishna et al., NeurIPS 2023. View reference paraphrases AI text and reduces DetectGPT’s detection accuracy from 70.3% to 4.6%. Sadasivan et al.Sadasivan et al., TMLR 2024. View reference show that paraphrasing breaks current detectors, and Liang et al.Liang et al., Patterns 2023. View reference report that asking ChatGPT to rewrite its text with more literary language brings detection close to zero. These attacks rewrite a finished text. In contrast, the skill never targets a detector and follows the writing of well-formed papers while the text is written.
另一条研究线测量科学写作中大模型的使用和检测误判的代价。Liang 等人Liang et al., COLM 2024. 查看参考文献和 Kobak 等人Kobak et al., Science Advances 2025. 查看参考文献根据 “delves” 这类词的词频估计,计算机科学摘要中最多 17.5%、2024 年生物医学摘要中至少 13.5% 经过大模型修改。前面提到的 Liang 等人还发现,检测器常把非英语母语者的写作误判为 AI,在托福作文上的平均误报率达 61.3%。这些研究在整个领域的层面讨论检测;我们补充了写法的视角:按人类规范论文的写法写,Pangram 判为 AI 的比例就明显下降。
Other studies measure LLM use in science and the cost of detector errors. Liang et al.Liang et al., COLM 2024. View reference and Kobak et al.Kobak et al., Science Advances 2025. View reference use the frequency of words such as “delves” to estimate the share of LLM-modified papers. Their estimates reach 17.5% of computer science abstracts and at least 13.5% of 2024 biomedical abstracts. The earlier study by Liang et al. also shows that detectors misclassify non-native English writing as AI, with a 61.3% average false positive rate on TOEFL essays. These studies treat detection at the level of a field. Our results add the view of writing style: writing like well-formed human-written papers reduces the share Pangram labels as AI.
08 · DISCLAIMER AND ACKNOWLEDGEMENT检测分数低,不改变应当披露的内容A lower score does not change what to disclose
能通过检测器的草稿仍然是大模型写的草稿。ICLR 2026 等会议要求披露对大模型的使用ICLR 2026 Program Chairs, ICLR Blog 2025. 查看参考文献,用这个 skill 写的稿件同样适用。检测分数也不该用来判定作者身份:第 4 节中两篇论文的人类写作也被判为 AI,而我们无从核实它们是怎么写的。
A draft that passes a detector is still an LLM draft. Venues such as ICLR 2026 require authors to disclose any use of an LLMICLR 2026 Program Chairs, ICLR Blog 2025. View reference, and this rule applies to drafts written with this skill. Detector scores should not decide authorship either: in Section 4, Pangram labels most of the human-written text of two papers as AI, although we cannot verify how it was written.
The authors conducted this research out of personal interest. The views expressed here are the authors’ own and do not represent those of any organization they are affiliated with. The authors declare no competing interests.
致谢Acknowledgement
感谢我的导师王翔、张岸提供蒸馏论文的许可,感谢早期使用者的反馈,感谢我的女朋友赵谢杨对 blog 设计的支持与帮助。
I thank my advisors Xiang Wang and An Zhang for permission to distill the papers, the early users for their feedback, and my girlfriend Xieyang Zhao for her support and help with the design of this blog.
Weixin Liang, Yaohui Zhang, Zhengxuan Wu, Haley Lepp, Wenlong Ji, Xuandong Zhao, Hancheng Cao, Sheng Liu, Siyu He, Zhi Huang, Diyi Yang, Christopher Potts, Christopher D. Manning, and James Y. Zou. Mapping the Increasing Use of LLMs in Scientific Papers. COLM 2024.↩︎↩︎
To cite this post, please use the following BibTeX entry:
@misc{sheng2026paperskill,
title = {The Art of Writing: Agents with Well-Curated Skills Can Cheat AI Detectors},
author = {Sheng, Leheng and Zhang, An and Wang, Xiang},
year = {2026},
howpublished = {Blog post},
url = {https://lehengthu.github.io/blog/2026/paper-skill-ai-detection/}
}
As shown in Figure A1, each of the six papers is tested in four steps, and each step mirrors how a researcher writes a paper. We scan each text once with the Pangram 4.1 web app on November 14, 2026.
实验按研究者写论文的流程设置The experiment follows how a researcher writes a paper
四个步骤,以及每一步这样设置的理由。The four steps, each with the reason for its setup.
六篇新论文Six new papers实验前几天才上 arXiv,模型没读过posted days before the experiment and unseen by the writer
内容提要Content brief像动笔前的笔记:只有事实,没有原句like a researcher’s notes: facts and none of the original sentences
生成两遍Generated twice同一模型、同一提要,只差 skillsame model and brief; only the skill differs
Pangram 检测Pangram scan和人类写作一起测,每份一次with the human-written text, one scan each
图 A1 · Figure A1 · 每一步都避免写作模型照搬原文:论文是新的,提要不含原句,生成时不能上网。Each step keeps the writer from copying the original: the papers are new, the briefs hold no original sentences, and the writer has no web access.
As shown in Figure A2, the two prompts share the output format and the content brief and differ only in their opening instructions. Each result in Figure 1 can be reproduced from them.
两份提示词只有开头的指令不同The two prompts differ only in their opening instructions
两份提示词的模板;每份提示词的结尾都是一篇论文的内容提要。The two prompt templates; each prompt ends with the content brief of one paper.
不使用 skillWithout the skill
You are drafting part of an ML conference paper.
Using only the facts in the content brief below, write:
- the Abstract (one paragraph, 150-250 words), and
- the Introduction (650-800 words, 4-8 paragraphs).
Write it the way you normally would for a strong ICLR / NeurIPS / ICML submission.
使用 skillWith the skill
You are drafting part of an ML conference paper with a paper-writing skill.
1. Read the skill first: `<skill>/SKILL.md`, then the files it asks for before drafting an Abstract and an Introduction: `<skill>/references/paper_types.md` (pick this paper's type), `<skill>/references/introduction.md` (write the argument chain first, then check rules IR1-IR15), `<skill>/references/contributions.md`, `<skill>/references/concepts.md`, `<skill>/references/prose.md` (slot templates, connectives, cohesion, sentence rules, habits to avoid), `<skill>/references/examples.md`, and the phrase bank `<skill>/assets/lexicon/lexicon.md` (take the wording for each rhetorical move from its section; prefer core and occasional entries). Check multi-word phrases you are unsure about with `python3 <skill>/tools/ps.py "phrase" -q`.
2. Using only the facts in the content brief below, write the Abstract (one paragraph, 150-250 words) and the Introduction (650-800 words, 4-8 paragraphs), following the skill's rules (argument chain, rule of two, the standard before the weaknesses, the key idea as the negation of the root cause, the First / Second / Third evidence roadmap, no contribution list, calibrated claims, banned words, sentence and cohesion rules).
3. The lexicon's example sentences come from published papers: use its entries as phrase patterns, but never copy an example sentence or clause.
两份提示词都接着写Both prompts continue with
Output format (the same for every writer in this study; it is not a style instruction):
- Plain text only: no LaTeX commands (write i.e. and e.g., never \ie or \eg), no markdown except the two headings `## Abstract` and `## Introduction`. If you use a list, put each item on its own line starting with `* `.
- No citations, reference markers or URLs. You may mention "Figure 1" or "Section 4" in words.
- Do not invent numbers beyond those in the brief.
- Reply with the two parts only, starting with `## Abstract`: no preamble, no notes, no closing remark.
Content brief:
- …
图 A2 · Figure A2 · 除 skill 路径缩写为 <skill>/ 外均为原文。不使用 skill 时不能用工具;使用 skill 时可以读取 skill 文件夹、查询短语。两者都不能上网。Verbatim, except that the skill’s path is shortened to <skill>/. Without the skill, the writer has no tools; with the skill, it can read the skill’s folder and look up phrases. Neither can access the web.
Without the skill, the writer answers each brief in a single turn. With the skill, it takes 24 to 33 turns per paper: it reads 7 to 9 of the skill’s files, browses the phrase bank and looks up phrases 8 to 18 times, such as “share one design” and “no longer”. One text generated without the skill has an overly long abstract and is regenerated once.
The skill’s main file walks the agent through eight steps, from the venue’s call for papers to a final self-review, while twelve reference files, a phrase bank, small tools and LaTeX templates hold the details. Each style rule records how widely human-written papers that follow academic conventions follow it. As shown in Figure A3, six parts shape the prose.
Skill 把规范论文变成规则、短语、次数和检查The skill turns well-formed papers into rules, phrases, counts and checks
决定行文的六个部分;所有数字都来自 skill 自己的文件。The six parts of the skill that shape the prose; all numbers are counted in the skill’s own files.
论证链Argument chain动笔前先写 8–10 条一句话论断8–10 one-line claims before any prose
已有工作共同的设计the design all prior work shares
它没达到的标准the standard it misses
两个缺陷two drawbacks
否定共同设计的核心想法a key idea that negates the shared design
短语库Phrase bank1,272 条原文短语,22 个分区1,272 attested phrases in 22 sections
常用core 659偶用occasional 440已弃用retired 71仅参考reference only 75禁用banned 27
词频Word frequency人类论文中的词数word counts in human papers
specifically48
however43
moreover34
furthermore5
each94
every4
口癖Habits to avoid27 种机器草稿的习惯,各有改法27 habits of machine drafts, each with a fix
, so→Therefore,
every→each
X, not Y→rather than
Table 2 lists→We report … in Table 2
句子规则Sentence rules15 项句子检查,外加衔接规则15 sentence checks plus cohesion rules
前置front主谓S + V细节 →details →重点point
句长中位 20 词;主语前最多 10 个词;一句一个主语;重点放在主句末尾;几乎每句都接住上一句。Median 20 words; at most ten words before the subject; one subject per sentence; the point at the end of the main clause; nearly all sentences link back.
工具Tools查短语,查草稿phrase lookup and draft checker
ps.py "to this end"多数论文都用 · 常用most papers · core
ps.py "delve into"旧写法 · 已弃用older usage · retired
check_paper.py字数、页数、必需声明、27 种口癖(逐行)word budget, page limit, required statements, 27 habits by line
另有:ICLR、NeurIPS、ICML 的 LaTeX 模板 · TikZ 框图样式包 · 绘图配色 · 8 种表格模板 · 各会议的页数预算Also: LaTeX templates for ICLR, NeurIPS and ICML · a TikZ style package for diagrams · a plot palette · eight table templates · page budgets per venue
图 A3 · Figure A3 · 次数来自 skill 的短语库和短语索引,统计来源论文的全部文字。Counts from the skill’s phrase bank and phrase index, over all text of its source papers.
工具。查询工具报告任意五词以内的短语在每篇论文中出现几次(索引只存词序列计数,不含句子),比如 “to this end” 属于常用,“delve into” 是已弃用的旧写法。检查器逐行报告草稿的字数预算、页数上限、必需声明和 27 种口癖。
Argument chain. Before any prose, the agent plans the introduction as 8–10 one-line claims: the design that prior work shares, the standard it misses, two drawbacks and a key idea that negates the shared design. The paragraphs and their openers follow from this chain, and five patterns adapt it to papers that propose a workflow, a closed-form method, a loss, a reward or a finding.
Phrase bank. The phrase bank holds 1,272 expressions taken from the papers, sorted into 22 sections by writing step, plus connectives and word classes. Each entry records how often each paper uses it and a tier: 659 core, 440 occasional, 71 retired, 75 reference-only and 27 banned.
Word frequency. The counts tell the agent which words such papers actually use. In the source papers, “specifically” appears 48 times, “however” 43 and “moreover” 34, while “each” appears 94 times and “every” only 4. Rare words such as “Furthermore” and “Notably” are banned.
Habits to avoid. Comparing a 24,000-word machine-written draft with the papers yields 27 habits. Each comes with the form the papers use instead: “, so” joining two clauses becomes “Therefore,”, “every” becomes “each” and “X, not Y” becomes “rather than”.
Sentence rules. Fifteen sentence checks and a set of cohesion rules are measured on the papers. The checks set a median of 20 words per sentence, at most ten words before the subject, one subject per sentence and the point at the end of the main clause. The cohesion rules link nearly all sentences back to the previous one.
Tools. A lookup tool reports how often any phrase of up to five words occurs in each paper, from a shipped index of word-sequence counts that holds no sentences. For example, it marks “to this end” as core and “delve into” as a retired, older usage. A checker reports the word budget, the page limit, the required statements and each match of the 27 habits, with line numbers.
句子更长,句首和短语来自来源论文Longer sentences, with the openers and phrasing of the source papers
To see what the skill changes, we measure the surface statistics that a classifier such as Pangram can pick up. Specifically, we measure them over the full abstracts and introductions of the 18 texts and count habits with the skill’s checker. These statistics only count words and use no model.
Without the skill, the generated sentences are short and even: 17.6 words on average with a standard deviation of 7.2, and 0.6% over 40 words. The human-written text averages 24.9 words with a standard deviation of 12.7 and 8.5% over 40 words. As shown in Figure A4, the text generated with the skill falls in between (22.7, 8.9 and 2.0%).
使用 skill 后句子更长,句首和短语来自来源论文With the skill, sentences are longer and take their openers and phrasing from the source papers
18 份文本完整摘要和引言上的六项表层统计。Six surface statistics over the full abstracts and introductions of the 18 texts of the six papers.
人类写作Human-writtenOpus 5.5 生成(不使用 skill)Generated by Opus 5.5 (without the skill)Opus 5.5 生成(使用 skill)Generated by Opus 5.5 (with the skill)
平均句长Mean sentence length词words
人类写作Human-written24.9
不使用 skillWithout skill17.6
使用 skillWith skill22.7
句长的标准差Spread of sentence length词,越小句子越均匀standard deviation in words; smaller is more even
人类写作Human-written12.7
不使用 skillWithout skill7.2
使用 skillWith skill8.9
超过 40 词的句子Sentences over 40 words占全部句子share of sentences
人类写作Human-written8.5%
不使用 skillWithout skill0.6%
使用 skillWith skill2.0%
以 “We” 开头的句子Sentences opening with “We”占全部句子share of sentences
人类写作Human-written14.7%
不使用 skillWithout skill16.0%
使用 skillWith skill7.1%
以 “However” 或 “Specifically” 开头的句子Sentences opening with “However” or “Specifically”占全部句子share of sentences
人类写作Human-written1.9%
不使用 skillWithout skill0.3%
使用 skillWith skill13.5%
也出现在 skill 来源论文里的四词序列Four-word sequences also in the skill’s source papers占全部四词序列share of four-word sequences
人类写作Human-written0.8%
不使用 skillWithout skill0.4%
使用 skillWith skill1.3%
图 A4 · Figure A4 · 四词序列只在 skill 的来源论文里计数。Four-word sequences count matches in the skill’s source papers only.
The sentence openers also differ. With the skill, 7.1% of the sentences open with “We”, compared with 16.0% without the skill and 14.7% in the human-written text. Moreover, 13.5% of the sentences generated with the skill open with “However” or “Specifically”, while the other two texts rarely do (0.3% and 1.9%). On both measures, the text generated with the skill moves toward the skill’s source papers, which use “specifically” 48 times and “however” 43 times, rather than toward the human-written text.
The phrasing moves toward the source papers as well. With the skill, 1.3% of the four-word sequences also occur in the source papers, about three times the share without the skill (0.4%) and more than the human-written text (0.8%). In contrast, the text generated with the skill shares only 2.5% of its eight-word sequences with the human-written text, about as much as without the skill (2.7%). About three quarters of these sequences also occur in the content brief. Therefore, the skill reuses phrasing from the source papers rather than copies the human-written text.
The skill also changes the style of the two position papers written from a topic alone. Their mean sentence length increases from 18.8 and 18.2 to 23.9 and 22.4 words, and the sentences that open with “We” decrease from 18 to 6 and 10.
机器草稿的口癖大多消失Most habits of machine-written drafts disappear
The habits change the most. As shown in Figure A5, the checker finds 75 habits in the text generated without the skill, 6 with the skill and 32 in the human-written text. These counts exclude domain terms and legitimate uses, which we remove by hand. Without the skill, the text writes “every” 20 times, joins two clauses with “, so” 15 times and puts “therefore” mid-sentence 10 times, compared with 0, 0 and 1 with the skill. The human-written text also uses “every” 11 times. In other words, the table records what the source papers avoid rather than what all authors avoid.
Skill 去掉了大部分机器草稿的口癖The skill removes most habits of machine-written drafts
Skill 检查器在六篇论文 18 份文本的摘要和引言中找出的口癖。Habits that the skill’s checker finds in the abstracts and introductions of the 18 texts of the six papers.
人类写作Human-writtenOpus 5.5 生成(不使用 skill)Generated by Opus 5.5 (without the skill)Opus 5.5 生成(使用 skill)Generated by Opus 5.5 (with the skill)
27 种口癖合计All 27 habits检查器找出的口癖habits the checker finds
人类写作Human-written32
不使用 skillWithout skill75
使用 skillWith skill6
把 every 当默认量词“every” as the default quantifier改用 eachuse “each”
人类写作Human-written11
不使用 skillWithout skill20
使用 skillWith skill0
, so 连接两个分句“, so” joining two clauses改用 Therefore,use “Therefore,”
人类写作Human-written2
不使用 skillWithout skill15
使用 skillWith skill0
句中的 thereforeMid-sentence “therefore”改用句首 Therefore,use a sentence-initial “Therefore,”
人类写作Human-written2
不使用 skillWithout skill10
使用 skillWith skill1
表示变化的动作动词Motion verbs for change如 grows、shrinks、falls tosuch as “grows”, “shrinks”, “falls to”
人类写作Human-written4
不使用 skillWithout skill8
使用 skillWith skill2
用 needs 或 extra 表示要求“needs” or “extra” for a requirement改用 requires、additionaluse “requires” or “additional”
人类写作Human-written0
不使用 skillWithout skill6
使用 skillWith skill0
图 A5 · Figure A5 · 用 skill 自带的检查器统计,已人工去掉 23 处领域术语和正常用法。Counted with the skill’s own checker, after removing 23 domain terms and legitimate uses by hand.
These measurements are consistent with the Pangram results. Without the skill, the generated text follows a frontier model’s default style, with short, even sentences and many habits, and the skill replaces it with the writing of its source papers. Pangram reads this style as more human even when it differs from the human-written text of the same paper.
The paper of Section 5 is generated from the prompt in Figure A6. Opus 5.5 runs in the non-interactive mode of Claude Code with at most 600 turns and can read and write files, run commands and use the web, but it cannot start subagents.
一段提示词,从想法写到排好版的论文One prompt, from an idea to a typeset paper
Opus 5.5 收到的完整任务。The full task that Opus 5.5 receives.
You are writing a complete ICML submission from scratch, in one session, with the paper-writing skill at <skill> (read its SKILL.md first and follow its workflow, Steps 0 to 7, reading each reference file at the step that names it). Your working folder is the current directory, <paper>; put every file of the paper project there.
1. Idea. Choose the research idea yourself: one original, specific machine-learning method (not a survey or a position paper) that fits one of the skill's paper types. It must be testable with real experiments that you run here: a CPU-only machine (15 cores, 48 GB RAM, no GPU), at most about 45 minutes of compute in total, on synthetic data or small public datasets you can download. Python 3 has numpy and scipy; create a virtual environment in .venv inside this folder and pip install anything else you need (matplotlib for the plots, scikit-learn or a CPU build of torch if the method needs them).
2. Experiments. Put the experiment code in code/ and its raw outputs in results/. Every number in the paper must come from those outputs. Never invent or estimate a number: if an experiment cannot be run, leave its table cells as \pend and say so, as the skill prescribes.
3. Venue and template. Follow the skill's Step 0 for ICML: check the current call for papers, fetch the latest official ICML style kit (templates/venue/icml/README.md), use the skill's ICML skeleton (templates/main_skeleton_icml.tex, templates/chapters_icml/ renamed to chapters/, templates/preamble.tex), and write an anonymous submission with the Impact Statement and an appendix.
4. Figures and tables. Draw the method diagram in TikZ on templates/housediagram.sty and build it with scripts/render_diagram.py --check; make the result plots with templates/plot_style.py from the files in results/; build the tables from templates/tables.tex.
5. References. Cite only real papers. Check each one on the web (arXiv or DBLP) before you add it, and format the .bib as the skill says (scripts/bib_to_dblp.py).
6. Build and check. Compile with tectonic until there are no errors, no undefined references and no overfull boxes; the main text must end exactly at the bottom of page 8 (references, Impact Statement and appendix after it). Run python3 <skill>/scripts/check_paper.py . --venue icml --stage submission and fix what it reports, as the skill's Step 7 describes. Look at the rendered pages (for example with PyMuPDF) and fix what you see.
7. Deliverables. The compiled PDF in this folder, and NOTES.md with: the title and the idea in one paragraph, the experiments you ran and how long they took, which numbers come from which result file, and the final check_paper.py summary.
Rules: work only inside this folder (you may read the skill folder; never modify it). Do not use git. Finish only when the PDF builds cleanly and the checks are addressed.
图 A6 · Figure A6 · 除 skill 路径缩写为 <skill>/、工作目录缩写为 <paper>/ 外均为原文。Verbatim, except that the skill’s path is shortened to <skill>/ and the working folder to <paper>/.