Blog
Paper-Writing Skill

写作的艺术:装备精心设计的 skill,写作智能体能骗过 AI 检测The Art of Writing: Agents with Well-Curated Skills Can Cheat AI Detectors

我们从人类撰写的规范论文中提炼写法,做成了一个学术写作 skill。装备该 skill 的智能体能写出流畅有逻辑的文字,且能降低 AI 检测率,甚至能一次写完一篇会议论文。我们开源如下:

We distilled the writing practices of human-written papers that follow academic conventions into an academic-writing skill. An agent equipped with the skill can write fluent, logical prose with a lower AI detection rate and can even write a complete conference paper in one run. We open-source the skill at the link below.

github.com/LehengTHU/humanize-ml-paper

TL;DR

我们开源了一个符合学术规范的写作 skill,装备该 skill 的智能体能极大提高通过 AI 检测器 Pangram 的概率。

We open-source a writing skill that follows academic conventions: agents equipped with it are far more likely to pass the AI detector Pangram.

  1. Skill 生成的文本降低了 Pangram 检测的概率。我们模拟了从草稿到成文的过程:在四篇论文上,Opus 5.5 不使用 skill 时平均 94% 的文字被判为 AI,使用 skill 时只有 29%,与人类写作持平。Text generated with the skill is less likely to be detected by Pangram. We simulate the writing process from a draft to a finished text. On four papers, Pangram labels 94% of Opus 5.5’s text as AI without the skill, compared with 29% with the skill, the same as the human-written text.
  2. 使用 skill 生成的文本符合学术规范。它按人类规范论文的写法组织和用词,skill 检查器找出的机器草稿口癖从 75 处降到 6 处。The text generated with the skill follows academic conventions. It is organized and phrased like human-written papers that follow academic conventions, and the skill’s checker finds 6 habits of machine-written drafts instead of 75.
  3. Skill 还支持 one-shot 生成一篇符合学术规范的会议论文。在高级用法中,智能体只凭一段提示词,约 82 分钟就独立写成一篇 ICML 2027 投稿,实验也是它自己跑的。The skill also supports one-shot generation of a conference paper that follows academic conventions. In Advanced Usage, an agent given one prompt runs the experiments and writes an ICML 2027 submission alone in about 82 minutes.
  4. Skill 能节省撰写时间,但代替不了人对逻辑的思考。如果内容本身就是 AI 生成的,用 AI 改也纠正不过来:人类写作本身就被判为 AI 的两篇论文,以及只给题目写成的论文,使用 skill 后仍有 95% 以上被判为 AI。The skill saves drafting time but cannot replace human thinking about the logic. If the content is AI-generated to begin with, revising it with AI cannot correct it. On two papers whose human-written text is itself labeled AI, and on papers written from a topic alone, Pangram still labels at least 95% of the text generated with the skill as AI.

适用范围:能通过检测的,一般是基于人写草稿的较短文本;直接由 AI 生成或篇幅较长的文章,被判为 AI 的比例仍然较高。

Scope: The texts that pass the detector are mostly short ones written from a human draft. Text generated directly by AI, or a long paper, is still likely to be labeled AI.

01 · BACKGROUNDAI 生成的内容很糟糕AI-Generated Contents Sucks

直接让大模型写出的论文往往很糟糕:说法空泛,还堆满 “delve”“underscore” 这类花哨的词Kobak et al., Science Advances 2025.
查看参考文献
。这样的文字正大量涌入学术圈:Pangram 估计,ICLR 2026 有 21% 的审稿意见完全由 AI 生成,9% 的投稿里 AI 生成的文字超过一半Emi, Pangram Blog 2025.
查看参考文献
。

Papers written directly by an LLM tend to read badly: they make generic claims and lean on ornate words such as “delve” and “underscore”Kobak et al., Science Advances 2025.
View reference
. Such text now floods academia. For example, Pangram estimates that 21% of the ICLR 2026 reviews were fully AI-generated and that 9% of the submissions contained more than half AI-generated textEmi, Pangram Blog 2025.
View reference
.

AI 检测器正是为此而出现的。比如 Pangram 在人写文档和前沿模型的输出上训练分类器,它的技术报告称准确率达到 99%Emi and Spero, arXiv 2024.
查看参考文献
。

AI detectors emerged to counter this. For example, Pangram trains a classifier on human documents and frontier-model outputs, and reports 99% accuracy in its technical reportEmi and Spero, arXiv 2024.
View reference
.

然而,AI 是提高效率的工具,我们不应片面地抛弃 AI 生成的内容。如果 AI 写的文字能和人写的一样流畅易懂,就应该用它来加速研究,让研究者把时间花在推敲逻辑上,而不是撰写上。为此,我们做了一个写论文的 skill,它从人类撰写的规范论文中提炼写法,让编程智能体照规范论文的方式起草论文(第 2 节)。我们首先探索:

However, as AI is a tool for working faster, we should not abandon AI-generated content wholesale. If AI-generated text could read as fluently as human writing, we should use it to speed up research, letting researchers spend their time on the logic rather than on drafting. To this end, we build a paper-writing skill that distills the writing of human-written papers that follow academic conventions, so that a coding agent drafts papers the same way (Section 2). We first explore:

精心整理的 skill,能否让智能体写出流畅易懂的文字,甚至骗过 AI 检测器?Can a well-curated skill make an agent write fluent, readable prose, and even cheat AI detectors?

02 · THE SKILLSkill 不仅润色文字,还对全文排布进行宏观约束The skill not only polishes the wording but also constrains the layout of the whole paper

这个 skill 是 Claude Code 这类编程智能体写论文前先读的一个文件夹。它把人类规范论文的写法蒸馏成流程、规则、短语和检查工具,既约束全文排布,比如各节骨架、页数预算和引言的论证链,也约束每一句的写法,比如句子规则、短语库和口癖表。

The skill is a folder that a coding agent such as Claude Code reads before writing a paper. It distills the writing of human-written papers that follow academic conventions into a workflow, rules, phrases and checking tools. These constrain both the layout of the whole paper, from the section skeleton to the argument chain of the introduction, and the wording of each sentence, from the phrase bank to the habits to avoid.

Humanizer 这类工具只逐句润色文字,ARIS 这类论文写作 skill 侧重安排写作流程;我们的 skill 则从全文排布到每句措辞系统地约束论文。

Tools such as Humanizer only polish the wording sentence by sentence, and paper-writing skills such as ARIS focus on planning the writing pipeline. In contrast, our skill constrains a paper systematically, from its layout to each sentence.

打开文件夹后,把鼠标移到卡片上或点按卡片,可以看到每一部分做什么;更多细节见附录。

Open the folder and hover over or tap a card to see what each part does; the Appendix has more details.

  • SKILL.md写作流程Workflow

    8 个步骤,从会议要求写到最后自查eight steps, from the call for papers to a final self-review

    主文件把写一篇论文拆成 8 步:确认会议并抓取当年的征稿要求、列提纲、分配页数、起草、做图表、写附录和声明、编译核对页数,最后逐条自查。每一步都写明该读哪份参考文档。The main file splits a paper into eight steps: confirm the venue and fetch this year’s call for papers, outline, budget the pages, draft, make the figures and tables, write the appendix and statements, compile and check the pages, and self-review. Each step names the reference file to read.

    起草顺序:预备知识 → 方法 → 实验 → 引言 → 摘要drafting order: preliminary → method → experiments → introduction → abstract
  • references/写作指南Writing guides

    12 份参考文档:结构、引言、实验、图表12 reference files on structure, introductions, experiments and figures

    12 份参考文档,写到对应步骤时才读:论文类型、结构和页数蓝图、引言、贡献表述、句子与衔接、实验、数学、图表、LaTeX 源码、术语、会议规则和示例。Twelve reference files, each read at the step that needs it: paper types, structure and page blueprints, introductions, contribution statements, sentences and cohesion, experiments, math, figures and tables, LaTeX source, terms, venue rules and examples.

    structure.md · introduction.md · prose.md · venues.md …structure.md · introduction.md · prose.md · venues.md …
  • introduction.md论证链Argument chain

    动笔前先写 8–10 条一句话论断8–10 one-line claims planned before any prose

    动笔前,先把引言写成 8–10 条一句话的论断:已有工作共同的设计、它没达到的标准、两个缺陷,以及否定这个共同设计的核心想法。每一环都成立了才开始起草,段落和段首句都由这条链决定。Before any prose, the agent writes the introduction as 8–10 one-line claims: the design all prior work shares, the standard it misses, two drawbacks and a key idea that negates the shared design. It drafts only when every link holds, and the paragraphs and their openers follow from the chain.

    共同设计 → 没达到的标准 → 两个缺陷 → 核心想法shared design → missed standard → two drawbacks → key idea
  • prose.md句子规则Sentence rules

    15 项句子检查,句长中位数 20 词15 sentence checks, with a median of 20 words

    15 项句子检查,都在人类规范论文上测量过:句长中位数 20 词,主语前最多 10 个词,一句只有一个主语,重点放在主句末尾。衔接规则要求几乎每一句都接住上一句。Fifteen sentence checks, each measured on human-written papers: a median of 20 words, at most ten words before the subject, one subject per sentence and the point at the end of the main clause. Cohesion rules make nearly every sentence link back to the previous one.

    前置 ≤ 10 词 · 主语 + 谓语 · 细节 → 重点front ≤ 10 words · subject + verb · details → point
  • prose.md口癖表Habits to avoid

    27 种机器草稿的口癖,每种附改法27 habits of machine-written drafts, each with a fix

    把一份 2.4 万词的机器草稿和人类规范论文逐一比对,找出机器反复出现的 27 种习惯,每种都给出论文里的写法。检查器会逐行标出草稿里的每一处。Comparing a 24,000-word machine-written draft with human-written papers turned up 27 habits that machines repeat, each with the form the papers use instead. The checker flags every occurrence in a draft, line by line.

    “, so” → “Therefore,” · “every” → “each” · “X, not Y” → “rather than”“, so” → “Therefore,” · “every” → “each” · “X, not Y” → “rather than”
  • assets/lexicon/短语库Phrase bank

    1,272 条原文短语,按写作步骤分成 22 区1,272 attested phrases in 22 sections by writing step

    从人类规范论文中摘出的 1,272 条原文表达,按写作步骤分成 22 区,从背景、已有工作的不足一直到结论,外加连接词和词类。每条标了状态,智能体写每一句前先查对应的分区。1,272 expressions taken from human-written papers, sorted into 22 sections by writing step, from the background and the gap in prior work to the conclusion, plus connectives and word classes. Each has a tier, and the agent checks the matching section before each sentence.

    659 常用 · 440 偶用 · 71 已弃用 · 75 仅参考 · 27 禁用659 core · 440 occasional · 71 retired · 75 reference-only · 27 banned
  • assets/lexicon/词频Word frequency

    “each” 出现 94 次,“every” 只有 4 次“each” appears 94 times, “every” only 4

    短语库记下每个词在人类规范论文里出现的次数,智能体据此挑这些论文真正会用的词。它们很少用的词,比如 “Furthermore” 和 “Notably”,被列为禁用。The phrase bank records how often each word appears in human-written papers, so the agent picks the words those papers actually use. Words they rarely use, such as “Furthermore” and “Notably”, are banned.

    specifically 48 · however 43 · moreover 34 · each 94 · every 4specifically 48 · however 43 · moreover 34 · each 94 · every 4
  • tools/ps.py短语查询Phrase lookup

    查任意短语在人类规范论文中出现几次how often any phrase occurs in human-written papers

    查任意五词以内的短语在人类规范论文里出现几次,背后是随 skill 发布的词序列计数索引,不含任何原句。拿不准一个说法时,智能体先查它。Reports how often any phrase of up to five words occurs in human-written papers, from a shipped index of word-sequence counts that holds no sentences. When unsure of a wording, the agent looks it up first.

    ps.py “to this end” → 多数论文都用 · 常用;“delve into” → 已弃用ps.py “to this end” → most papers · core; “delve into” → retired
  • scripts/check_paper.py草稿检查器Draft checker

    报告字数、页数、会议声明和每处口癖word budget, page limit, required statements and each habit

    读整篇 LaTeX 草稿,报告每章字数和预算、正文结束在第几页、会议要求的声明和匿名、术语和衔接、句子规则,以及 27 种口癖的每一处匹配和行号。Reads the whole LaTeX draft and reports the words per chapter against the budget, the page where the main text ends, the venue’s required statements and anonymity, terms and cohesion, the sentence rules, and each match of the 27 habits with its line number.

    [2 ‘every’] 4 line(s) → use: ‘each’[2 ‘every’] 4 line(s) → use: ‘each’
  • venues.md会议规范Venue rules

    ICLR、NeurIPS、ICML 的页数、匿名要求和必需声明page limits, anonymity and required statements for ICLR, NeurIPS and ICML

    ICLR、NeurIPS、ICML 的页数上限、匿名要求、必需声明和引用格式,外加一份单栏与双栏之间改投的核对清单。规则每年都变,所以第一步就要抓取当年的征稿要求核对。Page limits, anonymity, required statements and citation commands for ICLR, NeurIPS and ICML, plus a checklist for moving a paper between one and two columns. Rules change every year, so the first step fetches the current call for papers.

    ICLR 9 页 · NeurIPS 9 页 · ICML 8 页(双栏)ICLR 9 pp · NeurIPS 9 pp · ICML 8 pp, two columns
  • figures_tables.md图表规范Figure style

    论文的配色、绘图风格和框图画法the papers’ palette, plot style and diagram recipe

    规范论文常用的配色、绘图样式和框图画法:自家方法用饱和色,基线用浅色;框图用 TikZ 样式包画,一张图只有一个实心的焦点模块,渲染脚本会检查宽度、字体和配色。The palette, plot style and diagram recipe of well-formed papers: saturated colors for the method and tints for baselines; diagrams drawn in TikZ with one filled focal module, and a render script that checks the width, fonts and colors.

    蓝 #649CD8 / #BFD8EF · 粉 #FF85A1 / #FFDAE2blue #649CD8 / #BFD8EF · pink #FF85A1 / #FFDAE2
  • templates/模板Templates

    三个会议的 LaTeX 模板和表格样式LaTeX templates and table styles for the three venues

    三个会议的 LaTeX 骨架和分章文件、共用的导言区、8 种表格模板(主结果、消融、超参数等)、绘图样式和框图起步文件。实验还没跑完时,表格先用占位格。LaTeX skeletons with chapter files for the three venues, a shared preamble, eight table templates (main results, ablations, sweeps and more), the plot style and a diagram starter. Tables can hold placeholder cells until the experiments finish.

    main_skeleton.tex · main_skeleton_icml.tex · tables.tex · housediagram.stymain_skeleton.tex · main_skeleton_icml.tex · tables.tex · housediagram.sty

03 · RESULTSSkill 生成的文本降低了 Pangram 检测的概率Text generated with the skill is less likely to be detected by Pangram

这个 skill 最常见的用法是:思路和实验结果都有了,想快速写成一篇规范的论文。我们模拟这一写作过程,对每篇论文比较三份文本:人类写作,以及 Opus 5.5 不使用和使用 skill 生成的文本。本节报告四篇论文;人类写作本身就被判为 AI 的两篇放在第 4 节。

The skill is most often used when a researcher has the idea and the results and wants to write them up quickly as a paper. We simulate this writing process and compare three texts per paper: the human-written text and the texts that Opus 5.5 generates without and with the skill. This section reports four papers, and Section 4 covers the two whose human-written text is itself labeled AI.

场景模拟。我们选了这六篇论文分属推荐中的转化率预估18、条件生成模型的校准19、双层优化20、模仿学习21、联邦 DDoS 检测22和在线共形风险控制23。,每篇整理一份约 300 词的内容提要,只列事实、不含原句。Claude Opus 5.5 根据提要写两遍摘要和引言,两遍只差 skill,都在不能上网的新会话中完成;最后用 Pangram 检测这两份文本和人类写作。提要、生成文本和检测截图见本节末尾,流程和提示词见附录。

Simulated scenario. We choose They cover conversion-rate prediction in recommendation18, the calibration of conditional generative models19, bilevel optimization20, imitation learning21, federated DDoS detection22 and online conformal risk control23. and condense each into a content brief of about 300 words that lists the facts without the original sentences. Claude Opus 5.5 writes the abstract and introduction twice from each brief, with and without the skill, each time in a fresh session without web access. Pangram then scans both generated texts and the human-written text. The briefs, generated texts and Pangram screenshots are at the end of this section, and the setup and prompts are in the Appendix.

四篇论文上,使用 skill 后 AI 比例都大幅下降On four papers, the AI share drops sharply with the skill

Pangram 4.1 判为 AI 生成或 AI 辅助的文字占比,四篇模拟论文生成。Share of each text that Pangram 4.1 labels AI-generated or AI-assisted, over four simulated paper generations.

人类写作Human-writtenOpus 5.5 生成(不使用 skill)Generated by Opus 5.5 (without the skill)Opus 5.5 生成(使用 skill)Generated by Opus 5.5 (with the skill)
69%98%11%
0%80%31%
22%100%63%
23%98%12%
29%94%29%
四篇平均Average of 4
图 1 · Figure 1 · 点论文名可查看提要和生成文本。Each paper name links to its brief and generated texts.

使用 skill 后,Pangram 判为 AI 的比例从平均 94% 降到 29%,与人类写作相同(图 1)。这些论文不在 skill 的来源里,所以下降不来自模仿它们的写法;skill 具体改变了文风的哪些方面,见附录。

As shown in Figure 1, the skill reduces the share that Pangram labels AI from 94% to 29% on average, the same as the human-written text. The skill was distilled from none of these papers. Therefore, the drop does not come from imitating how they are written. The Appendix analyzes which aspects of the style the skill changes.

四篇论文的生成案例Generated texts of the four papers

原论文:Paper: Pavel Dolin, Xun Zou, Jiatian Wu, and Deguang Kong. Serving-Aligned Entire-Space Conversion Modeling via Click-Through Rate Knowledge Distillation. arXiv 2026. 我们逆向生成一个用于写论文的摘要和引言的 prompt 如下(使用 skill 和不使用 skill 的测试共享同一个生成 prompt)。We reverse-engineer the following prompt for writing the paper’s abstract and introduction; the runs with and without the skill share this generation prompt.

Content brief:
- Area: recommender systems; post-click conversion rate (pCVR) prediction for conversion-oriented ranking.
- Context: ranking may use the predicted click-through conversion rate (pCTCVR), the product of a predicted click-through rate (pCTR) and pCVR from separately trained models. Clicks are fast and dense, conversions slow and sparse, which motivates separate training schedules and independently developed pCTR models. Many pCVR models build on the Entire Space Multi-Task Model (ESMM), which trains a pCVR tower together with an auxiliary pCTR tower using pCTR and pCTCVR losses, but deploys only the pCVR tower.
- Problem: at serving time the pCVR tower is paired with a separately trained production pCTR model. When its predictions differ from those of the auxiliary tower, the pCTCVR used at serving differs from the one seen in training: a training-serving mismatch.
- Constraint: production pCTR models are large and expensive to train; retraining them, or training them jointly with the pCVR model, is impractical, especially when their predictions are shared across ranking stages.
- Method: fix the mismatch from the pCVR side with knowledge distillation (KD). ESMM's auxiliary pCTR tower is supervised with logged scores of the frozen serving pCTR model, mixed with the hard click labels; variants are named KD-alpha, where alpha weights the hard-label loss. Neither ESMM's architecture nor the serving pCTR model changes.
- Evidence: on the Ali-CCP dataset, KD-0.3, the best variant on serving pCTCVR, improves serving pCTCVR and pCVR ROC-AUC by 2.1% and 1.6%, and PR-AUC by 9.9% and 8.8%, over ESMM. Under negative downsampling that keeps 5% of non-clicks, it improves mean AUC by 1.2% and 0.9% and PR-AUC by 8.5% and 4.5% over downsampled ESMM. Code is released.
Opus 5.5 生成(不使用 skill)Generated by Opus 5.5 (without the skill)
Pangram 4.1 检测截图 / Pangram 4.1 result: KD for ESMM, generated by opus 5.5 (without the skill)Pangram 4.1 检测截图 · 点击看大图Pangram 4.1 result · click to enlarge

Abstract

Conversion-oriented ranking in recommender systems often scores items by the predicted click-through conversion rate (pCTCVR). This is the product of a predicted click-through rate (pCTR) and a post-click conversion rate (pCVR), each produced by a separately trained model. Many pCVR models build on the Entire Space Multi-Task Model (ESMM). ESMM trains a pCVR tower together with an auxiliary pCTR tower, but only the pCVR tower is deployed. At serving time this tower is paired with an independently developed production pCTR model. Whenever that model disagrees with the auxiliary tower, the pCTCVR used at serving differs from the one optimized during training, which creates a training-serving mismatch. Production pCTR models are large and expensive, and they are often shared across ranking stages, so retraining them or training them jointly with the pCVR model is impractical. We therefore address the mismatch from the pCVR side through knowledge distillation (KD). We supervise ESMM's auxiliary pCTR tower with logged scores of the frozen serving pCTR model, mixed with hard click labels. Neither ESMM's architecture nor the serving pCTR model changes. On the Ali-CCP dataset, our best variant improves serving pCTCVR and pCVR ROC-AUC over ESMM by 2.1% and 1.6%, and PR-AUC by 9.9% and 8.8%. Under negative downsampling that keeps 5% of non-clicks, it improves mean AUC by 1.2% and 0.9% and PR-AUC by 8.5% and 4.5% over downsampled ESMM. Code is released.

Introduction

Many recommender systems are built to drive conversions such as purchases, sign-ups or installs, and not just clicks. In these systems, items are commonly ranked by the predicted click-through conversion rate (pCTCVR), the probability that a user both clicks an item and then converts. pCTCVR factorizes naturally into a predicted click-through rate (pCTR) and a predicted post-click conversion rate (pCVR). Production systems usually obtain the two factors from separately trained models. The split is practical. Clicks are fast and dense, while conversions are slow and sparse, so the two signals call for different training schedules. As a result, pCTR models are often developed independently of pCVR models.

Many pCVR models build on the Entire Space Multi-Task Model (ESMM). ESMM trains a pCVR tower together with an auxiliary pCTR tower. It supervises the pCTR tower with click labels and the product of the two towers with click-and-conversion labels. This lets the pCVR tower learn over the entire impression space and not only over clicked samples. Only the pCVR tower is deployed, however. The auxiliary pCTR tower exists to shape training and is discarded at serving time.

This deployment pattern hides a problem. At serving time, the pCVR tower is not multiplied with the auxiliary pCTR tower it was trained with. It is multiplied with a separately trained production pCTR model. The two pCTR estimates can differ, since they come from different architectures, features, data and training schedules. When they differ, the pCTCVR that ranks items at serving time is not the one ESMM optimized during training. The pCVR tower has learned to complement one pCTR model and is then paired with another. We call this a training-serving mismatch. It weakens ESMM's main advantage: the pCTCVR objective that shapes the pCVR tower no longer matches the quantity actually used for ranking.

A direct fix would be to align the two pCTR models from the pCTR side. One option is to deploy the auxiliary tower instead of the production model. Another is to train the production pCTR model jointly with the pCVR model. Neither is realistic in practice. Production pCTR models are large and expensive to train, and their predictions are often shared across several ranking stages. Retraining them, or tying their training to a downstream pCVR model, would be costly and would risk regressions elsewhere in the system. Any practical solution must therefore treat the serving pCTR model as fixed.

We address the mismatch from the pCVR side using knowledge distillation (KD). Our key observation is that the auxiliary pCTR tower in ESMM does not have to be an independent click predictor. It only has to stand in, during training, for the pCTR model the pCVR tower will be paired with at serving time. We therefore supervise the auxiliary tower with logged scores of the frozen serving pCTR model, mixed with the hard click labels. We denote the resulting variants KD-alpha, where alpha weights the hard-label loss. The auxiliary tower then tracks the serving pCTR model, so the pCTCVR seen during training more closely resembles the pCTCVR used at serving time. The method changes neither ESMM's architecture nor the serving pCTR model. It only needs logged pCTR scores, which production systems typically already record.

We evaluate the approach on the Ali-CCP dataset, measuring performance in the setting that matters for deployment, where the pCVR model is paired with the serving pCTR model. KD-0.3 is the best variant on serving pCTCVR. Compared with ESMM, it improves serving pCTCVR and pCVR ROC-AUC by 2.1% and 1.6%, and PR-AUC by 9.9% and 8.8%. The gains hold under negative downsampling that keeps only 5% of non-clicks, a common practice for reducing training cost. In that setting, KD-0.3 improves mean AUC by 1.2% and 0.9% and PR-AUC by 8.5% and 4.5% over downsampled ESMM.

Our contributions are as follows:

  • We identify a training-serving mismatch in ESMM-style pCVR models. The auxiliary pCTR tower used during training differs from the separately trained pCTR model used at serving time, so the serving pCTCVR differs from the training objective.
  • We propose a simple remedy based on knowledge distillation. It supervises the auxiliary pCTR tower with logged scores of the frozen serving pCTR model mixed with hard click labels, and it requires no change to ESMM's architecture or to the serving pCTR model.
  • We show on Ali-CCP that this approach consistently improves serving pCTCVR and pCVR quality over ESMM, both with and without negative downsampling, and we release our code.
Opus 5.5 生成(使用 skill)Generated by Opus 5.5 (with the skill)
Pangram 4.1 检测截图 / Pangram 4.1 result: KD for ESMM, generated by opus 5.5 (with the skill)Pangram 4.1 检测截图 · 点击看大图Pangram 4.1 result · click to enlarge

Abstract

Conversion-oriented ranking often scores items by the predicted click-through conversion rate (pCTCVR), i.e., the product of a predicted click-through rate (pCTR) and a predicted post-click conversion rate (pCVR) from separately trained models. Many pCVR models build on the Entire Space Multi-Task Model (ESMM). ESMM trains a pCVR tower together with an auxiliary pCTR tower under pCTR and pCTCVR losses, but deploys only the pCVR tower. However, at serving time the pCVR tower is paired with a separately trained production pCTR model. When the production model's predictions differ from those of the auxiliary tower, the pCTCVR used at serving differs from the one seen in training (termed training-serving mismatch). Removing this mismatch from the pCTR side is impractical. Retraining the production pCTR model, or training it jointly with the pCVR model, is costly because such models are large and expensive to train, especially when their predictions are shared across ranking stages. To address this mismatch, we fix it from the pCVR side with knowledge distillation (KD), which makes pCVR training consistent and practical. Specifically, we supervise the auxiliary pCTR tower with logged scores of the frozen serving pCTR model, mixed with the hard click labels, while the ESMM architecture and the serving pCTR model remain unchanged. Experiments on a public dataset show that the best KD variant improves both ROC-AUC and PR-AUC of serving pCTCVR and pCVR over ESMM, including under negative downsampling of non-clicks. Our code is released.

Introduction

Conversion-oriented ranking in recommender systems often orders items by the predicted click-through conversion rate (pCTCVR), i.e., the product of a predicted click-through rate (pCTR) and a predicted post-click conversion rate (pCVR). In practice, the two factors come from separately trained models. Clicks are fast and dense, while conversions are slow and sparse, which motivates separate training schedules and pCTR models developed independently of the pCVR model. As a result, the pCTCVR that decides the ranking multiplies the outputs of two models that are never trained together.

Many pCVR models build on the Entire Space Multi-Task Model (ESMM). Specifically, ESMM trains a pCVR tower together with an auxiliary pCTR tower: a pCTR loss supervises the auxiliary tower with click labels, and a pCTCVR loss supervises the product of the two towers. After training, only the pCVR tower is deployed. This multi-task design has made ESMM a widely adopted basis for pCVR prediction. Whether through the pCTR loss or the pCTCVR loss, the auxiliary pCTR tower learns from labels alone, independently of the pCTR model used at serving.

Intuitively, effective pCVR training for pCTCVR ranking should be consistent (i.e., the pCTCVR it optimizes matches the pCTCVR used at serving) and practical (i.e., it leaves the production pCTR model untouched). ESMM-style training is practical by design, because it does not touch the production pCTR model. However, this design is not consistent. As illustrated in Figure 1, at serving time the pCVR tower is paired with the separately trained production pCTR model rather than with the auxiliary pCTR tower. When the predictions of the two pCTR models differ, the pCTCVR used at serving differs from the one seen in training, leading to a training-serving mismatch. As a result, the pCVR tower is optimized for a pCTCVR that differs from the one that ranks items.

This mismatch could be removed from the pCTR side, by retraining the production pCTR model or by training it jointly with the pCVR model. However, such remedies gain consistency at the cost of practicality. Production pCTR models are large and expensive to train, which makes retraining them, or training them jointly with the pCVR model, impractical. This cost is amplified when their predictions are shared across ranking stages, because a change to the pCTR model then affects each stage that uses its predictions. This gap calls for a remedy on the pCVR side that is both consistent and practical.

To this end, we fix the training-serving mismatch from the pCVR side with knowledge distillation (KD). The key idea is to let the auxiliary pCTR tower learn from the pCTR model used at serving, rather than from labels alone. Specifically, we supervise the auxiliary tower with the logged scores of the frozen serving pCTR model (i.e., the pCTR predictions recorded at serving), thereby reducing the gap between the pCTCVR seen in training and the one used at serving. To keep the click labels as the ground truth of the pCTR task, we mix the distillation loss with the hard-label loss and name each variant KD-α, where α weights the hard-label loss. The ESMM architecture and the serving pCTR model remain unchanged: the logged scores serve as fixed targets, and only the pCVR tower is deployed, as in ESMM. The distillation targets align training with serving, while the frozen logged scores leave the production pCTR model untouched; together, they make pCVR training consistent and practical.

We conduct experiments on the Ali-CCP dataset to examine the effectiveness of KD (cf. Section 4). First, KD-0.3, the best variant on serving pCTCVR, improves the ROC-AUC of serving pCTCVR and pCVR by 2.1% and 1.6%, respectively, and their PR-AUC by 9.9% and 8.8%, compared with ESMM. Second, under negative downsampling that keeps 5% of non-clicks, KD-0.3 improves mean AUC by 1.2% and 0.9% and PR-AUC by 8.5% and 4.5% compared with downsampled ESMM. Third, these gains come without retraining the production pCTR model or changing the ESMM architecture, because KD only changes the supervision of the auxiliary pCTR tower. We highlight that KD makes pCVR training consistent and practical, offering a pCVR-side remedy for the training-serving mismatch that leaves the serving pCTR model untouched.

04 · FAILURE CASESSkill 替代不了人的思考The skill does not replace human thinking

Skill 能改变文字怎么写,却改变不了内容写什么:如果内容本身就是 AI 生成的,用 AI 改也纠正不过来。下面两种场景里,使用 skill 生成的文本仍被判为 AI:人类写作本身就被判为 AI 的两篇论文,以及只给题目、从零写成的论文。

The skill changes how a text is written rather than what it says. If the content is AI-generated to begin with, revising it with AI cannot correct it. In the two settings below, Pangram still labels the text generated with the skill as AI: two papers whose human-written text is itself labeled AI, and papers written from a topic alone.

在这些场景里,使用 skill 生成的文本仍被判为 AIIn these settings, the text generated with the skill is still labeled AI

Pangram 4.1 判为 AI 生成或 AI 辅助的文字占比:人类写作被判为 AI 的两篇论文,以及两篇只给题目的论文。Share of each text that Pangram 4.1 labels AI-generated or AI-assisted, for two papers whose human-written text is labeled AI and two papers written from a topic alone.

人类写作Human-writtenOpus 5.5 生成(不使用 skill)Generated by Opus 5.5 (without the skill)Opus 5.5 生成(使用 skill)Generated by Opus 5.5 (with the skill)
图 2 · Figure 2 · 只给题目的两篇没有人类写作,检测的是其摘要和引言。The topic-only papers have no human-written text; Pangram scans their abstracts and introductions.

人类写作本身就被判为 AI 时,skill 也几乎没有降低 AI 比例When the human-written text is itself labeled AI, the skill barely reduces the AI share

ANE(双层优化)和 CCA(联邦 DDoS 检测)的人类写作,本身就被 Pangram 判为 87% 和 99% AI。在这两篇上,不使用 skill 生成的文本 100% 被判为 AI,使用 skill 后也只降到 95% 和 97%(图 2)。Skill 改了文风,口癖从 19 和 15 处降到 1 和 2 处,但没有改变内容。我们无从核实这两篇论文是怎么写成的;在这样的论文上,生成文本仍要由研究者自己读、自己改。

Pangram labels 87% and 99% of the human-written text of ANE (bilevel optimization) and CCA (federated DDoS detection) as AI. As shown in Figure 2, the text generated without the skill is labeled 100% AI on these papers, and the skill only reduces the share to 95% and 97%. The skill changes the style, reducing the habits from 19 and 15 to 1 and 2, but it does not change the content. We cannot verify how these two papers were written, and on such papers the researcher must still read and revise the generated text.

只给题目时,用不用 skill 都是 100% AIFrom a topic alone, both generated texts are 100% AI

我们还只给智能体一个题目(大模型评测中的基准污染、推理时计算的扩展),让它从零写一篇立场论文(position paper)。Pangram 把四篇论文的摘要和引言都判为 100% AI,用不用 skill 都一样(图 2)。这时观点、论证和例子全部出自模型,内容本身就是 AI 生成的,skill 只改了措辞和句式。

We also give the agent only a topic, benchmark contamination in LLM evaluation or the scaling of test-time compute, and let it write a position paper from scratch. As shown in Figure 2, Pangram labels the abstracts and introductions of all four papers as 100% AI, with or without the skill. In these papers, the claims, arguments and examples all come from the model. With such AI-generated content, the skill changes only the wording.

所以,skill 能把想清楚的内容写成流畅的文字、省下撰写的时间,却代替不了研究者:论文写什么、怎么论证,仍要由人来想,生成的文本也要由人读过、改过。

Therefore, the skill can turn what a researcher has thought through into fluent prose and save drafting time, but it does not replace the researcher. A person must still decide what the paper says and how it argues, and must read and revise the generated text.

两篇论文和两个题目的提示词与生成文本Prompts and generated texts of the two papers and the two topics

原论文:Paper: Jason Bohne, Paweł Polak, Gary Kazantsev, and David Rosenberg. Your Hypergradient is Skewed: Antithetic Neumann Estimation for Bilevel Optimization. arXiv 2026. 我们逆向生成一个用于写论文的摘要和引言的 prompt 如下(使用 skill 和不使用 skill 的测试共享同一个生成 prompt)。We reverse-engineer the following prompt for writing the paper’s abstract and introduction; the runs with and without the skill share this generation prompt.

Content brief:
- Area: stochastic bilevel optimization, used for hyperparameter tuning, data reweighting, robust learning, reinforcement learning and imbalanced classification.
- Context: gradient-based bilevel methods need the hypergradient, which involves the inverse of the inner Hessian. The standard scalable approach approximates it with a truncated Neumann series, a product of stochastic Hessian factors. Existing stochastic and variance-reduced bilevel methods judge hypergradient estimators only by their bias and variance.
- Problem: the product structure of the Neumann series adds a third source of error, skewness. Unlike ordinary stochastic gradients, whose asymmetry is averaged away by mini-batches, asymmetric noise in each Hessian factor enters the odd-order terms of the product; under weak contraction the third-moment error grows with the truncation depth, the inner condition number and the noise asymmetry.
- When it matters: corrupted labels, outliers, and heterogeneous or class-imbalanced data, which make the Hessian noise asymmetric. Variance-reduced methods reduce variance across iterations but do not act on the product inside a single estimate.
- Method name: the Antithetic Neumann Estimator (ANE). It pairs each stochastic Hessian sample with its mirror image around a reference Hessian and averages the two resulting inverse-Hessian estimates. It changes only the Hessian-vector step and combines with existing methods: ANE-SGD, and ANE-VR, which adds recursive variance reduction.
- Theory: with an exact reference, the antithetic product keeps only even-order noise terms and cancels all odd-order ones, also for non-commuting matrices; with an approximate reference, the remaining odd-order error is bounded linearly in the reference error. An iterate-level convergence bound shows that the inverse-Hessian third-moment term shrinks.
- Evidence: robust regression, data cleaning, offline reinforcement learning and imbalanced classification, 20 seeds each. ANE-based methods improve on stochastic and variance-reduced baselines on every task, lowering loss by 4-25% relative to the best baseline; the largest gain is a 25% lower test loss on robust regression. Synthetic controls show no gain when the noise is symmetric.
Opus 5.5 生成(不使用 skill)Generated by Opus 5.5 (without the skill)
Pangram 4.1 检测截图 / Pangram 4.1 result: ANE, generated by opus 5.5 (without the skill)Pangram 4.1 检测截图 · 点击看大图Pangram 4.1 result · click to enlarge

Abstract

Stochastic bilevel optimization underlies hyperparameter tuning, data reweighting, robust learning, reinforcement learning and imbalanced classification. Gradient-based bilevel methods need the hypergradient, which involves the inverse of the inner Hessian. The standard scalable approach approximates this inverse with a truncated Neumann series, which is a product of stochastic Hessian factors. Existing stochastic and variance-reduced methods judge such estimators only by their bias and variance. We show that the product structure adds a third source of error: skewness. Asymmetric noise in each Hessian factor enters the odd-order terms of the product. Under weak contraction, the resulting third-moment error grows with the truncation depth, the inner condition number and the degree of noise asymmetry. Corrupted labels, outliers and heterogeneous or class-imbalanced data all produce this kind of asymmetry. We propose the Antithetic Neumann Estimator (ANE), which pairs each stochastic Hessian sample with its mirror image around a reference Hessian and averages the two resulting inverse-Hessian estimates. With an exact reference, the antithetic product cancels every odd-order noise term, even when the matrices do not commute. With an approximate reference, the remaining odd-order error is bounded linearly in the reference error. ANE changes only the Hessian-vector step, so it plugs into existing methods. We study two instances: ANE-SGD and ANE-VR, which adds recursive variance reduction. We evaluate on robust regression, data cleaning, offline reinforcement learning and imbalanced classification, with 20 seeds per task. ANE-based methods lower loss by 4-25% relative to the best baseline on every task. Synthetic controls with symmetric noise show no gain, which supports skewness as the mechanism behind the improvement.

Introduction

Many learning problems have a nested structure. An outer objective depends on the solution of an inner optimization problem. Hyperparameter tuning is one example: validation loss is minimized over hyperparameters that shape a training problem. Other examples include data reweighting, robust learning, reinforcement learning and imbalanced classification, where outer variables such as sample weights, policies or class-dependent parameters steer an inner learner. Stochastic bilevel optimization gives a common framework for these problems. Gradient-based methods have become the standard way to solve them at scale.

The main computational step in these methods is the hypergradient, the gradient of the outer objective with respect to the outer variables. It involves the inverse of the inner Hessian. Forming or inverting that matrix is impractical in high dimensions. The usual scalable alternative approximates the inverse with a truncated Neumann series, which only needs stochastic Hessian-vector products. Each term of the series is a product of independently sampled Hessian factors. Many stochastic bilevel methods are built on this estimator, including variance-reduced ones. Their analyses judge it by two quantities: its bias, which comes from truncation and sampling, and its variance.

We argue that bias and variance leave out an important part of the error. The product structure of the Neumann series adds a third source, skewness. For ordinary stochastic gradients, asymmetry in the noise distribution is largely averaged away by mini-batching. The Neumann estimator behaves differently. When the noise in each Hessian factor is asymmetric, it enters the odd-order terms of the product and compounds across factors. Under weak contraction, the third-moment error grows with the truncation depth, the inner condition number and the degree of noise asymmetry. In short, the regimes that need long Neumann series are the same regimes in which skewness grows.

This is a practical concern, because the settings in which bilevel methods are most often used tend to make Hessian noise asymmetric. Corrupted labels, outliers, heterogeneous data and class imbalance all produce mini-batch Hessians whose deviations from the mean are lopsided rather than symmetric. Variance reduction does not fix this. Variance-reduced methods lower the variance of estimates across iterations, but they do not act on the product of factors inside a single estimate, which is where the skewness arises. An estimator can therefore have low variance and still carry substantial odd-order error.

We propose the Antithetic Neumann Estimator (ANE) to address this error directly. For each stochastic Hessian sample, ANE builds its mirror image around a reference Hessian. It then computes the inverse-Hessian estimate from both the original and the mirrored samples and averages the two. Because the mirrored deviations have the opposite sign, the odd-order contributions of the two estimates cancel when averaged, while the even-order contributions remain. ANE changes only the Hessian-vector step of the hypergradient computation, so it can be combined with existing bilevel methods. We study two instances. ANE-SGD uses the estimator inside a stochastic bilevel method. ANE-VR adds recursive variance reduction, so that variance and skewness are reduced together.

We support the method with theory at three levels. With an exact reference Hessian, the antithetic product keeps only even-order noise terms and cancels all odd-order ones. This holds even though the Hessian factors do not commute, so the cancellation is not limited to a scalar or diagonal case. With an approximate reference, as used in practice, the remaining odd-order error is bounded linearly in the reference error, so ANE degrades gracefully as the reference becomes less accurate. At the iterate level, a convergence bound shows that the third-moment term contributed by the inverse-Hessian estimate shrinks under ANE.

We test ANE on four tasks: robust regression, data cleaning, offline reinforcement learning and imbalanced classification, with 20 seeds for each. ANE-based methods outperform stochastic and variance-reduced baselines on every task. They lower loss by 4-25% relative to the best baseline, and the largest gain is a 25% lower test loss on robust regression. Synthetic controls show no gain when the noise is symmetric. This is consistent with our analysis and indicates that the improvement comes from cancelling skewness rather than from some other effect.

Our contributions are as follows:

  • We identify skewness as a third source of error in Neumann-series hypergradient estimators, separate from bias and variance. We show that under weak contraction it grows with truncation depth, condition number and noise asymmetry.
  • We propose the Antithetic Neumann Estimator, a drop-in replacement for the Hessian-vector step, and two methods built on it: ANE-SGD and ANE-VR.
  • We prove that ANE cancels all odd-order noise terms with an exact reference, including for non-commuting matrices. We also bound the remaining odd-order error linearly in the reference error and show that the inverse-Hessian third-moment term shrinks in the convergence bound.
  • We show consistent gains over stochastic and variance-reduced baselines on four bilevel tasks. Synthetic controls tie the gains to asymmetric noise.
Opus 5.5 生成(使用 skill)Generated by Opus 5.5 (with the skill)
Pangram 4.1 检测截图 / Pangram 4.1 result: ANE, generated by opus 5.5 (with the skill)Pangram 4.1 检测截图 · 点击看大图Pangram 4.1 result · click to enlarge

Abstract

Stochastic bilevel optimization underlies hyperparameter tuning, data reweighting, robust learning and reinforcement learning, and its hypergradient involves the inverse of the inner Hessian. Existing stochastic and variance-reduced methods approximate this inverse with a truncated Neumann series (i.e., a product of stochastic Hessian factors) and judge the resulting estimator only by its bias and variance. However, this product adds a third source of error, skewness (i.e., the third moment of the estimation error), because asymmetric noise in each Hessian factor enters the odd-order terms of the product. This error increases with the truncation depth, the inner condition number and the noise asymmetry. It also persists under variance reduction, which acts across iterations rather than inside each estimate. To address this, we propose the Antithetic Neumann Estimator (ANE), which makes hypergradient estimates symmetric and low-variance. Specifically, ANE pairs each stochastic Hessian sample with its mirror image around a reference Hessian and averages the two resulting inverse-Hessian estimates. Because it changes only the Hessian-vector step, ANE combines with plain stochastic updates (ANE-SGD) and with recursive variance reduction (ANE-VR). With an exact reference, the antithetic product cancels all odd-order noise terms, even for non-commuting matrices. With an approximate reference, the remaining odd-order error is bounded linearly in the reference error. Experiments on robust regression, data cleaning, offline reinforcement learning and imbalanced classification show that ANE-based methods outperform stochastic and variance-reduced baselines on each task, while synthetic controls show no gain under symmetric noise.

Introduction

Stochastic bilevel optimization, in which an outer objective depends on the solution of an inner problem, underlies hyperparameter tuning, data reweighting, robust learning, reinforcement learning and imbalanced classification. Gradient-based bilevel methods update the outer variables with the hypergradient, which involves the inverse of the inner Hessian. To keep this step scalable, the standard approach approximates the inverse with a truncated Neumann series. Specifically, each inverse-Hessian estimate sums products of the form (I - ηH_1)(I - ηH_2)...(I - ηH_k) up to a truncation depth K, where η is a step size and each stochastic Hessian factor H_j is drawn independently from a mini-batch. Stochastic bilevel methods use these factors directly. Variance-reduced methods also apply recursive corrections across iterations. Both families have shown promise in reducing hypergradient error, and both judge an estimator by its bias and variance.

Whether stochastic or variance-reduced, these methods share the same design: they build each inverse-Hessian estimate as a product of independently drawn Hessian factors, and they control its error only through its first two moments. Intuitively, an effective hypergradient estimator should be symmetric (i.e., its noise should not favor one direction within a single estimate) and low-variance (i.e., its estimates should vary little across iterations). Variance-reduced methods target the second property. However, the product structure adds a third source of error, skewness (i.e., the third moment of the estimation error), and existing methods leave it uncontrolled for two reasons.

On the one hand, asymmetric noise survives mini-batching. For ordinary stochastic gradients, mini-batches average such asymmetry away. In contrast, within the Neumann product the asymmetric noise of each independently drawn factor enters the odd-order terms, thus accumulating across the factors. Under weak contraction (i.e., when each factor reduces vector norms only slightly), the resulting third-moment error increases with the truncation depth, the inner condition number and the noise asymmetry. On the other hand, variance reduction does not reach the product. Recursive corrections reduce the variance across iterations, but they leave the product inside each estimate unchanged. As a result, a variance-reduced estimate can be low-variance and still skewed. This problem is amplified when corrupted labels, outliers, or heterogeneous and class-imbalanced data make the Hessian noise asymmetric.

To address these issues, we propose the Antithetic Neumann Estimator (ANE), which applies antithetic sampling (i.e., pairing each random draw with a mirrored copy so that their deviations offset) to the Neumann series to make hypergradient estimates symmetric and low-variance. The key idea is to draw the Hessian factors in mirrored pairs rather than independently, so that each inverse-Hessian estimate controls its third moment in addition to its first two. Specifically, ANE pairs each stochastic Hessian sample H with its mirror image 2R - H around a reference Hessian R, and averages the two resulting inverse-Hessian estimates. The odd-order noise terms then appear with opposite signs in the two products. However, the expected Hessian is unknown in practice, and the mirror is only as accurate as its reference. We therefore analyze both cases. With an exact reference, the antithetic product keeps only even-order noise terms and cancels all odd-order ones, even for non-commuting matrices. With an approximate reference, the remaining odd-order error is bounded linearly in the reference error. An iterate-level convergence bound also shows that the inverse-Hessian third-moment term decreases under ANE. Because ANE changes only the Hessian-vector step, it combines with existing methods: ANE-SGD uses plain stochastic updates, and ANE-VR adds recursive variance reduction. The antithetic pairing cancels odd-order terms inside each estimate, which mitigates skewness. The unchanged outer update keeps the variance reduction across iterations. Together, they make hypergradient estimates symmetric and low-variance.

We then run experiments on robust regression, data cleaning, offline reinforcement learning and imbalanced classification, with 20 seeds per task. First, ANE-based methods outperform stochastic and variance-reduced baselines on all four tasks, reducing the loss by 4-25% compared to the best baseline. Second, the largest gain, a 25% lower test loss, appears on robust regression, where outliers are likely to make the Hessian noise strongly asymmetric. Third, synthetic controls show no gain when the noise is symmetric, which suggests that the improvement comes from cancelling odd-order terms rather than from averaging two estimates. We highlight that ANE requires no change beyond the Hessian-vector step, making hypergradient estimates symmetric and low-variance within existing bilevel methods.

05 · ADVANCED USAGE装备 skill 的智能体一次写完一篇 ICML 论文With the skill, an agent writes a complete ICML paper in one run

Skill 还能带着智能体把一个想法写成排好版的投稿。我们只给装备 skill 的 Claude Opus 5.5 一段提示词(见附录),让它在一台只有 CPU 的 Mac 上独立完成:选定想法、写代码、下载数据、训练两个网络、跑实验、画图、上网核对参考文献,最后用 ICML 2027 模板排版。整个过程一次跑完,共 209 轮、约 82 分钟(实验约 37 分钟),API 费用 19.91 美元,没有人工干预。

The skill can also take an agent from an idea to a typeset submission. We give Claude Opus 5.5 with the skill a single prompt (see Appendix) and let it work alone in Claude Code on a CPU-only Mac. It chooses an idea, writes the code, downloads the data, trains two networks, runs the experiments, draws the figures, checks each reference on the web and typesets the paper in the ICML 2027 template. The single run takes 209 turns and about 82 minutes, 37 of them for the experiments, and costs $19.91 in API fees, without human intervention.

论文题为 SourceStein,研究测试时归一化如何混合源域和测试 batch 的统计量(图 3)。

Figure 3 shows the resulting paper, SourceStein, which studies how test-time normalization should mix the source and test-batch statistics.

SourceStein:一次运行写成的 ICML 2027 投稿SourceStein: an ICML 2027 submission written in one run

共 14 页:正文 8 页,声明与参考文献 1 页,附录 5 页。14 pages: eight of main text, one of statements and references, and five of appendix.

1 / 14摘要 · 1 引言Abstract · 1 IntroductionPDF
SourceStein 第 1 页 / page 1
SourceStein 第 2 页 / page 2
SourceStein 第 3 页 / page 3
SourceStein 第 4 页 / page 4
SourceStein 第 5 页 / page 5
SourceStein 第 6 页 / page 6
SourceStein 第 7 页 / page 7
SourceStein 第 8 页 / page 8
SourceStein 第 9 页 / page 9
SourceStein 第 10 页 / page 10
SourceStein 第 11 页 / page 11
SourceStein 第 12 页 / page 12
SourceStein 第 13 页 / page 13
SourceStein 第 14 页 / page 14
图 3 · Figure 3 · Claude Opus 5.5 生成,未经人工修改。页脚的 “Under review” 是 ICML 模板的默认文字,论文并未投稿。Generated by Claude Opus 5.5 without manual edits. The “Under review” footer is the default text of the ICML template, and the paper was not submitted.

实验是真实运行的:抽查的数字都能在原始输出里对上,方法和设置在看到第一份结果前就已定好,与预期不符之处论文也如实报告了。不过,论文未经人工审阅,实验规模也很小。它说明智能体能从一个想法写出格式完整、数字可追溯的稿件,但研究本身是否成立,仍要由研究者判断。

The experiments were actually run. Each number we spot-checked matches the raw outputs, the agent fixed the method and settings before it saw the first result, and the paper reports where the results miss its expectations. However, no human has reviewed the paper, and its experiments are small. It shows that an agent with the skill can produce a complete manuscript with traceable numbers from an idea, while whether the research holds up is still for a researcher to judge.

风险提示:对于一次完整的生成,尽管写出来像是人类写的,但因为篇幅过长,仍然会被判为 AI。

Risk: for a complete generation, although the writing reads as human, it is still labeled AI because of its length.

06 · CONCLUSION结论Conclusion

我们模拟了从草稿到成文的写作过程。在四篇论文上,从人类规范论文提炼的写作 skill 让生成文本被 Pangram 判为 AI 的比例降到与人类写作相当。不过,能通过检测的,一般是基于人写草稿的较短文本;直接由 AI 生成或篇幅较长的文章,被判为 AI 的比例仍然较高。因此,论文写什么、怎么论证,仍要由研究者来想。

We simulate the writing process from a draft to a finished text. On four papers, a writing skill distilled from human-written papers that follow academic conventions reduces the AI share of the generated text to that of the human-written text. However, the texts that pass the detector are mostly short ones written from a human draft, while text generated directly by AI, or a long paper, is still likely to be labeled AI. Therefore, the researcher must still work out what the paper says and how it argues.

本文有两点局限。一方面,结果只来自一个写作模型、六篇论文、两个题目和每份文本的一次检测,只能当作例子,不能当作比率。另一方面,从另一批规范论文提炼的 skill 是否同样有效,还有待研究。

This post has two limitations. On the one hand, the results are examples rather than rates, since they come from one writer model, six papers, two topics and a single scan of each text. On the other hand, whether a skill distilled from other well-formed papers works as well remains unexplored.

08 · DISCLAIMER AND ACKNOWLEDGEMENT检测分数低,不改变应当披露的内容A lower score does not change what to disclose

能通过检测器的草稿仍然是大模型写的草稿。ICLR 2026 等会议要求披露对大模型的使用ICLR 2026 Program Chairs, ICLR Blog 2025.
查看参考文献
,用这个 skill 写的稿件同样适用。检测分数也不该用来判定作者身份:第 4 节中两篇论文的人类写作也被判为 AI,而我们无从核实它们是怎么写的。

A draft that passes a detector is still an LLM draft. Venues such as ICLR 2026 require authors to disclose any use of an LLMICLR 2026 Program Chairs, ICLR Blog 2025.
View reference
, and this rule applies to drafts written with this skill. Detector scores should not decide authorship either: in Section 4, Pangram labels most of the human-written text of two papers as AI, although we cannot verify how it was written.

本研究出于作者的个人兴趣开展。文中观点仅代表作者本人,不代表作者所在的任何机构。作者声明不存在利益冲突。

The authors conducted this research out of personal interest. The views expressed here are the authors’ own and do not represent those of any organization they are affiliated with. The authors declare no competing interests.

致谢Acknowledgement

感谢我的导师王翔、张岸提供蒸馏论文的许可,感谢早期使用者的反馈,感谢我的女朋友赵谢杨对 blog 设计的支持与帮助。

I thank my advisors Xiang Wang and An Zhang for permission to distill the papers, the early users for their feedback, and my girlfriend Xieyang Zhao for her support and help with the design of this blog.

09 · REFERENCES参考文献References

  1. Sebastian Gehrmann, Hendrik Strobelt, and Alexander M. Rush. GLTR: Statistical Detection and Visualization of Generated Text. ACL 2019 System Demonstrations.↩︎↩︎
  2. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature. ICML 2023.↩︎↩︎
  3. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A Watermark for Large Language Models. ICML 2023.↩︎↩︎
  4. Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. Paraphrasing Evades Detectors of AI-Generated Text, but Retrieval is an Effective Defense. NeurIPS 2023.↩︎↩︎
  5. Xiaomeng Hu, Pin-Yu Chen, and Tsung-Yi Ho. RADAR: Robust AI-Text Detection via Adversarial Learning. NeurIPS 2023.↩︎↩︎
  6. Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu, and James Zou. GPT Detectors Are Biased Against Non-Native English Writers. Patterns 2023.↩︎↩︎
  7. Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. Can AI-Generated Text be Reliably Detected? TMLR 2024.↩︎↩︎
  8. Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature. ICLR 2024.↩︎↩︎
  9. Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text. ICML 2024.↩︎↩︎
  10. Vivek Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein. Ghostbuster: Detecting Text Ghostwritten by Large Language Models. NAACL 2024.↩︎↩︎
  11. Bradley Emi and Max Spero. Technical Report on the Pangram AI-Generated Text Classifier. arXiv 2024.↩︎ ↩︎↩︎ ↩︎
  12. Weixin Liang, Yaohui Zhang, Zhengxuan Wu, Haley Lepp, Wenlong Ji, Xuandong Zhao, Hancheng Cao, Sheng Liu, Siyu He, Zhi Huang, Diyi Yang, Christopher Potts, Christopher D. Manning, and James Y. Zou. Mapping the Increasing Use of LLMs in Scientific Papers. COLM 2024.↩︎↩︎
  13. Liam Dugan, Alyssa Hwang, Filip Trhlík, Josh Magnus Ludan, Andrew Zhu, Hainiu Xu, Daphne Ippolito, and Chris Callison-Burch. RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors. ACL 2024.↩︎↩︎
  14. Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, et al. Scalable Watermarking for Identifying Large Language Model Outputs. Nature 2024.↩︎↩︎
  15. Dmitry Kobak, Rita González-Márquez, Emőke-Ágnes Horvát, and Jan Lause. Delving into LLM-Assisted Writing in Biomedical Publications through Excess Vocabulary. Science Advances 2025.↩︎ ↩︎↩︎ ↩︎
  16. ICLR 2026 Program Chairs. Policies on Large Language Model Usage at ICLR 2026. ICLR Blog, 2025.↩︎↩︎
  17. Bradley Emi. Pangram Predicts 21% of ICLR Reviews are AI-Generated. Pangram Blog, 2025.↩︎↩︎
  18. Pavel Dolin, Xun Zou, Jiatian Wu, and Deguang Kong. Serving-Aligned Entire-Space Conversion Modeling via Click-Through Rate Knowledge Distillation. arXiv 2026.↩︎↩︎
  19. Benjamin Remy, Raphael Rossellini, Chihway Chang, and Rebecca Willett. Training Calibrated Conditional Generative Models with a Sample-Only Coverage Loss. arXiv 2026.↩︎↩︎
  20. Jason Bohne, Paweł Polak, Gary Kazantsev, and David Rosenberg. Your Hypergradient is Skewed: Antithetic Neumann Estimation for Bilevel Optimization. arXiv 2026.↩︎↩︎
  21. Cheng Pan, Nan Lu, Kai Arulkumaran, and Josie Hughes. Imitating the Imperfect: Offline-to-Online Robust Imitation Learning from Heterogeneous Demonstrators. arXiv 2026.↩︎↩︎
  22. Tapu Datta and Hafiz Imtiaz. Class-Competence Aggregation: Output-Layer Routing for Federated Multiclass DDoS Detection under Class-Concentrated Non-IID Data. arXiv 2026.↩︎↩︎
  23. Bowen Wang and Osvaldo Simeone. Capacity-Constrained Delayed Feedback for Online Conformal Risk Control. arXiv 2026.↩︎↩︎

10 · CITATION引用Citation

如需引用本文,可以使用下面的 BibTeX:

To cite this post, please use the following BibTeX entry:

@misc{sheng2026paperskill,
  title        = {The Art of Writing: Agents with Well-Curated Skills Can Cheat AI Detectors},
  author       = {Sheng, Leheng and Zhang, An and Wang, Xiang},
  year         = {2026},
  howpublished = {Blog post},
  url          = {https://lehengthu.github.io/blog/2026/paper-skill-ai-detection/}
}

A · APPENDIX附录Appendix

实验流程The experimental setup

六篇论文都按图 A1 的四步测试,每一步都对应研究者真实写论文的情形。检测用的是 Pangram 4.1 网页版(2026 年 11 月 14 日),每份文本测一次。

As shown in Figure A1, each of the six papers is tested in four steps, and each step mirrors how a researcher writes a paper. We scan each text once with the Pangram 4.1 web app on November 14, 2026.

实验按研究者写论文的流程设置The experiment follows how a researcher writes a paper

四个步骤,以及每一步这样设置的理由。The four steps, each with the reason for its setup.

六篇新论文Six new papers实验前几天才上 arXiv,模型没读过posted days before the experiment and unseen by the writer
内容提要Content brief像动笔前的笔记:只有事实,没有原句like a researcher’s notes: facts and none of the original sentences
生成两遍Generated twice同一模型、同一提要,只差 skillsame model and brief; only the skill differs
Pangram 检测Pangram scan和人类写作一起测,每份一次with the human-written text, one scan each
图 A1 · Figure A1 · 每一步都避免写作模型照搬原文:论文是新的,提要不含原句,生成时不能上网。Each step keeps the writer from copying the original: the papers are new, the briefs hold no original sentences, and the writer has no web access.

完整提示词The full prompts

两份提示词共用输出格式和内容提要,只有开头的写作指令不同(图 A2),图 1 的结果都可以据此复现。

As shown in Figure A2, the two prompts share the output format and the content brief and differ only in their opening instructions. Each result in Figure 1 can be reproduced from them.

两份提示词只有开头的指令不同The two prompts differ only in their opening instructions

两份提示词的模板;每份提示词的结尾都是一篇论文的内容提要。The two prompt templates; each prompt ends with the content brief of one paper.

不使用 skillWithout the skill
You are drafting part of an ML conference paper.

Using only the facts in the content brief below, write:
- the Abstract (one paragraph, 150-250 words), and
- the Introduction (650-800 words, 4-8 paragraphs).

Write it the way you normally would for a strong ICLR / NeurIPS / ICML submission.
使用 skillWith the skill
You are drafting part of an ML conference paper with a paper-writing skill.

1. Read the skill first: `<skill>/SKILL.md`, then the files it asks for before drafting an Abstract and an Introduction: `<skill>/references/paper_types.md` (pick this paper's type), `<skill>/references/introduction.md` (write the argument chain first, then check rules IR1-IR15), `<skill>/references/contributions.md`, `<skill>/references/concepts.md`, `<skill>/references/prose.md` (slot templates, connectives, cohesion, sentence rules, habits to avoid), `<skill>/references/examples.md`, and the phrase bank `<skill>/assets/lexicon/lexicon.md` (take the wording for each rhetorical move from its section; prefer core and occasional entries). Check multi-word phrases you are unsure about with `python3 <skill>/tools/ps.py "phrase" -q`.
2. Using only the facts in the content brief below, write the Abstract (one paragraph, 150-250 words) and the Introduction (650-800 words, 4-8 paragraphs), following the skill's rules (argument chain, rule of two, the standard before the weaknesses, the key idea as the negation of the root cause, the First / Second / Third evidence roadmap, no contribution list, calibrated claims, banned words, sentence and cohesion rules).
3. The lexicon's example sentences come from published papers: use its entries as phrase patterns, but never copy an example sentence or clause.
两份提示词都接着写Both prompts continue with
Output format (the same for every writer in this study; it is not a style instruction):
- Plain text only: no LaTeX commands (write i.e. and e.g., never \ie or \eg), no markdown except the two headings `## Abstract` and `## Introduction`. If you use a list, put each item on its own line starting with `* `.
- No citations, reference markers or URLs. You may mention "Figure 1" or "Section 4" in words.
- Do not invent numbers beyond those in the brief.
- Reply with the two parts only, starting with `## Abstract`: no preamble, no notes, no closing remark.

Content brief:
- …
图 A2 · Figure A2 · 除 skill 路径缩写为 <skill>/ 外均为原文。不使用 skill 时不能用工具;使用 skill 时可以读取 skill 文件夹、查询短语。两者都不能上网。Verbatim, except that the skill’s path is shortened to <skill>/. Without the skill, the writer has no tools; with the skill, it can read the skill’s folder and look up phrases. Neither can access the web.

不使用 skill 时,写作模型对每份提要一轮作答。使用 skill 时,它每篇先走 24 到 33 轮:读 7 到 9 个 skill 文件,浏览短语库,查询短语 8 到 18 次(如 “share one design”“no longer”)。有一份不使用 skill 生成的文本摘要偏长,我们重新生成了一次。

Without the skill, the writer answers each brief in a single turn. With the skill, it takes 24 to 33 turns per paper: it reads 7 to 9 of the skill’s files, browses the phrase bank and looks up phrases 8 to 18 times, such as “share one design” and “no longer”. One text generated without the skill has an overly long abstract and is regenerated once.

Skill 的六个部分The six parts of the skill

Skill 的主文件带智能体走完从查看投稿要求到最后自查的 8 个步骤,细节由 12 份参考文档、短语库、几个小工具和 LaTeX 模板负责。每条文风规则都标注了它在人类规范论文中有多普遍。决定行文的是其中六个部分(图 A3)。

The skill’s main file walks the agent through eight steps, from the venue’s call for papers to a final self-review, while twelve reference files, a phrase bank, small tools and LaTeX templates hold the details. Each style rule records how widely human-written papers that follow academic conventions follow it. As shown in Figure A3, six parts shape the prose.

Skill 把规范论文变成规则、短语、次数和检查The skill turns well-formed papers into rules, phrases, counts and checks

决定行文的六个部分;所有数字都来自 skill 自己的文件。The six parts of the skill that shape the prose; all numbers are counted in the skill’s own files.

论证链Argument chain动笔前先写 8–10 条一句话论断8–10 one-line claims before any prose
  1. 已有工作共同的设计the design all prior work shares
  2. 它没达到的标准the standard it misses
  3. 两个缺陷two drawbacks
  4. 否定共同设计的核心想法a key idea that negates the shared design
短语库Phrase bank1,272 条原文短语,22 个分区1,272 attested phrases in 22 sections
常用core 659偶用occasional 440已弃用retired 71仅参考reference only 75禁用banned 27
词频Word frequency人类论文中的词数word counts in human papers
specifically48
however43
moreover34
furthermore5
each94
every4
口癖Habits to avoid27 种机器草稿的习惯,各有改法27 habits of machine drafts, each with a fix
, so→Therefore,
every→each
X, not Y→rather than
Table 2 lists→We report … in Table 2
句子规则Sentence rules15 项句子检查,外加衔接规则15 sentence checks plus cohesion rules
前置front主谓S + V细节 →details →重点point

句长中位 20 词;主语前最多 10 个词;一句一个主语;重点放在主句末尾;几乎每句都接住上一句。Median 20 words; at most ten words before the subject; one subject per sentence; the point at the end of the main clause; nearly all sentences link back.

工具Tools查短语,查草稿phrase lookup and draft checker
ps.py "to this end"多数论文都用 · 常用most papers · core
ps.py "delve into"旧写法 · 已弃用older usage · retired
check_paper.py字数、页数、必需声明、27 种口癖(逐行)word budget, page limit, required statements, 27 habits by line

另有:ICLR、NeurIPS、ICML 的 LaTeX 模板 · TikZ 框图样式包 · 绘图配色 · 8 种表格模板 · 各会议的页数预算Also: LaTeX templates for ICLR, NeurIPS and ICML · a TikZ style package for diagrams · a plot palette · eight table templates · page budgets per venue

图 A3 · Figure A3 · 次数来自 skill 的短语库和短语索引,统计来源论文的全部文字。Counts from the skill’s phrase bank and phrase index, over all text of its source papers.
  • 论证链。动笔前,智能体先把引言规划成 8–10 条一句话论断:已有工作的共同设计、它没达到的标准、两个缺陷,以及否定这一设计的核心想法。段落和段首句都由这条链决定;另有五种范式,对应工作流、闭式方法、损失函数、奖励设计和实证发现类论文。
  • 短语库。从论文中摘出的 1,272 条表达,按写作步骤分成 22 个分区,外加连接词和词类。每条记录它在每篇论文中的次数和状态:常用 659 条、偶用 440 条、已弃用 71 条、仅见于参考论文 75 条、禁用 27 条。
  • 词频。次数告诉智能体规范论文实际用哪个词:来源论文里 “specifically” 出现 48 次、“however” 43 次、“moreover” 34 次;“each” 94 次,“every” 只有 4 次。“Furthermore”“Notably” 这类少见词被列为禁用。
  • 口癖。对比一份 2.4 万词的机器草稿和这些论文,整理出 27 种机器草稿的习惯,每种都给出论文里的写法:“, so” 连接分句改成 “Therefore,”,“every” 改成 “each”,“X, not Y” 改成 “rather than”。
  • 句子规则。15 项句子检查和一组衔接规则,都在这些论文上测量过:句长中位数 20 词,主语前最多 10 个词,一句一个主语,重点放在主句末尾,几乎每一句都接住上一句。
  • 工具。查询工具报告任意五词以内的短语在每篇论文中出现几次(索引只存词序列计数,不含句子),比如 “to this end” 属于常用,“delve into” 是已弃用的旧写法。检查器逐行报告草稿的字数预算、页数上限、必需声明和 27 种口癖。
  • Argument chain. Before any prose, the agent plans the introduction as 8–10 one-line claims: the design that prior work shares, the standard it misses, two drawbacks and a key idea that negates the shared design. The paragraphs and their openers follow from this chain, and five patterns adapt it to papers that propose a workflow, a closed-form method, a loss, a reward or a finding.
  • Phrase bank. The phrase bank holds 1,272 expressions taken from the papers, sorted into 22 sections by writing step, plus connectives and word classes. Each entry records how often each paper uses it and a tier: 659 core, 440 occasional, 71 retired, 75 reference-only and 27 banned.
  • Word frequency. The counts tell the agent which words such papers actually use. In the source papers, “specifically” appears 48 times, “however” 43 and “moreover” 34, while “each” appears 94 times and “every” only 4. Rare words such as “Furthermore” and “Notably” are banned.
  • Habits to avoid. Comparing a 24,000-word machine-written draft with the papers yields 27 habits. Each comes with the form the papers use instead: “, so” joining two clauses becomes “Therefore,”, “every” becomes “each” and “X, not Y” becomes “rather than”.
  • Sentence rules. Fifteen sentence checks and a set of cohesion rules are measured on the papers. The checks set a median of 20 words per sentence, at most ten words before the subject, one subject per sentence and the point at the end of the main clause. The cohesion rules link nearly all sentences back to the previous one.
  • Tools. A lookup tool reports how often any phrase of up to five words occurs in each paper, from a shipped index of word-sequence counts that holds no sentences. For example, it marks “to this end” as core and “delve into” as a retired, older usage. A checker reports the word budget, the page limit, the required statements and each match of the 27 habits, with line numbers.

句子更长,句首和短语来自来源论文Longer sentences, with the openers and phrasing of the source papers

为了看清 skill 改变了什么,我们在 18 份文本(六篇论文的完整摘要和引言)上测量了 Pangram 这类分类器能捕捉到的表层统计,并用 skill 的检查器统计口癖。这些统计只数文字,不用模型。

To see what the skill changes, we measure the surface statistics that a classifier such as Pangram can pick up. Specifically, we measure them over the full abstracts and introductions of the 18 texts and count habits with the skill’s checker. These statistics only count words and use no model.

不使用 skill 时,生成文本的句子短而均匀:平均 17.6 词,标准差 7.2,超过 40 词的只有 0.6%。人类写作平均 24.9 词,标准差 12.7,8.5% 超过 40 词;使用 skill 后介于两者之间(22.7 词、8.9、2.0%)(图 A4)。

Without the skill, the generated sentences are short and even: 17.6 words on average with a standard deviation of 7.2, and 0.6% over 40 words. The human-written text averages 24.9 words with a standard deviation of 12.7 and 8.5% over 40 words. As shown in Figure A4, the text generated with the skill falls in between (22.7, 8.9 and 2.0%).

使用 skill 后句子更长,句首和短语来自来源论文With the skill, sentences are longer and take their openers and phrasing from the source papers

18 份文本完整摘要和引言上的六项表层统计。Six surface statistics over the full abstracts and introductions of the 18 texts of the six papers.

人类写作Human-writtenOpus 5.5 生成(不使用 skill)Generated by Opus 5.5 (without the skill)Opus 5.5 生成(使用 skill)Generated by Opus 5.5 (with the skill)
平均句长Mean sentence length词words
人类写作Human-written24.9
不使用 skillWithout skill17.6
使用 skillWith skill22.7
句长的标准差Spread of sentence length词,越小句子越均匀standard deviation in words; smaller is more even
人类写作Human-written12.7
不使用 skillWithout skill7.2
使用 skillWith skill8.9
超过 40 词的句子Sentences over 40 words占全部句子share of sentences
人类写作Human-written8.5%
不使用 skillWithout skill0.6%
使用 skillWith skill2.0%
以 “We” 开头的句子Sentences opening with “We”占全部句子share of sentences
人类写作Human-written14.7%
不使用 skillWithout skill16.0%
使用 skillWith skill7.1%
以 “However” 或 “Specifically” 开头的句子Sentences opening with “However” or “Specifically”占全部句子share of sentences
人类写作Human-written1.9%
不使用 skillWithout skill0.3%
使用 skillWith skill13.5%
也出现在 skill 来源论文里的四词序列Four-word sequences also in the skill’s source papers占全部四词序列share of four-word sequences
人类写作Human-written0.8%
不使用 skillWithout skill0.4%
使用 skillWith skill1.3%
图 A4 · Figure A4 · 四词序列只在 skill 的来源论文里计数。Four-word sequences count matches in the skill’s source papers only.

句首也不同。不使用 skill 时,16.0% 的句子以 “We” 开头,和人类写作(14.7%)相当,使用 skill 后只有 7.1%;使用 skill 后 13.5% 的句子以 “However” 或 “Specifically” 开头,另外两种文本都很少这样开头(0.3% 和 1.9%)。这两项靠近的是 skill 的来源论文(其中 “specifically” 和 “however” 分别出现 48 次和 43 次),而不是这六篇论文的人类写作。

The sentence openers also differ. With the skill, 7.1% of the sentences open with “We”, compared with 16.0% without the skill and 14.7% in the human-written text. Moreover, 13.5% of the sentences generated with the skill open with “However” or “Specifically”, while the other two texts rarely do (0.3% and 1.9%). On both measures, the text generated with the skill moves toward the skill’s source papers, which use “specifically” 48 times and “however” 43 times, rather than toward the human-written text.

短语也靠近来源论文。使用 skill 后,1.3% 的四词序列也出现在来源论文里,约为不使用 skill 时(0.4%)的三倍,也高于人类写作(0.8%)。而与人类写作共享的八词序列只有 2.5%,和不使用 skill 时(2.7%)相当,其中约四分之三也出现在内容提要里。所以 skill 复用的是来源论文的说法,并没有照抄人类写作。

The phrasing moves toward the source papers as well. With the skill, 1.3% of the four-word sequences also occur in the source papers, about three times the share without the skill (0.4%) and more than the human-written text (0.8%). In contrast, the text generated with the skill shares only 2.5% of its eight-word sequences with the human-written text, about as much as without the skill (2.7%). About three quarters of these sequences also occur in the content brief. Therefore, the skill reuses phrasing from the source papers rather than copies the human-written text.

只给题目写成的两篇立场论文上,skill 同样改了文风:全文平均句长从 18.8 和 18.2 词增加到 23.9 和 22.4 词,以 “We” 开头的句子从 18 句降到 6 句和 10 句。

The skill also changes the style of the two position papers written from a topic alone. Their mean sentence length increases from 18.8 and 18.2 to 23.9 and 22.4 words, and the sentences that open with “We” decrease from 18 to 6 and 10.

机器草稿的口癖大多消失Most habits of machine-written drafts disappear

口癖的变化最明显。去掉领域术语和正常用法后,检查器在不使用 skill 生成的文本中找出 75 处口癖,使用 skill 后只有 6 处,人类写作有 32 处(图 A5)。不使用 skill 时有 20 次 “every”、15 次连接分句的 “, so” 和 10 次句中的 “therefore”,使用 skill 后分别是 0、0 和 1 次。人类写作也用了 11 次 “every”:这张表记录的是来源论文避开的写法,而不是所有作者都避开的写法。

The habits change the most. As shown in Figure A5, the checker finds 75 habits in the text generated without the skill, 6 with the skill and 32 in the human-written text. These counts exclude domain terms and legitimate uses, which we remove by hand. Without the skill, the text writes “every” 20 times, joins two clauses with “, so” 15 times and puts “therefore” mid-sentence 10 times, compared with 0, 0 and 1 with the skill. The human-written text also uses “every” 11 times. In other words, the table records what the source papers avoid rather than what all authors avoid.

Skill 去掉了大部分机器草稿的口癖The skill removes most habits of machine-written drafts

Skill 检查器在六篇论文 18 份文本的摘要和引言中找出的口癖。Habits that the skill’s checker finds in the abstracts and introductions of the 18 texts of the six papers.

人类写作Human-writtenOpus 5.5 生成(不使用 skill)Generated by Opus 5.5 (without the skill)Opus 5.5 生成(使用 skill)Generated by Opus 5.5 (with the skill)
27 种口癖合计All 27 habits检查器找出的口癖habits the checker finds
人类写作Human-written32
不使用 skillWithout skill75
使用 skillWith skill6
把 every 当默认量词“every” as the default quantifier改用 eachuse “each”
人类写作Human-written11
不使用 skillWithout skill20
使用 skillWith skill0
, so 连接两个分句“, so” joining two clauses改用 Therefore,use “Therefore,”
人类写作Human-written2
不使用 skillWithout skill15
使用 skillWith skill0
句中的 thereforeMid-sentence “therefore”改用句首 Therefore,use a sentence-initial “Therefore,”
人类写作Human-written2
不使用 skillWithout skill10
使用 skillWith skill1
表示变化的动作动词Motion verbs for change如 grows、shrinks、falls tosuch as “grows”, “shrinks”, “falls to”
人类写作Human-written4
不使用 skillWithout skill8
使用 skillWith skill2
用 needs 或 extra 表示要求“needs” or “extra” for a requirement改用 requires、additionaluse “requires” or “additional”
人类写作Human-written0
不使用 skillWithout skill6
使用 skillWith skill0
图 A5 · Figure A5 · 用 skill 自带的检查器统计,已人工去掉 23 处领域术语和正常用法。Counted with the skill’s own checker, after removing 23 domain terms and legitimate uses by hand.

这些测量与 Pangram 的结果吻合:不使用 skill 时,生成文本是前沿模型默认的文风,句子短而均匀、口癖多;skill 把它换成了来源论文的写法。即使这种文风与同一篇论文的人类写作不同,Pangram 也把它读得更像人写。

These measurements are consistent with the Pangram results. Without the skill, the generated text follows a frontier model’s default style, with short, even sentences and many habits, and the skill replaces it with the writing of its source papers. Pangram reads this style as more human even when it differs from the human-written text of the same paper.

一次性生成论文的提示词The prompt for the one-shot paper

第 5 节的论文由图 A6 的提示词生成。Opus 5.5 在 Claude Code 非交互模式下运行,最多 600 轮,可以读写文件、运行命令和上网,但不能启动子智能体。

The paper of Section 5 is generated from the prompt in Figure A6. Opus 5.5 runs in the non-interactive mode of Claude Code with at most 600 turns and can read and write files, run commands and use the web, but it cannot start subagents.

一段提示词,从想法写到排好版的论文One prompt, from an idea to a typeset paper

Opus 5.5 收到的完整任务。The full task that Opus 5.5 receives.

You are writing a complete ICML submission from scratch, in one session, with the paper-writing skill at <skill> (read its SKILL.md first and follow its workflow, Steps 0 to 7, reading each reference file at the step that names it). Your working folder is the current directory, <paper>; put every file of the paper project there.

1. Idea. Choose the research idea yourself: one original, specific machine-learning method (not a survey or a position paper) that fits one of the skill's paper types. It must be testable with real experiments that you run here: a CPU-only machine (15 cores, 48 GB RAM, no GPU), at most about 45 minutes of compute in total, on synthetic data or small public datasets you can download. Python 3 has numpy and scipy; create a virtual environment in .venv inside this folder and pip install anything else you need (matplotlib for the plots, scikit-learn or a CPU build of torch if the method needs them).

2. Experiments. Put the experiment code in code/ and its raw outputs in results/. Every number in the paper must come from those outputs. Never invent or estimate a number: if an experiment cannot be run, leave its table cells as \pend and say so, as the skill prescribes.

3. Venue and template. Follow the skill's Step 0 for ICML: check the current call for papers, fetch the latest official ICML style kit (templates/venue/icml/README.md), use the skill's ICML skeleton (templates/main_skeleton_icml.tex, templates/chapters_icml/ renamed to chapters/, templates/preamble.tex), and write an anonymous submission with the Impact Statement and an appendix.

4. Figures and tables. Draw the method diagram in TikZ on templates/housediagram.sty and build it with scripts/render_diagram.py --check; make the result plots with templates/plot_style.py from the files in results/; build the tables from templates/tables.tex.

5. References. Cite only real papers. Check each one on the web (arXiv or DBLP) before you add it, and format the .bib as the skill says (scripts/bib_to_dblp.py).

6. Build and check. Compile with tectonic until there are no errors, no undefined references and no overfull boxes; the main text must end exactly at the bottom of page 8 (references, Impact Statement and appendix after it). Run python3 <skill>/scripts/check_paper.py . --venue icml --stage submission and fix what it reports, as the skill's Step 7 describes. Look at the rendered pages (for example with PyMuPDF) and fix what you see.

7. Deliverables. The compiled PDF in this folder, and NOTES.md with: the title and the idea in one paragraph, the experiments you ran and how long they took, which numbers come from which result file, and the final check_paper.py summary.

Rules: work only inside this folder (you may read the skill folder; never modify it). Do not use git. Finish only when the PDF builds cleanly and the checks are addressed.
图 A6 · Figure A6 · 除 skill 路径缩写为 <skill>/、工作目录缩写为 <paper>/ 外均为原文。Verbatim, except that the skill’s path is shortened to <skill>/ and the working folder to <paper>/.