全文摘要：AI 把「写代码」变便宜了，但把「证明代码对」变贵了——一份 patch 让 AI 写可能只要 5 分钟，人验证却要 45 分钟。本文给出一份可落地的 6 层检查清单，从「让工具先当裁判」到「审意图不审语法」「审测试不审实现」「审边界与安全」，并用一个能跑的评审脚本把「审什么」变成可执行的规则。
目录
- 先说结论：代码便宜了，验证变贵了
- 为什么 AI 的代码要换一套审法
- 第 1 层：让工具先当裁判
- 第 2 层：审意图，不审语法
- 第 3 层：审测试与边界
- 第 4 层：审安全与幻觉依赖
- 第 5、6 层：拆分与回滚、该不该自己重写
- 一个能跑的 AI 代码评审器
- 症状 → 原因 → 解法对照表
1. 先说结论（The Verdict）
结论先给：AI 生成的代码，审查的重点不是「它写得对不对」，而是「它为什么看起来对」。
一份 AI patch 能在 5 分钟内写完，但验证它往往要 45 分钟。2026 年的残酷算式是：写代码的边际成本已经被压到接近零，证明正确的成本成了新的税。所以审核 AI 代码，本质是把你的注意力从「读代码」挪到「验证代码」上。
审核 AI 代码的六层清单，一句话概括：让机器先过一遍机器能过的，人只盯机器会犯的那些「看起来对」的错。
六层分别是：
打个比方 🎭：AI 像一个语速极快、偶尔胡说的助手。它 5 分钟给你 10 页纪要，你看起来效率爆炸，但真正花掉的是你逐句核对的那 45 分钟。审核 AI 代码，就是学会先不读那 10 页，而是先问「这些他是怎么来的」。
参考资料：
2. 为什么 AI 的代码要换一套审法（Why a Different Review）
先破一个误区：很多人以为 AI 代码的错是「语法错」或「编译不过」。恰恰相反，那些错 CI 一道就能拦住，根本不用人看。
真正危险的是幻觉正确性（hallucinated correctness）：代码能编译、能过所有测试、看起来完全正常，但它是错的。
GitHub 的官方指引把这类错列为最需警惕的 red flag：明显的幻觉（调用不存在的 API、引用越界变量）会被 CI 抓住；而危险的那个更隐蔽——分页的 off-by-one、某条从没被测试打到的分支上缺失的权限检查、在某个边界条件下短路掉的校验，这些全都藏在「通过测试」的伪装下。
参考资料：
- Agent pull requests are everywhere. Here's how to review them. -- GitHub Blog ⭐值得阅读
- Review AI-generated code -- GitHub Docs
为什么 AI 的错长得跟人不一样？ 三个结构性原因：
① 模式匹配，不是推理。 AI 生成代码是基于训练数据里的模式做预测，它不知道自己「预测错」了。它给出的代码带着「我很确定」的自信，哪怕那个 API 是它脑补出来的。
② 同源假设。 让同一个 AI 既写代码又写测试，两个产物共享同一套「什么是对」的假设。测试不是从需求倒推的，而是从代码倒推的——这种测试叫同义反复测试（tautological tests）。它通过，是因为它反向验证了自己的实现，而不是验证了需求。
③ 没有外部基准。 工业界应对 AI 代码质量问题的默认手段是「再上一个 AI reviewer」。但这在结构上是循环论证：生成器和评审器从同一个 artifact 出发，共享同一套训练分布，因此它们的失败是相关的。评审是在拿代码验证代码，而不是拿意图验证代码。
类比 🎭：这就像让同一个人既出考题又出答案。答案是「对」的，只是因为它是从考题里推出来的，但考题本身可能就出错了。
后果是量化的。 CodeRabbit 分析了 470 个真实 GitHub PR，发现 AI 生成代码产生的问题量约为人类代码的 1.7 倍——其中逻辑与正确性错误比人类代码多 75%。这些不是 toy benchmark，是生产仓库里的实测。
参考资料：
- The Specification as Quality Gate: Three Hypotheses on AI-Assisted Code Review -- arXiv
- AI broke QA and nobody noticed -- Jia Wei Ng
- Test-First Strategy for AI-Generated Code (2026) -- Alex Cloudstar
所以审法要换：机器先过机器能过的关，人只盯「看起来对」的错。
3. 第 1 层：让工具先当裁判（Let the Tools Judge First）
人不是好的编译器。别把十分钟花在肉眼看语法上，那是一个确定性工具几秒就能干完的事。
第 1 层就是让机器先跑一遍它该跑的：编译、lint、类型检查、已有测试套件。这一关的目的不是「证明代码对」，而是先砍掉所有「不对」的明显形态，把人的注意力留给后面几层。
对 Python 项目，第 1 层可以是：
python -m compileall . # 语法与编译
ruff check . # 静态 lint
mypy . # 类型检查
pytest -q # 现有测试套件
对前端 / Node 项目，第 1 层就是 CI 里那几行：
- run: npm ci
- run: npx tsc --noEmit
- run: npx eslint .
- run: npm test -- --coverage
关键规则：生成的代码过不了第 1 层，就不该合入。 这不是「AI 不行」，而是「不管是人写的还是 AI 写的，都要先过同一套 CI 关卡」。第 1 层跟审核人没有关系，它是一道自动闸门，帮你把「机器能判的错」和「只有人能判的错」分开。
⚠️ 但这道闸门有个盲区：它报错你当然能看见；可它没报错不代表代码对。能被编译器和 linter 抓住的是「明显的错」，抓不住的是「看起来对、但是错」。后者正是后面几层的战场。
参考资料：
- 7 Checks to Run Before Trusting AI-Generated Code -- KDnuggets ⭐值得阅读
- AI-Generated Code Quality: What Breaks, Why, and How to Ship It Safely -- Sarah D. Hashnode
4. 第 2 层：审意图，不审语法（Review the Intent）
过了第 1 层的闸门，代码已经能编译、能过测试了。但能跑 ≠ 解决了正确的问题。
这一层你要问的不是「这段代码怎么写得好不好」，而是**「这段代码在解决正确的问题吗？」** 它验证两个东西：是否贴合项目目的，是否符合你的架构约定。
GitHub 官方给的三个自审问题，直接照搬：
- 这段代码解决了正确的问题吗？ 有没有跑偏需求，还是只是「看起来在干活」。
- 它遵循了我们的约定吗？ 队伍里的 naming、error handling、分层、目录归属。
- 它有没有被硬塞进错误的层？ AI 擅长把逻辑放进「最顺手」的位置，而不是「应该」的位置——比如把业务规则写进 controller、把数据访问塞进 view。
关键前提：你必须先给 AI 足够的 context。 在生成之前，把 README、文档、最近的 PR 作为起点喂给 AI，告诉它「哪些来源可信、哪些不可用、给出好的示例」。这决定了第 2 层审起来是「对着一团陌生代码懵」还是「一判就中」。
审核方同样需要 context。若你只拿 PR 本身去审，那和 AI 生成时看到的一样单薄。你至少要拿到：仓库结构与依赖关系、架构约束与 service 边界、API contract 与向后兼容预期、编码规范、历史事故与 postmortem、上一轮 review comment 里接受的 trade-off。这些 context 决定你能否判断「这段代码到底该不该长这样」。
类比 🎭：审语法像改作文的错别字，审意图像看这篇作文有没有跑题。错别字交给词典（编译器）就行，跑不跑题只有知道题目的人能判断。
这里有个反直觉但重要的点：第 2 层往往要求你先怀疑「测试是不是也错了」。 如果 AI 把「需求 A」错误理解成了「需求 B」，它写出的测试也是验证需求 B 的——测试全绿，但做的不是你要的事。所以前两层的核心是：先确定「什么是对的」，再谈「代码对不对」。
参考资料：
- Review AI-generated code -- GitHub Docs ⭐值得阅读
- Agentic AI and Code Reviews: Toward a Pattern Language for Code Reviews in the Age of Agentic AI -- Readwise
- How to Build an AI Coding Workflow as a Software Engineer -- Dev Fieldnotes
5. 第 3 层：审测试与边界（Review the Tests, Then the Boundary）
先读测试，再读实现。 这是审核 AI 代码最关键的一条。
两个原因：
① 测试能帮你判断「AI 理解对了需求没有」。 如果 AI 把需求理解偏了，它的测试也会验证那个偏掉的需求。所以第一件事是看：这些测试真的在验证验收标准吗，还是只是从实现反推的回声？ 后者叫同义反复测试——它通过，只因为它验证的是实现本身在干什么，而不是该干什么。
② 边界是 AI 最常栽的地方。 AI 默认走 happy path。所以你要盯着测试里有没有真实覆盖这些边界：
- 空输入、单元素、最大长度、零、负数
- null / None / undefined、缺失字段
- 错误路径：坏输入时代码确实抛/返回特定错误，且测试断言的是那个具体错误，而不只是「会抛异常」
类比 🎭：这像审一份保险条款，重点不是看「正常理赔怎么走」（那当然写得通），而是看「地震、洪水、自杀这些极端情况赔不赔」。AI 把正常路径写得再漂亮都没意义，边界才是它真正会翻车的地方。
canary 信号： 如果 AI 写的测试都是用来证明实现、没有边界用例、且断言过于宽松（比如只断言「不抛异常」而不断言「返回特定值」），那这套测试基本没有验证价值。它不过是给「看起来对」的代码盖上了一个「看起来被测过」的章。
数据能证明这一点。 对开源项目 agent 生成测试的研究发现：AI 在边界用例的多样性上甚至超过人类（variety 0.62 vs 0.32），但在稳定性上落后——人类测试的 flaky 率为 0.30，AI 生成的为 0.41。也就是说，AI 会想到更多边界，但它的测试更不稳定、更脆，更容易误报。这恰恰说明：AI 能想到的边界和「可靠验证这些边界」是两回事。
参考资料：
- Reviewing AI-Generated Tests: A Code-Review Checklist 2026 -- QASkills.sh ⭐值得阅读
- AI-Generated Tests Are Testing the AI's Blind Spots, Not Your Code -- Dzaki Amri Zaidaan
- Beyond Test Presence: Assessing the Quality and Robustness of Agent-Generated Tests -- arXiv
6. 第 4 层：审安全与幻觉依赖（Review Security & Hallucinated Dependencies）
这一层是不能省的。因为 AI 生成代码时不做对抗性思考——它不会像攻击者那样想「我该怎么绕过这道校验」。但攻击者会。
审核时要扫的那几类，基本就是 OWASP 的 instinct 清单：
① 注入类（injection）。 AI 特别爱用字符串拼接拼 SQL / 查询，爱把用户输入直接塞进命令：
- SQL 注入：query 里的字符串拼接
- NoSQL 注入：来自用户 JSON 的 MongoDB 操作符
- XSS：dangerouslySetInnerHTML、未转义模板、LLM 渲染的原始 HTML
- 命令注入：exec、subprocess 里未校验的输入
② 授权缺失。 记住 authentication ≠ authorization。AI 很容易只写了「登录了没」，却漏了「这个用户有没有权限干这件事」。尤其是那些测试里从来没打到的分支。
③ 硬编码 secrets。 AI 常忘了把 key、token、password 塞进环境变量，而是直接写在源码里：
OPENAI_API_KEY = "sk-abc123def456" # 危险：硬编码密钥
DATABASE_URL = "postgres://user:password@localhost/db" # 危险：明文密码
④ 过度宽泛的输入绑定（mass assignment）。 把整个请求体直接绑定到对象，「用户传什么就设什么」，导致越权字段被覆盖。
⑤ 幻觉依赖（hallucinated dependencies）——这个最隐蔽。 AI 会引用一个根本不存在、或已过时的库。更危险的是：一个不存在的包名可以被攻击者注册——你 npm install 了 AI 编出来的名字，等于装了一个别人塞的包。所以新加的任何 dependency 都要单独审，确认它存在于 lockfile、license 合规、来源可信。
⑥ 针对代码生成 agent 本身的注入。 别忘了，现在很多代码是你让 agent 写的。agent 的 context 来自仓库、网络、连接的 tool。攻击者可以把隐藏指令塞进 issue body、PR 描述、AGENTS.md、MEMORY.md——这就叫 间接提示注入（indirect prompt injection），是对开发工作流本身的攻击。
类比 🎭：AI 像那个把「钥匙塞在门口花盆下」的人，觉得方便就行；攻击者则是挨个翻花盆的小偷。AI 不思考「谁会来翻」，所以这类坑它自己发现不了，只能由审的人从攻击者视角补上。
参考资料：
- Secure Coding with AI -- OWASP Cheat Sheet Series ⭐值得阅读
- The 15 Security Failures Your AI Coding Assistant Ships by Default -- Prodogon
- OWASP Top 10 for AI-Generated Code: A01-A10 Fixes (2026) -- Vibe Eval
- LLM Prompt Injection Prevention -- OWASP Cheat Sheet Series
7. 第 5、6 层：拆分与回滚、该不该自己重写（Scope, Rollback & Rewrite）
前 4 层审的是「这段代码本身对不对」，第 5、6 层审的是**「这段代码该不该以这个形态存在」**。
第 5 层：审拆分与回滚（Review Scope & Rollback）
AI 最大的毛病之一是scope 失控：一个 5 分钟的 patch，它顺手改了 10 个文件、删了 3 个无关函数、加了 2 个新依赖。所以这一层要看：
- diff 是否原子化：一个 PR 只做一件事，文件改动范围可控。
- 有没有意外删除：git diff | grep '^-'，对每个被删的符号全仓库搜一遍，确认它真的死了，而不是被 AI 误删。
- 回滚点是否存在：改之前留一个 recovery branch。
读 diff 的顺序也有讲究：按风险从高到低读，别从上到下读。 先扫删除（^-），再读新增，最后看被 AI 一顺带改掉的「无关区域」。
类比 🎭：这像收快递。你打开包裹第一眼看的是「有没有少东西」，而不是「赠品漂不漂亮」。AI 的 PR 最常「少东西」——少了边界处理、少了一行权限校验、少了一个原本在用的函数。
第 6 层：该不该自己重写（When to Rewrite Instead of Fix）
这是最容易踩坑的一层——沉没成本。AI 给你一大坨跑不通的代码，你本能想「修一修就好」。但很多时候，删掉重来比修补更便宜。
Kunal Ganglani 的 checklist 给了明确的「删除重写」判据，命中任一条就删：
- diff 超过 300 行，且改动不是机械性的
- 一个小 feature 新增了 2 个以上新依赖
- 读了 10 分钟还讲不清意图
- 引入了新抽象，却没有删掉旧抽象
- 测试全绿，但这段代码你用不了（比如沟通成本、可维护性超过它的价值）
为什么该删？ 因为「AI 生成的、藏着模糊意图的大 diff」，你修它花的认知成本，往往超过「给它一个更紧的 prompt 重新生成一遍」。你不是在修代码，你是在为一个模糊的约定做生产级兜底——这是最不划算的事。
⚠️ 一个必须记住的原则：别让同一个工具既写代码、又写「批准这段代码」的测试。 如果 AI 生成的代码和验证它的测试出自同一套假设，那这套验证是循环的。测试要由独立视角来写。
参考资料：
- Review AI-Generated Code Checklist [2026]: Beat the Bottleneck -- Kunal Ganglani ⭐值得阅读
- Review an AI Agent PR and Remove Unneeded Changes -- BetterToken
- How to Inspect AI-Generated Diffs Before Accepting -- AI Tools Guidebook
8. 一个能跑的 AI 代码评审器（A Runnable AI-Code Reviewer）
前面聊的都是「怎么审」，这一节把它落成一段直接能跑的脚本。它用正则 + 规则把「第 4 层安全与幻觉依赖」里最常被 AI 坑的几类 - 硬编码密钥、可疑 API、过度宽松绑定、危险 eval/exec、字符串拼 SQL - 自动扫出来。
这样你拿到一段 AI 代码，先跑这个脚本，把「机器能判的」先挑出来，再用人来做 2、3、5、6 层。只依赖标准库 🐍。
"""AI 代码初筛器：对一段 Python 源码做静态规则扫描，标记常见 AI 生成陷阱。
不替代人工评审，只做「第 1 层 + 第 4 层」的机器预筛：
1. 硬编码密钥 / token / password
2. 可疑或易误用的 API（eval、exec、pickle、dangerous 类）
3. 过度宽泛的输入绑定（mass assignment）
4. 字符串拼接 SQL（注入风险）
5. 危险子进程调用
"""
import re
import sys
from dataclasses import dataclass, field
# 每条规则：正则 + 严重级别 + 说明
@dataclass
class Rule:
"""一条静态检查规则。
属性:
pattern: 用于匹配源码的正则表达式
severity: 严重级别，'critical' 或 'warning'
message: 命中后的人读提示
"""
pattern: re.Pattern
severity: str
message: str
@dataclass
class Finding:
"""一次命中的检查结果。
属性:
line_no: 源码行号
severity: 严重级别
message: 提示信息
matched: 命中的原文片段
"""
line_no: int
severity: str
message: str
matched: str
# AI 生成代码最常见的安全陷阱规则表；pattern 匹配的是「危险写法」的特征
RULES = [
# 1) 硬编码密钥 / token / 密码，如 OPENAI_API_KEY = "sk-..."
Rule(re.compile(r'(api[_-]?key|secret|password|passwd|token|pwd)\s*=\s*["\'][^"\']{6,}["\']', re.I),
'critical', '疑似硬编码密钥/口令，应改为环境变量'),
# 2) 危险动态执行，AI 常把用户输入直接交给 eval/exec
Rule(re.compile(r'\b(eval|exec)\s*\('),
'critical', '危险动态执行：用户输入若进入 eval/exec 可导致任意代码执行'),
# 3) 不安全的反序列化，pickle 可被构造为代码执行
Rule(re.compile(r'\bpickle\.loads?\b'),
'critical', 'pickle 反序列化不安全，可能被构造为 RCE'),
# 4) 字符串拼接 SQL，最经典的注入入口
Rule(re.compile(r'\b(FROM|WHERE|INSERT|UPDATE|DELETE)\s+.*%|\+\s*[a-z_]+\s*\+\s*["\']'),
'critical', '疑似 SQL 字符串拼接，应改用参数化查询'),
# 5) 未校验输入直接进子进程命令，命令注入风险
Rule(re.compile(r'\b(os\.system|os\.popen|subprocess\.(run|call|Popen))\s*\('),
'warning', '子进程调用：确认命令未拼接未经校验的输入'),
# 6) 过度宽泛的输入绑定，把整个 request body 塞进对象
Rule(re.compile(r'\b(update|setattr)\s*\(.*\*\*,?\s*\w+'), # 如 Model(**request.json())
'warning', '疑似 mass assignment：将整块输入直接绑定到对象'),
]
def scan(source: str, filename: str = "<code>") -> list[Finding]:
"""对一段源码逐行执行规则扫描。
参数:
source: 待扫描的源码字符串
filename: 仅用于输出展示的文件名
返回:
扫描命中的 Finding 列表；无命中返回空列表
数据流示例：源码行 "key = 'sk-123'" → 规则1命中 → Finding(line, 'critical', ...)
"""
findings: list[Finding] = [] # 收集所有命中
for line_no, line in enumerate(source.splitlines(), start=1): # 逐行检查
for rule in RULES: # 遍历每条规则
m = rule.pattern.search(line) # 在当前行做正则匹配
if m: # 命中则记录一条结果
findings.append(Finding(line_no, rule.severity,
rule.message, m.group(0).strip()))
return findings
def report(findings: list[Finding]) -> None:
"""打印扫描结果，按严重级别排序。
参数:
findings: scan() 返回的命中列表
"""
if not findings: # 无命中
print("✅ 未发现明显 AI 生成陷阱（仍需人工评审意图与边界）")
return
order = {"critical": 0, "warning": 1} # 严重级别优先级
for f in sorted(findings, key=lambda x: order[x.severity]): # 先 critical 后 warning
tag = "🔴" if f.severity == "critical" else "🟠"
print(f"{tag} L{f.line_no} [{f.severity.upper()}] {f.message}")
print(f" 命中片段: {f.matched}")
# 一段测例：故意包含多种 AI 常见陷阱的伪代码
SAMPLE = '''
OPENAI_API_KEY = "sk-abc123def456" # 危险：硬编码密钥
sql = "SELECT * FROM users WHERE id = " + uid # 危险：SQL 字符串拼接
data = pickle.loads(payload) # 危险：pickle 反序列化
result = eval(user_expr) # 危险：eval 用户表达式
os.system("ping " + host) # 警告：子进程拼接输入
User(**request.json()) # 警告：mass assignment
'''
if __name__ == "__main__": # 直接运行脚本
report(scan(SAMPLE, "sample.py")) # 扫描样例并打印报告
实际输出：
🔴 L2 [CRITICAL] 疑似硬编码密钥/口令，应改为环境变量
命中片段: OPENAI_API_KEY = "sk-abc123def456"
🔴 L3 [CRITICAL] 疑似 SQL 字符串拼接，应改用参数化查询
命中片段: WHERE id = " + uid
🔴 L4 [CRITICAL] pickle 反序列化不安全，可能被构造为 RCE
命中片段: pickle.loads(payload)
🔴 L5 [CRITICAL] 危险动态执行：用户输入若进入 eval/exec 可导致任意代码执行
命中片段: eval(user_expr)
🟠 L6 [WARNING] 子进程调用：确认命令未拼接未经校验的输入
命中片段: os.system("ping " + host)
🟠 L7 [WARNING] 疑似 mass assignment：将整块输入直接绑定到对象
命中片段: User(**request.json())
这个脚本的价值不在于规则多全，而在于它把「第 4 层安全」里原本靠人眼扫的东西，变成了一个可复用的确定性检查。你完全可以把它接进 CI，让它在 review 之前先跑一遍——这就是第 1 层「让工具先当裁判」的落地。
类比 🎭：这个脚本像个安检门，先帮你把「带刀带枪」的拦下来。安检门挡不住「伪装成旅客的罪犯」（对应意图与边界层），但能让你别再花时间在「查刀」上。拿它当第一道过滤，人只处理剩下的高价值判断。
参考资料：
- Bandit: Audit AI-Generated Python for Security Flaws -- CodeCut ⭐值得阅读
- Comparing Human and LLM Generated Code: The Jury is Still Out! -- arXiv
9. 症状 → 原因 → 解法对照表（Symptom Table）
把前面六层压缩成一张能直接对号入座的表。建议按「我最近最像哪一行」来读，而不是从头读到底 📋。每一行都是一个真实的、会在生产里冒出来的坑。
⚠️ 一个必须记住的口径：别把「能编译、能过测试」当成「代码对」的证据。 第 1 层机器闸门只砍掉「明显错」，真正值钱的是后面五层——它们决定这段代码该不该以这个形态存在。
参考资料：
- 2026年AI编码工具实战：六款主流Agent与IDE深度评测 -- 尧图网站定制
- Review AI-Generated Code Checklist [2026]: Beat the Bottleneck -- Kunal Ganglani ⭐值得阅读
什么时候该用它？ 当你拿到一段「看起来都对」的 AI 代码，先用这张表对号入座，把最像你的那几行指出来逐层去查。什么时候不该用？ 当这段代码是你在自己维护多年的仓库里改既有逻辑时，别机械套六层，而要先补足 context——context 不够，任何检查清单都审不出真问题。
本专栏《AI开发范式》聚焦如何让 AI 真正成为开发效率的放大器，而不是返工源头。 从「怎么让 AI 写得更稳」到「怎么写才不让 AI 带偏架构」，每篇都配一段可直接运行的代码，每月更新两
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Full text summary: AI made writing codes cheaper, but costed "certifying codes" more - a patch that allowed AI to write in five minutes, and people to authenticate 45 minutes. This paper gives a list of six tiers of landable inspection, ranging from "Let the tool be the referee" to "Defence test" and "Defence border and security" in the "Defence test" syntax, and translates "what to judge" into enforceable rules with a running script.
Contents
- Let's start with the conclusion: the code is cheap, the verification is expensive.
- Why should the AI code be changed?
- 1st floor: make the tool the referee
- Level 2: Intent, syntax.
- Level 3: trial test and boundary
- Level 4: Security of trial and reliance on hallucinations
- 5th and 6th floor: split and Rollback, should you rewrite it yourself?
- A running AI code reviewer.
- Symptoms, cause, solution comparison.
1. First conclusion (The Verdict)
The conclusion begins with: "Ai generation code, the review is not about "it written right" but "why it looks right."
An AI Patch can be finished in 5 minutes, but it often takes 45 minutes to verify. The cruel calculation of 2026 is that the marginal cost of writing codes has been pushed to near zero, proving that the right cost has become a new tax. So the essence of auditing the AI code is to shift your attention from "read the code" to "validation code".
Review the six-tier list of AI codes, summed up in one sentence: let the machine go through the machine first, and people only look at the "right" mistakes the machine would make.
The six levels are:
Example: AI is like an assistant who talks very quickly and occasionally. It's 5 minutes to give you 10 pages of records that you look efficient and explode, but what really cost you is 45 minutes to cross-check. The AI code is to learn not to read the 10 pages, but to ask "how did he get here?"
References:
Why a Different Review
One error: many people think that the error of the AI code is a "grammatic error" or "uncompilable." On the contrary, the wrong C.I. can stop it together, without looking at it.
The real danger is hallucinating correctness: code can be compiled, pass all tests, and it looks perfectly normal, but it's wrong.
GitHub's official guidelines list such errors as the most alert red flag: the obvious illusion (move non-existent API, quote cross-border variables) will be captured by CI; the dangerous one is more hidden - the off-by-one page, the missing permission check on a branch that has never been tested, and the verification of short circuits under certain border conditions.
References:
Here's how to review them.
- Review AI-generated code -- GitHub Docs
Why does AI look different? Three structural reasons:
1 Model matches, not reasoning. AI generation code is based on a pattern in training data, and it doesn't know that it's wrong. It gives the code with "I'm sure" self-confidence, even if the API is made up of it.
2 Co-source assumptions. One AI wrote both code and test, sharing the same "what's right" hypothesis. Tests are not pushed backwards from demand, but from codes - they are called tautogenical tests. It is adopted because it validates its realization in reverse, not demand.
3 No external benchmarks. The industry’s default response to the AI code quality issue is “another AI response”. But this is structured as a circular argument: since the generator and the assessor start from the same artifact and share the same training distribution, their failure is relevant. The evaluation was to authenticate the code with a code, not with an intent.
Similarity: It's like giving the same person both a test and an answer. The answer is yes, just because it came out of the examination, but the examination itself may be wrong.
The consequences are quantization. CodeRabbit analyzed 470 real GitHub PRs and found that the number of problems caused by the AI generation code was about 1.7 times that of the human code - 75 per cent more logical and correct than human code. These are not toy benchmarks, but measurements from production warehouses.
References:
- The Specification as Quality Gate: Three Hypotheses on AI-Assisted Code Review -- arXiv
- AI broke QA and nobody noticed -- Jia Wei Ng
- Test-First Strategy for AI-Generated Code (2026) -- Alex Cloudstar
So the trial has to change: the machine goes through the machine first, and people only look at the "right" mistakes.
3. First floor: let the Tools Judge First
People are not good compilers. Don't spend 10 minutes looking at grammar, it's a certainty tool that can be done in seconds.
The first level is to let the machine run it first: compile, Lint, type check, test packages already available. The purpose of this is not to "prove the code ", but to cut off all obvious patterns of "incorrectness" and leave attention behind.
For the Python project, level 1 could be:
# Syntax and compiled
# Static lint
# Type check
Pytest-q # Existing Test Package
For the front / Node project, the 1st floor is the lines of CI:
- run: npm ci
- run: npx tsc --noEmit
- run: npx eslint .
- run: npm test -- --coverage
Key rule: The code that is generated should not fit into the first layer. It's not "AI can't" but "it's written by people or "AI" before the same set of CI level." Level 1 has nothing to do with the examiner, it's an automatic gate that separates you from "mechanical error" and "only human error".
But there's a blind zone at the gate: of course it's wrong, you can see it; but it's not wrong, it doesn't mean the code is right. It's "manifestly wrong" to be caught by the compiler and the linger, but it's "right, but wrong." The latter are the battlefields in the back.
References:
Checks to Run Before Trusting AI-Generated Code-KDnugets
- AI-Generated Code Quality: What Breaks, Why, and How to Ship It Safely -- Sarah D. Hashnode
4. Second floor: Intent of the trial, syntax (revision the Intent)
After the first level of the gate, the code has been compiled and tested. But can run the rook solve the right problem.
This level is not about "How does this code work?" I don't know. ** It verifies two things: whether they fit the purpose of the project and are in accordance with your framework agreement.
GitHub's three official self-censorship questions are reproduced directly:
- Did this code solve the right problem? Are there running needs or are they just "looking working"?
- Did it follow our agreement? In the team, Naming, Error Handling, Layers, Catalogs.
- Did it get stuck on the wrong level? AI is very good at putting logic in the "best of luck" rather than in the "should" position - for example, writing rules of business into controller and stuffing access to data into view.
Key premise: You have to give AI enough contact. Prior to generation, README, documents, and the latest PR are given as a starting point to AI, telling it "what sources are credible, which are not available, and which are good examples". This determines whether the 2nd tier is a "spreading of a strange code" or a "one sentence".
The examiner also needs contact. If you take the PR itself to trial, it's as thin as the AI generated. At least you need to get: warehouse structure and dependency, structure constraints and service boundaries, API compatibility with backward expectations, code codes, historical accidents and postmotem, track-off accepted in the last round of reporting. These details determine whether you can judge whether this code should be long or not.
Similarity: The reading of syntax is a misword of the writing, and the intention is to see if the writing is uncharted. It's all about the dictionary (codifier), and only those who know the subject can judge it.
There is a counterintuitive but important point here: Level 2 often requires you to question whether the test is wrong. If AI misreads "demand A" as "demand B," it writes that the test is also for "demand B" - testing is green, but not for you. So the core of the first two levels is to make sure what's right and then to talk about the code.
References:
- Review AI-generated code- GitHub Docs worth reading
- Agentic AI and Code Reviews: Toward a Pattern Language for Code Reviews in the Age of Agentic AI -- Readwise
- How to Build an AI Coding Workflow as a Software Engineer -- Dev Fieldnotes
5. Level 3: Review the Tests, then the Boundary
Read the test, then read it. This is one of the most critical things about auditing the AI code.
Two reasons:
One test will help you judge that "AI understands need is not." If AI misreads the need, its test will also validate the need for the bias. So the first thing to look at is: are these tests really authenticating acceptance criteria, or are they just echoing backsliding? The latter is called synonym testing — it passes only because it proves what it is doing, not what it should do.
2 Boundary is the most common place in AI. AI defaults to go happy path. So you're gonna keep an eye on whether these boundaries are actually covered in the test:
- Empty input, modular, maximum length, zero, negative
- Full / None / undefined, missing field
- Error path: The code does throw/return a specific error when bad input and the test asserts that it was the specific error, not just the "unusual."
Similarity: This is like an insurance clause, which focuses not on "normal settlement" (which certainly makes sense), but on "uncompensated extremes such as earthquakes, floods, suicides." It doesn't make sense for AI to write a normal path, and the border is where it actually rolls over.
Canary signal: If AI's tests are used to prove that they have been achieved, that there are no border examples and that they are too liberal (e.g. to say "no abnormality" and not "return to a specific value"), the tests have little validation value. It's just a "looks right" chapter on the code.
The data prove this. A study of the open source project angent generation test found that AI outpaced humans (variety 0.62 vs. 0.32), in terms of diversity in the use of border examples, but lagged behind in stability - the flaky rate of human testing was 0.30, and AI generated 0.41. In other words, AI would think of more borders, but its tests were more unstable, more narrow and more misleading. This is precisely the difference between the boundaries that AI can think of and the "reliable verification of these borders".
References:
- Reviving AI-Generated Tests: A Code-Review Checklist 2026-QASkills.sh is worth reading.
- AI-Generated Tests Are Testing the AI's Blind Spots, Not Your Code -- Dzaki Amri Zaidaan
- Beyond Test Presence: Assessing the Quality and Robustness of Agent-Generated Tests -- arXiv
6. Level 4: Security of trial and reliance on hallucinations
This floor is inescapable. Because AI generates code without confrontational thinking -- it doesn't think like the assailant, "How am I supposed to get around this?" But the attackers will.
The types to be cleaned during the audit are basically the OWASP checklist:
1 Injection Category (injection). AI particularly likes to spell SQL/ Query with string, and to insert user input into the command:
- SQL Injecting: string fusion in query
- NoSQL Injection: MongoDB Operator from user JSON
- XSS: originally SetInnerHTML, unmodified template, LLM rendering original HTML
- Command injection: unverified input in exec, subprocess
2 Mandate missing. Remember that. It's easy for AI to write "not login," but it's missing "Is this user doing permission?" Especially those branches that never hit.
3 Hard-coding requirements. AI often forgets to stick key, token, password into the environment variable, but writes directly in the source code:
OPENAI API KEY = "sk-abc123def456"
== sync, corrected by elderman == @elder man
4 Excessive input binding (mass attachment). The whole body of the request is tied directly to the object and "the user sends whatever he wants", resulting in overstepping field.
5 hallucinated dependence - the most hidden. AI will cite a library that does not exist or is outdated. Even more dangerous is the fact that a non-existent package name can be registered by the attacker -- you npm intostall made up the name of AI, which is a package that someone else stuffed. So any new dependency is to be examined separately to confirm that it exists in Lockfile, license compliance, and that its sources are credible.
6 For the code generation of angent itself. Don't forget, now there's a lot of code that you let angent write. Agent's contact comes from the warehouse, the network, the connected tool. The attackers can shove hidden commands into issue body, PR description, AGENTS.md, MAMORY.md... This is called indirect injection, an attack on the development of workflow itself.
A: Ai, like the one who put the key under the flowerpot at the door, it's convenient; the attackers are the thieves who went through the flowerpot. AI doesn't think "who's going to turn it over," so it can't find out by itself, and it can only be replaced by the assailant's perspective.
References:
- Secure Coding with AI-- OWASP Cheat Sheet Series worth reading.
- The 15 Security Failures Your AI Coding Assistant Ships by Default -- Prodogon
- OWASP Top 10 for AI-Generated Code: A01-A10 Fixes (2026) -- Vibe Eval
- LLM Prompt Injection Prevention -- OWASP Cheat Sheet Series
7. 5th and 6th floors: split and Rollback, should rewrite (Scope, Rollback & Rewrite)
The first four are "Is this the code itself" and the fifth and sixth are** "Is this code supposed to exist in this form?"**
Floor 5: Review split and Rollback
One of the biggest problems of AI is scope out of control: a five-minute watch that changed 10 files, deleted 3 unrelated functions and added 2 new dependes. So this floor depends on:
- diff whether or not to atomize: a PR to do only one thing, the range of file changes is manageable.
- Is there an accident to delete: git diff | grep '-, search every missing symbol in the warehouse to make sure it's really dead, rather than being incorrectly deleted by AI?
- Whether there is a roll-back point: leave one before changing.
The order of diff reading is also detailed: from high to low reading at risk, not from top to bottom. Scanning (^-) and then reading the new section, and finally looking at the "not related area" that has been modified by AI.
Similar: This is like a delivery. The first thing you opened the package was "Is there anything missing" instead of "Is the gift not pretty?" The PR of AI is the most common "less things" - less border processing, less line permission verification, less functional.
Layer 6: Shouldn't rewrite itself
It's the easiest layer to step on -- the costs of sinking. AI gives you a lot of code that can't get through. You could have said, "Just fix it." But sometimes it's cheaper to delete it than to fix it.
The checklist of Kunal Ganglani gave a clear "Deletion rewrite" ruling and deleted either:
- Diff over 300 rows, and changes are not mechanical
- A little feature added two or more new dependencies.
- Yes, 10 minutes without a clear intention.
- introduced a new abstract, but did not delete the old abstract.
- All green, but you can't use it.
Why should it be deleted? Because "Ai created a big diff that hides a vague intention," you fix it at a much higher cognitive cost than "a tighter prompt regenerated." You're not fixing the code, you're doing a production-level drive for a vague agreement -- it's the least cost-effective thing.
A principle that must be remembered: do not let the same tool write both the code and the "approval" test. If AI generates a code and validates its tests from the same set of assumptions, the validation is circular. Tests are written from an independent perspective.
References:
- Review AI-Generated Code Checklist [2026]: Beat the Bottleneck-Kunal Ganglani worth reading
- Review an AI Agent PR and Remove Unneeded Changes -- BetterToken
- How to Inspect AI-Generated Diffs Before Accepting -- AI Tools Guidebook
8. A running AI code assessor (A Runable AI-Code Reviewer)
It's all about "how," and this section turns it into a straight run script. It uses regular + rules to automatically sweep out the types of AI pits most often - hard coded keys, suspected API, overly loose binding, dangerous eval/exec, stringing SQLs.
So you get an AI code, you run this script, you pick out the "mechanical" first, and you use people to make two, three, five, six. Only rely on the standard library.
"""AI code starter: static scan of a Python source code, marking common AI generation traps.
Instead of replacing the manual review, only a "1st + 4th floor" machine prescreening:
1. Hard code key / token / password
2. Suspected or vulnerable API (eval, exec, pickle, dangerous)
3. Excessive input binding (mass attachment)
String fusion SQL (injection of risk)
5. Hazardous sub-process calls
"""
import re
import sys
from dataclasses import dataclass, field
# Each rule: regular + grave level + description
@dataclass
class Rule:
"""A static check rule.
Properties:
Pattern: Regular expression for matching source code
Severity: Serious, 'critical' or 'warning'
Message: Read the hint after the hit.
"""
pattern: re.Pattern
severity: str
message: str
@dataclass
class Finding:
""The result of a hit.
Properties:
Line no: Source Line Number
Severity: Serious
message: hint message
Matched: Original Story of Life
"""
line_no: int
severity: str
message: str
matched: str
# AI Generates the most common security trap rule sheet; pattern matches the character of "hazardous writing"
RULES = [
#1) Hard code key / token / password, e. g. OPENAI API KEY = "sk-..."
Rule(re.compile(r'(api[_-]?key|secret|password|passwd|token|pwd)\s*=\s*["\'][^"\']{6,}["\']', re.I),
'critical', 'suspected hard-coded key/port, should read environmental variable'),
#2) Hazard dynamic execution, AI often direct user input to eval/exec
Rule(re.compile(r'\b(eval|exec)\s*\('),
'critical', 'hazard dynamic execution: the user enters into eval/exec to cause any code execution',
#3) Unsafe inverse sequence, pickle can be constructed as code execution
Rule(re.compile(r'\bpickle\.loads?\b'),
'critical', 'pickle inverse sequence is unsafe and may be constructed as RCE'),
#4) String fusion SQL, classic injection portal
Rule(re.compile(r'\b(FROM|WHERE|INSERT|UPDATE|DELETE)\s+.*%|\+\s*[a-z_]+\s*\+\s*["\']'),
'critical', 'suspect SQL string spell, should be changed to parameterized query'),
#5) Unvalidated input direct-entry process command, command risk injection
Rule(re.compile(r'\b(os\.system|os\.popen|subprocess\.(run|call|Popen))\s*\('),
'warning ', 'the sub-process calls: confirm that the commands have not been spelled for unverified input'),
#6 Over-broad input binding and stuff the whole body into the object
Rule (re.compile(r'b\setattr)\s*(.**, ?s*w+'), #like Model(**request.json())
'warning', 'suspect mass attachment: input the whole piece directly to the object'),
]
def scan(source: str, filename: str = "<code>") -> list[Finding]:
""Scanning the rules on a line-by-line line.
Parameters:
Source string to scan
filename: Only for the output of files displayed First Name
Return:
Scaning list of Findings;back empty list of hits
Example of data stream: source line "key = 'sk-123'" → Rule 1 hits Finding
"""
# Collect all hits
# Check line by line
# Through every rule
m = rule.pattern.search(line) # Matches in the current line
If m:
findings.append(Finding(line_no, rule.severity,
rule.message, m.group(0).strip()))
return findings
def report(findings: list[Finding]) -> None:
""Scanning results, sorted by severity.
Parameters:
Findings: scan() returned hit list
"""
If not findings: Medium
"Print"
return
# Serious priority
For f in sorted (findings, key=lambda x: order[x.severity]:
tag = "🔴" if f.severity == "critical" else "🟠"
print(f"{tag} L{f.line_no} [{f.severity.upper()}] {f.message}")
Print(f) hit clip: {f.metched})
# A test: false codes that deliberately contain multiple AI common traps
SAMPLE = '''
OPENAI API KEY = "sk-abc123def456"
sql = "SELECT * FROM users id = " + uid # Danger: SQL string spell
# Danger: pickle inverse
# Danger: eval user expression
os.system("Ping "+host") # Warning: subprocess adhesive input
# Warning: mass attack
'''
If name = " main ": # Directly run script
# Scan sample and print report
Actual output:
L2 [CRITIC] Suspected hard-coding key/port, should read environment variable
Snippet: OPENAI API KEY = "sk-abc123def456"
🔴L3 [CRITIC] Suspected SQL string spell, should be changed to parameterized query
Snippet: WERE id = " + uid
🔴L4 [CRITICAL] pickle inverse sequence is unsafe and may be constructed as RCE
Snippet: pickle.loads (payload)
🔴L5 [CRITICAL] Hazard Dynamic Execution: User input into eval/exec can cause any code execution
Snippet: eval (user expr)
🟠L6 [WARNING] Sub-process call: confirm that the command has not been spelled for unverified input
Snippet: os.system
🟠L7 [WarnING] suspect mass attachment: input the whole piece directly to the object
Snippet: User(**request.json())
The value of this script is not so much that it is full of rules, but that it turns what's supposed to be covered by human eyes in Level 4 security into a reusable certainty check. You can connect it to the CI and let it run before the review -- this is the landing of the 1st floor, "Let the tool be the referee."
Similarity: This script is like a security door to stop you with a knife and a gun. The security gate can't stop "criminals disguised as travellers" (responsible intent and border level), but it can stop spending time on "checking knives." Take it as the first filter, and people only deal with the remaining high-value judgements.
References:
- Bandit: Adit AI-Generated Python for Security Laws -- CodeCut
- Comparing Human and LLM Generated Code: The Jury is Still Out! -- arXiv
Symptom Table
Compress the first six floors into a table with direct pairing. It's suggested to read "what I've been most like lately" instead of reading from the beginning. Each line is a real pit that comes out of production.
⚠️ A caliber that must be remembered: don't use "compile, pass the test" as evidence of "code-correction". The first layer of machine gates only cut off "manifestly wrong" and the real value is the back five -- they decide whether or not the code should exist in this form.
References:
- 2026 AI coding tool: six mainstream Agent and IDE in-depth assessments - customizing the Yautu website
- Review AI-Generated Code Checklist [2026]: Beat the Bottleneck-Kunal Ganglani worth reading
When are you going to use it? When you get an AI code that's all right, you're going to have to use this table for a pair of numbers, and you're going to point out the lines that are most like you. When should it not be used? When you change the logic in a warehouse that you've been maintaining for years, don't mechanize six layers, but fill up the context-context first is not enough, and no inspection list can be considered true.
This column, "AI Development Model ", focuses on how AI can really be an enabler for development efficiency, not back to work. From "How to make AI more stable" to "how to write to not allow AI to be partial", each piece is equipped with a direct running code, updated twice a month.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.