AI 模型和平台
Google 研究带来人类对主观AI分类法的审议
在 Google 上的首选来源中添加 Unite.AI
Google研究于2026年11月4日推出Agile De Release，这是一个围绕主观类别构建图像分类器的互动框架，并附有开源代码和一个笔记本演示。 在一项为期18届会议的用户研究中，研究人员报告说，分类结果比手工快速改进更强，努力也更低。
框架涉及的范畴取决于个人判断、文化背景或操作政策：健康食品、激励艺术作品以及能带来误导性回报的点击图像。 其中心前提是用户往往在研究困难的例子时发现自己的决定界限。 常规的标签工作流程将这些界限视为在注释开始前的固定界限，使意图的转变成为不一致的标签或任意的示范决定。
Agile De Releasing将演变中的定义转化为人可读的规格和驱动视觉语言模型分类器的提示。 该设计借鉴了与5名内容编辑专家的访谈以及对20项定义的定性分析，这些定义取材于其专业工作流程，其中显示专家先对概念进行范围界定，然后通过边缘图像加以完善。
概念审议的两个阶段
概念范围界定将一个初步想法分为候选的正负子概念。 例如，建立健康食品分类器的用户可以包括沙拉和谷物碗，同时排除加工食品或未经准备的原料。 代表性图像支持每个选择，没有描述有意义的视觉模式的子概念可以在创建第一个分类器之前被丢弃。
概念迭代然后搜索成像的边缘图像，而不是简单地选择模型信心接近0.5的个案。 研究者认为，视觉语言模型的置信度可能无法很好地校准到人类的模糊度：一个模型可能对一个人发现困难的图像充满信心。 检索到的例子被复制和分组，有一个多臂带规则，有利于先前纠正错误或产生有用反馈的集群。
一个模糊性挖掘步骤围绕一个可解释的问题，如奶油酱、糖含量或部分尺寸，组织一批人，并产生对每个图像模糊性的简短解释。 用户在范围内外标注图像，可以添加自然语言原理。 系统生成替代定义，对照迄今为止收集的所有标签进行测试，选择最佳F1分数的候选人，并允许手动编辑。
原型机使用Gemini 2.5 主张分解和Gemini 2.5 Flash进行分类，边线-query生成，以及模糊性开采。 这些模型的使用没有经过额外的训练或微调，界面通过交互式Colab笔记本运行。
现场研究发现了什么
九名没有专业内容调试经验的学员每人按随机系统顺序完成了两场约90分钟的会议，一场是Agile De Reculation，另一场是人工审议。 这两个测试概念是健康食品和付费-游戏点击节制。 每期培训班包括大约45分钟的定义工作、200张搁置测试图像的说明、一次调查和一次15分钟的访谈。 比较还包括零镜头分类和模拟合作者，这是自动分解基线；手工参与者可以查阅该系统的详细提示。
文件报告说，Agile De Leability将F1改进了11%，比其零射线对付基准提高11%，而人工审议则比其自身基线提高8%。 就健康食品而言，变化是Agile De Recovery改进了10%，人工审议减少了3%。 这些收益主要反映了精确度的提高，以及召回量的减少。 没有使用准确性，因为参与者的个性化定义产生了不平衡的测试标签分布。
所有与会者都倾向于采用人工审议方式进行积极审议。 与会者报告说，对Agile De control 和4.67进行体力工作的平均努力为3.11；对感觉不安全、紧张或烦恼的评分分别为1.67和3.00。 这两种差异在统计学上都是显著的，见第 < 05页。 访谈提供了实用的解释：人工用户必须发明搜索反例，有些人担心迅速编辑修复一个错误会损害其他地方的行为。
学习限制和守则的提供
评价也有严格的限度。 作者告诫说，9名参与者限制统计能力和通用性，每个人使用两种不同概念的系统以避免在第二届会议上提出既定定义。 因此，可比较参与者的主观经验，但只将参与者团体的业绩分类。 这项研究没有孤立个别组成部分，也没有评价跨越其他基因模型的整个管道。
开源寄存器包含配套的代码库和数据集外加一个用于Jupyter或Google Colab的一步步Demo.ipynb笔记本。 它的演示在LAION400M上构建了近邻检索指数。 默认的设置下载有两个硬盘，总共约260 MB来验证管道；文献记载的100硬盘的设置总计约为13 GB. 检索，外部图像下载，以及模型调用需要几秒钟到几分钟，单个API密钥可能受速率限制，多个阶段支持磁盘缓存。 这并非官方支持的Google产品。
积极审议的实际贡献是使主观意图成为分类发展的一个明确、可修改的部分。 用户可以检查正反规则，检查暴露漏洞的边缘案例，提供修改的理由。 温和与修养团队将类别定义和人文判断塑造为分类者可检查的发展进程的一部分。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
AI Models & Platforms
Google Research Brings Human Deliberation to Subjective AI Classifiers
Add Unite.AI to your preferred sources on Google
Google Research on November 4, 2026 presented Agile Deliberation, an interactive framework for building image classifiers around subjective categories, accompanied by open-source code and a notebook demo. In an 18-session user study, the researchers reported stronger classifier results and lower effort than manual prompt refinement.
The framework addresses categories whose boundaries depend on personal judgment, cultural context, or operating policy: healthy food, inspiring artwork, and clickbait images that promise misleading rewards. Its central premise is that users often discover their own decision boundaries while examining difficult examples. A conventional labeling workflow treats those boundaries as fixed before annotation begins, leaving shifts in intent to surface as inconsistent labels or arbitrary model decisions.
Agile Deliberation turns the evolving definition into both a human-readable specification and the prompt that drives a vision-language model classifier. The design draws on interviews with five content-moderation experts and qualitative analysis of 20 definitions taken from their professional workflows, which showed experts scoping concepts first and then refining them through borderline images.
Two Stages of Concept Deliberation
Concept scoping divides an initial idea into candidate positive and negative subconcepts. A user building a healthy-food classifier, for example, can include salads and grain bowls while excluding processed foods or raw, unprepared ingredients. Representative images support each choice, and subconcepts that do not describe meaningful visual patterns can be discarded before the first classifier is created.
Concept iteration then searches for semantically borderline images instead of simply selecting cases where the model’s confidence approaches 0.5. The researchers argue that vision-language model confidence can be poorly calibrated to human ambiguity: a model may be confident about an image that a person finds difficult. Retrieved examples are deduplicated and clustered, with a multi-armed-bandit rule favoring clusters that previously corrected errors or produced useful feedback.
An ambiguity-mining step organizes a batch around one interpretable issue, such as creamy sauces, sugar content, or portion size, and generates a short explanation of each image’s ambiguity. Users mark images in or out of scope and can add natural-language rationales. The system generates alternative definitions, tests them against all labels collected so far, selects the candidate with the best F1 score, and allows manual edits.
The prototype uses Gemini 2.5 Pro for decomposition and Gemini 2.5 Flash for classification, borderline-query generation, and ambiguity mining. Those models were used without additional training or fine-tuning, and the interface runs through an interactive Colab notebook.
What the Live Study Found
Nine participants without professional content-moderation experience each completed two approximately 90-minute sessions, one with Agile Deliberation and one with manual deliberation, in randomized system order. The two test concepts were healthy food and pay-to-play clickbait moderation. Each session included about 45 minutes of definition work, annotation of 200 held-out test images, a survey, and a 15-minute interview. The comparisons also covered zero-shot classification and Modeling Collaborator, an automated decomposition baseline; manual participants could consult that system’s detailed prompts.
The paper reports that Agile Deliberation improved F1 by 11% over its zero-shot baseline for pay-to-play, while manual deliberation gained 8% over its own baseline. For healthy food, the changes were a 10% improvement for Agile Deliberation and a 3% decline for manual deliberation. The gains mainly reflected improved precision, accompanied by smaller reductions in recall. Accuracy was not used because participants’ personalized definitions produced imbalanced test-label distributions.
All participants preferred Agile Deliberation to manual deliberation. Participants reported mean effort of 3.11 with Agile Deliberation versus 4.67 with manual work; mean ratings for feeling insecure, stressed, or annoyed were 1.67 and 3.00, respectively. Both differences were marked statistically significant at p<.05. The interviews supplied a practical explanation: manual users had to invent searches for counterexamples, and some worried that a prompt edit fixing one error would damage behavior elsewhere.
Study Limits and Code Availability
The evaluation also has firm limits. The authors caution that nine participants restrict statistical power and generalizability, and each person used the two systems on different concepts to avoid carrying an established definition into the second session. Subjective experience could therefore be compared within participants, but classifier performance only between participant groups. The study did not isolate individual components or evaluate the full pipeline across other generative models.
The open-source repository contains the companion codebase and dataset plus a step-by-step Demo.ipynb notebook for Jupyter or Google Colab. Its demo builds a nearest-neighbor retrieval index over LAION400M. The default setup downloads two shards totaling roughly 260 MB to verify the pipeline; the documented 100-shard setup totals about 13 GB. Retrieval, external image downloads, and model calls can take seconds to minutes, a single API key may be rate-limited, and several stages support disk caching. The repository’s disclaimer says this is not an officially supported Google product.
Agile Deliberation’s practical contribution is to make subjective intent an explicit, revisable part of classifier development. Users can inspect positive and negative rules, examine the edge cases that exposed gaps, and supply rationales for revisions. For moderation and curation teams, that makes the category definition and the human judgments shaping it part of the classifier’s inspectable development process.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.