翻译｜林绮蓓
编辑｜蔡芳芳
10 月 13 日，Anthropic 工程经理、Claude Cowork 工程负责人 Felix Rieseberg 发表了一篇博文，建议大家在使用 AI 时“更有野心”。他分享这篇文章的 X 帖子，已获得超过 500 万次浏览。在他看来，这并不只是让 AI 把事情做得更快，还需要重新想一想：眼前的工作最终要达成什么目标？能不能把更多事情交给 AI，让它自己寻找方法？
Felix 自己就经历过这样的转变。以前，他会逐一查看程序崩溃记录，挑出问题，再交给 Claude 修复。后来，他意识到自己不过是在给 Claude 当“传送带”：既然 Claude 能修复问题，为什么不让它直接读取崩溃数据，自己判断哪些需要修复？
这种反思，也影响了他对 Agent 工作流的看法。
近日，Felix 做客 Silicon Valley Girl，与主持人 Marina Mogilko 深入讨论了如何更充分地发挥 AI 的能力。他所在的团队曾用大约一周半开发出 Claude Cowork，而如今，他已不再亲自写代码，更多精力放在产品原型、用户需求和团队管理上。Felix 认为，我们可能花了太多精力告诉 AI 应该怎样工作，却没有给它足够的工具、数据和自主判断空间。在这次访谈中，Felix 提到了不少具体的做法。
当 AI 能够承担越来越多的工作，我们又该如何判断它做得对不对？尤其是在自己并不熟悉的领域，怎样避免被它一本正经的错误答案误导？Felix 分享了一个办法：“对抗式审查”——让多个 Claude 分别挑错，再由另一个 Claude 判断这些意见是否成立。
此外，他还建议，不必再刻意告诉 AI“你是某个领域的专家”，与其让模型扮演专家，不如直接为它提供专业工具和数据。在他看来，这种角色扮演式的提示词已经是 2024 年的做法，但如今的大模型已经是通才，刻意设定专家角色反而可能限制其能力的发挥。
访谈中，两人还聊到了软件创业和工程师的未来。当过去需要团队花几个月开发的软件，一个人也能在短时间内做出来，创业者的优势在哪里？对于刚进入行业的年轻工程师，哪些能力又会变得更加重要？Felix 结合自己的产品和招聘经验，给出了他的看法。以下是访谈完整译文。
太长不看版：
主持人 Marina： 不再亲自写代码后，你每天主要做什么？
Felix：智能体会先简要汇报当天的工作。我现在兼顾产品、工程、团队管理和数据分析，大部分时间用来尝试产品想法，让 Claude 做出不同版本，再评估哪些值得继续。
Marina：你怎么判断一个想法好不好？主要看哪些信号？
Felix：过去要先权衡开发和维护成本，现在可以让 AI 快速做出原型，试用后再判断。我看重的是，用户能否很快理解产品的用处，并借助它完成以前做不到的事。功能不是越多越好，好产品往往反而需要更少的按钮。
Marina：你会怎么判断，哪些任务交给 AI，哪些由人完成？哪些适合人机配合？
Felix：我会区分自动化和能力增强：前者帮助人节省时间和成本，后者让人做到原本因缺少技能或知识而做不到的事。我更关注后者。用好 AI，需要重新考虑工作流程，明确目标和评价标准，而不只是把旧任务交给新工具。
Marina：写作等创意工作很难描述和评价，该怎么告诉 AI 什么才算好？
Felix：不仅要看结果，也要看过程是否可靠，AI 碰巧猜对的不算做好。创意工作确实更难评价，但优秀创作者也有长期打磨的方法。与其笼统地要求 Claude“写好一点”，不如让它按照专业人士的步骤工作，让 AI 使用同样的方法和工具。这样往往效果更好。
Marina：面对自己不熟悉的领域，怎样避免被 AI 自信的错误回答误导？
Felix：可以用对抗式审查：一个 Claude 完成任务，另外几个对话专门找错，找错有时比创造更容易，所以此时可以用能力稍弱的模型。再由另一个 Claude 核查这些批评，把成立的意见交回去判断是否修改。如果自己能判断结果好不好，就没必要弄懂 AI 的每个步骤。
Marina：哪类知识工作，你已经能完全放心交给智能体？需要专门训练对应工作的智能体吗？
Felix：就我的使用经验，事实核查的工作已经可以完全信任 Claude。我会要求它为每个数字找到可靠的一手来源，这通常只需一个普通对话就够了。模型本身就是通才，不必为每项任务设定专家角色，过多的角色设定反而可能限制我们的用法。
Marina：你现在还用 Skills 吗？
Felix：经常用，但会谨慎选择。模型更新很快，为上一代模型写的 Skill 可能已经过时，有时删掉效果反而更好。也要考虑模型需要什么数据和工具。以前我逐个把崩溃问题交给 Claude 修复，后来发现自己只是在当“传送带”，于是让它直接查看全部崩溃数据，自行判断哪些需要修。
Marina：所有内容最后还是要我审核，怎样才能让 Claude 学会我是如何判断的？
Felix：先让它了解你过去给过的反馈，再和你分别审查同一份材料，对比意见有哪些异同，逐步调整流程，不必立刻完全放手。团队协作可以使用 Slack 里的 Claude，让多人共同参与。
Marina：软件越来越容易做，创业者的护城河在哪里？
Felix：开发成本降低了，但做好产品的关键没变：理解用户，了解使用场景，并做出让人愿意付费的产品。最初的点子可能会变，创始人理解需求、做出产品的能力更重要。
Marina：个人怎样才能更充分地发挥模型的能力？
Felix： 先想清楚眼前的工作最终要实现什么，再试着把更大的目标交给模型，不断探索它的能力边界。当然，这不等于直接让它“去赚钱”。但如果始终只用它总结邮件、调整语气，就很难真正发挥模型的作用。
做出一千个原型之后，什么才是好产品？
Marina：Felix，非常感谢你来参加节目，很高兴今天能和你交流。我是 Claude 的重度用户，频道的各项运营工作都在用它。团队成员也都在 Claude 里建立了各自的项目，处理自己负责的工作。对内容创作者来说，我的团队规模算比较大，有 35 个人，我希望大家提高效率，产出更多内容。今年，我们大概新开了三个社交媒体渠道，还推出了一份邮件通讯。一年半前，我们也开始运营 LinkedIn，这个平台以文字内容为主，AI 帮了不少忙。所以，我希望通过今天的交流，让自己、观众和听众都能更好地使用 AI。
Felix：据我了解，你已经用得很熟练了。
Marina：但总还有提升的空间，对吧？
Felix：确实，我自己每天也有新的体会。开发 Claude 最有意思的一点就是，你很难预料用户会拿它做什么。这和我以前参与开发的产品很不一样。比如 Slack，我大致知道人们会怎么用。但用户用 Claude 做出的事情，经常超出我的想象，有些甚至是我原本认为不可能做到的。作为产品开发者，看到用户用自己做的产品取得这样的成果，会特别有成就感。
Marina：有没有什么特别大胆的操作，让你看完也想试试看？
Felix：倒不一定是想照着做。不过，两天前，一位叫 Pavo 的用户让我印象深刻。我记得他是一名产品经理，他把大量 NASA 数据交给 Claude，让它寻找一颗新行星，结果还真找到了。据说，NASA 目前正在核实，还没有确认。我既不是天体物理学家，对这件事的了解也不够充分，所以也不敢说得太肯定。但看起来，Claude 可能找到了一颗此前从未被发现的行星。
Marina：这太有意思了。
Felix：我以前完全没想过，还能这样用 Claude。但仔细想想，对我们这些从事知识工作的人来说，让 AI 分析大量数据，从中寻找有价值的信息，其实是很常见的做法，我们每天都在做。只是我没想到，也可以让它分析公开的太空数据，找找有没有什么是我们此前没有注意到的。这个想法真的很美妙。
Marina：能不能说说你现在一天的工作是怎么样的？你参与开发了 Claude Cowork，却说自己已经不再亲自写代码了。那你早上开始工作时，通常会做些什么？智能体会先向你汇报当天的安排吗？
Felix：是的，它们确实会向我简要汇报当天有哪些事情。我现在看问题的角度也变了，比几年前做传统工程经理时，更关注人的情绪，更愿意花时间理解别人的感受，以及他们做事的方式。同时，我发现自己越来越少需要找其他专业人士帮忙，产品经理、工程经理、团队管理者和数据科学家的工作，现在我都能承担一部分。
至于每天具体做什么，我大部分时间都在做原型。我们会尝试很多想法，先琢磨几天，看看是不是个好主意，再让 Claude 把各种方案都做出来。有了原型，我们就可以逐一评估，判断哪些想法好、哪些不好。所以，做原型确实占了我每天很大一部分时间。
Marina：那你怎么判断一个想法好不好？主要看哪些信号？
Felix：这一直是个难题，我也没有一个全面的答案。但现在，判断一个想法值不值得做考虑的因素确实和以前不同了。过去十到十五年的产品开发，尤其是软件开发，很多决定的背后其实都是成本上的权衡：产品能不能做出来？后续维护和持续投入要花多少钱？需要多少时间？以及选择做这件事，会不会错过其他机会？工程经理得把这些都考虑清楚，才能决定要不要做。现在，我们可以更快地看到实际效果，直接让 AI 先做一个能试用的版本，亲自体验一下看看实际效果如何，再作判断。
但人的注意力仍然有限。比如，你对 AI 产品很有判断力，我一天也许只能请你看一个。我能轻松做出一千个，却没办法让你看完一千个。就算你愿意花一整天陪我看，到后来也会看不进去，觉得：“好了，我不想再看了。”所以，什么才算好产品？我觉得有句老话在 AI 时代尤其适用：完美，是没有什么可以再删去，而不是把所有功能都加进去。我们可以做出你想要的任何按钮，但我认为，好产品的按钮反而更少。
判断一个产品有没有用，我通常会把它交给几个人试试。如果他们很快就明白了它的用处，还能用它完成以前做不到的事，我就知道这个产品做对了。这才是我真正看重的信号。
Marina：那你会怎样判断，自己真正要解决的是什么问题？比如，你现在来到我的公司，会怎么决定哪些任务完全交给 AI，哪些留给人来做，哪些只让 AI 处理其中一部分？
Felix：首先，我会把自动化和能力增强分开看。有些任务占用了你的时间，让你没法去做自己喜欢的工作；还有些任务成本很高，自动化之后能省下很多钱或时间。在这些情况下，自动化确实很有帮助。我觉得，AI 目前创造的经济价值，大部分可能都来自这里。我们判断用 AI 值不值得，通常也是看自动化能带来多少收益。
不过，我更感兴趣的是能力增强，也就是让人们借助 AI，做到以前做不到的事情。你原本可能缺少某种技能、方法，或者不了解相关知识，无法独立完成，现在 AI 能帮你做到了。我去一家公司时，最让我印象深刻的，往往不是他们用了什么工具、工具能做什么，而是他们怎样调整了自己的工作方式。
AI 是一种很特别的产品。要真正用好它，你得重新想想自己是怎么工作的：最终想得到什么结果？怎样才算做好？拿到结果之后又该怎么判断？那些让我印象最深的人，通常都很善于反思自己的工作流程。
Marina：但这很难。拿我的工作来说，很多人用 AI 起草文章，可什么才算一篇好文章？知识工作也能有明确的评价标准吗？写代码也许比较容易判断，你知道自己想实现什么，也知道代码不能有漏洞。但对于知识工作，尤其是需要创造力的工作，你怎么告诉智能体，什么才算好？
Felix：确实很难。我经常举一个例子：假设你的工作是播报天气，你让智能体查一下外面的温度。这里就得想清楚，我们究竟该怎样评价它的表现。如果模型只是猜了一个温度，恰好猜对了，我们不应该让它过关，反而应该认为这种做法有问题。相反，如果它给出的温度差了半度，但找对了地点，查了可靠的数据来源，整个过程有理有据，那也应该比碰巧猜对更好。
但写作，以及其他很依赖审美判断的工作，确实更难评价。要证明一份成果比另一份更好并不容易。所以，我很喜欢和那些写得好、同时又愿意使用 AI 的作者交流。
Marina：他们会怎么做？
Felix：看他们工作很有意思。我的朋友 Adam Davidson 曾在几家很不错的媒体担任专职作者。他会用各种提示词让 Claude 帮他推敲文章，比如：“把我的论点放大一百倍。”“试着改动一下这篇文章。”“哪些地方你最不认同？”“最有力的反对意见是什么？”我不是职业作者，最容易想到的做法，就是把文章交给 Claude，说一句“帮我改好”。但我越来越发现，做知识工作，不能只是要求 AI 给出一个好结果，还要先想一步：这个领域最优秀的人是怎么工作的？能不能让 AI 完成其中的那些步骤？
Marina：这样可以模仿他们的工作流程，但还是缺少审美判断，所以我觉得仍然需要很多……
Felix：对，我同意。不过，有意思的是，真正优秀的作者，大多不认为自己写得好，只是因为有什么神奇的天赋。他们往往会说：“不是的，这是下了很多功夫才写出来的。”
大多数创作者都有自己的一套方法。如果你问他们：“你的作品好，是因为天生就擅长，自然而然就能做出来，还是因为你有一套经过多年打磨的方法？”前一种说法会让他们觉得受到冒犯。但和创作者聊得越多，我就越发现，他们很为自己的方法感到自豪。创作是很辛苦的，要把作品做好，有很多具体的步骤需要逐一完成，并不是坐下来，一本好书就突然出现了。你去作者的工作室看看，会看到很多白板，上面写满了资料和想法。
我在不同领域都发现，与其只告诉 Claude 想要什么样的结果，不如让它按照专业人士的步骤来做，效果往往很好。这样说可能有点抽象，我举个具体的例子。最近，大家应该看到了 AI 在动态视觉、视频和 3D 方面的进步。Opus 5.5 的能力确实提升了很多，但用户能做出更好的作品，还有一个重要原因，就是开始让 Claude 使用 Blender，直接告诉它：“请使用专业动画师用的工具。”结果就相当不错。这里真正起作用的，是创作所用的媒介、方法和工具，而不完全是模型本身理解了什么才是一部好的动画电影。
别让 Claude 给自己的工作打分
Marina：你刚才说，可以让 AI 按照专业人士的方式工作。可即使它会上网查资料，很多信息也没有公开，你怎么知道它给出的结果可信？比如，我想再尝试一种内容形式，或者像刚才说的那样寻找新星体，但我对此一无所知，只告诉它“像 NASA 的研究人员一样工作”。怎样才能避免它把错误答案说得很笃定，让我信以为真？
Felix：这是个很好的问题，很多人都会遇到这种情况。我有一个简单、实用的办法，就是对抗式审查。在编程、数学，以及其他能够用数字衡量效果的领域，这种方法已经被证明相当有效。做法是，先让一个 Claude 完成任务，再启动另外十个，专门来找它的问题。它们的任务就是提出质疑，证明这个结果为什么不对，基本思路和红队测试、做空类似。
Marina：你会用同一个模型来审查，还是换一个模型？
Felix：要看具体任务。不过，找出问题有时候比创造新东西更容易，人也是一样。让我评价一部新手机好不好，远比让我从头设计一部手机容易。所以，审查时通常可以用能力稍弱一些的模型，也能得到很好的效果。这是一个很实际的建议。
Marina：那提示词具体怎么写？如果已经有一套工作流程，需要额外加上什么要求？
Felix：假设你已经有了一份演示文稿或文档。就拿我今天早上写的那篇博客来说，我用的是 Claude Projects，这是我们的一个新产品，可以更方便地同时开展多个线程的工作。我会直接告诉 Claude：“启动五个线程，让每个线程都去证明这篇文章有问题：可能是我的思路不对，可能是事实有误，也可能是我理解错了什么。”它们带着这个明确的目标去审查，就是要找出我哪里错了。
Marina：那收到这么多意见，你怎么处理？我准备采访时也会这样做，只不过会用不同的模型，让它们看看我是不是在某个问题上钻得太深了。但最后收到的反馈各不相同，这个模型这么说，另一个又有别的意见，还有一个说的完全不同，结果还是得由我来决定。你怎么判断哪些意见值得采纳？
Felix：你应该会喜欢这个答案，也大概能猜到我要说什么了：还是交给 Claude。写代码时，我们会设置一个“裁判”。先让那些负责挑错的智能体列出主要意见，再交给另一个 Claude，让它逐一核查，判断这些意见是否成立。
接下来，根据你的工作流程，可以把有效的意见交回最初完成任务的 Claude，让它决定要不要修改。它也能判断，不必每条建议都接受。我们写代码时经常遇到，我自己做知识工作时也有同样的体会：有些意见听起来合理，却忽略了某个关键点，或者实际上没法据此修改，也没有多大帮助。到了这一步，我通常就知道，原来的成果已经相当不错了。如果你自己拿不准做得好不好，又不容易核实，就可以用这种办法。
不过，我用 AI 做的很多东西，其实不需要自己完全了解制作过程。对于每天使用的东西，我们本来就有自己的判断。比如，我重新做了自己的网站，里面用了很多 3D 素材。我从来没用过 Blender，完全不知道怎么操作，但我能判断这些东西看起来好不好、整体感觉对不对，这可能就够了。你可以把自己当作客户，不必弄懂每一步是怎么做的。生活中也常有这样的情况：无论是请人施工、找律师，还是看医生，我们未必有专业知识去判断对方的工作做得好不好，更多时候只能根据自己的感受提出反馈。
所以，如果你不知道怎样判断结果好不好，就多花一些 Token，让 AI 帮你做更多审查。如果你自己就能判断，那可能已经够了，不一定非要弄懂它是怎么做出来的。
Marina：我觉得，还是要挑出少数最重要的项目，去了解每一步是怎么做的，而不是所有事情都想控制。
Felix：对，还是那个老问题：能不能放手，不再事事过问。
Marina：确实很难，现在用智能体也会遇到这个问题。有没有哪类知识工作，你已经能很放心地交给它们，完全信任它们做出的结果？
Felix：事实核查，比如编辑工作中的事实核查，我现在完全信任它的结果。我会让 Claude 核实演示文稿里提到的所有事实，也不用反复检查很多轮，更不需要找五十个智能体，一个就够了。我会告诉它：“请为我提到的每个数字找到一手来源，并确认来源可靠。”这项工作，我现在可以完全放心交给它。
从替 Claude 搬运问题，到让它自己找答案
Marina：你说的这个智能体，是专门训练来做事实核查的，还是平时处理各种任务时用的那个？
Felix：就是开一个 Claude 对话。我接下来这个看法可能有点争议，你应该会喜欢：我觉得，我们有点过于强调“智能体”了。模型本身就是很出色的通才，这一点已经得到过很多次验证。比如，行业刚开始认真研究模型的数学能力时，我记得 Google DeepMind 是最早投入的公司。他们训练了专用模型，搭建了配套的执行框架，让它参加数学奥林匹克竞赛，拿到了银牌，在当时已经是最好的成绩。结果四五个月后，几家主要的通用模型也拿到了金牌。
我能理解大家为什么要给智能体设定角色。告诉它“你是一名数据科学家”，这样我们就更容易理解它能做什么。但被我们称为“数据科学家”的这个模型，其实也有能力处理顶尖生物学家、数学家等其他专业人士面对的问题。如果每次使用 AI，都要先给它安排好一套角色，有时反而会限制我们对它的使用。所以，我刚才提到的很多工作，都没有专门给智能体设定身份。
Marina：但给智能体提供什么数据，难道不重要吗？我们通常会给每个智能体上传特定的资料。还是说，你的 Claude 能调用所有工具，所以不需要再分别准备？可这样一来，它每回答一个问题，需要考虑的数据会不会太多了？
Felix：数据确实很多，但它已经很会判断哪些是自己需要的了。我不太赞成先把数据分成几份，再给每个智能体分配一份。我们自己工作也是一样，要把事情做好，往往需要了解整个业务，只看到一小部分信息是不够的。比如，我是工程经理，如果只能看代码，能发挥的作用就比较有限；但如果还能看到客户数据、了解客户沟通的情况，就能做得更好。有时销售团队和客户开会，我也想参加，听听他们怎么说。我还希望了解详细的产品使用情况，最好能看到业务数据，知道哪些做法有效、哪些没用。智能体也应该能获取这些信息。
但这不意味着要把所有数据一次性塞给它。给它工具，再把每个工具能查到什么数据说明白就行。比如，告诉它：“如果需要业务数据，可以用这个工具查询。”这样，它就能在需要时自己去查，不必先把所有内容读一遍。
Marina：那具体该怎么设置？听你这么说，我在想，是不是应该调整现在的做法，不再为每个项目单独准备一套资料。比如，我们有一个评估嘉宾的项目，里面放了完整的流程说明和以前各期节目的脚本，用来了解不同嘉宾的表现。我是不是可以直接告诉 Claude：“这个文件夹里有全部脚本，Notion 里有每位嘉宾的表现资料，这里是我们的评分方法。”把这些准备好，以后每次要评估嘉宾，就直接开一个新对话，不用再交代一遍背景，只说“帮我评估这位嘉宾”，它就能去做？
Felix：我不太愿意告诉别人，工作流程一定要怎么安排。我一直觉得，只要现在的方法对你有效，就没必要改。
Marina：可模型在进步，我也想跟着进步。这套方法已经用了一年，虽然一直有效，但一年已经很长了。
Felix：明白。如果你想知道我的建议，或者我自己是怎么做的，我还是会先说，每个人都应该选适合自己、用着顺手，也确实能帮上忙的方法。不过，模型现在确实更擅长调用工具了，知道什么时候该用、该怎么用，也不容易陷入反复调用的循环。它不会一发现有个工具能查数据，就想着：“那我一直查，直到把所有数据都查完。”现在，它们已经很会合理地使用工具，既能把事情做好，也不会浪费太多功夫。
所以，我现在不太会在开始前就替模型决定，这项任务需要哪些数据。这样一来，同时开展多个对话、处理多项工作也方便得多，整个过程简单了很多。
Marina：你还用 Skills 吗？
Felix：用，我们现在用得还是挺多的。不过，Anthropic 的官方建议和我个人的看法一致：使用 Skills 要谨慎。我们给模型写的指令，很多其实是针对上一代模型的。AI 更新得太快，一套原本很好用的 Skill，可能很快就不再适用了。有时把它完全去掉，模型给出的结果反而更好。
Marina：我记得有人也说过类似的话：先别管这些，直接写提示词，看看效果怎么样。
Felix：对，这个办法确实有效。当然，前提是你有办法评估结果，知道什么算好、什么算不好。我会仔细判断，哪些时候需要 Skills，哪些时候不需要。还有一个很好的思考角度：除了用户体验，也想想“模型体验”。假设你就是 Claude，接到了眼前这个任务，你需要什么才能把它做好？你很快就会发现，自己需要查看原始数据，需要一台电脑，可能还得安装一些软件，也希望能自己作判断，而不是先听别人告诉你他们已经得出了什么结论。
Marina：我希望直接看到原始材料，就像你说的，知道对方到底说了什么，再自己判断。
Felix：是的，只是前期要多做些准备。一年半前，我还是自己查看程序的崩溃记录，每发现一个问题，就交给 Claude 修复。很快我就意识到，自己不过是在给 Claude 当传送带，把问题一个个搬到它面前。与其这样，不如让它直接查看全部崩溃数据，自己判断哪些需要修、哪些不用修。虽然一开始得花时间把这些配置好，但很值得，因为之后就能让它承担更多工作。我从 AI 身上获得的很多实际价值，都来自这样的改变。
Marina：有些人听到这里，可能会觉得，要用好 AI，得先做很多复杂的配置。那对于今天才开始用 AI 做知识工作的人，是不是可以更简单一些？比如，直接让它查看邮件和自己写过的邮件通讯，从中了解自己的表达风格和业务情况，代替事先专门整理这些说明？
Felix：我觉得，现在开始用 Claude，最好的办法就是先下载桌面应用。里面有内置浏览器，可以登录你平时使用的系统，不必一开始就配置连接器、下载全部数据，或者做其他准备。先把键盘交给 Claude，让它操作一下你的电脑，就已经能做很多事情，也能看出哪些任务做得好、哪些还不行。
用上一段时间，你可能会发现，Claude 已经能通过浏览器，帮你完成公司内部系统或外部软件里的某些工作。这时，再考虑怎样让它操作得更方便：比如，接上连接器，让它直接调用相关功能，就不用在网页上一步步点击了。但刚开始，用浏览器就足以尝试很多事情。
Marina：我再确认一下：你说的“智能体”，其实就是在 Claude 里开启不同的对话，而不是事先设置一个研究助手、一个报税助手，分别负责不同的工作，对吗？
Felix：我确实会用项目来区分不同的工作，但不会给它们分别设定角色。我从来不会告诉 Claude：“你是一位非常优秀的某某专家。”2024 年很流行这样写提示词，比如：“Claude，你是一名优秀的工程师，请修复这里的问题。”
Marina：但我们刚才不是也谈到，可以让它像这个领域最优秀的人那样工作吗？
Felix：我的做法是，先让 Claude 告诉我，这个领域最优秀的人会用什么工具，再要求它使用同样的工具。现在很多模型仍有一个问题，拿数据科学或会计来说，如果你只是告诉它“表现得像一名优秀的会计师”，它就可能开始角色扮演。但我们真正需要的，是专业人士用的方法、工具和知识，不只是他们说话的方式。我不需要 Claude 装成专家，也不需要它说得像专家。我甚至希望它能用五岁孩子也听得懂的话来解释，但处理问题时，用的仍然是这个领域最好的工具。
Marina：现在还有哪些事情，你不放心交给智能体做？
Felix：这个问题不太好回答，因为我们总可以增加检查和验证。对我来说，更需要区分的是：这件事不能放心交给 Claude，还是不能只靠一个对话就完成？后面这种情况其实多得多。比如，我很少让 Claude 在同一个对话里评价自己刚做完的工作，这种自我评价目前还不太可靠。无论你用的是 Claude Code、Claude Cowork、普通聊天，还是项目里的对话，Claude 通常都不太擅长判断自己的工作做得好不好。
所以，我会尽量找外部信号来判断。实在没有其他办法，也可以另开一个 Claude 对话来检查，效果已经证明相当不错。但我更希望找到实际的外部证据，看它到底有没有把事情做好。有时，这也需要一点耐心，给 Claude 一些时间，看看最后的效果。比如，在你的领域，我只是猜测，播放量、订阅人数可能就是重要的指标。
Marina：对，我们看的是播放量。每次给 Claude 布置任务，我都会告诉它，这次需要达到多少播放量。但我们现在还没解决好怎么把反馈传回去。我会给团队提意见，团队修改后发布的内容，已经不是 Claude 最初给我们的版本了。可能七成内容没变，另外三成是我加的，体现了我的判断和偏好。怎样才能让 Claude 也学到这部分反馈？
Felix：我们有个办法，效果还不错。先问一下，你们平时在哪里给反馈？是当面说，还是通过其他方式？
Marina：我们平时在 Telegram 上给反馈，但 Claude 没法直接读取。我们有个机器人收集聊天记录，可我还是得手动让 Claude 去查看这些记录，找出里面的反馈。现在用起来还不太顺畅，甚至我们已经发出去的内容，也得专门让它去看。
Felix：我们写代码时也有类似的情况。团队里有很多工程师，你通常知道手上的工作适合找谁来审查，也知道每个人会重点看什么。Claude 很擅长从他们过去给出的反馈里，总结出这些要求和偏好。
Marina：比如 Slack 上的反馈？
Felix：是的，GitHub、Slack，或者其他任何留下反馈的地方。
Marina：你没有专门让它去看反馈？只是给它 Slack 的访问权限，它就会自己找吗？
Felix：如果 Claude 能调用相关工具，就可以自己收集你过去给过的反馈，不用你再逐条整理。你可以让它分三步完成：先收集你以前给过的反馈，再从中总结出你重视的规则和原则。有时还可以多问一步：“有没有哪些原则，是我一直在遵循，却连自己都没有意识到的？”以后让 Claude 审查工作时，就把这份清单交给它作为参考。我们试过这种办法，确实能让不同对话中的 Claude 按照更符合我们要求的标准提出意见。
对于编程以外的工作，或者不以文字为主的任务，这种做法可能就没那么容易了。但类似的办法还是可以尝试。比如，把客户会议的文字记录交给它，请它评价这次沟通进行得怎么样。我也经常为 Anthropic 做招聘面试，这就是一种典型的知识工作，也很值得尝试。
Marina：可这样还是它在教我，我想做的是让它学会我的判断。我觉得，很多开始用 AI 的人都遇到过这种情况，我自己也是：发现效率提高了，就想做更多事情，但又没有哪个智能体能让我百分之百放心。所有内容都要先经过团队，最后再由我审核，结果我反而成了公司最大的瓶颈。
Felix：坦白说，我觉得我们现在的产品已经能很好地帮你做这件事，前提是让 Claude 看一段时间，了解你平时怎么给反馈。我对此很有信心。刚开始，你可以和它分别审查同一份材料，再把意见放在一起比较。这个实验我几乎会推荐给所
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
I'll translate it.
Editor Cai Fang
On October 13th, Felix Rieseberg, Project Manager of Anthropic and Director of Claude Cowork Project, published an article suggesting that you “be more ambitious” in using AI. He shared the article with the X post and has received over 5 million views. In his view, it's not just to get AI to do things faster, but to think again: What is the ultimate goal of the work ahead? Could you give more to AI and let it find its own way?
Felix himself has experienced this transformation. He used to look at the program crash record one by one, pick up the problem and hand it over to Claude to fix it. Later, he realized that he was just giving Claude a "transmission belt": since Claude could fix the problem, why not let him read the crash data and decide what needed to be repaired?
This reflection also influenced his view of Agent workflow.
In recent days, Felix, a guest of Silicon Valley Girl, and the moderator, Marina Mogilko, discussed in depth how to make fuller use of AI. His team had developed Claude Cowork for about a week and a half, and now he no longer writes the code in person, focusing more on product prototypes, user needs and team management. Felix thought that we might have spent too much effort telling AI how to work, but not giving it enough tools, data and discretion. During this interview Felix mentioned a number of specific practices.
When AI can take on more and more work, how do we judge it to do it right? How can it be avoided to be misled by a serious wrong answer, especially in areas that it is not familiar with? Felix shared a method of "confrontational review" - allowing several Claudes to pick the wrongs, and another Claude to judge whether these opinions are valid or not.
In addition, he suggested that AI should no longer be deliberately told that “you are an expert in a certain field”, rather than having the model act as an expert, it would be better to provide it with professional tools and data directly. In his view, this role-playing prompt had been done in 2024, but today LLM has become a generalist, deliberately setting the role of an expert might instead limit its ability.
During the interviews, they also talked about software start-up and the future of engineers. Where is the advantage of an entrepreneur when the software that used to be developed by a team takes months and one can do it in a short time? What is more important for young engineers who have just entered the industry? Felix gave his views in the context of his product and recruitment experience. Below is a full translation of the interview.
Too long to read:
What do you do most of the day without writing the code in person?
Felix: AI agent gives a brief briefing on the day. Now I'm trying to balance product, engineering, team management and data analysis, and most of the time I'm trying to get Claude to make different versions and then assess what is worth continuing.
Marina: How do you judge an idea? What are the main signals?
Felix: When development and maintenance costs are weighed in the past, it is now possible for AI to quickly make prototypes, test them and judge them. It is important to see whether users can quickly understand the usefulness of the product and use it to do what they were previously unable to do. The more functional the better, the better the better the better the better the better often require fewer buttons.
Marina: How will you judge which tasks are assigned to AI and which are performed by people? What's good for people?
Felix: I will distinguish between automation and capacity enhancement: the former helps people save time and costs, and the latter enables people to do things that they cannot do because they lack skills or knowledge. I'm more concerned about the latter. With good AI, there is a need to reconsider the workflow process to clarify objectives and evaluation criteria, rather than simply entrusting old tasks to new tools.
Marina: It's hard to describe and evaluate creative work like writing. What should I tell AI?
Felix: It depends not only on the results, but also on the reliability of the process, and AI happens to be a good guess. Creative work is indeed more difficult to evaluate, but excellent creators also have long-term methods of grinding. Instead of asking Claude to "write better," let it work according to professional steps and let AI use the same methods and tools. This often works better.
Marina: How do you avoid being misled by AI's self-confident wrong answer in the face of an area you don't know?
Felix: A confrontational review can be used: one Claude accomplishes the task, and several conversations are dedicated to finding mistakes, which are sometimes easier to find than to create, so at this point a slightly less capable model can be used. And then another Claude checks out these criticisms and sends back the firm opinion to decide whether to change it. If you can judge the results, there is no need to understand each step of AI.
Marina: What kind of knowledge do you have at your disposal? Does AI agent require specialized training for counterparts?
Felix: As far as I'm concerned, the work of fact-checking is already fully trusted by Claude. I would ask it to find a reliable first-hand source for each number, which usually requires only one general dialogue. Models themselves do not have to create expert roles for each mission, and too many role-setting may limit our use.
Marina: Do you still use Skills now?
Felix: Always, but carefully. Models are updated quickly, and Skill, written for the previous generation, may be outdated, and sometimes it is better to delete them. Consideration also needs to be given to what data and tools the model requires. I used to hand over the crash problem to Claude to fix it, and then I realized I was just doing the "transmission belt," and then I made it look at all the crash data and figure out what needed to be fixed.
Marina: How do I get Claude to learn what I'm thinking?
Felix: Let it know what feedback you have given in the past, then review the same material separately from you, compare the difference between the views and gradually adjust the process without having to let go immediately. Teamworks can use Claude from the Slack to bring people together.
Marina: Software is becoming easier to do, where is the moat of entrepreneurs?
Felix: Development costs have decreased, but the key to good products has not changed: understanding users, understanding the use scene and making products that make people happy with paid. The initial ideas may change, and the ability of the founders to understand demand and produce is more important.
Marina: How can individuals more fully utilize the capabilities of models?
Felix: First think of what the job is going to be, then try to give the bigger goal to the model and keep exploring its power boundaries. Of course, that doesn't mean it's going to make money. But if it is always used only to summarize mail and to adjust tone, it will be difficult to really act as a model.
What's a good product after a thousand prototypes?
Marina: Felix, thank you very much for coming to the show and it's a pleasure to have a chat with you today. I'm a heavy user of Claude's, and it's all running the channel. Team members have set up their own projects in Claude to handle their own work. For content creators, my team is quite large, with 35 people, and I want you to be more efficient and more productive. This year we have about three new social media channels and an e-mail newsletter. A year and a half ago, we also started operating LinkedIn, a platform based on words, and AI helped a lot. So I want to make it better for myself, for the audience and for the audience.
Felix: I understand you've used it very well.
Marina: But there's always room for improvement, right?
Felix: Indeed, I myself have a new experience every day. The most interesting thing about developing Claude is that you can hardly predict what the user will do with it. This is very different from the product I used to develop. For example, Slack, I'm about to know how people use it. But what users do with Claude often goes beyond my imagination, some of which I thought would be impossible. As a product developer, there is a particular sense of achievement when it comes to seeing users achieve such results with their own products.
Marina: Is there a particularly bold operation that you want to try after?
Felix: Not necessarily. But two days ago, I was impressed by a user named Pavo. I remember he was a product manager, and he gave Claude a lot of NASA data to look for a new planet, and he actually found it. It is said that NASA is currently being verified and has not yet been confirmed. I am neither astrophysicist nor fully aware of the matter, so I am not too sure. But it looks like Claude may have found a planet that was never discovered before.
Marina: This is so interesting.
Felix: I never thought I'd use Claude like this before. But think about it, it's very common for us intellectuals to have AI analyze a lot of data and look for valuable information from it, and we do it every day. It's just that I didn't think it would be possible to analyze open space data and find out what we didn't notice before. It's a wonderful idea.
Marina: Can you tell me about your day job? You were involved in the development of Claude Cowork, but you said you no longer wrote the code. So what do you usually do when you start working in the morning? Will AI agent tell you about the day?
Felix: Yes, they will indeed brief me on what happened on that day. I now look at the problem from a different perspective, more concerned with human emotions than I did a few years ago as traditional engineering manager, more willing to spend time understanding how others feel and how they do things. At the same time, I find myself less and less in need of the help of other professionals, and I can now take part in the work of product managers, engineering managers, team managers and data scientists.
As for what I do on a daily basis, I've been working mostly on prototypes. We'll try a lot of ideas, think about it for a few days, see if it's a good idea, and then Claude will come up with all the options. With the prototype, we can assess one by one what is good and what is not. So, the prototype does take a lot of my time every day.
Marina: How do you judge an idea? What are the main signals?
Felix: It's been a problem, and I don't have a comprehensive answer. But now, the factors that determine whether an idea is worth considering are really different. The development of products over the last 10 to 15 years, especially software development, is the real trade-off behind many decisions: can products be made? How much is it going to cost for follow-up maintenance and ongoing investment? How much time does it take? And choosing to do it, will you miss other opportunities? The project manager had to consider all of these before deciding whether to do it or not. Now, we can see the actual effects more quickly, and let AI do a trial version first, and see how it works, and then make a judgement.
But human attention remains limited. For example, you're very judgmental about the AI product, and I might only be able to show you one a day. I can easily make a thousand, but I can't get you through a thousand. Even if you wanted to spend the whole day with me, you wouldn't be able to see it, and you thought, "Well, I don't want to see it anymore." So, what's the good product? I think there's an old saying that applies especially in the AI era: perfect, there's nothing to delete, not to add all the functions. We can make any button you want, but I think the good product button is less.
I usually give it to a few people. If they soon understood its usefulness and could do what they had not done before, I would know that the product was right. That's the signal I really value.
Marina: So how do you judge what you really want to solve? How, for example, do you come to my company now to decide which tasks are entirely assigned to AI, which are left to people and which only AI handles some of them?
Felix: First of all, I will look at automation and capacity enhancement separately. Some of the tasks take up your time and make it impossible for you to do the job you like; others are costly and automatically save a lot of money or time. In these cases, automation is indeed helpful. I think that AI's currently creating economic value, most probably from here. We judge that using AI is not worth it, and we usually see the benefits of automation.
But what I'm more interested in is empowerment, that is, getting people to use AI to do things they couldn't do before. You might have been lacking some skills, some methods, or some knowledge that you couldn't do on your own, and now AI can help you. When I went to a company, I was most impressed not by what tools and tools they could do, but by how they had adapted their way of working.
AI is a very special product. To really use it, you have to rethink how you work: what is the end result? How do you do that? What are you gonna say when you get the results? Those who remember me most are usually very good at reflecting on their workflow process.
Marina: But it's hard. In terms of my work, a lot of people use AI to draft articles. Can knowledge work also have clear evaluation criteria? It's probably easier to judge, you know what you want to do, and you know that code can't be flawed. But how can you tell AI agent about knowledge work, especially work that requires creativity?
Felix: It's hard. I often give an example: assuming your job is to broadcast the weather, you let AI agent check the temperature outside. Here we have to figure out exactly how we should evaluate its performance. If the model just guesses the temperature, we should not let it cleared, but rather think it's a problem. On the contrary, if it gave a half-degree difference in temperature, but found the right place, found a reliable source of data and the whole process was justified, it should also be better than a coincidence.
However, writing, as well as other work that relies heavily on aesthetic judgement, is more difficult to evaluate. It is not easy to prove one outcome better than the other. So, I like to talk to those who write well and are willing to use AI.
Marina: What would they do?
Felix: It's fun to see them work. My friend Adam Davidson worked as a full-time author in several very good media outlets. He'd use all kinds of prompt to get Claude to pry his article, like, "Smuggle my argument a hundred times." "Try to change this article." "Where do you disagree?" "What's the strongest objection?" I'm not a professional writer, and the easiest thing to think about is handing over the article to Claude and saying, "Help me fix it." But I'm finding out more and more that doing knowledge can't just ask AI to give a good result, but to think first: how do the best people in this field work? Can you get AI to finish those steps?
Marina: It's a way to imitate their workflow process, but there's still a lack of aesthetic judgment, so I think much more needs to be done.
Felix: Yeah, I agree. Interestingly, however, most of the really good authors don't think they're good, just because they have some amazing talent. They often say, "No, it's a lot of work."
Most creators have their own methodology. If you ask them, "Your work is good because you're born good, and you can do it naturally, or because you have a method that's been grinding for years?" But the more I talk to the creators, the more I find that they're proud of their methods. It's hard to create, and there's a lot of concrete steps that need to be done, not sit down, and a good book suddenly appears. You go to the author's studio and you'll see a lot of whiteboards with information and ideas.
And I found out in different fields that instead of just telling Claude what kind of results she wanted, it'd be good to do what professionals did. This may be somewhat abstract, and I will give a specific example. Recently, you should have seen the progress of AI in dynamic visualization, video and 3D. Opus 5.5 does improve its ability, but users can do better, and one important reason is to get Claude to use Blender to tell him: "Please use the tools used by professional animators. The results are quite good. What really works here is the media, methods and tools used to create, not just the models themselves that understand what is a good animation film.
Don't let Claude give himself a job. Score
Marina: You just said that you could get AI to work in a professional way. But even if it looked online and much information was not made public, how did you know it was credible? For example, I'd like to try another form of content, or look for a new star, as I just said, but I don't know anything about it but tell it "work like NASA researchers." How do I avoid it telling me the wrong answer is certain?
Felix: It's a good question, a lot of people will experience it. I have a simple, practical approach: a confrontational review. This approach has proven to be quite effective in programming, mathematics and other areas where numerical impact can be measured. The way to do it is to get one Claude to finish the job, and then start another ten to find it. Their task is to question why this result is not correct, and the basic idea is to test the red team and make a blank.
Marina: Will you review it with the same model or change it?
Felix: Depending on the specific tasks. However, it is sometimes easier to identify problems than to create new things, and so are people. Let me evaluate a Q79 machine. It's a lot easier to design a cell phone from the beginning. As a result, the review can usually be carried out using models that are somewhat less capable and have good results. This is a very practical suggestion.
Marina: What about prompt? If there is already a workflow programme, what are the additional requirements?
Felix: Suppose you already have a presentation or document. Taking the blog I wrote this morning, I'm using Claude Projects, which is a new product that can work more easily across multiple lines. I'll just tell Claude, "Let's start five threads and get every thread to prove that the article is wrong: I may be wrong, I may be wrong, I may be wrong.” They look at it with this clear objective to find out what I was wrong.
Marina: So what do you do with all the comments? I'll do the same when I'm going to interview, but I'm going to use different models to see if I'm getting too deep on an issue. But the final feedback varied, and the model said that there were other opinions and that there was a completely different opinion, and the result was left to me. How do you judge what is worth taking?
Felix: You're supposed to love this answer and guess what I'm going to say: give it to Claude. When you write the code, we set up a referee. First, those who were responsible for the error, AI agent, set out the main points, then hand them over to another Claude, so that it could verify them one by one and determine whether they were valid.
Next, according to your workflow, you can hand over the valid opinion to Claude, who initially completed the mission, so that it can decide whether to change it or not. It could also judge that it was not necessary to accept every recommendation. We often meet when we write codes, and I feel the same way when I do knowledge work: Some of them sound reasonable, but ignore a key point, or they cannot actually be modified on that basis and are not helpful. At this point, I usually know that the original results are quite good. If you are unable to do well on your own, and it is not easy to verify, you can use this approach.
But I do a lot of things with AI, and I don't really need to know everything about it. We have our own judgment about what we use every day. For example, I remade my website, which uses a lot of 3D asset. I've never used Blender, I don't know how to do it, but I can judge how well these things look, how they feel as a whole. You can think of yourself as a client, you don't have to figure out how to do it every step. It is also common in life to invite people to work, to seek a lawyer or to see a doctor, and we do not necessarily have the expertise to judge that the other party's work is not good, and more often we can only give feedback on how we feel.
So if you don't know how to judge the results, then spend more token, let AI help you with more scrutiny. If you can judge by yourself, that might be enough, not necessarily to understand how it came out.
Marina: I think it's time to pick out a few of the most important projects to see how each step is done, not how everything is controlled.
Felix: Yeah, it's the old question: can't let go, let go of things.
Marina: It's hard, and it's a problem with AI agent now. Are there any types of intellectual work that you can hand over to them with great confidence and full confidence in their results?
Felix: I now have full confidence in the results of fact-checking, such as in editorial work. I'll have Claude verify all the facts in the presentation, and I won't have to double-check many rounds, let alone find 50 AI agents, one is enough. I'll tell it: “Please find a first-hand source for each of the numbers I mentioned and confirm that the source is reliable.” I can now give it my full confidence in this work.
From taking over Claude's handling problems to letting him find his own answers.
Marina: Is this AI agent you're talking about, is it special training for fact-checking, or is it the one used to handle tasks?
Felix: It's a Claude conversation. My next comment may be somewhat controversial, and you should like it: I think we're a little overemphasised on “AI agent”. The model is itself an excellent generalist and has been tested many times. For example, when the industry started to study the mathematical capabilities of models, I remember Google DeepMind was the first company to invest. They trained specialized models and built an accompanying implementation framework that allowed them to participate in the Math Olympiad and to win silver medals, which were already the best at the time. As a result, four or five months later, several major generic models were awarded gold.
I understand why you set a role for AI agent. Tell it, "You're a data scientist," so we can understand more easily what it can do. But the model that we call "data scientists" is also capable of dealing with the problems faced by top biologists, mathematicians and other professionals. If each time AI is used, you have to set up a set of roles for it, which sometimes limits our use. So much of the work I have just mentioned has not been specifically identified for AI agent.
Marina: But what data is available to AI agent is not important? We usually upload specific information to each AI agent. Or is your Claude able to call all the tools, so there's no need for separate preparation? But is it too much data to consider for each question?
Felix: The data are really large, but it's already very good at judging what it needs. I'm not in favour of split the data into a few, then distributing one to each AI agent. The same is true of our own work, which often requires an understanding of the whole business, and it is not enough to see a fraction of the information. For example, I am an engineering manager, who can play a more limited role if he only looks at the code, but can do better if he can also see client data and know how the client communicates. Sometimes the sales team has meetings with clients, and I want to join them, hear what they say. I would also like to know the details of the use of the product, and it would be useful to see operational data and to know what works and what does not. AI agent should also have access to this information.
But that doesn't mean that all data will be stuffed to it once and for all. Give it the tools, and make it clear what data each tool can find. For example, it is told: “If business data are needed, this tool can be used for searching.” In this way, it can look into it itself if it needs to do so, without having to read everything out first.
Marina: What exactly should we do? Listening to you, I wonder if it would be appropriate to adjust the current practice and not to prepare a separate set of information for each item. For example, we have a project to assess the performance of guests, which contains a complete process description and scripts from previous programmes to understand the performance of different guests. Can I just tell Claude, "This folder has all the scripts, Notion has every guest's performance, and here's our rating." Get these ready, and every time you assess a guest, you start a new conversation, without having to go back to the background and say, "Assess this guest for me," it can do?
Felix: I hate to tell anyone what to do with workflow. I always felt there was no need to change as long as the current approach worked for you.
Marina: But models are progressing, and I want to follow. The methodology has been in use for a year and, although it has been in effect, it has been long.
Felix: Yes. If you want to know what I suggest or how I do it myself, I will say first that everyone should choose a way that is for himself, that is easy, and that really helps. The model is, however, now better at using tools, knowing when and how to use them, and is not easily caught in a cycle of repeated calls. And it doesn't come up with a tool to check the data, and it's like, "Well, I'll keep looking until I'm done with all the data." Now, they are very good at using tools, doing things right and not wasting too much effort.
So, I'm not sure I'm going to be able to decide what data this task requires before it starts. In this way, it is much easier to have multiple dialogues and to deal with multiple tasks at the same time, and the process is much simpler.
Marina: Do you still use Skills?
Felix: Yes, we use a lot of it now. But the official advice of Anthropic is in agreement with my personal opinion: use Skills with caution. We write instructions for models, many of which are actually for older generations. AI was too fast to update, and a set of Skills, which was a good one, might soon no longer apply. Sometimes it's completely removed, and the model gives better results.
Marina: I remember someone who said the same thing: Let's just forget about it. Just write prompt, see how it works.
Felix: Yes, it does work. Of course, if you have a way of assessing what's good and what's bad. I'll figure out when Skills is needed and when. There is also a good perspective: beyond the user experience, think about the “model experience”. Assuming you're Claude, what do you need to do with this job? You'll soon find that you need to look at raw data, have a computer, have some software, and want to judge yourself, rather than hear what they've come to.
Marina: I want to see the raw material, as you say, know exactly what the other person said, and judge for himself.
Felix: Yes, just a little more preparation in the front. A year and a half ago, I looked at the crash record of the program, and whenever I found a problem, I gave it to Claude to fix it. Soon I realized that I was just giving Claude a conveyor belt and moving the problem to it. Instead, it should be allowed to look directly at the total crash data and determine which needs to be repaired and which does not. While it would have to take time to get these configurations right at the outset, it would have been worth it, because it would then be able to take on more work. I got a lot of real value from AI from this change.
Marina: Some people hear this, and they might think, to use AI, they have to make a lot of complicated configurations. Would it be easier for someone who just started using AI today? For example, would it be able to look directly at e-mails and e-mails that it had written, to learn about its style of expression and business, instead of pre-packaging those notes?
Felix: I think the best way to start with Claude is to download desktop applications. It contains a built-in browser that logs on to the system you normally use, without having to configure the connector at the outset, download all the data or make other preparations. Hand over the keyboard to Claude, let it run your computer, and you can already do a lot of things, and you can see what's good and what's not.
For some time now, you might find that Claude has been able to help you with some of the work in the internal system or the external software. At this point, consider how to make it more operational: For example, by connecting the connector to a direct call to the relevant function, there is no need to click one step at a time on the web page. But at first, it's enough to try a lot of things with a browser.
Marina: I want to confirm: when you say "AI agent," you actually start a different conversation in Claude, instead of pre-positioning a research assistant and a tax assistant to do different jobs, right?
Felix: I would really use projects to distinguish between different jobs, but I wouldn't give them separate roles. I never told Claude, "You're a very good expert." It's popular in 2024 to write prompt, for example: "Claude, you're a great engineer, fix the problem here."
Marina: But didn't we just talk about making it work like the best people in this field?
Felix: My approach is to have Claude tell me what tools the best people in this field will use, and then ask it to use the same tools. Now there's a problem with many models, in terms of data science or accounting, if you just tell it to act like a good accountant, it could start playing. But what we really need is the methods, tools and knowledge that professionals use, not just the way they speak. I don't need Claude to pretend to be an expert, and I don't need it to sound like an expert. I would even like it to be explained in words that five-year-olds understand, but the best tools in this field are still used to deal with problems.
Marina: What else do you want to do now?
Felix: That's not a very good answer, because we can always add checks and checks. What's more important to me is: can't this be left to Claude, or can't it be done by just one conversation? The latter is actually much more. For example, I rarely let Claude evaluate the work that he's done in the same conversation, and this self-evaluation is not very reliable. Whether you use Claude Code, Claude Cowork, regular chats, or dialogue in the project, Claude is usually not very good at judging how well she works.
So I'll try to find an external signal to judge. There's really no other way to do it, or to open another Claude dialogue, which has proven to be quite good. But I'd rather find practical, external evidence to see if it did. Sometimes it takes a little patience to give Claude some time to see the end effect. For example, in your field, I'm just guessing that the number of players, the number of subscribers, may be an important indicator.
Marina: Yeah, we're looking at the volume. Every time you give Claude a job, I'll tell her how much to play this time. But we still haven't figured out how to get the feedback back. I'll advise the team, and the team changed the content of publish, not the original version Claude gave us. Maybe 70% of the content hasn't changed, and I added 30%, reflecting my judgment and preferences. How can Claude learn this part of the feedback?
Felix: We have an idea. It worked. First of all, where do you give feedback? In person, or in other ways?
Marina: We usually give feedback on Telegram, but Claude can't read it directly. We have a robot that collects chat records, but I still have to manually get Claude to look at them and find the feedback. It's not working very well now, and even what we've sent out has to be seen.
Felix: We have a similar situation when we write codes. There's a lot of engineers in the team, and you usually know who's good at the job and what everybody's focused on. Claude is very good at extracting these demands and preferences from their past feedback.
Marina: Like the feedback on Slack?
Felix: Yes, GitHub, Slack, or any other place where feedback is left.
Marina: You didn't specifically let it see the feedback? Just give it the Slack interview permission, will it find it?
Felix: If Claude can call the tools, he can collect the feedback you've given in the past, and you don't have to sort it out. You can make it happen in three steps: first by collecting your previous feedback and then drawing from it the rules and principles that you value. Sometimes one more step can be asked: “Are there any principles that I have been following without even being aware of?” This list will be referred to Claude for information at a later stage. We've tried this way, and it's true that Claude in the different dialogues is able to put forward ideas that are more in line with our requirements.
This may not be so easy for jobs other than programming or for tasks that are not text-based. But similar approaches can be tried. For example, it was asked to evaluate how well the communication had proceeded by handing it the transcripts of the client meeting. I've been doing job interviews for Anthropic, which is a typical knowledge job, and it's worth trying.
Marina: But it's still teaching me, and all I want to do is let it learn my judgment. I think that's what happened to many people who started using AI, and I myself: I'm trying to do more when I find out it's more efficient, but none of the AI agents gives me 100% assurance. I'm the biggest bottleneck in the company.
Felix: Frankly, I think our product is doing you a good job of doing this, if Claude sees for a while how you usually give it back. I'm very confident. At the outset, you can examine the same material separately from it and compare it with the comments. I'll almost recommend this experiment.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.