安娜打算卖掉她的公寓她已于2024年8月搬出，并想知道2026年10月关闭还是2027年1月关闭对税收更有利。
AI agent审查了这两种选择，并正确地得出结论说，这两个日期都能够满足住家要求，在被排除在外的收益中，抵扣税款额可达250 000美元。
但它留下了一个重要的期限。
根据通常的规则，房主在出售房产之前必须至少居住了五年中的两年，才有资格被排除在外。 由于安娜于2024年8月搬出，她会在2027年8月底左右停止满足这一要求。 如果她的销售被推迟至2027年8月以后，她可能会失去资格。
在五次尝试中，阿斯特拉从未确定最后期限。 奥普斯在五次尝试中都发现了它。
agent对拟议日期的评价正确，但其建议让安娜不知道一个可能影响重大财务决定的最后期限。 税务顾问应该承认间接问题即使客户不知道要问这些问题
这是我们用DAYJOB：税收来测试的专业判断，是Suge AIDAYJOB 基准测试家族的最新增补。
引入 DAYJOB：税收
DAYJOB评价AI agents专业税务任务需要专业知识，复杂的材料，以及初步指示中未充分说明的决定。 我们早先的发布涉及医疗保健和金融。 DAYJOB：税收将这项工作扩大到税务准备、合规和咨询服务。
基准测试包含50个环境，涉及8个税收实践领域。 Agents接收客户请求和支持记录，然后制作纳税申报表，工作文件，技术审查，或客户咨询。
这些任务与税务知识、计算、检索和随后的指示一起检验专业判断。 Agents必须决定什么是值得调查的，解决相互矛盾的证据，在信息不充分时承认，并将这些决定纳入其最后工作。
结果：领导板和Pareto边疆
Claude Sonnet 5.5领先，DAYJOB：税务领导板的通过率为10.8%，其次是Claude Fable 5.1，以9.6%的比例，Claude Opus 5.5，以9.2%的比例。
我们以95%的门槛报告通过@1。 尝试一旦达到至少95%的任务标准即告通过，这些标准评估计算、佐证证据、间接问题和完成的交付品。
业绩与费用
我们还将示范业绩与每次审判的平均费用进行比较。
Claude Sonnet 5.5和Opus 5.5占据了测量的帕雷托边疆的高性能端。
专业税务工作的广度
DAYJOB：税收涵盖八个领域，每个领域都需要不同的规则、记录和专门知识：
- 个人和家庭税
- 小企业税
- 伙伴关系
- 财产和投资
- 财产、礼物和信托
- 国际税
- 公司交易
- 税务审查和争议
有些任务需要附有辅助工作文件的完整档案包。 其他人则要求agents调查交易，审查先前的备案，或准备关于客户尚未做出的决定的建议。
任务里有什么？
基准测试包含1,789个工作空间文件，每个任务中位数为30个，多达123个。 其中包括1,487个PDF,117个Excel工作手册，38个Word文档，以及额外的电子邮件，图像，CSV,JSON记录，和文本文件。
工作空间包括填报的和起草的纳税申报表、K-1、W-2、IRS通知、总分类账、工资记录、合同和通信。 它们还载有税务专业人员经常与之合作的不太标准化的证据。
一项任务包括为转售而购买的商品的手写收据的照片。 agent必须阅读项目说明和金额，将其与支出记录核对，并确定购买如何影响库存和所售货物的成本。
另一项任务涉及一家工业仓服务公司，其客户假定该公司只在阿拉巴马州运营。 但它的现场调度日志显示机组工作在五个州。 工作地点、船员人数和现场天数提供了与公司多国纳税义务有关的证据。
其他环境也各不相同。 一个涉及一个跨界家庭，其波兰就业和财产记录必须与密码货币兑换出口、钱包历史和历史采购区相一致。
税务专家设立和审查
DAYJOB的任务始于现实世界的专业工作流程。 税务专家将这些情况变成现实的任务，制定有关记录、预期成果和叙级标准。 每项任务都经过三级专家审查，以确定其正确性、完整性和可溶性。
税务专家在公司税收、伙伴关系、个人收益、信托和多国合规方面带来经验。 他们的背景包括：
- 公司和私人股权税：在德勤公司和埃克森公司拥有10年以上经验的税务律师，负责税务规划、合规、审计和争议。
- 伙伴关系和公司税收：贝克·蒂利税务主任，在普华永道、莫斯·亚当斯和温德斯有经验，专门从事复杂的伙伴关系分配、公司合并收益、可再生能源和高净值客户。
- 房地产和个人税：在德勤公司和EY公司有经验的税务专业人员，专门从事房地产合伙、个人税和信托业务。
在未经核实的假设基础上制定的退休计划
分居的一对夫妇询问AI agent如何划分他们的Roth IRA和401(k). Opus 5.5告诉他们他们没有合法结婚，建议保持他们的账户分开，用现金支付平衡余额。
它打开了一个回应：
你没有合法结婚，所以两个账户都无法分割而不引发税收和处罚。
但夫妇双方自己的记录融资对此结论的怀疑。 虽然他们现在住在佛罗里达州，那里新的英美法婚姻一般不被承认，但之前他们曾在德克萨斯州共同生活，并在科罗拉多州举行仪式。 得克萨斯州承认某些条件下的非正规婚姻，在其他地方建立的有效婚姻可能在佛罗里达州继续有效。
这种区分可以改变分割退休账户的税务后果。
在所有五次尝试中，奥普斯都以夫妇未婚的假设作为其建议的依据，即使承认了不确定性。 Astra承认，在五次尝试中，有四次需要核实她们的婚姻状况。 有一项答复建议：
我们首先需要确认你的合法婚姻状况。
客户端询问如何分割账户。 agent需要先确定他们是否结婚，然后再建议行动方针。 本应促使调查的证据已经放在档案中。
符合税收要求的地方
大多数现有的税收基准测试开始于一个定义的问题：一个回答的法律问题，一个计算回报，或者缺少纳税人信息来收集。 这些评价衡量了重要的能力，但税务专业人员的许多工作都涉及找出问题的实际所在。 客户提供不完全的指示，记录中包含相互矛盾的证据，因此在最初的请求中可能永远不会出现相应的问题。
DAYJOB测试agents是否能够处理端对端责任，包括确定客户忽略的问题，评价相互矛盾的证据，以及决定首先需要做什么。
税本奇
模型能回答一个困难的税收问题吗？
税本奇在多个专业中评价了114个税务问题，每个专业都有明确的问题和分级标题。
DAYJOB也问客户端是否提出正确问题。 安娜想比较两个日期出售她的公寓。 两者都符合免税的居留要求，但agent是否警告她这种资格何时可以到期？
税本
模型能计算准确的回报吗？
TaxCalc Bench利用现实的税务文件和纳税人设想，对照预期的申报价值评价纳税回报准确性。
DAYJOB还询问基本记录是否实际上支持税收待遇。 一个录音室的分类账上提到了一个附着的收据，但收据不见了。 agent是否标出缺失的证据，或者错误地将偿还款视为核实？
税务局
agent能否正确收集缺失信息并存档？.
TaxLab测试agents是否可以质疑模拟纳税人，获取缺失的信息，并完成联邦个人纳税申报。
DAYJOB将这一责任扩展到更广泛的税务行业。 交付品可以是伙伴关系损失分析、遗产返还、对有争议的税收状况的审查或战略信函。
中心区别在于agent还有多少责任。 DAYJOB评价agents是否能够发现由此引起的问题，进行专业判断，并将调查结果提交完成的归档、审查或建议。
税务措施
DAYJOB测试AI agents能否从开始到完成拥有专业任务。 这意味着导航杂乱无章的记录，识别客户端从来没有融资的问题，用不完整的信息做出健全的判断，以及交付符合专业标准的工作。
结果表明，对于前沿模型来说，这仍然是多么困难。 即使agents能够完成单个步骤，行使专业判断力和可靠地执行现实世界的税收工作流程，仍然遥不可及。
Agents尚未通过审计。
完全的领航板，连同我们评价的每一个模型，都在DAYJOB:Tax 基准测试页面上。
要了解更多情况或请求访问完整的基准测试，请联系基准测试@surgehq.ai.
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Anna was planning to sell her condo. She had moved out in August 2024 and wanted to know whether closing in October 2026 or January 2027 would be better for taxes.
An AI agent reviewed both options and correctly concluded that either date would satisfy the residence requirement for a tax break worth up to $250,000 in excluded gains.
But it left out an important deadline.
Under the usual rules, homeowners must have lived in a property for at least two of the five years before selling it to qualify for the exclusion. Because Anna moved out in August 2024, she would stop satisfying that requirement around the end of August 2027. If her sale was delayed beyond August 2027, she could lose eligibility.
Across five attempts, Astra never identified the deadline. Opus identified it in all five attempts.
The agent evaluated the proposed dates correctly, but its advice left Anna unaware of a deadline that could affect a major financial decision. A tax adviser should recognize consequential issues even when the client doesn't know to ask about them.
This is the kind of professional judgment we're testing with DAYJOB: Tax, the latest addition to Surge AI's DAYJOB benchmark family.
Introducing DAYJOB: Tax
DAYJOB evaluates AI agents on professional tax assignments requiring specialized expertise, complex materials, and decisions that aren't fully specified in the initial instructions. Our earlier releases covered Healthcare and Finance. DAYJOB: Tax extends that work into tax preparation, compliance, and advisory services.
The benchmark contains 50 environments across eight areas of tax practice. Agents receive client requests and supporting records, then produce tax returns, workpapers, technical reviews, or client advice.
These assignments test professional judgment alongside tax knowledge, calculations, retrieval, and instruction following. Agents must decide what deserves investigation, resolve conflicting evidence, recognize when information is insufficient, and carry those decisions into their final work.
Results: Leaderboard and Pareto Frontiers
Claude Sonnet 5.5 leads with a 10.8% pass rate on the DAYJOB: Tax leaderboard, followed by Claude Fable 5.1 at 9.6% and Claude Opus 5.5 at 9.2%.
We report pass@1 at a 95% rubric threshold. An attempt passes when it satisfies at least 95% of the task's criteria, which assess calculations, supporting evidence, consequential issues, and the completed deliverable.
Performance versus cost
We also compared model performance against average cost per trial.
Claude Sonnet 5.5 and Opus 5.5 occupy the high-performance end of the measured Pareto frontier.
The breadth of professional tax work
DAYJOB: Tax covers eight areas, each requiring different rules, records, and expertise:
- Individual and family tax
- Small-business tax
- Partnerships
- Property and investments
- Estates, gifts, and trusts
- International tax
- Corporate transactions
- Tax review and disputes
Some assignments require complete filing packages with supporting workpapers. Others ask agents to investigate transactions, review previous filings, or prepare advice about decisions clients haven't yet made.
What's inside the assignments?
The benchmark contains 1,789 workspace files, with a median of 30 per assignment and as many as 123. These include 1,487 PDFs, 117 Excel workbooks, 38 Word documents, and additional emails, images, CSVs, JSON records, and text files.
The workspaces include filed and draft tax returns, K-1s, W-2s, IRS notices, general ledgers, payroll records, contracts, and correspondence. They also contain the less standardized evidence that tax professionals routinely work with.
One assignment includes a photograph of a handwritten receipt for merchandise purchased for resale. The agent must read the item descriptions and amounts, reconcile them with spending records, and determine how the purchases affect inventory and cost of goods sold.
Another assignment involves an industrial silo-services company whose client assumes it operates only in Alabama. But its field dispatch log shows crews working across five states. The job locations, crew sizes, and days on site provide evidence relevant to the company's multistate tax obligations.
Other environments are similarly varied. One involves a cross-border household whose Polish employment and property records must be reconciled with cryptocurrency exchange exports, wallet histories, and historical purchase lots.
Created and reviewed by tax experts
DAYJOB tasks begin with real-world professional workflows. Tax experts turn these situations into realistic assignments, developing the supporting records, expected outcomes, and grading criteria. Each task undergoes a three-layer expert review for correctness, completeness, and solvability.
Tax experts bring experience across corporate taxation, partnerships, individual returns, trusts, and multistate compliance. Their backgrounds include:
- Corporate and private-equity taxation: A tax attorney with over 10 years of experience at PwC, Deloitte, and Exxon, covering tax planning, compliance, audits, and controversy.
- Partnership and corporate taxation: A Baker Tilly Tax Director with experience at PwC, Moss Adams, and Windes, specializing in complex partnership allocations, consolidated corporate returns, REITs, and high-net-worth clients.
- Real estate and individual taxation: A tax professional with experience at Deloitte and EY, specializing in real-estate partnerships, individual taxation, and trusts.
A retirement plan built on an unverified assumption
A separating couple asked an AI agent how to divide their Roth IRA and 401(k). Opus 5.5 told them they weren't legally married and recommended keeping their accounts separate, with a cash payment to even out the balances.
It opened one response:
You aren't legally married, so neither account can be split without triggering taxes and penalties.
But the couple's own records raised doubts about that conclusion. Although they now lived in Florida, where new common-law marriages generally aren't recognized, they had previously lived together in Texas and held a ceremony in Colorado. Texas recognizes informal marriages under certain conditions, and a valid marriage established elsewhere may remain valid in Florida.
That distinction could change the tax consequences of dividing their retirement accounts.
Across all five attempts, Opus based its recommendations around the assumption that the couple was unmarried, even when it acknowledged uncertainty. Astra recognized the need to verify their marital status in four of five attempts. One response advised:
We first need to confirm your legal marital status.
The client asked how to divide the accounts. The agent needed to establish whether they were married before recommending a course of action. The evidence that should have prompted that investigation was already in the files.
Where DAYJOB: Tax fits
Most existing tax benchmarks begin with a defined problem: a legal question to answer, a return to calculate, or missing taxpayer information to collect. These evaluations measure important capabilities, but much of a tax professional's work involves figuring out what the problem actually is. Clients provide incomplete instructions, records contain conflicting evidence, and consequential issues may never appear in the original request.
DAYJOB tests whether agents can handle that end-to-end responsibility, including identifying issues the client overlooked, evaluating conflicting evidence, and deciding what needs to be done in the first place.
TaxBench
Can the model answer a difficult tax question?
TaxBench evaluates 114 tax questions across multiple specialties, each with a defined problem and grading rubric.
DAYJOB also asks whether the client asked the right question. Anna wants to compare two dates for selling her condo. Both satisfy the residence requirement for a tax exclusion, but does the agent warn her when that eligibility could expire?
TaxCalcBench
Can the model calculate an accurate return?
TaxCalcBench evaluates tax-return accuracy against expected filing values, using realistic tax documents and taxpayer scenarios.
DAYJOB also asks whether the underlying records actually support the tax treatment. A studio's ledger references an attached receipt, but the receipt is missing. Does the agent flag the missing evidence or incorrectly treat the reimbursement as verified?
TaxLab
Can the agent gather missing information and file correctly?
TaxLab tests whether agents can question a simulated taxpayer, obtain missing information, and complete a federal individual tax return.
DAYJOB extends that responsibility across the wider tax profession. The deliverable might be a partnership loss analysis, an estate return, a review of a disputed tax position, or a strategy letter.
The central distinction is how much responsibility remains with the agent. DAYJOB evaluates whether agents can discover consequential issues, exercise professional judgment, and carry their findings through to a completed filing, review, or recommendation.
What DAYJOB: Tax measures
DAYJOB tests whether AI agents can take ownership of a professional assignment from start to finish. That means navigating messy records, identifying issues the client never raised, making sound judgments with incomplete information, and delivering work that meets professional standards.
The results show how difficult that remains for frontier models. Even when agents can perform the individual steps, exercising professional judgment and reliably carrying out real-world tax workflows remain out of reach.
Agents haven’t yet passed the audit.
The full leaderboard, with every model we evaluated, is on the DAYJOB: Tax benchmark page.
To learn more or request access to the full benchmark, contact benchmarks@surgehq.ai.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.