Asana 削减模型在使用GPT‐6.1 Sol的浏览器测试中花费76x
Asana在Codex中使用GPT‐6 Astra，使其浏览器agent 76x更便宜，在测试中更快地提供了客户更有能力的模型。
76x
最佳GPT-6.1 降低模型成本
5x
GPT-6.1 Sol工作流程加快浏览器运行速度
$0.47
优化GPT-6.1 Sol工作流程的平均模型成本估计
由于在Codex运行实验中的GPT‐6 Astra,Asana优化了在GPT‐6.1 Sol的浏览器agent工作流程，以运行76x更便宜，5x更快。
Asana通过StackAI(在新窗口打开)，帮助客户实现跨业务应用程序的自动化工作，这个平台它收购(在新窗口打开). 使用StackAI，客户可以构建导航网站的工作流程，填写表格，并收集信息而无需写入代码。 在阿萨纳的规模上，这些工作流程中小幅的低效率加起来。
阿萨纳的StackAI CTO，博士弗兰克·伊达尔戈(Frank Hidalgo)着手使浏览器agent运行更快，更便宜。 他在Codex中指示GPT‐6 Astra调查agent，测试改进，并比较结果。 他估计要花一到两个月的人工工作，大约需要一个星期。
Asana的144个运行研究(在新窗口打开)测试了GPT‐6.1 Sol和另外三个前沿模型，称为A型、B型和C型。 GPT‐6.1上出现的优化工作流程，平均模型成本为0.47美元，每运行约4分钟，比B型模型上最初的生产设置便宜76x，快5x。
“这就是人类和agents的队伍在实践中的样子。 一位工程师设定了方向，GPT-6 Astra主持了实验，结果通过Command到生产。 这说明Asana 带来人类和agent的团队如何生存。”
为快速移动，伊达尔戈首先在Codex中使用GPT‐6 Astra来映射代码库，并解释agent如何构建每个模型请求。 GPT‐6 Astra发现agent缓存了它的固定指令和工具定义，但不是它所收集的页面文本和截图的不断增长的历史，所以每个请求都以全额的价格对历史感到不满。
agent还几乎在每一步都放下了更古老的截图和修剪文本。 每次编辑都改变了历史，因此仅仅刻画历史是无济于事的，失去这些事实可能要求agent重温它已经读过的网页。
Hidalgo审查了GPT‐6 Astra提出的修补方法，并选择了三个进行测试：
- 将缓存扩大到agent的浏览历史
- 增加可保留的案文数量
- 分批而不是每一步删除截图
GPT‐6 Astra开始快速测试，以确定哪些变量很重要。 因为代码不是为受控实验设计的，所以它随后重构了代码，因此一个前端和后端可以并行支持许多工作流程，每个工作流程都有自己的设置。
Astra进行了全面研究：历史预算为12万和48万个字符，以及6个缓冲和截图政策，每个模式在四个模式中各测试三次(见下表)。 业绩最佳的政策允许截图累积到20个，然后才恢复到最近的。 这使得先前的历史保持不变，因为清除量之间的长度较长。 与更大的历史预算相结合，成为优化的工作流程。 每个配置都执行同样的任务：从一个公开的演示目录中收集32本书籍的6个字段，代表一些Asana 客户在StackAI运行的内容。
GPT‐6 Astra运行工作流程并审查请求、使用记录和产出，另外的示范会议审查了工作。 每一课的请求，数据痕迹和结果都记录在Asana软件发送平台Command(在新窗口打开)中，这样小组就可以审查之后的完整研究。 从"司令部"看，发现结果变成了票，然后拉出请求，改版开始制作。
“这需要我一到两个月的手。 GPT-6 Astra在Codex中写道
对于模型B，优化后估计模型成本从至少36.21美元(一些原始运行在完成前达到阶梯限制)降至每跑1.24美元，减少了29x. GPT‐6.1 Sol的优化工作流程仍然便宜2.6x，为0.47美元。 优化工作流程中的每一个运行都完成了任务，并返回了正确的答案。
仅在GPT‐6.1 Sol，由于历史预算较大，新的缓存和截图政策将4x的费用从每跑1.97美元降至0.47美元。 每通电话约3x价格低廉，因为89%的输入来自缓存，价格为未标价的5%. 跑步也变得更快：在B型车的最初设置上至少22.5分钟，在GPT‐6.1 Sol的优化工作流程上大约4分钟。
调查还显示历史管理如何影响agent是否给出了答案。 给予 GPT%6.1 保留其浏览历史的空间更大一点，使产生答案的运行次数从历史预算较小的18个中的3个增加到预算较多的18个，每个都有正确的答案。 对伊达尔戈来说，业务价值正在让客户获得更快，更有能力的模型，同时保持运营成本的可持续性。
“成本用于限制我们能够为这些工作量提供客户的模型。 通过提高agent的效率，我们可以给客户一个更好，更快的模型，同时降低我们的运营成本".
Asana在StackAI中拥有对浏览器导航的修改的发布，并正在开发工具，使类似的实验更容易重复。 随着时间的推移，团队计划将这项测试纳入平台的评价，因此客户和内部团队可以在配置其agents时比较成本，运行时间，并回答质量。
“速度不再是瓶颈，人类的注意力就是瓶颈。 我们接近一个每个工程师都是总理的世界
Asana现在在 Codex 中使用 GPT‐6 Astra 在发布前测试产品特征：Astra 导航平台，尝试不同输入，为人类QA审查员报告错误。 Hidalgo认为这是新的软件开发生命周期的基础，许多云agent会话的测试功能是平行的。
Asana (在新窗口打开) 和 StackAI (在新窗口打开) 博客上都有完整的研究。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Asana cuts model costs 76x in browser tests with GPT‑6.1 Sol
Using GPT‑6 Astra in Codex, Asana made its browser agent 76x cheaper and 5x faster in tests to offer customers more capable models.
76x
Lower estimated model costs with the optimized GPT-6.1 Sol workflow
5x
Faster browser runs with the optimized GPT-6.1 Sol workflow
$0.47
Average estimated model cost with the optimized GPT-6.1 Sol workflow
With GPT‑6 Astra in Codex running experiments, Asana optimized its browser agent’s workflow on GPT‑6.1 Sol to run 76x cheaper and 5x faster.
Asana helps customers automate work across business applications through StackAI(opens in a new window), a platform it acquired(opens in a new window). Using StackAI, customers can build workflows that navigate websites, fill out forms, and gather information without writing code. At Asana’s scale, small inefficiencies in these workflows add up.
Asana’s StackAI CTO, Frank Hidalgo, PhD, set out to make the browser agent faster and cheaper to run. He directed GPT‑6 Astra in Codex to investigate the agent, test improvements, and compare the results. Work he estimates would have taken one to two months by hand took about a week.
Asana’s 144-run study(opens in a new window) tested GPT‑6.1 Sol and three other frontier models, called here Models A, B and C. The optimized workflow that emerged on GPT‑6.1 Sol averaged $0.47 in estimated model costs and about four minutes per run, 76x cheaper and 5x faster than the original production setup on Model B.
“This is what teams of humans and agents look like in practice. An engineer set the direction, GPT-6 Astra ran the experiments, and the results went through Command to production. This demonstrates how Asana brings human and agent teams to life.”
To move quickly, Hidalgo started by using GPT‑6 Astra in Codex to map the codebase and explain how the agent built each model request. GPT‑6 Astra discovered that the agent cached its fixed instructions and tool definitions, but not the growing history of page text and screenshots it gathered, so every request resent that history at full price.
The agent also dropped older screenshots and trimmed text at nearly every step. Each edit altered the history, so caching the history alone would not have helped, and losing those facts could require the agent to revisit pages it had already read.
Hidalgo reviewed GPT‑6 Astra’s proposed fixes and selected three to test:
- Extending caching to the agent’s browsing history
- Increasing the amount of text it could retain
- Removing screenshots in batches rather than at every step
GPT‑6 Astra began with quick tests to establish which variables mattered. Because the code was not designed for controlled experiments, it then refactored the code so one frontend and backend could support many workflows in parallel, each with its own settings.
Astra conducted the full study: history budgets of 120,000 and 480,000 characters and six caching and screenshot policies, each tested three times on each of the four models (see the table below). The best-performing policy allowed screenshots to accumulate to 20 before cutting back to the most recent one. This kept earlier history unchanged for longer stretches between removals. Combined with the larger history budget, it became the optimized workflow. Each configuration performed the same task: collecting six fields for each of 32 books from a public demo catalog, representative of what some Asana customers run in StackAI.
GPT‑6 Astra ran the workflows and examined the requests, usage records and outputs, and separate model sessions reviewed the work. Every session’s requests, data traces and results were recorded in Command(opens in a new window), Asana’s software delivery platform, so the team could review the complete study afterward. From Command, the findings were turned into tickets, then pull requests, and the changes went to production.
“This would have taken me one to two months by hand. With GPT-6 Astra in Codex, it took about a week: I’d set a /goal before going to bed and review the results in the morning.”
For Model B, the optimization reduced estimated model cost from at least $36.21 (some original runs reached the step limit before they finished) to $1.24 per run, a reduction of 29x. The optimized workflow on GPT‑6.1 Sol was 2.6x cheaper still, at $0.47. Every run in the optimized workflow completed the task and returned the correct answer.
On GPT‑6.1 Sol alone, with the larger history budget, the new caching and screenshot policy reduced cost 4x, from $1.97 to $0.47 per run. Each call was about 3x cheaper, because 89% of the input came from cache at 5% of the uncached price. Runs also became faster: at least 22.5 minutes on the original setup on Model B, roughly four minutes with the optimized workflow on GPT‑6.1 Sol.
The investigation also showed how history management affected whether the agent produced an answer at all. Giving GPT‑6.1 Sol more room to retain its browsing history increased the number of runs that produced an answer from three of 18 with the smaller history budget to all 18 with the larger budget, each with the correct answer. For Hidalgo, the business value is giving customers access to faster, more capable models while keeping operating costs sustainable.
“Cost used to limit which models we could offer customers for these workloads. By making the agent more efficient, we can give customers a better, faster model while lowering our operating costs.”
Asana has released the changes to browser navigation in StackAI and is developing tools to make similar experiments easier to repeat. Over time, the team plans to incorporate this testing into the platform’s evaluations, so customers and internal teams can compare cost, runtime, and answer quality when configuring their agents.
“Shipping speed is no longer the bottleneck; human attention is. We’re close to a world where every engineer is a PM leading a fleet of agents.”
Asana is now using GPT‑6 Astra in Codex to test product features before release: Astra navigates the platform, tries different inputs and reports bugs for human QA reviewers. Hidalgo views this as the foundation for a new software development lifecycle, with many cloud agent sessions testing features in parallel.
The complete study is available on the Asana(opens in a new window) and StackAI(opens in a new window) blogs.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.