Token 成本为何不告诉你企业AI是否工作

中文版正在随流水线自动整理，稍后刷新即可；可先阅读下方英文原文。

本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。

Thought Leaders

Why Token Costs Don’t Tell You Whether Enterprise AI Is Working

Add Unite.AI to your preferred sources on Google

Enterprise AI is moving through a familiar cycle. At first, organizations went all in, reaching for the largest and most capable models because the technology was exciting and the bill had not yet arrived. When it did arrive, the response was equally predictable: teams began rationing access, reducing experimentation, and searching for less expensive models.

Ali Ghodsi, cofounder and CEO of Databricks, has described the early stage of this cycle as “tokenmaxxing,” where consuming more tokens is treated as evidence of progress. The more important stage is “valuemaxxing,” which means understanding what that spending produces and directing it toward the work that matters.

This is not simply a new name for cost optimization. Cost optimization asks how an organization can spend less, while valuemaxxing asks how it can get more from what it spends. That distinction will become increasingly important as AI moves from isolated experiments into everyday enterprise operations.

The Largest Model Is Not Always the Right Model

Many organizations started their AI programs by using the most powerful model available for almost every task. This approach made sense during the early experimental phase, when teams were trying to understand what the technology could do and model capability was more important than predictability.

As deployments grew, however, the cost of using a frontier model for every request became difficult to justify. Some organizations reacted by limiting access or asking teams to do less, but broad restrictions can suppress useful experimentation along with genuine waste.

A more mature approach is emerging. Teams are beginning to match smaller, specialized models to routine work while reserving more capable models for tasks that require complex reasoning or carry a high cost of failure. Over time, some organizations may also deploy models within their own infrastructure to gain greater control over cost, performance, and sensitive data.

The important question is not whether a large or small model is better. It is whether the model is appropriate for the work and whether the result justifies the total cost of producing it.

Tokens Show Activity, Not Value

Most enterprise AI dashboards are built around consumption. They show how many tokens were used, which models handled them, and how much the organization spent. Those numbers are necessary, but they do not reveal whether the work produced a useful result.

A team that consumes twice as many tokens is not necessarily creating twice as much value. It may be handling more difficult work, using an unnecessarily powerful model, or relying on an agent that repeatedly retries failed tasks. A workload with high model costs may still represent an excellent investment if it prevents outages, resolves customer issues, or gives employees back hours of valuable time.

The same problem has existed in observability for years, only with gigabytes instead of tokens. Teams began by collecting everything because maximum visibility appeared to be the safest strategy. When storage and processing bills increased, they started sampling data, reducing retention, and dropping context to control the expense.

In both cases, the response to rising cost has often been to do less. What organizations need instead is the ability to identify which activity is valuable, which is wasteful, and which could become more efficient without sacrificing the outcome.

The Real Cost Extends Beyond Inference

A model call is rarely the complete AI workload. An agent may retrieve documents, query a database, search telemetry, call external services, and generate new application or infrastructure activity before producing an answer.

Consider an agent investigating why checkout latency increased after a deployment. It might examine logs, compare metrics, inspect traces, retrieve deployment records, and analyze downstream services. If its first theory fails, it may adjust the time window, search under different service names, and follow several alternative causes.

One human instruction can therefore create hundreds of machine-generated operations. Those operations consume compute, storage input and output, network capacity, and third-party API quotas, none of which may appear in the organization’s model-spending dashboard.

This means a seemingly inexpensive model can support an expensive workflow. A more capable model might cost more per call but reach the correct conclusion in fewer steps, requiring fewer retries and less supporting infrastructure. The only way to make that comparison is to observe the complete task.

Connect Activity, Cost, and Outcomes

Valuemaxxing requires organizations to connect three types of information that are often measured separately.

The first is AI activity, including model selection, token consumption, latency, tool calls, retries, and parallel branches. The second is the infrastructure and financial cost created by that activity, including compute, storage, network use, database load, and external API charges. The third is the outcome, which should indicate whether the work was completed successfully and whether anyone found the result useful.

Not every outcome needs to be translated into an exact dollar amount. A practical measure may be enough, such as whether an answer was accepted, whether a person had to redo the work, whether an incident was resolved more quickly, or whether an automated workflow completed without intervention.

For example, an incident investigation that generates hundreds of queries may be worthwhile if it helps engineers restore a critical service 30 minutes sooner. A much smaller workflow may still be wasteful if it produces a report that nobody reads. Token consumption alone cannot distinguish between the two.

Once organizations can connect activity with outcomes, they can make deliberate decisions about where to use frontier models, where smaller models are sufficient, and where a workflow needs to be redesigned.

Visibility Should Not Require Surrendering Control

Understanding AI value requires access to sensitive context. Prompts, completions, source code, customer information, and production telemetry may all be part of the same workflow.

Organizations should not have to export this information to an outside service simply to understand how their AI systems are performing. The prompts and production context that make measurement useful may be the same information that security, privacy, and compliance teams cannot allow to leave the organization’s environment.

Enterprise leaders should therefore ask where their AI and observability data is processed, who can access it, how long it is retained, and whether the analysis can happen within their existing security boundary. Good governance depends on visibility, but that visibility must not create a new data-control problem.

Keeping analysis close to the underlying data also makes it easier to connect AI activity with the applications, infrastructure, and business processes it affects. That complete view is what allows teams to understand not only what they spent, but what the spending returned.

Optimize for Value, Not Volume

The organizations that succeed in the next phase of enterprise AI will not necessarily be those that use the most tokens or spend the least. They will be the ones that can see which models and workloads produce meaningful outcomes, then use that knowledge to guide deployment.

This requires moving beyond isolated model-cost dashboards. Teams need to observe the full path from a user request through model calls, tools, infrastructure, and the eventual result, while keeping sensitive information under appropriate organizational control.

Tokenmaxxing was a useful phase because it encouraged experimentation and revealed what AI could do. Valuemaxxing is the discipline that must follow, combining visibility, cost, and outcomes so organizations can keep experimenting without losing control of their spending or their data.

Enterprise AI does not need to become smaller simply because the first bills have arrived. It needs to become more deliberate, with effort matched to difficulty, models matched to tasks, and spending matched to measurable value.

Auto-fetched and translated by AI Atlas. All rights belong to the original author.