中文版正在随流水线自动整理,稍后刷新即可;可先阅读下方英文原文。
本文由 AI图鉴自动抓取并翻译,版权归原作者所有;如需查看原文请点下方链接。
AI图鉴首页 › AI 资讯 › 僵尸统计:AI引用的数字来自何处
文章 URL: https://trakkr.ai/trakkr-research/zombie-stats 互联网档案馆的存檔,存档日期2013-12-22. 评论 URL:https://news.ycombinator.com/roble?id=50290283 评论点:67#评论点:28
中文版正在随流水线自动整理,稍后刷新即可;可先阅读下方英文原文。
本文由 AI图鉴自动抓取并翻译,版权归原作者所有;如需查看原文请点下方链接。
One number, followed.
We asked ChatGPT, Perplexity, Gemini, Claude and Google’s AI Overviews 200 questions that call for a number, the kind people ask every day.Note 1A statistic is a share, rate, average, count or cost that an assistant states as fact. We left out advice (“save 20% of your income”) and prices. Then we clicked the citations, and kept clicking. Here is one trail, from the answer to the end.
Google AI Overviews
What percentage of salespeople give up after one follow-up?
44% of salespeople give up after just one follow-up attempt.
- 44%
- 44%
- 44%
- 44%
- 44%
44% of salespeople give up after one follow-up attempt (source).
80 Percent of Sales Require 5 Follow-up Calls after Meeting; 44 Percent Salespeople Give Up After One Follow-up
No study behind itNo survey, no sample, no date.
- In the answer Every assistant got the same question: “What percentage of salespeople give up after one follow-up?” All five assistants gave the same figure, and only ChatGPT said the evidence behind it was unclear.
- One click in Google AI Overviews cited a sales firm’s stats page, from 2020. The number is right there. Its “source” link goes to a writing sample.
- Two clicks in That leads to a freelance writer’s sample article, online by 2017. The number is in its headline. It lists two slide decks, a blog list and two news stories. None is a study of follow-ups.
- The end of the trail That’s where it ends. No survey, no sample, no study: every other route to this number we tried stopped at a page that cites no research. It’s a number still walking around with nothing underneath it. That’s a zombie stat.
Most numbers aren’t zombies. But you can’t tell from the answer: a zombie arrives with the same confidence and the same citation, and here, with all five assistants agreeing. So we did the same for 1,285 statistics.
Fewer than half get there.
We took a random sample of the statistics each assistant cited, 1,285 in all, and walked every trail the same way: the cited page, then that page’s source, up to five pages deep.Note 2Each assistant’s sample was drawn at random from the statistics it cited: ChatGPT 108, Perplexity 277, Gemini 300, Claude 300, Google AI Overviews 300. The five-assistant rate weights each assistant by how many cited statistics it quoted, so it describes the statistics people were shown. Where a page named a source without a link, we searched for it.
40% reached a study that says the same thing. Another 8% reached the study, but the number or its context had changed on the way. The rest never got there: 40% stopped at a source we couldn’t check, because it was locked, gone or silent on the number, and 11% led nowhere at all.Note 3An AI model judged every trail, and a stronger one re-judged 300. Then people re-checked 150 trails at random, with a third check wherever the two disagreed. People found originals the model missed: by the model alone, 23% were sourced. Every rate here is corrected with the hand checks, and its range carries the uncertainty from both.
ChatGPT did best, and quoted the fewest numbers: 1.3 per answer, against 6.8 for Claude. The other four landed between 38% and 46%, too close to rank.
It isn’t a quirk of our questions. Of 300 statistics in Google AI Overviews answers to real everyday searches, 40% were sourced, the same share.Note 4Google AI Overviews answers to 52,930 everyday English searches that Trakkr tracked between 25 September 2026 and 25 October 2026. 4.4% carried a statistic, and we traced 300 of those the same way. Only counts and rates leave our systems.
AI cites the copy, not the study.
Most of the time, the AI wasn’t making numbers up: the page it cited had the figure 82% of the time. The trouble is which page. Only 21% of the pages cited for a number were the research itself.
More than half were company blogs. Add news stories and statistics roundups, and most citations land on pages built to collect figures from elsewhere, often without a link back.
Where the trail starts decides where it ends. From a research firm or a government page, it reached a matching study 61% of the time. From a company blog, 37%; from a statistics roundup, 35%. Even research firms fell short 38% of the time, often because the full report sat behind a form.
A citation tells you where the AI found a number, not where the number came from.
Numbers change on the way.
Reaching the study didn’t always mean the number held. Of the statistics that got there, 17% had changed on the way. The digits moved only about one time in four. Usually it was the context: what was measured, the year, the source, or who was asked.
What changed, across every change we logged
- The number changed
- 26%
- What was measured changed
- 20%
- The year changed
- 17%
- The source changed
- 16%
- Who was asked changed
- 14%
- The place changed
- 4%
Fifteen women judging health websites become every website’s first impression. Three companies become a rule of business. Here are the clearest trails we followed, each one checked by hand.Note 5We name the original studies. The pages that repeated them are described by type, not named. Every step in these trails was checked by hand on the open web, with archived copies kept.
The zombies.
Ask five assistants the same question and you’ll often get five different numbers. On 28 of the 71 questions where all five gave a percentage, we got four or five different figures.Note 6We compared each assistant’s headline figure for the same question. A range counts at its middle, and figures within 2 points count as the same.
It wasn’t one odd question. The typical gap between the highest figure and the lowest was 16 points.
Agreement feels like proof. It isn’t. All five gave the same figure on 10 questions, and of the 5 we traced, 2 were sourced. The statistics they repeated most, quoted by three or more, did no better: 11 of 29.
And four were quoted by all five assistants, with no study anywhere behind them.
What to do with an AI statistic.
Treat it as a lead, not a source. Six habits, three for each side of the citation.
If you use them
- Follow the citation, then follow it again. If the cited page is a list of statistics or a company blog, use its own link to go one page further. Keep going until you reach the study itself, or until you can’t.
- Check who was asked, how many, and when. Most of the changes we found were in the context, not the digits: one country’s poll, a ten-year-old study, fifteen people.
- Don’t take agreement as a check. Five assistants giving the same number isn’t five sources. The numbers they repeated most were no better sourced than the rest.
If you publish them
- Put the method next to the number. The sample, the date and a link to the source, on the same page as the figure, so the next page, and the next answer, can trace it back to you.
- Keep the findings in the open. Reports behind a form were a common dead end: even trails that started at a research firm fell short 38% of the time. Put the headline numbers and the method on an open page.
- See which page AI cites for your numbers. It is often a roundup or a blog that copied you, sometimes with an older figure. That page decides what people hear.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.
Blumify:音频语音方向的 AI 工具,Hacker News 新品(热度 90)。
BreadcrumbBreadcrumb:效率方向的 AI 工具,Hacker News 新品(热度 32)。
HandwiredHandwired:编程开发方向的 AI 工具,Hacker News 新品(热度 25)。
L.O.S.S. AIL.O.S.S. AI:效率方向的 AI 工具,Hacker News 新品(热度 25)。
免费版:每日 AI 搜索额度
InstaBox5 台免费云电脑(每台 4 vCPU/8GB)
E2BAgent 沙箱:注册送 $100 赠金
DaytonaAgent 沙箱:注册送 $200 算力
Hacker News · 10-29
Grace Blackwell 需要一个新的排程器:27% 的可用GPU 来自相同的架子Hacker News · 10-29
LLM 语音Hacker News · 10-29
Gemini 4 Carbon 是人工分析智能指数v5上的新1号Hacker News · 10-29
美国在短短3个月内增加了20.2千瓦的电池储存记录Hacker News · 10-29
链式PHP2壳开发根 25 服务器,包括美国政府服务器Hacker News · 10-29
标题、摘要与正文均来自该信源的公开页面,版权归原作者所有;本站自动聚合与翻译,仅作出处跳转。