86%的字,你可能只懂38%的动画线
报道会告诉你口语词的百分比,而不是你能听多少行。 以弗里伦和其他7个动画测量。
说想看Frieren,所以学习节目中最常用的500个单词。 一个覆盖计算器称你现在知道85.9%的语句。
你做茶和新闻播放。 以下是你听到的三件事:
模糊字落于前五百门外。 跳过或敲击来揭示他们。
到了第三次暂停找一个字,你的茶已经凉了。 有了这500个字,你就知道前3000行中的1,151个字。 这是38.4%。 其他1,849行各包含至少一个你不知道的词。
两个数字都是真的
覆盖面是大多数学习网站引用的数字。 是口语的百分率,你知道的,所以说一千遍的粒子数到一千次。 Frieren列表中最顶端的词都是语法片段,比如\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\ 你只会听8.5%的台词
三个数字说明了同样的500个字:
manga-kotoba以这种方式建立其覆盖表。 jpdb和jiten也显示一个覆盖数字,虽然他们的帮助页面没有给出准确的公式。
覆盖范围与线路之间的差距来自线路长度。 一条弗里伦线平均有7个字。 须知七根随顺。 在85.9%的覆盖率,你了解整条线路的机会约为35%. 这是0.859乘以自身7倍。 它接近我们测得的38.4%的线路。
试试看你想看的节目
厚线:在所选节目中理解的线条。 骗局:该节目的报导。 暗线:其他7条理解的线。 对于每个曲线,我们假设你先学会那个节目最常用的词。
所有8个显示都有这个缺口。 连载实验 Lain 覆盖率达到95% 最少的字,1,191. 即使这样,你也会只听77.5%的台词。
有些显示需要的字比其他的要少得多
串行实验莱恩需要8个字中最少的一句话:大约400个字可以得到你一半的台词。 朱朱苏凯森需要980.
我们以3000条台词封杀每场演出,这样长篇演出就不会因为有更多剧集而更难看。 如果晚点节目变难,这里看起来比它容易得多。
你错过的许多台词都只差一个字
复至佛来伦。 你错过了1 849行。 其中1 042个字缺失,为56.4%。 在所有显示中,这一数字从50.2%上升到63.9%。
从500字到1000字帮助你远远超出报道暗示。 覆盖面只增加6.5分,因为这些新词不常出现。 线条理解得22.1分,因为一个新词往往是你需要尾随行的最后一个词。
你不能用几个字来填补这些单词的空白。 这1,042行遗漏了804条不同的词。 从一个单词得到的最多是4行,来自QQ(regret),QQ(保证)和QQ(收集)等单词。
两千个普通的日语单词最多能让你得到38%的行数
作为粗略的比较,我们从单词freq列表中尝试了普通日语中最常用的2000个单词。 他们给你23.4%到37.6%的线路。 然后我们试演了两千个单词他们得到74.3%到93.3%的线路。 连环实验 Lain 的字数不到2000个,所以它不在第二个范围。
一个节目的顶级词可以是不寻常的。 弗里伦榜上第304位是QQ,一位不守誓言的和尚,这就是人物们一直称海特尔。 21号为Q(魔力),4位领先于H.
前几集的榜单以后还有三分之二的工作
说你从一个赛季学到一个节目的顶级词,然后继续到下一个赛季。 我们在四座显示测试了近于此的东西有足够的线路我们从最初的3000条线路中建立了每个清单然后在接下来的3000条线路上测试在弗里伦接下来的3000条线路上,旧的榜单获得22.3%,而从这些线路本身构建的榜单获得33.0%. QQ(exam)一词只出现在这些第一行中一次,在他们之后出现51次,因为角色稍后会参加mage考试。 穿过显示,下降5.5比14.0点。
几个意思的词使这些数字过高
许多词有几种含义,语言学家称之为多语种。 我们计算出一个词以已知的,以表示行用。 在500个单词中,这两个弗里伦行都算作理解:(我的手在颤抖)和手に取るように分かる(我能像书一样阅读). 你可以完全从第一行知道仍然在第二行迷路一字不差地说,它称"仿佛把它拿在手中". 我们没有测量有多少行有这个问题。 光是这一点,人们所了解的线路数字就太高了。
我们怎么做到的
食谱
——通过删除歌曲,格式化标签,括号中的任何内容和音乐指出来清理字幕。 将文字分为句号行。 删除重复行。
- 将每行都与日本免费的标致器Sudachi分割成单词。 我们使用SudachiPy 0.6.11与SudachiDict-core 20260428在分裂模式C. Our Python版本为3.13.7. 使用每个单词的词典表,因此QQ(went)和QQ(go)算作一个单词。
- 跳过标注,数字和任何东西 Sudachi标签作为一个适当的名称。
- 跳过填充器,如 (shh)和ハハッ(haha)。 当Sudachi将其标记为插音、副词或名词时,声音才算作填充符。 我们还在计算结束语法,它有一个不同的标记。 文章和日常回复
- 放下没有字的台词,按插曲排序,保留前3000行。
- 计算每个字出现的次数,然后排列它们。 按字母顺序断开领带。
- 把顶尖的N字当成已知覆盖范围是已知的单词事件除以所有单词事件。 所理解的行是每个词已知的行的百分比。
整个事情在Python的29行
管道安装 sudachipy sudachidict core
# lines.txt:每行1条字幕线,按插曲顺序,歌曲和重复被删除
# 填充器.txt:您自己的不计数声音列表,每行一个
从收藏进口柜台
从 sudachipy 导入词典,拆分模式
导入重置
tok = 词典 (.) 创建 ()
填充器=设置(打开("填充器.txt",编码="utf-8")). read (.split ()).
def 单词( 行):
输出 = []
m 单位为tok.tokenize(行,SplitMode.C):
pos = m.part_of_speech()
如果 pos [0] 在 ("QQ", "XXX", "XXX" ) 中:继续 # puntuation
如果 pos [1] 在 (“ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
如果不是再研究(r"[0-9A-Za-zぁ-コト一-々-々]]),m. surface():继续
如果填充器中的m.表面()和 pos[0] 中的(“+”、“+”、“+”、“+”):继续
输出。 附录( m. districtary form())
返回出来
线条 = [l.strip ()] 表示l在开口的("lines.txt",编码="utf-8").
线条 = [l for l in if l][3: 00] # 前3000行有真词
计数 = 计数器(w 表示行为w)
known = {w for w, _ in sorted(count.items(), key=lambda x: (-x[1], x[0]))[:500]}
覆盖=和(n为w,n为计数。 items()如果w为已知)/和(count.vales())
understand=和 (所有(w)在行中为w)在行中为 / len(行)
打印(f) 覆盖:{覆盖:1. } 行理解:{ 理解:1.
运行在我们的弗里伦线上,这个打印率为85.9%和38.4%. 我们不存储未处理的字幕文件。 自己的字幕和填充列表会给您带来略有不同的数字。
所有的数字
每个细胞的显示覆盖,然后行理解,在那许多已知的词。
已知的词语需要达到一定比例的行理解:
怎么会这样呢?
- "明白"意味着你懂每个字它忽略了语法,坎吉读数,说话速度和上下文。
- 我们假设你完全知道节目中最常用的词并且没有忘记任何词如果你知道其他的字,或者忘记一些,这些数字会更低。 在观看之前你还需要一份节目的词表。
- 每条不同的线都算一次在节目中,你会听到股票线像QQ(Yeah) 一次又一次,这可能会提高线理解。 我们没有测量这个。
- 台词来自每个节目的最早剧集,没有跳过任何剧集。 链锯人和串行实验 Lain的线路不足3000条,所以我们使用了他们所有的线路。 线条较少意味着得分较高。 在500个单词中,其他显示在切到莱恩长度时得分高达4.8分。 系列实验 Lain的字幕质量较低,因此其数字可靠性较低。
- 一些字幕文件重复了略微不同措辞的行,我们只删除了确切的重复。 删除密件太修改了500字理解的行数,最多为2.2点。
- 决定什么才算填充者是判决如果将填充器算作单词,500字理解的行最多有2.7点之差。 变化是双向的,因为我们数线是转移的。
- 计算所听到的表格,使QQ和QQ分别计算,将5.3至7.7分的行数降低500字。
- 一个词无论它有多重要都算得上一个词这使得它覆盖和线条理解都过高我们没有计算每一行的用意。 作为部分测试,我们分别计算了萨达奇语给出的每个部分的一个单词,将用500个单词的1.2到2.5点理解的线条降低。 这一测试只捕捉到名词与动词的区别。
- 杉田会犯错的它有时会把削减一个字变成碎片。 它也把一些名字当作普通的词:QQ(Frieren)是她自己名单上的第31位。 计算每一个词Sudachi在500个单词中最多0.6分时都认不出已知的新增。
学习fromx是一个闪卡应用,从动漫中教日语。 每张卡片都使用一条你除了一个新词之外什么都懂的线,这就是我们开始计算线的原因。 上面的数字计算每个词一次,无论它有多重要。 这在应用程序里是行不通的:当某人在称时,知道QQ是"那"是没用的?这是两张单独的牌。 还没找到每个这样的词我们正在努力在早期访问期间,它提供一个节目,Hunter X Hunter.
本文由 AI图鉴自动抓取并翻译,版权归原作者所有;如需查看原文请点下方链接。
With 86% of the words, you might understand only 38% of an anime's lines
Coverage tells you the percentage of spoken words you know, not how many lines you can follow. Measured in Frieren and seven other anime.
Say you want to watch Frieren, so you learn the 500 most common words in the show. A coverage calculator says that you now know 85.9% of the words spoken.
You make tea and press play. Here are three of the things you hear:
Blurred words fall outside the top 500. Hover or tap to reveal them.
By the third pause to look up a word, your tea has gone cold. With those 500 words, you know every word in 1,151 of the first 3,000 lines. That's 38.4% of them. The other 1,849 lines each contain at least one word you don't know.
Both numbers are true
Coverage is the number most learning sites quote. It's the percentage of spoken words that you know, so a particle spoken a thousand times counts a thousand times. The words at the very top of the Frieren list are all grammar pieces like の (of, 's) and は (topic marker), so the top 100 words alone give you 68.3% coverage. You'd follow only 8.5% of the lines.
Three figures describe the same 500 words:
manga-kotoba builds its coverage tables this way. jpdb and jiten show a coverage figure too, though their help pages don't give the exact formula.
The gap between coverage and lines comes from line length. A Frieren line has 7 words on average. You need to know all seven to follow it. At 85.9% coverage, your chance of knowing a whole line is about 35%. That's 0.859 multiplied by itself seven times. It's close to the 38.4% of lines we measured.
Try it on a show you like
Thick line: lines understood in the chosen show. Dashed: coverage in that show. Pale lines: lines understood in the other seven. For each curve, we assumed you learn that show's most common words first.
All eight shows have this gap. Serial Experiments Lain reaches 95% coverage with the fewest words, 1,191. Even then, you'd follow only 77.5% of the lines.
Some shows need far fewer words than others
Serial Experiments Lain needs the fewest words of the eight: about 400 gets you half the lines. Jujutsu Kaisen needs 980.
We capped every show at 3,000 lines so that a long show doesn't look harder for having more episodes. If a show gets harder later on, it looks easier here than it is.
Many of the lines you miss are only one word away
Back to Frieren. You missed 1,849 lines. Of those, 1,042 are missing a single word, which is 56.4%. Across all the shows, that figure runs from 50.2% to 63.9%.
Going from 500 to 1,000 words helps you far more than coverage suggests. Coverage only gains 6.5 points, because these new words don't come up often. Lines understood gain 22.1 points, because one new word is often the last one you needed to follow a line.
You can't fill these one-word gaps with a handful of words. Those 1,042 lines are missing 804 different words between them. The most you get from one word is 4 lines, from words like 後悔 (regret), 約束 (promise) and 収集 (collecting).
2,000 common Japanese words get you at most 38% of the lines
As a rough comparison, we tried the 2,000 most common words of general Japanese, from the wordfreq list. Those got you 23.4% to 37.6% of lines. We then tried each show's own top 2,000 words. They got 74.3% to 93.3% of lines. Serial Experiments Lain has fewer than 2,000 different words, so it isn't in that second range.
A show's top words can be unusual. Number 304 on the Frieren list is 生臭坊主, a monk who doesn't keep his vows, which is what the characters keep calling Heiter. Number 21 is 魔法 (magic), four places ahead of 私 (I).
A list from earlier episodes still does two-thirds of the job later on
Say you learn a show's top words from one season, then carry on to the next. We tested something close to this on the four shows with enough lines. We built each list from the first 3,000 lines, then tested it on the next 3,000. On Frieren's next 3,000 lines, the old list gets 22.3%, while a list built from those lines themselves gets 33.0%. The word 試験 (exam) appears only once in those first lines and 51 times after them, because the characters take a mage exam later on. Across those shows, the drop is 5.5 to 14.0 points.
Words with several meanings make these figures too high
Many words have several meanings, which linguists call polysemy. We counted a word as known whichever meaning a line uses. At 500 words, both of these Frieren lines count as understood: 手が震えています (my hands are shaking) and 手に取るように分かる (I can read it like a book). You could know 手 (hand) perfectly from the first line and still be lost in the second. Word for word, it says "as if taking it in your hand." We didn't measure how many lines have this problem. On this count alone, the lines-understood figures are too high.
How we did it
The recipe
- Clean the subtitles by removing songs, formatting tags, anything in brackets and music notes. Split the text into sentence-sized lines. Remove duplicate lines.
- Split every line into words with Sudachi, a free Japanese tokenizer. We used SudachiPy 0.6.11 with SudachiDict-core 20260428 in split mode C. Our Python version was 3.13.7. Use each word's dictionary form, so 行った (went) and 行く (go) count as one word.
- Skip punctuation, numbers and anything Sudachi tags as a proper name.
- Skip fillers like シッ (shh) and ハハッ (ha ha). A sound only counts as filler when Sudachi tags it as an interjection, adverb or noun. We still count the grammar ending ん, which has a different tag. Particles and everyday replies like うん (yeah) still count as words.
- Drop lines with no words left, sort by episode and keep the first 3,000 lines.
- Count how often each word appears, then rank them. Break ties alphabetically.
- Treat the top N words as known. Coverage is known word occurrences divided by all word occurrences. Lines understood is the percentage of lines where every word is known.
The whole thing in 29 lines of Python
# pip install sudachipy sudachidict_core
# lines.txt: one subtitle line per row, in episode order, songs and repeats removed
# fillers.txt: your own list of sounds not to count, one per row
from collections import Counter
from sudachipy import Dictionary, SplitMode
import re
tok = Dictionary().create()
fillers = set(open("fillers.txt", encoding="utf-8").read().split())
def words(line):
out = []
for m in tok.tokenize(line, SplitMode.C):
pos = m.part_of_speech()
if pos[0] in ("補助記号", "空白", "記号"): continue # punctuation
if pos[1] in ("固有名詞", "数詞"): continue # names, numbers
if not re.search(r"[0-9A-Za-zぁ-ヿ一-々]", m.surface()): continue
if m.surface() in fillers and pos[0] in ("感動詞", "副詞", "名詞"): continue
out.append(m.dictionary_form())
return out
lines = [words(l.strip()) for l in open("lines.txt", encoding="utf-8")]
lines = [l for l in lines if l][:3000] # the first 3,000 lines with a real word
count = Counter(w for line in lines for w in line)
known = {w for w, _ in sorted(count.items(), key=lambda x: (-x[1], x[0]))[:500]}
coverage = sum(n for w, n in count.items() if w in known) / sum(count.values())
understood = sum(all(w in known for w in line) for line in lines) / len(lines)
print(f"coverage: {coverage:.1%} lines understood: {understood:.1%}")
Run on our Frieren lines, this prints 85.9% and 38.4%. We don't store the unprocessed subtitle files. Your own subtitles and filler list will give you slightly different numbers.
All the numbers
Each cell shows coverage, then lines understood, at that many known words.
Known words needed to reach a given percentage of lines understood:
What could make this wrong
- "Understood" means you know every word. It ignores grammar, kanji readings, speaking speed and context.
- We assume you know exactly the show's most common words and haven't forgotten any. If you knew other words instead, or forgot some, these figures would be lower. You'd also need a list of the show's words before watching.
- Each different line counts once. In the show you'd hear stock lines like うん (yeah) again and again, which would likely raise lines understood. We didn't measure this.
- The lines come from each show's earliest episodes, with no episode skipped. Chainsaw Man and Serial Experiments Lain have fewer than 3,000 lines, so we used all of theirs. Fewer lines mean a higher score. At 500 words, the other shows score up to 4.8 points higher when cut to Lain's length. Serial Experiments Lain's subtitles are lower quality, so its figures are less reliable.
- Some subtitle files repeat a line in slightly different wording, and we only removed exact duplicates. Removing the close copies too changes lines understood at 500 words by at most 2.2 points.
- Deciding what counts as filler is a judgment call. If you count fillers as words, lines understood at 500 words differ by 2.7 points at most. The change goes in both directions because the lines we count shift.
- Counting forms as heard, so that 行った and 行く count separately, lowers lines understood by 5.3 to 7.7 points at 500 words.
- A word counts once however many meanings it has, which makes both coverage and lines understood too high. We have no count of which meaning each line uses. As a partial test, we counted a word separately for each part of speech Sudachi gives it, which lowers lines understood by 1.2 to 2.5 points at 500 words. That test only catches differences like noun against verb.
- Sudachi makes mistakes. It sometimes cuts a word into a fragment. It also treats some names as ordinary words: フリーレン (Frieren) is number 31 on her own list. Counting every word Sudachi doesn't recognize as known adds at most 0.6 points at 500 words.
learnfromx is a flashcard app that teaches Japanese from anime. Each card uses a line where you know everything except the one new word, which is why we started counting lines. The figures above count each word once, however many meanings it has. That wouldn't work in the app: knowing あれ as "that" doesn't help when someone says あれ? (huh?). These are two separate cards. It doesn't catch every such word yet, we're working on it. During early access it offers one show, Hunter x Hunter.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.