客户甩过来一份200页的产品手册PDF："帮我们做个能查这个的AI助手。"
我的第一反应：简单，第3篇RAG搭过了——切chunk、存向量库、检索问答，搞定。
结果上线第一天就被投诉：查表单数据查不到，跨章节的对比问题答不对，"帮我总结全文"直接超出token限制。
200页文档和10页文档，不是"量变"，是"质变"。 长文档处理的坑，远比想象中多。
先说结论
Java人的理解：长文档 ≈ 大数据量查询。 你不会把100万条记录全SELECT出来再处理，而是先WHERE过滤再聚合。AI处理长文档也是：先检索筛选再生成，不是一股脑全塞。
坑1：PDF提取出来全是乱码和断行
翻车现场
用PyPDF2提取200页PDF：
from PyPDF2 import PdfReader
reader = PdfReader("product-manual.pdf")
text = ""
for page in reader.pages:
text += page.extract_text()
结果：
产品名称 型号 价格
---------- ----- -----
智能门锁 A200 899
（续表）
颜色：深空灰 重量：2.3kg
表格被打散成文本流，跨页表格直接断开，多栏排版左右混排。 用这种数据做RAG，检索效果可想而知。
正确做法：用专用PDF解析工具
# 方案1：unstructured（推荐，支持表格/图片/布局）
from unstructured.partition.pdf import partition_pdf
elements = partition_pdf(
"product-manual.pdf",
strategy="hi_res", # 高精度模式
infer_table_structure=True, # 识别表格结构
extract_images_in_pdf=True, # 提取图片
)
# 按类型处理
for element in elements:
if element.category == "Table":
# 表格保留HTML结构
table_html = element.metadata.text_as_html
# 转成Markdown表格更好读
table_md = html_table_to_markdown(table_html)
elif element.category == "Title":
# 标题信息
title = element.text
elif element.category == "NarrativeText":
# 正文
text = element.text
方案2：Marker（PDF转Markdown，效果最好）
# pip install marker-pdf
from marker.converters.pdf import PdfConverter
from marker.models import create_model
from marker.output import text_from_rendered
converter = PdfConverter(artifact_dict=create_model())
rendered = converter("product-manual.pdf")
markdown_text, metadata = text_from_rendered(rendered)
Marker的优势： 把PDF转成结构化Markdown，表格保留为Markdown表格，标题层级完整，代码块完整。直接用第23篇的Markdown结构切分就行。
成本对比：
坑2：跨章节问题答不准——检索只看片段不看上下文
翻车现场
用户问："A200和A300有什么区别？"
A200的参数在第3章，A300的参数在第5章。检索top-4 chunk：
Chunk 1: "A200智能门锁，支持指纹解锁..." (第3章，sim=0.78)
Chunk 2: "A200技术参数：重量2.3kg..." (第3章，sim=0.72)
Chunk 3: "常见问题：如何重置密码..." (FAQ章，sim=0.45)
Chunk 4: "售后服务政策..." (第8章，sim=0.41)
A300的信息根本没检索到！ 因为用户的问法里只提了"A200和A300"，而A300那个chunk的相似度没进top-4。
正确做法：父文档检索（Parent Document Retriever）
核心思想：用小chunk做检索（精准），命中后返回大chunk（完整上下文）给LLM。
from langchain.retrievers import ParentDocumentRetriever
from langchain.storage import InMemoryStore
# 小chunk：用于精准检索
child_splitter = RecursiveCharacterTextSplitter(chunk_size=200)
# 大chunk：给LLM看完整上下文
parent_splitter = RecursiveCharacterTextSplitter(chunk_size=800)
# 向量库存小chunk，文档库存大chunk
vectorstore = Chroma(embedding_function=embeddings)
docstore = InMemoryStore()
retriever = ParentDocumentRetriever(
vectorstore=vectorstore,
docstore=docstore,
child_splitter=child_splitter,
parent_splitter=parent_splitter,
)
# 检索时：用小chunk匹配，返回对应的大chunk
docs = retriever.invoke("A200和A300有什么区别？")
# 返回的是包含A200和A300信息的完整段落
为什么有效？ 小chunk（200字符）匹配更精准——"A300"这个关键词在小chunk里密度更高。匹配到后，把包含这个小chunk的大chunk（800字符）取出来，大chunk里有A300的完整参数。
进阶：多向量检索——一个文档存多个检索向量
from langchain.retrievers.multi_vector import MultiVectorRetriever
# 对每个大chunk生成摘要，用摘要做检索向量
summaries = []
for doc in large_chunks:
summary = llm.invoke(f"用一句话总结：{doc.page_content}")
summaries.append(summary)
# 向量库存摘要向量，文档库存原文
retriever = MultiVectorRetriever(
vectorstore=vectorstore,
docstore=docstore,
id_key="doc_id",
)
# 摘要向量 → 检索命中 → 返回对应原文
效果： 用户问"A200和A300区别"，摘要向量里"A200和A300智能门锁对比"精准命中，返回包含两者完整参数的原文。
坑3：全文摘要——token超限或摘要不全
翻车现场
用户问"帮我总结一下这份手册的核心内容"。
方案1：把200页全文丢给LLM → token超限（200页≈50万token） 方案2：只取前10页摘要 → 漏掉后半部分内容 方案3：分章摘要再合并 → 每章摘要还行，合并时又超限
正确做法：Map-Reduce分片摘要
from langchain.chains.summarize import load_summarize_chain
# Map-Reduce：先分片摘要，再合并
chain = load_summarize_chain(
llm,
chain_type="map_reduce",
verbose=True
)
# 自动处理：分片→各片摘要→合并摘要
summary = chain.run(docs)
Map-Reduce的问题： 合并时可能丢失细节。更好的方案是Refine（渐进式精炼）：
chain = load_summarize_chain(
llm,
chain_type="refine",
verbose=True
)
# Refine逻辑：
# 第1个chunk → 生成初始摘要
# 第2个chunk + 当前摘要 → 更新摘要
# 第3个chunk + 当前摘要 → 更新摘要
# ...
# 最终摘要不断被"精炼"，越来越完整
Refine vs Map-Reduce：
200页手册用Refine更合适： 串行处理虽然慢，但摘要更完整。
实际优化： 先用Map-Reduce做每章摘要，再用Refine合并章节摘要——两阶段处理：
# 阶段1：每章独立摘要（可并行）
chapter_summaries = []
for chapter in chapters:
summary = llm.invoke(f"用3句话总结以下内容：\n{chapter}")
chapter_summaries.append(summary)
# 阶段2：合并章节摘要（Refine）
final_summary = chapter_summaries[0]
for i in range(1, len(chapter_summaries)):
final_summary = llm.invoke(
f"当前摘要：{final_summary}\n\n"
f"新内容：{chapter_summaries[i]}\n\n"
f"将新内容融入当前摘要，保持完整性和连贯性。"
)
坑4：长上下文模型的陷阱——能塞进去不等于能用
翻车现场
2025年模型上下文越来越长：GPT-4o 128K，Claude 200K，Gemini 1M。
"那我直接把200页文档塞进prompt不就行了？不用RAG了！"
llm = ChatOpenAI(model="gpt-4o-128k")
response = llm.invoke(f"根据以下文档回答问题：\n{full_document}\n\n问题：{query}")
3个问题：
- 成本爆炸： 200页≈50万token输入，GPT-4o的输入价格\(2.5/百万token → 一次查询\)1.25
- 中间内容被忽略： 研究表明，LLM对长上下文中间部分的信息"注意力衰减"——像读一本长篇小说，中间章节记不清
- 响应变慢： 50万token输入，推理时间10-30秒
正确做法：长上下文≠不用RAG，而是RAG+长上下文互补
# 最佳实践：先用RAG筛选相关片段，再用长上下文模型做深度理解
# Step 1: RAG检索top-10（比平时多取一些）
retriever = vectorstore.as_retriever(search_kwargs={"k": 10})
relevant_chunks = retriever.invoke(query)
# Step 2: 用长上下文模型处理多个chunk（不用截断）
context = "\n\n---\n\n".join([chunk.page_content for chunk in relevant_chunks])
llm = ChatOpenAI(model="qwen-long") # 长上下文模型，便宜
response = llm.invoke(
f"根据以下文档片段回答问题，如果多个片段有冲突请指出：\n\n"
f"{context}\n\n"
f"问题：{query}"
)
什么时候用长上下文模型替代RAG？
实用建议： 用便宜的长上下文模型（qwen-long、deepseek）做RAG的后端，比用贵的短上下文模型+大量chunk更划算。
回头看
长文档处理的核心原则和Java处理大数据量一样：不要全量加载，先过滤再处理。 RAG就是AI应用的"WHERE条件"，长上下文模型是"聚合函数"，两者结合才能高效处理大文档。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Clients threw over a 200-page product manual PDF: "Do us an AI assistant who can look into this."
My first reaction: Simple, 3th RAG went through - Chechunk, Vector, Retrieving Questions and Answers.
As a result, on the first day on the line, complaints were filed: the forms were not available, cross-chapter comparative questions were not answered correctly, and the "full summary" went directly beyond the Token limit.
200 pages of documents and 10 pages of documents, not "mass change," but "mass change". There are far more pits to process in long files than expected.
Let's start with the conclusion.
Java understands: long files ≈ big data search. You're not going to get a million records out of SELEECT before you process it, but you're going to filter it first. The same is true of AI processing long files: they are filtered before they are generated, not an entire brain.
Pipe 1: PDF pulls out all the pieces and lines.
Turn the car over.
Two hundred pages of PDF from PyPDF2:
from PyPDF2 import PdfReader
reader = PdfReader("product-manual.pdf")
text = ""
for page in reader.pages:
text += page.extract_text()
Results:
Product name Model price
---------- ----- -----
Smart door lock A200 899
(continued)
Colour: Dark Space Ash Weight: 2.3kg
Tables are dispersed into text streams, cross-page tables are cut off and multiple columns are lined to the left. Using this data to do the RAG, the search results are understandable.
Good practice: using a dedicated PDF resolution tool
# Option 1: unstructured (recommended, supporting table/photo/ layout)
from unstructured.partition.pdf import partition_pdf
elements = partition_pdf(
"product-manual.pdf",
# High precision mode
Infer table structure=True, # recognize table structure
Extract images in pdf=True, #extract pictures
)
# By type
for element in elements:
if element.category == "Table":
# Table retains HTML structure
table_html = element.metadata.text_as_html
# It's better to read the Markdown form
table_md = html_table_to_markdown(table_html)
elif element.category == "Title":
# Title Information
title = element.text
elif element.category == "NarrativeText":
# The text
text = element.text
Option 2: Marker
# pip install marker-pdf
from marker.converters.pdf import PdfConverter
from marker.models import create_model
from marker.output import text_from_rendered
converter = PdfConverter(artifact_dict=create_model())
rendered = converter("product-manual.pdf")
markdown_text, metadata = text_from_rendered(rendered)
Marker's Advantages: Convert PDF into a structured Markdown, and keep the table as a Markdown table, full title level, complete code block. Just use the Markdown structure of 23rd.
Cost comparison:
Pipe 2: cross-section questions not answered - search only for footage without context
Turn the car over.
The user asked, "What's the difference between A200 and A300?"
A200 parameters in chapter 3, A300 parameters in chapter 5. Search Top-4 chunk:
Chunk 1: "A200 smart door lock to support fingerprint unlocking..."
Chink 2: "A200 Technical Parameter: 2.3kg weight..." (Chapter 3, sim=0.72)
Chunk 3: "A common question: how to reset the password..."
Chunk 4: "Post-sale service policy..."
A300 didn't even get it! 'Cause the user's query mentioned "A200 and A300" and the similarity of the A300 chunk didn't go Top-4.
Correct: Parent Document Retriever
Core idea: Retrieval (precision) with little chunk, return to Big chunk (full context) to LLM after hit.
from langchain.retrievers import ParentDocumentRetriever
from langchain.storage import InMemoryStore
# Little chunk: For precision search
child_splitter = RecursiveCharacterTextSplitter(chunk_size=200)
# Big chunk: Show LLM the full context
parent_splitter = RecursiveCharacterTextSplitter(chunk_size=800)
# Vector inventory small chunk, document inventory large chunk
vectorstore = Chroma(embedding_function=embeddings)
docstore = InMemoryStore()
retriever = ParentDocumentRetriever(
vectorstore=vectorstore,
docstore=docstore,
child_splitter=child_splitter,
parent_splitter=parent_splitter,
)
# Retrieving: matching with little chunk and returning the corresponding big chunk
Docs = retriever.infoke.
# Back is a complete paragraph with A200 and A300 messages
Why does it work? Little chunk (200 characters) matches the more precise -- the key "A300" is more dense in little chunk. When matched, the large chunk (800 characters) containing this small chunk contains the full A300 parameter.
Progress: Multi- vector retrieval - multiple search vectors for one document
from langchain.retrievers.multi_vector import MultiVectorRetriever
# Generate summary for each big chunk and use summary as a search vector
summaries = []
for doc in large_chunks:
Summary = lllm.invoke (f) "Summary by one sentence: {doc.page content}")
summaries.append(summary)
# Vector inventory summary vector, document inventory original
retriever = MultiVectorRetriever(
vectorstore=vectorstore,
docstore=docstore,
id_key="doc_id",
)
# summary vector # # # summary vector # # # Retrieving hit # # # return the corresponding original #
Effect: The user asks "A200 and A300" and "A200 and A300 smart door locks" in summary vectors, and returns the original text containing the full parameters.
Pit 3: Full text summary - Token Superstition or summary incomplete
Turn the car over.
The user asked, "Help me summarize the core of the manual."
Option 1: Throw 200 pages of the full text to LLM → token (200 pages ≈500,000 token) Option 2: take only the first 10 pages of summary and leave out the latter part of Option 3: sub-chapter summary merge Z110Z plus summary plus the consolidation is too limited
Correct approach: Map-Reduce sub-section summary
from langchain.chains.summarize import load_summarize_chain
# Map-Reduce: Split summary, merge
chain = load_summarize_chain(
llm,
chain_type="map_reduce",
verbose=True
)
# Automatic processing: each piece of summary merges summary
summary = chain.run(docs)
Map-Reduce's problem: may be lost in the merger. The better option is Refine:
chain = load_summarize_chain(
llm,
chain_type="refine",
verbose=True
)
# Refine logic:
# The first chunk # generation of initial summary
# 2nd chunk + current summary
# 3rd chunk + current summary
# ...
# Eventually summary keeps being refined and more complete
Refine vs Map-Reduce：
200 pages of the handbook is more appropriate with Refine: Serial processing, although slow, is more complete with summary.
Actual optimization: Map-Reduce will be used for each chapter of Z110Z, followed by Refine for a combined chapter of Z110Z - two phases:
# Phase 1: Each chapter is independent of summary (may be in parallel)
chapter_summaries = []
for chapter in chapters:
Summary = llm.invoke (f) "Summary in three sentences:
chapter_summaries.append(summary)
Phase 2: Merge Chapter summary
final_summary = chapter_summaries[0]
for i in range(1, len(chapter_summaries)):
final_summary = llm.invoke(
"The current summary:
"f" New content: {chapter summaries[i]}\n"
f "Integrate new content into the current summary and maintain integrity and consistency."
)
Pocket 4: The trap of the long context model - which can be plugged in is not as good as it can be used.
Turn the car over.
The year 2025 model context is growing longer: GPT-4o 128K, Claude 200K, Gemini 1M.
"Well, why don't I just stick 200 pages of documents in the prompt? No more RAG!"
llm = ChatOpenAI(model="gpt-4o-128k")
Response = lllm.invoke (f) answers questions based on:
Three questions:
- Cost explosion: 2 pages ≈5 million token input, GPT-4o input price \(2.5/million token \ 1 \) 1.25
The study shows that LLM's message on the middle of the long context is "rejected" -- like reading a long novel, the middle chapter is not clear.
- Slower response: 500,000 token input, reasoning time 10-30 seconds
Correct: Long context does not use RAG, but rather RAG+ Long context complements
# Best practice: filter the relevant footage with RAG, then understand in depth with a long context model
# Step 1: RAG Retrieval Top-10 (more than usual)
retriever = vectorstore.as_retriever(search_kwargs={"k": 10})
relevant_chunks = retriever.invoke(query)
# Step 2: Treat multiple chunks with a long context model
context = "\n\n---\n\n".join([chunk.page_content for chunk in relevant_chunks])
# Long context model, cheap
response = llm.invoke(
f "Answer questions based on the following document clips, if there is a conflict between several clips, indicate: \\n\"
f"{context}\n\n"
"Question: Question"
)
When will you replace RAG with a long context model?
Practical advice: Using a cheap long context model (qwen-long, Deepseek) for the back end of RAG is more cost-effective than using a expensive short context model plus a large number of chunks.
Look back.
The core principles for long document processing are the same as the size of Java's large data processing: do not load in full, filter and then process. RAG is the "WHERE condition" that AI applies, and the long context model is a "polymer function" that combines to handle large documents efficiently.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.