面向复杂场景的大模型应用落地与能力优化实践
大模型在真实业务中的落地，难点往往不在模型本身，而在工程化：如何管理多轮上下文、控制推理成本、稳定输出结构化结果。本文以一个"智能客服知识问答"为例，演示从 Prompt 设计到检索增强、再到结果校验的完整链路。
一、场景与问题定义
业务方要求：根据企业内部知识库回答用户问题，支持多轮追问，且回答必须引用知识原文，禁止编造。
三个核心痛点：
- 幻觉：模型自由发挥，答案脱离知识库；
- 上下文失控：多轮对话中历史消息无限累积，Token 成本高、噪声大；
- 输出不可解析：下游系统需要结构化字段，直接输出文本难以对接。
二、解决方案架构
采用 RAG（检索增强生成）+ 滑动窗口记忆 + JSON 结构化输出 三段式设计。
用户提问
│
▼
① 记忆裁剪（滑动窗口，保留最近 N 轮）
│
▼
② 向量检索（知识库 Top-K 召回）
│
▼
③ 结构化 Prompt（限定引用原文，强制 JSON）
│
▼
④ 结果校验（字段完整性 + 引用存在性）
三、核心代码演示
1. 上下文裁剪
def trim_history(history: list, max_rounds: int = 5):
"""仅保留最近 max_rounds 轮对话，防止 Token 膨胀。"""
return history[-max_rounds * 2:] # 每轮含 user + assistant 两条
2. 向量检索召回
def retrieve(query: str, index, top_k: int = 3):
"""对 query 编码后从向量库召回最相关的知识片段。"""
vec = index.encode([query])
scores, ids = index.search(vec, top_k)
return [knowledge[doc_id] for doc_id in ids[0]]
3. 结构化生成与校验
import json, re
SYSTEM_PROMPT = """你是客服助手。请严格依据【参考资料】回答，
不得编造。输出格式必须为 JSON：
{"answer": "回答正文", "citations": ["引用片段"]}"""
def generate_answer(query, context, history, llm):
messages = trim_history(history) + [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user",
"content": f"参考资料：{context}\n问题：{query}"},
]
raw = llm.chat(messages)
return validate_response(raw, context)
def validate_response(raw, context):
try:
data = json.loads(re.search(r"\{.*\}", raw, re.S).group())
except Exception:
raise ValueError("输出非法 JSON，需重试")
# 校验引用确实来自知识库，遏制幻觉
assert all(c in context for c in data["citations"]), "引用不在知识库中"
return data
4. 统一入口
def ask(query, history, index, llm):
context = retrieve(query, index)
result = generate_answer(query, context, history, llm)
history.append({"user": query, "assistant": result["answer"]})
return result
四、效果与收益
- 幻觉率：引入"引用必须命中知识库"的硬校验后，明显下降；
- 成本：滑动窗口使单次请求 Token 量保持稳定，长对话不再线性膨胀；
- 可对接：JSON 输出可直接落入客服工单、质检系统，无需二次解析。
五、优化要点小结
- 始终给模型"可验证的上下文"，而非只靠指令约束；
- 用结构化输出 + 后置校验兜底，而非完全信任模型；
- 对长对话做显式裁剪，平衡信息量与成本。
以上方案稍作改造，即可迁移到工单分类、财报问答、文档摘要等各类复杂场景。
海量精选技术文档和实战案例持续更新，敬请关注【风骏时光少年】
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Application of LLM applications to complex scenes and optimization practices
The difficulty of LLM landing in real business is often not in the model itself, but in engineering: how to manage multiple rounds of context, control the cost of reasoning, and stabilize the output structure. This is an example of a complete link from Prompt designed to retrieve enhanced, then results verified.
Definition of scenes and problems
The business community requested that user questions be answered according to the enterprise ' s internal knowledge base, that multi-cycle queries be supported, and that the answer be to quote the original knowledge and to prohibit fabrication.
Three core pain points:
- Ignorance: the model is free and the answer is out of the knowledge base;
- context is out of control: an inexhaustible accumulation of historical news in multiple rounds of dialogue, high cost and high noise;
- Output is non-resolvable: the downstream system needs to be structured field, and the direct output text is difficult to connect.
II. SOLUTION STRUCTURE
Use RAG (Retrieval Enhancement Generating) + Slide Window Memory + JSON Structure Output three-part design.
User Questions
│
▼
1 Memory Crop (Slide Window, Keep Recent N Wheel)
│
▼
2 Vector Retrieval (Top-K recall)
│
▼
3 Structured Prompt (prescriptive original quote, mandatory JSON)
│
▼
4 Results validation (field completeness + references existential)
III. Core code presentations
1. Disarming context Cut
def trim_history(history: list, max_rounds: int = 5):
""only keep the recent max rounds dialogue to prevent Token from swelling."
# Each round contains two of us
2. Vector retrieval recall
def retrieve(query: str, index, top_k: int = 3):
"The most relevant piece of knowledge from the vector library is retrieved after encoded query."
vec = index.encode([query])
scores, ids = index.search(vec, top_k)
return [knowledge[doc_id] for doc_id in ids[0]]
3. Structured generation and validation
import json, re
SYSTEM PROMPT = """ You're the help of the customer. Please rely strictly on [references].
No fabrication. The output format must be JSON:
"Answer": "Answer the Text," "Citiations": "Refer to the Story"
def generate_answer(query, context, history, llm):
messages = trim_history(history) + [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user",
"content": "f" reference: {context} Question: Query"},
]
raw = llm.chat(messages)
return validate_response(raw, context)
def validate_response(raw, context):
try:
data = json.loads(re.search(r"\{.*\}", raw, re.S).group())
except Exception:
Rice ValueError ( "Export illegal JSON, need retry")
# Validation of references comes from the knowledge base to contain hallucinations
Assert all (c in context for c in data)
return data
4. Unified entrance
def ask(query, history, index, llm):
context = retrieve(query, index)
result = generate_answer(query, context, history, llm)
history.append({"user": query, "assistant": result["answer"]})
return result
IV. Effects and benefits
- Perception rate: the introduction of a hard-checked reference to the knowledge base required to hit has significantly decreased;
- Cost: The slide window stabilizes the single request volume of Token, and long conversations are no longer linear expansion;
- Available: JSON output can fall directly into the passenger manifest, quality control system, without secondary analysis.
Summary of points for optimization
- Always giving the model "verifiable context" rather than merely restraining by command;
- Using structured output + post-validation loops rather than fully trusted models;
- Visible tailoring of long conversations to balance the volume and cost of information.
The above programme is slightly modified to allow migration to complex scenarios such as worksheet classification, financial questions and answers, and document summary.
The mass selection technology files and operational cases are constantly updated.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.