新智元报道
前一阵，GPT-6 Astra 画 CAD 的视频在 X 上实火。
比如这个涡轮增压器，每个零件都单独建模，不仅能正常转动，还能一键原地拆解展开，视觉效果相当炸裂。
一时间，GPT-6 Astra 的 BenchCAD 跑分达到了惊人的 95.9%，评论区不少人已经在讨论 CAD 工程师的再就业问题了。
不过，也有工程师出来泼了盆冷水：这些零件，一个都别往生产线送。因为，画得像，不代表画得对。
内行看门道，外行看热闹。表面上看，模型已经能够复刻 CAD 的视觉界面，但从专业工程师的视角来看，这些模型在科学原理、几何结构以及部件之间的关系上，是否真的准确？有没有一个专业的测评基准，能够检验 AI 对三维科学对象的观察与理解能力，并评估其生成的三维对象是否符合科学与工程要求？
最近发现了一个 benchmark，还真能评测 AI 在建筑设计中的 CAD 能力。它会像专业工程师一样，走一遍零件的检查流程，从结构关系到几何细节逐项核验，判断模型生成的结果是否符合要求。
这个 benchmark 叫 ScienceVision。它的评测范围不止于工程制图，还覆盖生物、化学、医学影像、建筑和机械 CAD 共 5 个科学领域、22 个子领域，包含 250 道基于真实三维数据构建的题目，要求模型通过自主操作三维对象，完成观察、测量与分析。
考得最好的 GPT-6-Astra（搭配 Codex）只拿到了 0.53 分。满分 1 分。
BenchCAD 上 95.9 分的选手，到了三维科学理解与推理的考场上，只过了一半。
ScienceVision 出自 UniPat AI 和视觉推理实验室 Elorian AI，是一个基于专家级视觉问答（VQA）的跨学科视觉三维理解与推理benchmark，团队透露它被开发出来的出发点是一个问题：AI 能像科学家一样观察和推理吗？
根据现有的结果来看，评测覆盖了 GPT-6 Astra、Claude Fable 5.1、Gemini 4 Argon、Qwen3.8 Max 等当前最新、最强的 12 个开源闭源前沿大模型，没有哪个模型能在五个学科同时领先。
这到底是些什么题，能把一众模型卡成这样？
ScienceVision 的 demo 展示了五个学科各一段检视操作实录，其中所使用的 3D 工具是按照领域专家处理三维源数据的实际操作方式建模的，模仿了最真实的 3D 观察场景。
生物那段，是把整个蛋白质藏起来，只留一个柠檬酸分子和旁边几个离子，量它们挨得多近。
化学那段最短，8 步：框出一个晶胞，用对称操作看它怎么一层层堆下去。
医学影像那段，CT 从三个方向来回切，找肾脏上那个病灶的边在哪。
CAD 那段是一台叉车，零件一个个点开，数有几个长得一样。
建筑那段，是顺着一条坡道量长度和高差，看坡度超没超标。
五段 demo 看下来，就像是各领域的人类专家在审核自己领域的 3D 视觉问题。动作本身都不复杂：转、点、量，但难的是每一步都得先想清楚往哪转、点什么、量哪里，如果不经历比较 solid 的观察、理解、推理，是很难做出正确判断的。
光看视频不过瘾的话，可以自己上手。
这五道题的原始文件和检视工具，ScienceVision 的 blog 上原样开放了：你可以自己点开题目，切 3D 或平面视图，体验如何通过真实的三维检视过程，评估 AI 的科学观察能力。下方展示上手实测，大家可以感受一下：
体验入口：https://unipat.ai/blog/ScienceVision
题看完了，再来说说为什么要出这套题。
过去其实有不少 3D 理解与推理基准，覆盖室内场景理解、具身交互、抽象结构推理等方向，但它们主要服务于日常场景感知、机器人操作或工业设计等通用任务。
然而，真正面向科学领域的 3D 理解与推理基准几乎空白，仅有医学影像和机械工程等少数应用场景的专门基准，且尚未形成体系化的科学 3D 推理评测。
但科学研究过程天然对真实三维世界中的观察、推理能力有要求。
这种能力要求贯穿于不同学科的实践中：化学家要定位原子、测量几何关系；放射科医生要在不同切片间追踪结构；建筑工程师要切换视角、排除遮挡、确认构件的连接关系。
类似的活儿没有一样是看一眼图就能干完的。人类科学家需要根据过往经验，决定往哪看、怎么看、量什么，看完了再用专业知识下结论。放在 AI Scientist 身上，也是相同的道理。
ScienceVision 考察 AI 的逻辑也一样：题目和数据摆在眼前，自己决定看哪里、怎么查。其中的关键之处的道理有点类似于「斯坦门茨画线」：画一条线值1美元，知道在哪儿画线值9999美元。这就能看出哪个模型更有水平了，而可靠的观察和理解，即是检验的第一步。
在过往科学领域 3D 理解和推理的评测方面，不同学科的问题设定、任务输入和评估方式是五花八门的，比如3D文件的格式不统一，渲染出来的交互逻辑也不一样，表达形式更是不一样。
ScienceVision 要填的正是这个空白，做出一个统一的跨学科 3D 视觉理解评测。
ScienceVision 涵盖五个学科：生物学、化学、医学影像、机械 CAD 和建筑学。每个学科 50 道题，共 250 道，横跨 22 个子领域。
每道题由四部分组成：一份真实的 3D 源数据（分子坐标、CT 体数据、机械装配体或建筑模型），一个领域问题（看什么、在哪找、用什么单位和坐标），一份作答要求（返回什么、什么格式），再加一个专家标好的参考答案。
答案用确定性规则核验，不用语言模型当裁判。没有「大致正确」这回事，到就是到，没到就是没到。
一个带确定性校验的 benchmark，其实还有一层潜力：它几乎可以原地变成一个强化学习环境：答对给奖励，答错不给，不需要另一个模型当裁判。
不过，出这套题本身是一件极其困难的事。
首先，选出科学上有意义、确实依赖三维证据的问题。构题需要从具体科学对象出发，明确观察目标、判定条件和所需证据，并确认题目能够在给定环境中解答。每道题都要同时满足专业上成立、视觉目标清楚、答案可以复核。
其次，让不同领域的数据都能被正确观察和测量。分子坐标、CT 体数据、机械 CAD 和建筑模型具有不同的格式、尺度、坐标约定和操作方式。转换与展示过程中，需要保留关键结构信息，并提供适合各类对象的查看、隔离、切片和测量能力。
最后，依靠领域专家建立可信的答案和证据过程。专家需要检查问题真实性和相关性，实际观察对象、计算或确认答案，记录关键步骤，并参与审核与返修。五个领域的专业规则不同，这些工作需要相应领域的知识和经验。
整套流程是机器和人接力干的。
先由自动化管线出题：从三维资产库里挑出适合出题的对象，试一下在查看器里能不能正常看、能不能量，再按模板生成候选题，过一遍规则筛选，打包。
然后专家接手：在 3D 标注平台上审题、改题，亲自上手转、隔离、测量，写下参考答案，操作轨迹顺手录下来，最后再过一道质量审核，不合格的打回。专家打回的意见又反过来喂给出题管线，让它下一轮少犯同样的错。
造题、专家审核、自动修题、改进管线，一圈一圈转下来，才有这 250 道。
最终，这套题的质量靠的是人。
UniPat AI 搭建了覆盖五个领域的专家平台，维护 66 名领域专家，累计投入 482.98 小时。其中化学 113 小时，机械 CAD 116 小时，医学影像 103 小时，建筑 77 小时，生物 74 小时。
专家们审查问题、标注答案、记录操作过程，并参与多轮审核与返修。
这恰好呼应了 ScienceVision 的核心定位：它要测的是 AI 有没有专家级的观察能力，所以答案本身需由专家建立才更有说服力。
前面 demo 里那样的操作记录，250 道题每一道都附带了，而且逐步保存了关键帧。
这些记录把科学观察从一个只看得到结论的黑箱，变成了可以逐步回放的过程。将来要训练 AI 学会「看」，这就是现成的对标数据。
考法上，ScienceVision 在同一套 250 道题上比较了三种证据获取方式：
第一种，给模型原始三维数据，让它自己写代码分析。相当于给了你全部原始实验数据，你自己算。
第二种，在数据基础上再给一套交互检视工具，可以旋转视角、隔离结构、切片浏览、测量距离和角度。相当于不仅给了你数据，还给了你一台显微镜。
第三种，只给六张固定角度的二维渲染图。看你自己能从照片里看出什么？
三种方式用同一套评分标准。
这就可以直接回答一个问题：数据更多、工具更全，模型的表现到底会不会更好？
来看结果，最高分 0.53（GPT-6-Astra 搭配 Codex），刚过满分的一半。而且这还不是答对率，它是包含部分得分的平均指标，真正完全答对的比例还要更低。
学科之间的差异比总分更有意思。
没有哪个模型五科全优。GPT-6 Astra 在建筑、机械 CAD 和化学三科领先（0.458、0.580、0.609），生物被 Claude Fable 5.1 以 0.650 超过，医学影像则是 Claude Opus 5.5 以 0.399 领先。
CT 本来就是一叠切片，放射科医生读的也是切片，模型训练时见过海量的二维医学影像。
但一个带轴序和体素索引的三维数组，模型就没有那么熟悉。
只给六张固定角度的渲染图，分数掉得更狠：最好的 Claude Fable 5.1 也只有 0.30，GPT-6 Astra 0.25。
换句话说，给照片，模型看不出来；给三维数据，分数是上去了，但论文自己提醒，这些分可能是代码从坐标里算出来的，算不算「看见了」还两说。
扎心的是，给了工具也一样。
榜单前五的模型里，加上交互检视工具后涨得最多的 GPT-6.1 Sol 也只涨了 0.02，GPT-6 Astra 两种设置差不到 0.01，Claude Opus 5.5 还掉了 0.02。
论文里还有个细节：即便在工具模式下，模型照样会去读原始坐标文件。也就是说，你给它一台显微镜和一张数据表，它选数据表。
医学影像是最意料之外的，模型的操作次数是专家的三倍左右，得分却是五个学科里最低的。
这说明当前的前沿模型即使获得了最充分的 3D 数据和工具支持，面对真实科学任务的观察与推理，也还差得很远。
点得多，不代表会观察、会推理。
在 ScienceVision 之前，科学多模态评测不是没有，但大多只管一个学科，只看二维图片，评分规则也各搞一套。
ScienceVision 的独特价值，可以归成三件事：
第一，题是真三维的。模型拿到的是分子坐标、CT 体数据、机械装配体、建筑模型这样的源文件，不是平面截图。答题得用领域知识去解释三维结构证据，有些题还要先从观察里把目标认出来，再接上拓扑和几何推理，一环扣一环。
第二，工具是照着专家工作流做的。视角切换、目标隔离、切片浏览、科学测量，这些检视操作通过 MCP 开放给模型，具体能做什么随查看器而定。看哪、怎么看、量什么，模型自己决定，没人替它挑证据。
第三，评分是确定性的，五个学科同一套。每道题有参考答案和预设规则，答案的各个组成部分分开核验，再按模型配置和学科汇总得分。不靠另一个模型打分，化学分子和建筑构件的成绩，第一次能放到一起比。
与之最相关的 SciVisAgentBench 同样支持三维交互检查，但评测目标不同：
ScienceVision 在五个学科中都采用同一类问题，即受领域知识约束的结构性问题，加上结构化的领域问题和逐题参考答案评分，SciVisAgentBench 则主要围绕科学可视化软件中的任务展开，评估 Agent 生成的可视化与分析结果是否符合任务要求。
可以说，ScienceVision 第一次在统一框架下衡量出了 AI 三维科学观察的能力边界。
知道差在哪，才知道怎么追。
下次再有人宣布「AI 科学家来了」，可以先让它做完这 250 道题再说。感兴趣的可以点下面的链接了解细节：
GitHub 开源链接：
https://github.com/UniPat-AI/ScienceVision/
Benchmark 链接：
https://huggingface.co/datasets/UnipatAI/ScienceVision
论文链接：
https://github.com/UniPat-AI/ScienceVision/blob/main/paper/sciencevision.pdf
编辑：所罗门 桃子
不喜欢
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
New Ji-won.
A while ago, GPT-6 Astra drew a video of CAD on X fire.
This turbine booster, for example, has a stand-alone model of each component, which not only moves normally, but also disassemblys in situ, with considerable visual fragmentation.
At one point, the GPT-6 Astra BenchCAD ran at an alarming 95.9 per cent, and many people in the comment area are already discussing the re-employment of CAD engineers.
However, engineers came out and poured cold water: none of these parts were delivered to the production line. Because, like, it doesn't mean it's right.
In-house doorways, outsiders. On the surface, the models can already recreate CAD's visual interface, but from the perspective of professional engineers, are they really accurate in terms of science, geometry and the relationship between components? Is there a professional benchmark against which to test AI ' s ability to observe and understand 3D scientific objects and to assess whether the 3D objects it generates meet scientific and engineering requirements?
The recent discovery of a benchmark is a good measure of AI's CAD capability in architectural design. It would go through the inspection process of the parts, like a professional engineer, and check each item from the structure in relation to geometry to determine whether the results produced by the model were satisfactory.
This benchmark is called ScienceVision. It covers more than engineering mapping. It also covers five scientific fields, 22 sub-areas of CAD, including 250 topics based on real three-dimensional data, requiring models to be observed, measured and analysed through the autonomous operation of 3-dimensional objects.
The best GPT-6-Astra (with Codex) got only 0.53 points. One full score.
BenchCAD, 95.9, is only halfway through the three-dimensional science of understanding and reasoning.
ScienceVision, from UniPat AI and Elorian AI, an interdisciplinary visual three-dimensional understanding and reasoning based on expert-level visual questions and answers (VQA), revealed that its starting point was a question: can AI observe and reason like scientists?
Based on the results available, the evaluation covered the latest and strongest 12 open-source front lines LLM, such as GPT-6 Astra, Claude Fable 5.1, Gemini 4 Argon, Qwen 3.8 Max, and none of the models can lead at the same time in five disciplines.
What the hell is this? It's a whole bunch of models.
ScienceVision's demo shows a review of the operations in each of the five disciplines, using a 3D tool that models the actual way in which field experts process 3D source data, imitating the most realistic 3D observation scene.
That part of the creature is to hide the whole protein, leaving only one lemon acid molecule and several ions next to it, and to measure them much closer.
The chemical one is the shortest, eight steps: frame a crystal, use symmetry to see how it piles down.
The medical footage, the CT cut back and forth from three directions, looking for the side of the stove on the kidney.
CAD, that's a forklift, the parts are running at one point, and a few of them look the same.
The construction was based on a ramp of length and height, depending on whether the slope was above standard.
Five paragraphs demo look like human experts in various fields are reviewing 3D visual problems in their own field. The actions themselves are not complicated: turn, point, measure, but the hard thing is to figure out where to turn, what to do, where to measure, and it is difficult to make the right judgment without going through the observation, understanding and reasoning of a comparison of solid.
You can do it yourself if you can't just watch the video.
The original file and inspection tool for the five topics, the ScienceVision blog, is open as it is: you can click the subject, cut the 3D or plane view, and experience how to assess AI ' s scientific observation capacity through a real three-dimensional review process. Here's what you can see:
Access: https://unipat.ai/blog/ScienceVision
When we're done with the question, then why do we have to do this?
In the past, there were many 3D understanding and reasoning benchmarks covering the direction of indoor landscape understanding, interaction, abstract structural reasoning, but they mainly served common tasks such as routine scene perception, robotic operation or industrial design.
However, the true science-oriented 3D understanding and reasoning benchmark is almost blank, with only a few applications such as medical imagery and mechanical engineering being specifically benchmarked, and no systematic scientific 3D reasoning assessment has yet been developed.
But the scientific research process naturally requires the ability to observe and reason in a real three-dimensional world.
This capability is embedded in the practice of different disciplines: chemists need to locate atoms and measure geometry; radiologists need to track structures between different slices; and construction engineers need to switch perspectives, remove masks and confirm connections to components.
Nothing like this can be done by looking at a picture. On the basis of past experience, human scientists need to decide what to look at, how to look at, what to measure, and then come to conclusions with expertise. It's the same thing with AI Scientist.
The same logic is for ScienceVision to visit AI: the subject and the data are in front of you, and it's up to you to see where and how to look. The key point is somewhat similar to the "Stanmantz line": drawing a line worth $1 and knowing where to draw $9999. This shows which model is more horizontal, and reliable observation and understanding is the first step of the test.
In the assessment of understanding and reasoning in the past in the field of science, problems in different disciplines are set, tasks entered and evaluated in a variety of ways, for example, the format of the 3D document is not uniform, the interactive logic is not the same and the expression is different.
It's exactly this gap that ScienceVision is going to fill in to make a unified interdisciplinary 3D visual understanding assessment.
ScienceVision covers five disciplines: biology, chemistry, medical imagery, mechanical CAD and architecture. Each of the 50 subjects, 250 in total, spans 22 sub-areas.
Each Question consists of four parts: a real 3D source data (molecule coordinates, CT data, mechanical assembly or construction model), an area question (see what, where, what units and coordinates to use), a request for answers (return what, what format) and an expert reference answer.
The answer is verified by the rules of certainty and does not refer to the language model. There's no "generally right" thing. We're here, we're not there.
A benchmark with a definitive validation has a potential: it can almost become an enhanced learning environment: the answer is yes, the answer is no, and no other model is needed as a judge.
However, it is extremely difficult in itself to come up with this set of questions.
First, the issue of scientific relevance and true reliance on three-dimensional evidence is selected. The building blocks need to be based on specific scientific subjects, with clear observations of the objectives, the conditions for determination and the evidence required, and to confirm that the topics can be answered in a given environment. Each of these questions must be accompanied by professional merit, clear vision and reviewable answers.
Second, data from different fields can be correctly observed and measured. Molecular coordinates, CT data, mechanical CAD and construction models have different formats, scales, coordinates and modes of operation. During the conversion and presentation process, key structural information needs to be maintained and the ability to view, isolate, slice and measure suitable for various categories of objects is provided.
Finally, it relies on field experts to establish credible answers and evidence processes. The experts need to examine the authenticity and relevance of the questions, physically observe, calculate or confirm the answers, document the key steps, and participate in review and correction. Professional rules differ in five areas, and these require knowledge and experience in the respective areas.
The whole process is machine and human.
Questions are first posed by automated pipelines: pick the right subjects from the three-dimensional asset store, try to see them properly in the viewer, quantify them, then create candidates by template, filter them through the rules and pack them.
The experts then took over: the issues were considered, changed, personally, isolated, measured, the answers were written, the trajectories were recorded, and finally a quality review was carried out and the calls were not qualified. The expert ' s comments, in turn, feed the tube so that it does not make the same mistake in the next round.
This is the 250-track route, which is created, reviewed by experts, automatically modified, improved pipes, and turned around.
In the end, the quality of the issue depends on people.
UniPat AI established an expert platform covering five areas, maintaining 66 area experts, with a cumulative investment of 482.98 hours. Of which 113 hours chemical, 116 hours mechanical, 103 hours medical, 77 hours construction, 74 hours biological.
Experts review questions, indicate answers, document operational processes and participate in multiple rounds of review and refurbishment.
This echoes ScienceVision's core positioning: It's about AI having an expert-level observation capability, so the answers themselves need to be built by experts to be more convincing.
Each of the 250 lines is attached to the operating records of the previous demo, and the key frames are gradually preserved.
These records turn scientific observations from a black box where only conclusions can be seen to be drawn, into a process that can be progressively replayed. In the future, I'm going to train AI to "see," which is the data that's available.
On the test, ScienceVision compares three types of evidence acquisition with the same set of 250 questions:
First, give the model the original three-dimensional data so that it can write its own code analysis. It's like giving you all the raw experimental data, you figure it out.
The second is an interactive review tool based on data that can rotate perspectives, isolation structures, slice views, measurements of distance and angles. It's like giving you not only data, but also a microscope.
Third, only six two-dimensional renderings of a fixed angle are given. See what you can see from the pictures?
The three approaches are based on the same set of rating criteria.
This is a question that can be answered directly: will models perform better if they have more data and tools?
For the results, the highest score was 0.53 (GPT-6-Astra mix Codex), just over half the score. Moreover, this is not a yes-to-no, it is an average indicator with partial scores, and it is even lower in the true total.
Differences between disciplines are more interesting than total scores.
None of the models are perfect. GPT-6 Astra leads in construction, mechanical CAD and chemistry (0.458, 0.580, 0.609), life exceeds Claude Fable 5.1 by 0.650, while medical images lead Claude Opus 5.5 by 0.399.
CT is a stack of slices, and radiologists are reading them, and modeling has seen a big 2D medical image.
But a three-dimensional array with an axle sequence and body index is less familiar with the model.
Only six replicas of a fixed angle were given, and the score fell harder: the best Claude Fable 5.1 was 0.30, GPT-6 Astra 0.25.
In other words, to give the picture, the model does not see it; to give the three-dimensional data, the score is up, but the paper reminds itself that these points may have been calculated by the code from the coordinates.
That's the thing about the tools.
The top five models, with the most significant GPT-61 Sol, added to the interactive inspection tool, have only increased by 0.02, and the GPT-6 Astra settings are less than 0.01, and Claude Opus 5.5 has dropped 0.02.
There is also a detail in the paper: even in tool mode, the model will read the original coordinate file. In other words, you give it a microscope and a data sheet, and it selects the data sheet.
Medical images are the most unexpected, with models operating about three times as often as specialists, with the lowest score in five disciplines.
This suggests that the current front-line model, even with the fullest support of 3D data and tools, is far from being observed and reasoned in the face of real scientific tasks.
It's a lot more. It doesn't mean it's observational. It's reasoning.
Prior to ScienceVision, scientific multi-modular assessments were not non-existent, but mostly focused on one discipline, looking only at two-dimensional pictures, with one set of scoring rules.
The unique value of ScienceVision can be attributed to three things:
First, the question is three-dimensional. The model was provided with sources such as molecular coordinates, CT data, mechanical assembly, construction models, not a plane screenshot. The answers have to be based on knowledge of the field to explain the evidence of the three-dimensional structure, and some of the questions have to be followed by observations that identify the target and expand and geometrical reasoning.
Second, the tools were made by expert workflow. Perspective switching, target isolation, slice browsing, scientific measurements, these inspections are open to models through MCP, depending on the viewer. See, see, measure, and the model decides for itself, and nobody picks evidence for it.
Thirdly, the rating is definitive and the five disciplines are the same. Each of the questions has reference answers and preset rules, and the various components of the answers are examined separately, and scores are aggregated by model configuration and discipline. For the first time, the performance of the chemical molecules and building components could be compared instead of another model.
The most relevant of these is SciVisAgentBench, which also supports a three-dimensional interactive examination, but with different objectives:
ScienceVision uses the same type of questions in each of the five disciplines, namely, structural issues that are subject to field-based knowledge, along with structured area-specific questions and question-by-issue questions, and SciVisAgentBench focuses on tasks in scientific visualization software, assessing whether the visualization and analysis produced by Agent meet mission requirements.
It can be said that ScienceVision measured for the first time the boundaries of AI 3D scientific observation under a unified framework.
You don't know where to go until you know where to go.
The next time someone announces that "AI Scientists are here," we can let it finish the 250. The following links are available for details:
GitHub Open Source Link:
https://github.com/UniPat-AI/ScienceVision/
Benchmark Link:
https://huggingface.co/datasets/UnipatAI/ScienceVision
Articles Link:
https://github.com/UniPat-AI/ScienceVision/blob/main/paper/sciencevision.pdf
Editor: Solomon Peach
I don't like it.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.