新智元报道
想象一项任务：把相机交给AI，让它进入一间从未见过的虚拟房间。没有预先给定的物体坐标，也没有替它安排好的参观路线。它需要决定朝哪里转、向哪里移动、何时再拍一张图，并最终说清楚各个目标物体在三维空间中的位置。
论文链接：https://arxiv.org/abs/2610.26659
代码链接：https://github.com/JesseZZZZZ/SpatialScout
数据集链接：https://huggingface.co/datasets/jessezhaoxizhang/SpatialScout
SpatialScout把这样的探索过程变成了可以记录、评分和比较的研究任务。每一次观察是否有用、不同视角能否相互印证、提交的坐标是否可靠，都会在最后的物体级地图中留下痕迹。
图1 一次空间探索由相机行动、多视角观测和物体位置估计共同构成。
让模型亲自选择下一次观察：从RGB图像出发，通过相机控制与几何工具估计物体中心。探索策略和空间推断被放在同一个交互闭环里考察。
人走进屋子，四处看一看、绕上一圈，通常就能对整体布局形成连贯的认识：桌子靠近哪面墙，椅子在桌子的哪一侧，转过身以后刚才的物体还在什么方向。这样的三维空间表征，让接近、绕行、伸手拿取等动作有了依据。
物理AI同样需要这层基础。智能体要在真实环境里持续行动，就必须把零散的视觉线索组织成稳定的空间关系。连基本位置和布局都难以通过观察确定，后续的行动规划、物体操作与环境交互也就缺少可靠的参照。SpatialScout关注的主动空间理解能力，正是迈向物理AI的一块基石。
正式评测选取80个室内场景、3,140个目标实例，来自iTHOR、RoboTHOR、ArchitecTHOR、ReplicaCAD和ProcTHOR。房间类型既有厨房、客厅、卧室、浴室，也有公寓和多房间住宅。固定的场景与初始相机设置，让不同模型面对一致的任务条件。
图2 数据集物体类别词云。此图按完整90场景目录的3,440个实例统计，经原有归一化规则汇总为143类；正式结果采用其中的80场景评测协议。
椅子、桌子、橱柜、书本、灯具和日常用品共同构成了这些环境。物体的尺度、摆放位置和可见程度各不相同，模型需要不断调整视角，在局部细节与房间整体布局之间建立联系。
每个任务开始时，模型会拿到目标实体清单与初始图像。实体库告诉它需要寻找哪些对象，但不透露坐标。模型可以主动拍摄、移动相机、读取相机内参与位姿，并将多视角中的对应观测交给三角测量工具；确认位置后，可随时分批提交并继续修正。每个任务最多进行100轮模型交互。
评分同时关注「找得全不全」和「放得准不准」：全部目标与已提交目标各有四项指标——平均IoU，以及IoU超过0%、50%、90%的比例。未提交目标在全部物体指标中记为零。IoU使用平移至预测中心的原有渲染几何包围盒计算，重点衡量中心定位与目标覆盖。
表1 基于正常执行finish_mapping结束的轨迹，按任务求平均（macro）。
当前7个模型、8种配置的比较中，Opus 5.5的任务平均IoU为41.71%，Astra（xhigh）为33.16%，Sonnet 5.5为32.28%。最高得分仍留下明显提升空间，也让观察决策与空间推断中的薄弱环节更值得研究。
图3 从准确布局到物体缺失、位置偏移，重建画面直观呈现了不同的成功与失败形式。
看过一个物体，并不意味着已经确定了它的空间位置。如果各次观察之间缺少稳定的对应，图像数量的增加就可能只带来更多局部印象。模型还需要辨认同一物体在不同视角中的同一位置，并选择能够提供有效视差的拍摄点，才能建立可靠的几何约束。
图4 探索过程中不同类型工具的调用分布，展示了观察、相机移动与建图操作之间的调度差异。
工具调用分布展示出值得关注的现象：部分模型在探索中反复观察、频繁移动相机，后续的mapping操作却没有充分跟上。它们可能收集了不少图像，却未能有效组织跨视角信息，用几何计算把「看见了」落实为「定位到了」。
三角测量尤其考验精细化空间感知。输入需要来自不同拍摄位置的对应图像点；物体中心标记偏移、遮挡和视差不足，都可能影响估计。在这一环节，观察是否精准、计算是否及时、结果是否经过核对，比单纯增加拍摄次数更值得关注。
在共同的80个任务上，Astra最终提交的物体中，99.33%此前获得过成功三角测量支持；DeepSeek为63.48%。这一差异为「观察怎样转化为定位证据」提供了可分析的切入口。
这些轨迹线索支持一个改进方向：提升跨视角对应的精度，并加强观察、计算与提交之间的协同。工具次数本身不能直接测量观测质量，上述分析属于描述性证据。图中mapping还包含位置提交与结束建图，不应解释为纯计算量；后期柱高下降也会受任务结束影响。
SpatialScout的作用还在于让过程变得可追踪：缺失的物体可能指向覆盖不足，偏移的布局可能指向坐标推断不稳，而大量观察后的低分则提示需要检查视觉对应与几何工具使用。研究者可以沿着这些线索，设计更有针对性的探索策略。
持续拍摄、长上下文推理和多轮工具交互都会消耗资源。SpatialScout将重建IoU与每任务平均费用放在同一张图中，用Pareto前沿帮助观察不同预算下的质量选择。
图5 模型的质量—费用分布。平均费用以美元计，横轴使用对数刻度，虚线连接Pareto前沿配置。
按当前保存的价格与token记录估算，Astra（medium）平均约2.47美元/任务，Astra（xhigh）约3.06美元/任务；Opus5.5约为27.90美元/任务。DeepSeekV4.1 Flash的均值约为0.027美元/任务，其对应的定位得分也较低。这组结果让探索策略的质量收益与资源投入能够被同时讨论。
费用使用OpenRouter固定价格快照，并仅对token记录完整的任务求均值。Cost图中的IoU为按物体汇总的micro指标，前文表格为逐任务平均的macro 指标；费用是价格快照下的估算，不代表实际账单。Astra medium延续先前IoU与新运行费用均值的既定统计口径。
从选择一个新视角，到确认一组空间坐标，再到形成可用的物体级地图，主动空间理解需要一整条能力链。SpatialScout提供固定任务、交互工具与评分流程，方便研究者接入自己的模型，把新的视角规划、多视角推理和几何工具策略放到同一个环境里比较。
这个基准能帮助更多研究把注意力放到真实行动所需的空间基础上：让智能体知道自己在哪里、对象在哪里，并据此决定接下来该怎样观察与行动。
参考资料：https://arxiv.org/abs/2610.26659
编辑：LRST
不喜欢
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
New Ji-won.
Imagine a mission: give the camera to AI and let it enter a virtual room that it has never seen before. There were no pre-determined coordinates of the object, nor were the tour routes arranged for it. It needs to decide where to turn, where to move, when to take another picture, and eventually clarify the position of each target object in three-dimensional space.
Paper link: https://arxiv.org/abs/2610.26659
Code link: https://github.com/JesseZZZZ/SpatialScout
Data set link: https://huggingface.co/datasets/jessezhaoxizhang/SpatialScout
SpatialScout has turned this exploration process into a research mission that can be documented, rated and compared. Each observation, whether it is useful, whether different perspectives are mutually verifiable and whether the coordinates submitted are reliable, leaves a mark on the final object-scale map.
Figure 1 One space exploration consists of camera operations, multi-perspective observations and object position estimates.
Let the model personally select the next observation: from the RGB image, estimate the object centre by camera control and geometry tool. Exploration strategies and spatial extrapolations are examined in the same interactive closed circle.
When people walk into the house and look around and circle around, they usually have a consistent view of the overall layout: The table is near which wall, the chair is on which side of the table, and what direction is the object just after turning around? Such a three-dimensional spatial representation provides a basis for approaching, detouring, reaching and taking.
Physical AI also needs this foundation. For AI agent to continue to operate in a real environment, it is necessary to organize scattered visual clues into stable spatial relationships. Basic location and layout are difficult to determine by observation, and subsequent operational planning, object operation and environmental interaction lack reliable reference. SpatialScout's focus on active space understanding is a cornerstone of the movement towards physical AI.
80 indoor scenes and 3,140 target examples were officially evaluated from iTHOR, Robothor, ArchitecTHOR, RepicaCAD and ProcTHOR. The types of rooms are kitchens, living rooms, bedrooms, bathrooms, apartments and multi-room dwellings. Fixed scenes and initial camera settings that allow different models to face consistent mission conditions.
Figure 2 Dataset category word cloud. The figure is based on 3,440 examples from the complete 90-scenario catalogue, which are grouped under the original generic rule of 143; the official results are based on 80 scene assessment protocols.
The chairs, tables, cabinets, books, lamps and daily supplies together constitute these environments. The scale, placement and visibility of the object vary, and the model needs to constantly adjust its perspective and establish links between local details and the overall layout of the room.
At the beginning of each mission, the model receives a list of target entities and initial images. The physical library told it what objects it needed to look for, but did not disclose the coordinates. The model allows for active filming, mobile cameras, reading of the camera ' s engagement position and handing over the corresponding observations from multiple perspectives to the triangulation tool; once the location is confirmed, it can be submitted in batches and continue to be amended. A maximum of 100 rounds of model interaction per mission.
In addition, the rating is focused on "unfinished search" and "unpredictable release": All the targets and each of the presented targets have four indicators — average IOU, and IOU over 0 per cent, 50 per cent, and 90 per cent. Targets not submitted are recorded as zero in all object indicators. IoU uses flat to calculate the original rendering geometric encirclement box with a focus on the location of the centre and target coverage.
Table 1 is based on an average mission (macro) based on normal execution of the finish track of finish mapping.
For the current comparison of seven models and eight configurations, the average IoU for Opus 5.5 tasks was 41.71 per cent, Astra (xhig) was 33.16 per cent and Sonet 5.5 was 32.28 per cent. The maximum scores still leave room for significant improvement and make it more worthwhile to observe weaknesses in decision-making and spatial extrapolation.
Figure 3 presents different forms of success and failure in the visualization of the reconstruction picture, from accurate layout to missing objects and shifting positions.
Seeing an object does not mean that its space position has been determined. If there is no stable correspondence between the observations, the increase in the number of images may give only more local impressions. Models also need to identify the same position of the same object in different perspectives and to select sites that can provide effective vision to establish reliable geometric restraints.
Figure 4. The distribution of the different types of tools in the exploration process shows the movement differences between observation, camera movement and charting operations.
The distribution of tool call shows a phenomenon of concern: some models are repeatedly observed in exploration, moving cameras frequently, and subsequent mapping operations are not fully up to date. They may have collected a lot of images, but they have not been able to effectively organize cross-perspective information and use geometrical calculations to translate "see" into "positioned".
Triangular measurements test precision spatial perception in particular. The input requires a corresponding image point from different locations; the central mark of the object is deviating, shielding and inadequate vision, which may affect estimates. At this stage, the accuracy of the observations, the timeliness of the calculations and the verification of the results are more of concern than simply an increase in the number of shots taken.
Of the 80 common missions, 99.33 per cent of the objects eventually submitted by Astra had previously been supported by successful triangulation; and 63.48 per cent of DeepSeek. This difference provides an analytical entry point for "observing how to translate into positioning evidence".
These trajectories support an improvement direction: improving accuracy across perspectives and enhancing synergy between observations, calculations and submissions. The number of tools per se cannot directly measure the quality of observations, and the above analysis is descriptive evidence. Mapmapping also contains locational submissions and end-of-mission maps, which should not be interpreted as pure calculations; the fall in post-production column height will also be affected by the end of the mandate.
SpatialScout also plays a role in making the process traceable: missing objects may point to under-coverage, shifting layouts may point to indeterminate coordinates, and large-scale observations of low-scoring points point to the need to check visual resonance with geometric tools. Researchers can follow these leads and design more targeted exploration strategies.
Continuous filming, long context reasoning and multi-wheeling of tools will consume resources. SpatialScout places the reconstruction of IoU in the same picture as the average cost per mission, using the Paretto front to help observe quality choices under different budgets.
Figure 5 Quality-cost distribution of models. The average cost is in United States dollars, with a logarithmic scale used for cross-axis, and a no-line connection to the Paretto front configuration.
Based on current preservation prices and token records, Astra (mediam) averages about $2.47/mission, Astra (xhigh) about $3.06/mission, and Opus 5.5 about $27.90/mission. The average value of DeepSeekV4.1 Flash is about $0.027/task, and its corresponding positioning scores are low. This set of results allows the qualitative benefits of exploration strategies to be discussed at the same time as resource inputs.
The cost is based on a fixed price snapshot of OpenRouter and averages only the task for which token records are complete. The IoU in the Costa chart is an object-based micro indicator and the previous table is a mission-by-mission average macro indicator; the cost is a price snapshot estimate and does not represent the actual billing. Astra medium continues the previously established statistical calibration of IOU with the average of new operating costs.
From the selection of a new perspective to the identification of a set of spatial coordinates, to the formation of a usable object-level map, the active space understanding requires an entire chain of capabilities. SpatialScout provides fixed missions, interactive tools and scoring processes that facilitate researchers ' access to their own models and compare new perspective planning, multi-perspective reasoning and geometric tools strategies in one environment.
This benchmark helps to focus more research on the space needed for real action: let AI agent know where he is and where he is, and decide on what to do next.
References: https://arxiv.org/abs/2610.26659
Edit: LRST
I don't like it.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.