Browse State-of-the-Art › 3D Question Answering (3D-QA)
3D Question Answering (3D-QA)
17 papers with code · 3 benchmarks · 5 datasets archive 2025-07-28
A 3D-QA task requires models to answer a question when given all the information of a 3D scene. Here, models use the 3D spatial information, such as RGB-D scans or point cloud data. We also require models to specify the 3D-bounding boxes of objects that are related to this question answering. This prevents models from answering questions by relying on the textual priors of the trained questions without examining the scene. However, unlike 3D dense captioning, we do not require models to target one described object for each question. This is because multiple objects can be used to answer certain questions. For example, the question “What color is the chairs around the table?” is related to multiple objects. This question is also answerable as long as the chairs around the unique table in the scene have the same color. In such scenarios, we require models to answer the question addressing multiple 3D-bounding boxes.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
3 leaderboard tables shown for this task, 3 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| ScanQA Test w/ objects (18 rows) | BridgeQA | Bridging the Gap between 2D and 3D Visual Question Answering: A... | code | Syntology ran 10 of 13 samples · 3 unverified | Compare |
| SQA3D (13 rows) | LLaVA-3D | LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with... | — | — | Compare |
| 3D MM-Vet (5 rows) | ShapeLLM-13B | ShapeLLM: Universal 3D Object Understanding for Embodied Interaction | code | Syntology ran 9 of 17 samples · 8 unverified | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
5 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Most implemented papers archive 2025-07-28
17 shown of 17 papers with code (22 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.
-
17 Apr 2023 13 repositories listed Syntology ran 16 of 51 samples · 35 unverifiedInstruction tuning large language models (LLMs) using machine-generated instruction-following data has improved zero-shot capabilities on new tasks, but the idea is less explored in the multimodal field.
-
1 Sep 2023 5 repositories listed Syntology ran 13 of 20 samples · 7 unverified · 7 pointer-only (licence)We introduce Point-Bind, a 3D multi-modality model aligning point clouds with 2D image, language, audio, and video.
-
24 Jul 2023 5 repositories listed Syntology ran 5 of 10 samples · 5 unverified · 10 pointer-only (licence)Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs.
-
27 Feb 2024 3 repositories listed Syntology ran 9 of 17 samples · 8 unverifiedThis paper presents ShapeLLM, the first 3D Multimodal Large Language Model (LLM) designed for embodied interaction, exploring a universal 3D object understanding with 3D point clouds and languages.
-
28 Nov 2023 3 repositories listed Syntology ran 7 of 10 samples · 3 unverifiedWith the rapid development of Multi-modal Large Language Models (MLLMs), a number of diagnostic benchmarks have recently emerged to evaluate the comprehension capabilities of these models.
-
31 Aug 2023 3 repositories listedThe unprecedented advancements in Large Language Models (LLMs) have shown a profound impact on natural language processing but are yet to fully embrace the realm of 3D understanding.
-
6 Aug 2024 2 repositories listedWe present LLaVA-OneVision, a family of open large multimodal models (LMMs) developed by consolidating our insights into data, models, and visual representations in the LLaVA-NeXT blog series.
-
13 Dec 2023 2 repositories listed Syntology ran 11 of 11 samples · 0 unverified · 5 pointer-only (licence)Recent advancements in 3D Large Language Models (LLMs) have demonstrated promising capabilities for 3D scene understanding.
-
4 Dec 2023 2 repositories listed Syntology ran 10 of 15 samples · 5 unverifiedWe conduct extensive experiments to evaluate the performance and generalizability of our model.
-
28 Mar 2025 1 repository listed Syntology ran 1 of 8 samples · 7 unverifiedOur evaluation of state-of-the-art 3D-VL models on Beacon3D reveals that (i) object-centric evaluation elicits true model performance and particularly weak generalization in QA; (ii) grounding-QA coherence remains…
-
5 Mar 2025 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)In this paper, we propose a Dual-vision Scene Perception Network (DSPNet), to comprehensively integrate multi-view and point cloud features to improve robustness in 3D QA.
-
30 Nov 2024 1 repository listedEfforts to enhance MLLMs, such as incorporating point cloud features, have been made, yet a considerable gap remains between the models' learned representations and the inherent complexity of 3D scenes.
-
4 Sep 2024 1 repository listed Syntology ran 3 of 3 samples · 0 unverified · 1 pointer-only (licence)Comprehensive evaluations on MSQA and MSNN highlight the limitations of existing vision-language models and underscore the importance of handling multi-modal interleaved inputs and situation modeling.
-
24 Feb 2024 1 repository listed Syntology ran 10 of 13 samples · 3 unverified · 13 pointer-only (licence)In 3D Visual Question Answering (3D VQA), the scarcity of fully annotated data and limited visual content diversity hampers the generalization to novel scenes and 3D concepts (e.
-
18 Nov 2023 1 repository listed Syntology ran 3 of 13 samples · 10 unverifiedHowever, several significant challenges remain: (i) most of these models rely on 2D images yet exhibit a limited capacity for 3D input; (ii) these models rarely explore the tasks inherently defined in 3D world, e.
-
8 Aug 2023 1 repository listed Syntology ran 4 of 6 samples · 2 unverified3D vision-language grounding (3D-VL) is an emerging field that aims to connect the 3D physical world with natural language, which is crucial for achieving embodied intelligence.
-
20 Dec 2021 1 repository listedWe propose a new 3D spatial understanding task of 3D Question Answering (3D-QA).
Syntology lines on 13 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections