INHerit-SG: Incremental Hierarchical Semantic Scene Graphs with RAG-Style Retrieval
Published in ICRA 2026 Workshop on Robots Meet Prior Maps (Oral, Best Presentation Finalist), 2026
Abstract
Driven by advancements in foundation models, semantic scene graphs have emerged as a prominent paradigm for high-level 3D environmental abstraction in robot navigation. However, existing approaches are fundamentally misaligned with the needs of embodied tasks. As they rely on either offline batch processing or implicit feature embeddings, the maps can hardly support interpretable human-intent reasoning in complex environments. To address these limitations, we present INHerit-SG. We redefine the map as a structured, RAG-ready knowledge base where natural-language descriptions are introduced as explicit semantic anchors to better align with human intent. An asynchronous dual-process architecture, together with a Floor-Room-Area-Object hierarchy, decouples geometric segmentation from time-consuming semantic reasoning. An event-triggered map update mechanism reorganizes the graph only when meaningful semantic events occur. This strategy enables our graph to maintain long-term consistency with relatively low computational overhead. For retrieval, we deploy multi-role Large Language Models (LLMs) to decompose queries into atomic constraints and handle logical negations, and employ a hard-to-soft filtering strategy to ensure robust reasoning. This explicit interpretability improves the success rate and reliability of complex retrievals, enabling the system to adapt to a broader spectrum of human interaction tasks. We evaluate INHerit-SG on a newly constructed dataset, HM3DSem-SQR, and in real-world environments. Experiments demonstrate that our system achieves state-of-the-art performance on complex queries, and reveal its scalability for downstream navigation tasks.
🏆 Oral Presentation · Best Presentation Finalist — ICRA 2026 Workshop on Robots Meet Prior Maps
This paper presents INHerit-SG, an incremental hierarchical semantic scene graph generation method with RAG-style retrieval capabilities.
Real-Robot Demos
Captured on a real robot during deployment and data collection.
Recommended citation:
Yuk Tung Samuel Fang, Zhikang Shi, Jiabin Qiu, Zixuan Chen, Jieqi Shi, Hao Xu, Jing Huo, Yang Gao. (2026). "INHerit-SG: Incremental Hierarchical Semantic Scene Graphs with RAG-Style Retrieval." ICRA 2026 Workshop on Robots Meet Prior Maps. Oral Presentation, Best Presentation Finalist.