CONVR 315: From Visual Perception to Regulatory Cognition: A VLM and Knowledge Graph-Integrated

A VLM and Knowledge Graph-Integrated Framework for Construction Safety Monitoring
Authors: Xiaowen Guo, Peter Kok-Yiu Wong, Jack C.P. Cheng
Department of Civil Engineering, The Hong Kong University of Science and Technology, Hong Kong S.A.R. (China)

Abstract:
This paper presents an AI framework that automates construction safety regulation retrieval by integrating visual scene analysis with structured regulatory knowledge. To overcome the limitations of general-purpose Vision-Language Models (VLMs) and manual knowledge curation, this study introduce a structured VLM analyzer guided by the 4M1E principle for generating focused site descriptions and an LLM-based agent for automatically parsing regulations into a dynamic knowledge graph. This graph represents safety rules as discrete, complete concepts. A semantic retrieval is conducted to match the scene description against these rule nodes. Our method achieves an F1-score of 97.57% for knowledge graph generation and regulation retrieval accuracy to 88.57%. The results validate that aligning structured visual perception with a semantically organized knowledge base is crucial for accurate and automated safety monitoring.

Видео CONVR 315: From Visual Perception to Regulatory Cognition: A VLM and Knowledge Graph-Integrated канала Digital Roads of the Future