XAI + Construction Site Safety Monitoring System

XAI-SMS: eXplainable AI-Integrated Construction Site Safety Monitoring System

Summary

This research aims to explore the practical effectiveness of existing Explainable AI (XAI) techniques in construction safety applications, with a focus on fall detection—one of the leading causes of workplace injuries. Despite the increasing adoption of AI-driven safety solutions, their opaque decision-making processes pose challenges in trust, usability, and regulatory compliance. To bridge this gap, this study proposes the development of an XAI-integrated vision-based fall detection alert framework designed to deliver practical and actionable insights tailored for enhancing construction site safety. The Craig and Diane Martin National Center for Construction Safety, University of Kansas, will support this project by facilitating real-world implementation and validation. The project will focus on generating preliminary findings on the practicality of an XAI-integrated vision-based safety monitoring system, emphasizing three widely used explainability approaches: feature saliency-based, perturbation-based, and concept-based explanations. This framework is designed to triangulate the construction safety insights, ensuring a comprehensive and interpretable understanding of safety risk. Stakeholder feedback from construction safety professionals will be collected to assess the practical utility and acceptance of XAI-driven alerts.

Funding Agency:
South Dakota State University | 2026 RSCA Challenge Fund

Team:
Chulwoo Pack (PI) | McComish Dept. of EECS, SDSU
Phuong Nguyen (Co-PI) | Construction & Concrete Industry Management, SDSU
Chien-Ho Ko (Collaborator) | Department of Civil, Environmental & Architectural Engineering University of Kansas
Omeshamisu Anigala (PhD Student) | McComish Dept. of EECS, SDSU Preethi Amasa (M.S. Student) | McComish Dept. of EECS, SDSU

Duration:
2025-2026

Total Funding:
$15,000

External Resources:

Multimodal Video Anomaly Detection Forthcoming
XAI + Video Anomaly Detection Forthcoming
Multimidal Dataset Cleaning Forthcoming

Video anomaly detection is crucial for applications like surveillance and autonomous systems. Traditional methods often rely solely on visual cues, missing valuable contextual data. This paper presents Perceptual and Analytical Representation Learning (PEARL), a novel method that combines perceptual (raw sensory input) and analytical (higher-level context) modalities. Specifically, we integrate visual information with object tracking data, along with the tracking data-specialized normalization method, DOT-Norm, leveraging ID switching to capture high-level contexts of abnormal movements. We evaluate early- and late-fusion strategies to enhance anomaly detection, particularly for irregular movements marked by frequent track ID switches. Our approach, tested on the UCSD-Ped1 dataset, outperforms the state-of-the-art by improving precision (+0.082), recall (+0.104), F1 score (+0.149), and AUC (+0.053). These findings highlight the potential of integrating analytical tracking data with perceptual video frames in a multimodal learning approach for anomaly detection, paving the way for future applications and research where knowledge-driven analytical modalities are crucial.