How Generative AI Could Help Make Construction Sites Safer
Construction sites are among the most dangerous workplaces in the United States, with over 1,000 worker deaths annually due to slips, trips, and falls—the leading causes of fatal accidents in the industry. Last winter, a tragic incident on Martha’s Vineyard saw a 32-year-old worker, Jose Luis Collaguazo Crespo, fall from a ladder and lose his life. This grim reality highlights the urgent need for better safety monitoring and prevention measures on construction sites.
The Safety-Productivity Dilemma
Despite safety being often touted as the top priority on job sites, entrepreneur and executive Philip Lorenzo points out a persistent tension between safety and productivity. Speaking at Construction Innovation Day 2025 at the University of California, Berkeley, he emphasized how shortcuts and risk-taking often undermine safety protocols in favor of meeting deadlines or cutting costs. It is in this challenging environment that technological solutions like artificial intelligence (AI) are proving to be crucial.
Introducing Safety AI: A Generative AI for Construction Safety
Philip Lorenzo is developing an innovative AI tool for DroneDeploy, a San Francisco-based company known for its software that creates daily digital models of construction sites through reality capture—videos and images documenting work progress. The new tool, called Safety AI, employs generative AI to analyze daily imagery and flag onsite conditions that violate Occupational Safety and Health Administration (OSHA) regulations.
What sets Safety AI apart is its reported 95% accuracy in identifying specific OSHA violations. Since its launch in October 2024, Safety AI has been deployed on hundreds of construction sites across the US, with customized versions adapted for building regulations in Canada, the UK, South Korea, and Australia.
Beyond Object Recognition: AI That “Reasons”
Most existing AI safety tools rely on teams of human annotators—often based in low-wage countries—to label images by drawing bounding boxes around objects like ladders and hard hats. These datasets train the algorithms to recognize objects, a technique known as object detection.
In contrast, Safety AI uses a visual language model (VLM), a type of generative AI that combines a large language model (LLM) with vision processing capabilities. This allows the AI not just to identify objects but to “reason” about what is happening in an image. For example, it can assess how a worker is using a ladder and determine if their behavior complies with OSHA rules, such as whether they are properly using three points of contact or improperly standing on the top step.
This level of logical reasoning enables Safety AI to perform more nuanced analyses—such as answering layered questions about worker behavior—to conclude whether safety practices are being followed.
Training the AI: Building a “Golden Data Set”
Lorenzo’s team has compiled a vast repository of tens of thousands of images depicting OSHA violations from customers who have given permission for their data to be used. This “golden data set” is critical to training the VLM to detect a variety of common risks.
Human safety experts help train the AI by posing specific questions about test images and refining its responses. If the model misses a violation or raises false alarms, the team adjusts the training inputs or prompts, effectively teaching the AI to approach safety analysis much like an experienced human inspector.
Challenges and Limitations
Despite its promise, Safety AI is not infallible. Chen Feng, who leads New York University’s AI4CE lab specializing in construction robotics and scene understanding, acknowledges that while 95% accuracy is encouraging, the remaining 5% failure rate poses critical challenges. A 2024 study coauthored by AI pioneer Yann LeCun highlighted some systematic shortcomings in VLMs, including difficulties with 3D spatial reasoning from 2D images and a lack of common-sense understanding about scenes.
Lorenzo admits that VLMs have major flaws in spatial reasoning, so Safety AI supplements generative AI with traditional machine learning techniques such as image segmentation and photogrammetry (creating 3D models from 2D images) to better understand site layouts. The AI is especially trained in ten different problem areas, including ladder safety, which accounts for nearly a quarter of fall-related deaths in construction.
While the AI may miss some edge cases, its assistance is valuable for overworked safety managers responsible for multiple sites, who benefit from receiving timely digital alerts instead of making frequent in-person inspections.
Workers’ Concerns and the Human Element
Aaron Tan, a concrete project manager in the San Francisco Bay Area, sees potential for tools like Safety AI to save time and enhance safety monitoring. However, he also notes that workers may fear such technology becomes “bossware” – a surveillance tool used to punish rather than protect employees. Previous experiences with security cameras, for example, have led to concerns about privacy and constant monitoring.
Lorenzo agrees that AI will not replace human oversight but rather serve as an additional set of “digital eyes” to support safety professionals. The software’s success depends on careful integration and maintaining human judgment in monitoring and decision-making.
Traditional Methods Still in Play
Not all companies have embraced generative AI. Izhak Paz, CEO of Safeguard AI in Jerusalem, remains committed to traditional machine-learning methods augmented by human intervention. His approach involves collecting extensive labeled footage to train algorithms specifically tailored to recognize particular hazards, followed by rigorous risk assessments.
Safeguard AI integrates real-time video feeds from cameras at construction sites and sends actionable alerts to site managers’ mobile devices. The system’s practical and reliable design has made it popular among mid-market builders managing multiple sites across Israel, the US, and Brazil.
The Future of AI in Construction Safety
As generative AI tools like Safety AI evolve, they hold great promise for reducing preventable injuries and fatalities on construction sites. However, challenges with AI accuracy, spatial reasoning, and worker privacy remain critical issues to address. The best outcomes are likely to arise from a hybrid approach where AI augments, but does not replace, experienced human safety inspectors.
By combining cutting-edge AI advancements with established safety practices and mindful implementation, the construction industry may be poised to significantly improve workplace safety and save lives in the coming years.
