How Generative AI Could Enhance Safety on Construction Sites
By Andrew Rosenblum | Published July 2, 2025 | MIT Technology Review
Construction sites are inherently risky environments—each year in the United States alone, over 1,000 construction workers die from job-related accidents, making it one of the most hazardous industries. Fatal falls from ladders, in particular, are a leading cause of these tragedies, accounting for nearly a quarter of deaths on the job. Now, groundbreaking advances in generative artificial intelligence (AI) promise to improve safety monitoring and reduce these risks, though experts caution that technology should augment, not replace, human oversight.
From Tragedy to Technology
Last winter, a 32-year-old worker named Jose Luis Collaguazo Crespo died after falling from a ladder at a housing construction site in Martha’s Vineyard, Massachusetts. His death is a stark reminder that despite widespread acknowledgment that “safety is the number-one priority,” shortcuts and unsafe practices persist in the rush to maintain productivity.
Philip Lorenzo, an entrepreneur leading innovation at the San Francisco–based company DroneDeploy, spoke about this challenge during Construction Innovation Day 2025 at the University of California, Berkeley. DroneDeploy is developing an AI-driven safety monitoring tool called Safety AI, designed to detect OSHA (Occupational Safety and Health Administration) violations on construction sites through daily “reality capture” imagery—comprehensive digital models created from videos and photos of the worksite.
According to Lorenzo, Safety AI flags safety concerns with approximately 95% accuracy, identifying conditions that break OSHA rules. Since its launch in October 2024, the tool has been deployed on hundreds of sites across the U.S., with customized versions adapted for regulations in Canada, the UK, South Korea, and Australia.
Going Beyond Object Detection with Visual Language Models
While many existing AI safety tools use traditional computer vision to detect objects like ladders or hard hats, Safety AI employs an advanced form of generative AI known as a visual language model (VLM). This is a type of large language model (LLM) equipped with a vision encoder, enabling it to analyze images in context rather than simply recognize objects.
“The VLM doesn’t just identify a ladder; it ‘reasons’ about what’s happening in the scene,” Lorenzo explains. For example, the model can determine whether a worker is standing unsafely on the top rung or if they are using a ladder improperly as stilts to move around. Through a multi-layered line of inquiry—more than a dozen steps of questioning—the AI assesses safety conditions and cross-references relevant OSHA regulations.
This sophisticated reasoning capability is supported by a “golden data set” of tens of thousands of labeled images of safety violations, painstakingly compiled over years from consenting customer sites. Human safety professionals collaborate closely with the AI, providing feedback and refining its responses through an iterative training process that aims to teach the model “how to think” safely on a construction site.
Challenges Remain: The Missing 5%
Despite promising results, Safety AI’s approximate 95% accuracy rate leaves a critical 5% margin where unsafe conditions could be missed or falsely flagged. This limitation highlights key challenges with current VLM technology, particularly when applied to complex, dynamic environments like construction sites.
Chen Feng, director of New York University’s AI4CE lab, notes that VLMs face difficulties with spatial reasoning and interpreting three-dimensional scene structures from two-dimensional images. Studies, including a 2024 paper coauthored by AI pioneer Yann LeCun, point to “systematic shortcomings” of VLMs when tackling tasks beyond straightforward object detection.
To address this, Safety AI integrates traditional computer vision techniques, such as image segmentation and photogrammetry (which builds 3D models from 2D images), to supplement the VLM’s imperfect spatial reasoning. Lorenzo acknowledges that some edge cases will elude perfect detection, but argues that the system still provides significant value as a “digital pair of eyes” for safety managers who might oversee multiple sites simultaneously.
Adoption and Worker Trust
Workers and safety managers see value in AI safety tools for offering timely alerts and reducing the need for numerous on-site visits. Aaron Tan, a project manager in the San Francisco Bay Area, believes tools like Safety AI can help overworked personnel monitor large construction portfolios more efficiently, potentially saving lives.
However, concerns remain among workers about surveillance and privacy. Tan recounts past resistance to the use of cameras, with employees fearing “Big Brother” oversight and the perception that such technologies are “bossware” designed to catch them making mistakes instead of promoting safety.
Balancing New and Established Technologies
Not all companies are rushing to adopt cutting-edge generative AI. Izhak Paz, CEO of Safeguard AI located in Jerusalem, opts for older, more traditional machine-learning methods in his construction safety solution. He values the reliability and human-in-the-loop model that allows his system to better handle deviations and anomalies by leveraging human oversight during algorithm refinement—a process that can take weeks or months depending on the hazard category.
Safeguard AI deploys its technology on roughly 3,500 sites spanning Israel, the U.S., and Brazil, using real-time video feeds and AI agents to assess risk and provide actionable alerts to site managers via mobile devices. Paz suggests this approach strikes the right balance between automation and human judgment, especially tailored for mid-market builders managing multiple active projects.
The Future of Construction Site Safety
Emerging generative AI tools like Safety AI demonstrate the potential to revolutionize construction safety by providing automated, context-aware analysis of complex, ever-changing sites. Yet, the technology is not foolproof and benefits greatly from human expertise combined with multiple AI methodologies.
As the construction industry continues to embrace digital innovation, the most effective safety strategies will likely blend advanced AI with experienced human supervision, earning the trust of workers and managers alike while ultimately striving to save lives and reduce preventable accidents on the job.
Related stories:
-
How AI Can Help Spot Wildfires: New technologies could detect fires faster, giving first responders a head start.
-
Profiling Buildots: A Tel Aviv-based company innovating in construction tech.
Image Credit: Courtesy of DroneDeploy
