Auburn researcher wins $417K NSF award for explainable video surveillance

Pan He, an assistant professor of computer science and software engineering at Auburn University, has received a $417K National Science Foundation award to develop more transparent AI-powered video surveillance systems. The project, conducted with Muchao Ye from the University of Iowa, aims to use vision-language models to identify important events, synthesize evidence across cameras and time, and produce conclusions that operators can understand and verify.

The research targets a persistent problem in modern surveillance: video volumes have outpaced human monitoring capacity, while many automated tools issue alerts without clear reasoning. He’s approach would allow a system to flag events such as a vehicle traveling the wrong way, point to the relevant footage and explain the observations behind the finding. The tools are also designed to support natural-language search, letting users describe an event and retrieve matching clips instead of reviewing long recordings manually.

Human oversight remains central to the work. He said automated systems can make mistakes because of lighting, camera angles, crowds, shadows or unfamiliar environments, so people must be able to question results and make final decisions. The team will evaluate the technology in workplace safety and traffic-monitoring settings, with attention to privacy, responsible deployment and whether the tools improve decision-making without encouraging misplaced trust.