Detecting anomalies from roadside video streams

Abstract

The interconnected nature of road networks implies that anomalies on narrow residential roads can ripple through the entire traffic system, particularly in high‐ traffic areas as common for the Maltese Islands. Detecting anomalies in such en‐ vironments using roadside cameras is challenging due to the multitude of normal and anomalous events, changes in illumination, obstructions, complex anomalies, and difficult viewing angles. This thesis investigates anomaly detection methods tailored to the realistic road and data limitations typical of Maltese urban roads. Classical anomaly detection, which identifies anomalies from structured data, and deep learning‐based techniques, which detect anomalies directly from video input, were evaluated. The literature review revealed limited evaluations on realistic datasets for both methods. The classical method was developed to filter out ID switch artifacts and identify specific anomalies using a combination of filtering, DBSCAN clustering, masking, and rule‐based techniques. For the deep learning method, an AE model with the STAE [1] architecture was chosen for its ability to capture temporal rep‐ resentation. Both methods were evaluated on video datasets collected in Malta and a relabeled Street Scene [2] dataset. The classical method demonstrated high reliability in detecting anomalies in structured data, achieving an 82% true positive rate and a 3% false positive rate for a local dataset. However, the data acquisition method did not accurately record all anomalies, reducing the true positive rate for actual video anomalies. The deep learning method showed strong performance across all datasets, achiev‐ ing an 83% AUC and a 25% EER for a dataset recorded in the same location. Per‐ formance was slightly reduced for locations with heavy shadows, as shown on a second local dataset. Segmenting frames into tiles and augmenting datasets improved performance in shadow‐affected conditions, as did masking irrelevant regions. An event‐level comparison showed both methods performed similarly in detecting non‐typical vehicle paths. The classical method excelled at identifying non‐typical object locations and was more robust against changes in scene dynam‐ ics, is more modular, and easier to debug. The deep learning method was better at detecting non‐typical slow‐moving and non‐typical vehicles and was more resilient to variations in the data acquisition method within the Intelligent Traffic System (ITS). However, neither method effectively detected unforeseen anomalies. Over‐ all, this thesis provides valuable insights and guidance for choosing the most ap‐ propriate anomaly detection methods tailored to different types of anomalies in complex urban road environments.