Neuromorphic event-based vision: Sparse and spiking networks for efficient vision systems

This thesis investigates the potential of event-based vision systems in Driver Monitoring Systems (DMS), addressing the critical need for fast, efficient sensors in automotive safety. The journey begins with conventional computer vision algorithms designed for frame-based cameras, gradually evolving toward methods optimized for the sparse, asynchronous data produced by event cameras. As these unconventional sensors operate fundamentally different, they bring new possibilities, but also require tailored approaches in data processing and neural network design. This exploration led to a progressively deeper understanding of sparsity in neural models, from dense networks to sparse processing units, and ultimately to the adoption of binary spike representations known for their optimal efficiency in encoding event-driven data. To harness the full potential of event data, the research moved from initial experiments with traditional networks to developing specialized architectures uniquely suited to event- driven inputs. Early efforts included a multitask network for driver facial analytics, tackling tasks like gaze estimation and head pose detection. These initial approaches were soon complemented by an exploration of YOLO for out-of-cabin object detection, which revealed the limitations of conventional networks in leveraging event-based data. Recognizing these challenges, the thesis shifted focus toward architectures and methods explicitly designed for event processing, including a submanifold convolutional network for driver distraction detection, a Sigma Delta Neural Network (SDNN) for event stream super-resolution, and spiking-based networks for Distraction Detection & Face Detection. The result is a novel hybrid model that combines Spiking Neural Networks (SNN) and Artificial Neural Net- works (ANN) to balance temporal precision with the robustness needed for real-world DMS applications. Noise resilience techniques, such as event bias tuning and real-time filtering, were also developed to improve data quality under challenging conditions. The findings show that both the event representation and the processing pipeline are essential to realize the advantages of event cameras. Many of the methods presented here are among the first of their kind in the event domain, consistently achieving competitive state-of-the-art results across diverse DMS tasks. This thesis lays a strong foundation for future event-based DMS systems, demonstrating that optimized event representations and processing architectures hold significant promise for practical, high-efficiency deployment.

Funder

Research Ireland

Publisher

University of Galway

Rights

Attribution-NonCommercial-NoDerivatives 4.0 International

cbn

Collections

University of Galway Theses (PhD Theses)