Object segmentation: from neuromorphic sensing to neuromorphic machine learning

Autonomous robotic systems are increasingly deployed in a variety of applications, ranging from industrial automation to search and rescue missions. A fundamental challenge for these systems lies in effective perception and navigation, especially in environments with complex dynamics or limited visibility. Neuromorphic vision systems address these challenges, featuring attributes like high dynamic range, low latency, and microsecond-level temporal resolution. However, utilizing neuromorphic vision for advanced autonomous perception in robotics remains a largely untapped area. The objective of this dissertation is to advance the field of autonomous perception in robotics through the application of neuromorphic vision systems. In achieving this aim, three major contributions are introduced:(i) First, this work introduces Bimodal SegNet, a novel encoder-decoder architecture equipped with cross attention mechanisms. This architecture is tailored to enhance multi-modal signal processing, particularly for the task of segmentation. (ii) Second, the Graph Mixer Neural Network (GMNN)is developed, which features a unique Collaborative Contextual Mixing (CCM) layer. This innovation enhances panoptic segmentation capabilities in event-based vision systems by effectively leveraging spatiotemporal correlations. (iii) Lastly, a dynamic thresholding technique is proposed for spiking neural networks. This approach incorporates bio-plausible elements such as Spike Frequency Adaptation and Burst Suppression, aiming to boost both the efficiency and performance of the system. Extensive tests on both publicly available and custom-collected ESD datasets demonstrate the efficacy of these contributions. Quantitative evaluations based on commonly accepted metrics show that the approaches introduced in this dissertation outperform current state-of-the-art methods. The research findings from this dissertation present a step forward in overcoming the limitations of current autonomous robotic perception systems, particularly in challenging and visually constrained environments. These contributions set the stage for more robust, efficient, and intelligent autonomous systems, unlocking new possibilities for their deployment in real-world scenarios.