DESTformer: Energy-Efficient Monocular Depth Estimation with Spiking Transformer for Edge Devices

Monocular depth estimation is a crucial task in computer vision with numerous applications in autonomous driving, robotics, and augmented reality. Traditional methods often rely on frame-based approaches, which can be computationally expensive and lack efficiency in dynamic scenes. We propose a novel event-based monocular depth estimation framework, namely DESTformer, using spiking neural network (SNN) as the backbone architecture. Our approach leverages the advantages of event-based cameras, which asynchronously capture pixellevel brightness changes (events) instead of traditional frames. These events are processed by a spiking transformer encoder that mimics the asynchronous and event-driven nature of the data. The spiking transformer extracts global features and generates sparse representations, which are then fed into the decoder for depth estimation refinement. The SNN based encoder allows for efficient processing of event data while the transformer-based backbone maintains competitive accuracy in depth estimation. We demonstrate the effectiveness of our DESTformer model in an edge device - Nvidia Jetson Orin Nano, which allows up to 3.76× speed up and 4.2× lower energy consumption compared to existing methods opening up new possibilities for real-time depth estimation in dynamic environments, with implications for various applications in computer vision and robotics.