Nighttime Semantic Segmentation with Instance-level Data Augmentation: a Case Study of the Dark Zurich Benchmark

Semantic segmentation has been a core learning task in the autonomous driving technology stack. However, current deep learning-based models do not perform well at nighttime due to the low illumination. In this study, we present an instance-level data augmentation method to increase the quantity and diversity for the low-resource classes to feed more instances of these classes to the training algorithm, with an aim to encourage the model to learn more features and patterns to better distinguish the low-resource classes presented in the original training set. We validate the method on the Dark Zurich dataset, a typical dataset that contains driving scene images taking at daytime e, twilight, and nighttime. We take the ''person'' class as an example and apply the instance-level data augmentation method. Experimental results have shown significant improvement compared to the SOTA, lifting the IoU by 4.52%. The results demonstrate the efficacy of the proposed method, indicating that the augmenting low-resource classes at the instance level is a promising strategy and can be an effective complement alongside other performance boosting methods.