TransER: Hybrid Model and Ensemble-based Sequential Learning for Non-homogenous Dehazing

Image dehazing is one of the most challenging imaging inverse problems that estimates the haze-free images from hazy ones. While recent transformer/convolutional neural network-based methods have shown excellent performance in handling both homogeneous and non-homogeneous dehazing problems, these networks are often trained end-to-end to estimate the haze-free image directly and require a large number of parameters. In this work, we propose a novel, lightweight two-stage deep network for non-homogeneous dehazing. In particular, our proposed method, denoted as TransER, consists of two separate deep neural networks which are TransConv Fusion Dehaze (TFD) model in Stage I and Lightweight Ensemble Reconstruction (LER) network in Stage II. The first model (TFD) using transformer-based encoder and decoders generates two estimates of the haze-free image: a parameter-based dehazed output based on the physical modeling of the problem and a pseudo haze-free output generated directly by the model in an end-to-end fashion. LER in stage II reconstructs the final dehazed output fusing the two estimates from stage I. We incorporate knowledge distillation to develop a teacher network with the same architecture as LER, allowing it to supervise the intermediate features. Extensive experiments performed on challenging real and synthetic scene image datasets (NTIRE 2019-2023, and RESIDE-indoor) demonstrate that TransER can outperform many state-of-the-art competing methods while using a significantly lower number of parameters. The source code is available at https://github.com/trungpsu1210/TransER.