Wavelet-Driven GAN with Attention Mechanism for Improved Raindrop Removal
Keywords:
Single Image Raindrop Removal, Complex Wavelet Transform (CWT), GAN, AttentionAbstract
The adhesion of raindrops to camera lenses creates serious distortions and image aberrations, posing a major problem for computer vision systems used in autonomous driving, surveillance, and environmental monitoring. This paper presents a powerful multi-scale restoration architecture, D2W-GAN, that combines the Dual-Tree Complex Wavelet Transform (DT-CWT) with a U-shaped network, an attention scheme, and Generative Adversarial Networks (GANs) to remove raindrops. Current techniques tend to struggle to remove artifacts introduced by raindrops and maintain overall image fidelity. The suggested method minimally distorts raindrop distortions by using DT-CWT, which simultaneously provides fine high-frequency and large-scale low-frequency components, thereby improving the separation of the distortion. The addition of attention mechanisms enables the network to focus more on the regions of the raindrops, thereby improving the restoration accuracy. Moreover, refinements to the adversarial learning framework yield more realistic, consistent results. Experimental tests show that D2W-GAN is superior to many contemporary methods and produces higher-quality restorations, as measured by PSNR and SSIM. In general, wavelet-based feature extraction, feature attention, and adaptive optimization have been successfully integrated to achieve effective image restoration under unstable, demanding conditions.
Downloads
References
Z. Shi et al., “Resfusion: Denoising diffusion probabilistic models for image restoration based on prior residual noise,” in The Thirty-Eighth Annual Conference on Neural Information Processing Systems, 2023, pp. 2773–2783. DOI: 10.52202/079017-4153.
O. Ozdenizci and R. Legenstein, “Restoring vision in adverse weather conditions with patch-based denoising diffusion models,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 8, pp. 10 346–10 357, 2023. DOI: 10.1109/TPAMI.2023.3238179.
Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” in International Conference on Learning Representations, 2020. DOI: 10.48550/arXiv.2011.13456.
J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in Neural Information Processing Systems, vol. 33, pp. 6840–6851, 2020. DOI: 10.48550/arXiv.2006.11239.
J. Ho, C. Saharia, W. Chan, D. J. Fleet, M. Norouzi, and T. Salimans, “Cascaded diffusion models for high fidelity image generation,” Journal of Machine Learning Research, vol. 23, no. 47, pp. 1–33, 2022. DOI: 10.48550/arXiv.2106.15282.
R. Qian, R. T. Tan, W. Yang, J. Su, and J. Liu, “Attentive generative adversarial network for raindrop removal from a single image,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 2482–2491. DOI: 10.1109/CVPR.2018.00263.
Y. Quan, S. Deng, Y. Chen, and H. Ji, “Deep learning for seeing through window with raindrops,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 2463–2471. DOI: 10.1109/ICCV.2019.00255.
J. Nie, J. Xie, J. Cao, and Y. Pang, “Context and detail interaction network for stereo rain streak and raindrop removal,” Neural Networks, vol. 166, pp. 215–224, 2023. DOI: 10.1016/j.neunet.2023.06.015.
H. Wang, S. Zhao, L. Yu, and X. Yang, “Unsupervised network for single image raindrop removal,” in Computer Vision and Pattern Recognition, 2024. DOI: 10.48550/arXiv.2403.20208.
H. Xia, Y. Lan, S. Song, and H. Li, “Raindrop removal from a single image using a two-step generative adversarial network,” Signal Image and Video Processing, vol. 16, no. 3, pp. 677–684, 2022. DOI: 10.1007/s11760-021-02068-7.
Z. Tu et al., “Maxim: Multi-axis mlp for image processing,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 5769–5780. DOI: 10.1109/CVPR52688.2022.00569.
Y. Huang et al., “Wavedm: Wavelet-based diffusion models for image restoration,” IEEE Transactions on Multimedia, vol. 26, pp. 7058–7073, 2024. DOI: 10.1109/TMM.2024.3361235.
P. Isola, J. Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1125–1134. DOI: 10.1109/CVPR.2017.632.
C. Xu, J. Gao, Q. Wen, and B. Wang, “Generative adversarial network for image raindrop removal of transmission line based on unmanned aerial vehicle inspection,” Wireless Communications and Mobile Computing, vol. 2021, no. 1, p. 6668771, 2021. DOI: 10.1155/2021/6668771.
Y. Qiao, M. Shao, H. Liu, and K. Shang, “Mutual channel prior guided dual-domain interaction network for single image raindrop removal,” Computers & Graphics, vol. 112, pp. 132–142, 2023. DOI: 10.1016/j.cag.2023.03.018.
S. I. Niwas, P. Palanisamy, and K. Sujathan, “Complex wavelet as nucleus descriptors for automated cancer cytology classifier system using ann,” in 2010 IEEE International Conference on Computational Intelligence and Computing Research, 2010, pp. 1–5. DOI: 10.1109/ICCIC.2010.5705848.
S. I. Niwas, P. Palanisamy, K. Sujathan, and E. Bengtsson, “Analysis of nuclei textures of fine needle aspirated cytology images for breast cancer diagnosis using complex daubechies wavelets,” Signal Processing, vol. 93, no. 10, pp. 2828–2837, 2013. DOI: 10.1016/j.sigpro.2013.04.018.
X. Liu, M. Suganuma, Z. Sun, and T. Okatani, “Dual residual networks leveraging the potential of paired operations for image restoration,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 7007–7016. DOI: 10.1109/CVPR.2019.00718.
J. Xiao, X. Fu, A. Liu, F. Wu, and Z. J. Zha, “Image de-raining transformer,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 11, pp. 12 978–12 995, 2022. DOI: 10.1109/TPAMI.2022.3170894.
R. Li, R. T. Tan, and L. F. Cheong, “All in one bad weather removal using architectural search,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 3175–3185. DOI: 10.1109/CVPR42600.2020.00323.
Y. Alami Mejjati, C. Richardt, J. Tompkin, D. Cosker, and K. I. Kim, “Unsupervised attention-guided image-to-image translation,” Advances in Neural Information Processing Systems, vol. 31, 2018. DOI: 10.48550/arXiv.1806.02311.
J. Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2223–2232. DOI: 10.1109/ICCV.2017.244.
H. Tang, D. Xu, N. Sebe, and Y. Yan, “Attention-guided generative adversarial networks for unsupervised weather conditions,” International Journal of Computer Vision, pp. 1–23, 2024. DOI: 10.1007/s11263-024-01964-0.
S. Chen, T. Ye, J. Bai, E. Chen, J. Shi, and L. Zhu, “Sparse sampling transformer with uncertainty-driven ranking for unified removal of raindrops and rain streaks,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 13 106–13 117. DOI: 10.1109/ICCV51070.2023.01206.
Z. Li and Q. Gu, “I-vit: Integer-only quantization for efficient vision transformer inference,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 17 065–17 075. DOI: 10.1109/ICCV51070.2023.01565.
T. Wang et al., “Gridformer: Residual dense transformer with grid structure for image restoration in adverse weather conditions,” International Journal of Computer Vision, pp. 1–23, 2024. DOI: 10. 1007 / s11263 - 024 -01964-0.
