Cascade Pixel Transformer with Distance-Driven Spatial Fusion for Hyperspectral Image Classification Using Limited Training Samples
Keywords:
Hyperspectral images, computer vision, deep learning, classification, transformer, pixel-level transformer, sparse feature embedding, spatial fusionAbstract
Classification of hyperspectral images is a crucial component of remote sensing, as it facilitates characterization of land surfaces. The limited availability of labeled data necessitates the design of architectures that can train with a minimal number of samples. Existing pixel- and patch-based methods advanced the field; however, they struggle to provide optimal results with limited training samples due to architectural limitations in learning complex spectral features and spatial contextual information from a small number of samples. To overcome these limitations, the Cascade Pixel Transformer Network (CPTNet) is proposed in this paper. CPTNet introduces a novel pixel-level transformer architecture that comprises transformer units in a cascade configuration to capture long-range spectral features. This work proposes an innovative input feature embedding strategy that combines PCA with a fully connected layer with pruned weights. CPTNet includes a distance-driven spatial fusion block that effectively aggregates the spectral features of all neighboring pixels in the input patch, which results in efficient spatial contextual information usage without relying on overfitting spatial patterns. Extensive experiments involving three benchmark datasets demonstrated the superiority of CPTNet as compared to state-of-the-art methods. The proposed CPTNet improves overall accuracies for three benchmark datasets by 3.31%, 1.71%, and 0.84% compared to the respective second-best approaches. The work proposed in the paper advances the field of hyperspectral classification by robust integration of PCA, weight pruning, pixel-level transformer units, and a distance-driven spatial fusion strategy. The source code repository for CPTNet can be found in https://github.com/profCRB/CPTNet.
Downloads
References
S. Peyghambari and Y. Zhang, “Hyperspectral remote sensing in lithological mapping, mineral exploration, and environmental geology: An updated review,” in Journal of Applied Remote Sensing, vol. 15, no. 3, p. 031501, 2021, doi: 10.1117/1.JRS.15.031501.
U. Heiden, W. Heldens, S. Roessner, K. Segl, T. Esch, and A. Mueller, “Urban structure type characterization using hyperspectral remote sensing and height information,” in Landscape and Urban Planning, vol. 105, no. 4, pp. 361–375, 2012, doi: 10.1016/j.landurbplan.2012.01.001.
W. Yang, T. Nigon, Z. Hao, G. D. Paiao, F. G. Fernandez, D. Mulla, and ´ C. Yang, “Estimation of corn yield based on hyperspectral imagery and convolutional neural network,” in Computers and Electronics in Agriculture, vol. 184, p. 106092, 2021, doi: 10.1016/j.compag.2021.106092.
T. Ishida, J. Kurihara, F. A. Viray, S. B. Namuco, E. C. Paringit, G. J. Perez, Y. Takahashi, and J. J. Marciano, “A novel approach for vegetation classification using UAV-based hyperspectral imaging,” in
Computers and Electronics in Agriculture, vol. 144, pp. 80–85, 2018, doi: 10.1016/j.compag.2017.11.027.
K. Nagasubramanian, S. Jones, A. K. Singh, S. Sarkar, A. Singh, and B. Ganapathysubramanian, “Plant disease identification using explainable 3D deep learning on hyperspectral images,” in Plant Methods, vol. 15, p. 98, 2019, doi: 10.1186/s13007-019-0479-8.
P. Duan, X. Kang, P. Ghamisi, and S. Li, “Hyperspectral remote sensing benchmark database for oil spill detection with an isolation forest-guided unsupervised detector,” in IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1–11, 2023, doi: 10.1109/TGRS.2023.3268944.
S. Armoogum, D. A. Dewi, V. Armoogum, N. Melanie, and T. B. Kurniawan, “Deep Learning-Based Loan Approval Prediction Using Artificial Neural Network (ANN) and Feature Importance Analysis,” in
J. Digit. Mark. Digit. Curr., vol. 3, no. 1, pp. 38–56, Feb. 2026. doi: 10.47738/jdmdc.v3i1.55.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is All You Need,” in Proc. 31st Int. Conf. Neural Inf. Process. Syst. (NeurIPS), Long Beach, CA, USA, 2017, pp. 6000–6010, doi: 10.48550/arXiv.1706.03762.
Y. Chen, Z. Lin, X. Zhao, G. Wang, and Y. Gu, “Deep learning-based classification of hyperspectral data,” in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 7, no. 6, pp. 2094–2107, 2014, doi: 10.1109/JSTARS.2014.2329330.
L. Mou, P. Ghamisi, and X. X. Zhu, “Deep recurrent neural networks for hyperspectral image classification,” in IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 7, pp. 3639–3655, 2017, doi: 10.1109/TGRS.2016.2636241.
W. Li, G. Wu, F. Zhang, and Q. Du, “Hyperspectral image classification using deep pixel-pair features,” in IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 2, pp. 844–853, 2017, doi:
1109/TGRS.2016.2616355.
H. Lee and H. Kwon, “Going deeper with contextual CNN for hyperspectral image classification,” in IEEE Transactions on Image Processing, vol. 26, no. 10, pp. 4843–4855, 2017, doi: 10.1109/TIP.2017.2725580.
Z. Zhong, J. Li, Z. Luo, and M. Chapman, “Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework,” in IEEE Transactions on Geoscience and Remote Sensing,
vol. 56, no. 2, pp. 847–858, 2018, doi: 10.1109/TGRS.2017.2755542.
Z. Ge, G. Cao, X. Li, and P. Fu, “Hyperspectral image classification method based on 2D–3D CNN and multibranch feature fusion,” in IEEE Journal of Selected Topics in Applied Earth Observations
and Remote Sensing, vol. 13, pp. 5776–5788, 2020, doi: 10.1109/JSTARS.2020.3024841.
S. Ghaderizadeh, D. Abbasi-Moghadam, A. Sharifi, N. Zhao, and A. Tariq, “Hyperspectral image classification using a hybrid 3D-2D convolutional neural networks,” in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 7570–7588, 2021, doi: 10.1109/JSTARS.2021.3099118.
U. Rahardja and Q. Aini, “Enhancing Blockchain Security Through Smart Contract Vulnerability Classification Using BiLSTM and Attention Mechanism,” in Journal of Current Research in Blockchain, vol. 3, no.1, pp. 28–45, Feb. 2026, doi: 10.47738/jcrb.v3i1.56.
M. Furqan, N. Katuk, and D. Hartama, “Multiclass Skin Lesion Classification Algorithm Using Attention-Based Vision Transformer With Metadata Fusion,” in Journal of Applied Data Sciences, vol. 7, no. 1, pp. 203–217, 2025, doi: 10.47738/jads.v7i1.1017.
Y. Dong, Q. Liu, B. Du, and L. Zhang, “Weighted feature fusion of convolutional neural network and graph attention network for hyperspectral image classification,” in IEEE Transactions on Image Processing, vol. 31, pp. 1559–1572, 2022, doi: 10.1109/TIP.2022.3144017.
M. E. Paoletti, S. Moreno-Alvarez, Y. Xue, J. M. Haut, and A. Plaza, “AAtt-CNN: Automatic attention-based convolutional neural networks for hyperspectral image classification,” in IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1–18, 2023, doi: 10.1109/TGRS.2023.3272639.
A. Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in arXiv preprint arXiv:2010.11929, 2021, doi: 10.48550/arXiv.2010.11929.
H. Yan, E. Zhang, J. Wang, C. Leng, A. Basu, and J. Peng, “Hybrid Conv-ViT network for hyperspectral image classification,” in IEEE Geoscience and Remote Sensing Letters, vol. 20, pp. 1–5, 2023, doi:
1109/LGRS.2023.3287277.
L. Sun, G. Zhao, Y. Zheng, and Z. Wu, “Spectral–spatial feature tokenization transformer for hyperspectral image classification,” in IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–14, 2022, doi: 10.1109/TGRS.2022.3144158.
S. K. Roy, A. Deria, C. Shah, J. M. Haut, Q. Du, and A. Plaza, “Spectral–spatial morphological attention transformer for hyperspectral image classification,” in IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1–15, 2023, doi: 10.1109/TGRS.2023.3242346.
X. Zheng, H. Sun, X. Lu, and W. Xie, “Rotation-Invariant Attention Network for Hyperspectral Image Classification,” IEEE Transactions on Image Processing, vol. 31, pp. 4251–4265, 2022, doi: 10.1109/TIP.2022.3177322.
D. Hong, Z. Han, J. Yao, L. Gao, B. Zhang, A. Plaza, and J. Chanussot, “SpectralFormer: Rethinking Hyperspectral Image Classification With Transformers,” in IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–15, 2022, doi: 10.1109/TGRS.2021.3130716.
S. Shinde and H. Patidar, “Hyperspectral image classification for vegetation detection using lightweight cascaded deep convolutional neural network,” in Journal of the Indian Society of Remote Sensing, vol. 51, pp. 2159–2166, 2023, doi: 10.1007/s12524-023-01754-5.
M. Ahmad, A. M. Khan, M. Mazzara, S. Distefano, M. Ali, and M. S. Sarfraz, “A fast and compact 3-D CNN for hyperspectral image classification,” in IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1–5, 2022, doi: 10.1109/LGRS.2020.3043710.
J. Xu, J. Zhao, and C. Liu, “An effective hyperspectral image classification approach based on discrete wavelet transform and dense CNN,” in IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1–5, 2022, doi: 10.1109/LGRS.2022.3181627.
M. Cihan and M. Ceylan, “IRSN: Involutional residual spectral network for hyperspectral image classification,” in IEEE Geoscience and Remote Sensing Letters, vol. 21, pp. 1–5, 2024, doi: 10.1109/LGRS.2023.3341497.
M. Ahmad, M. H. F. Butt, M. Usama, H. A. Altuwaijri, M. Mazzara, S. Distefano, and A. M. Khan, “Multi-head spatial-spectral mamba for hyperspectral image classification,” in Remote Sensing Letters, vol. 16, no. 4, pp. 339–353, 2025, doi: 10.1080/2150704X.2025.2461330.
M. Ahmad, M. Usama, M. Mazzara, and S. Distefano, “WaveMamba: Spatial-Spectral Wavelet Mamba for Hyperspectral Image Classification,” in IEEE Geoscience and Remote Sensing Letters, vol. 22, pp. 1–5, 2025, doi: 10.1109/LGRS.2024.3506034.
H. Wu, X. Yu, and Z. Zeng, “SSBFNet: A Spectral–Spatial Fusion With BiFormer Network for Hyperspectral Image Classification,” in The Visual Computer, vol. 41, no. 8, pp. 5391–5404, 2025, doi: 10.1007/s00371-024-03728-1.
P. Gamba, “A collection of data for urban area characterization,” Proc. IGARSS 2004 - IEEE Int. Geosci. Remote Sens. Symp., vol. 1, 2004, p. 72, doi: 10.1109/IGARSS.2004.1368947.
M. Grana, M. A. Veganzons, and B. Ayerdi, “Hyperspectral Remote ˜Sensing Scenes.” www.ehu.eus. Accessed: Nov. 5, 2024. [Online.] Available: https://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes.
M. F. Baumgardner, L. L. Biehl, and D. A. Landgrebe, “220 Band AVIRIS hyperspectral image data set: June 12, 1992 Indian Pine test site 3,” Purdue University Research Repository (PURR), Sep. 2015, doi: 10.4231/R7RX991C. [Online]. Available: purr.purdue.edu/publications/1947/1.
