Attention Blocks Improve White Matter Hyperintensity Semantic Segmentation using U-Nets

Authors

Keywords:

attention blocks, artificial intelligence, alzheimer's disease, semantic segmentation, white matter hyperintensities

Abstract

White matter hyperintensities (WMHs) are a common finding on magnetic resonance (MR) images in older individuals, appearing as high-signal intensity regions on fluid-attenuated inversion recovery (FLAIR) imaging. People with high WMH volume are at increased risk for dementia and stroke, controlling for vascular risk factors, but WMH burden is not reliably assessed in clinical practice. Manual segmentation of WMHs is accepted as the gold standard (or ground truth), however, it is a laborious and time-consuming method. Newer machine learning (ML)-based approaches are being proposed as alternatives to manual segmentation. Among these approaches, U-Net convolutional neural networks have demonstrated good WMH segmentation performance. However, even state-of-the-art ML models sometimes fail to correctly identify WMHs and their boundaries with sufficient accuracy. Attention blocks have emerged as a potential solution for improving the performance of U-Net models by enhancing the ability of the model to focus on relevant features in the data. We investigated the effectiveness of attention blocks in U-Net models for WMH segmentation compared to three other models (U-Net++, U-Net3+, and a standard U-Net). Attention blocks significantly improved the F-measure score for WMH segmentation (0.811 vs 0.789 for next best model, p=0.04) in a diverse brain imaging dataset. This study demonstrates that attention blocks enhance U-Net models used for WMH identification and classification.

Downloads

Download data is not yet available.

Author Biographies

Kauê Tartarotti Nepomuceno Duarte, University of Calgary

Kaue TN Duarte holds a B.Sc. degree in SystemAnalysis and Technology from the University ofCampinas in S˜ao Paulo, Brazil (2014). His M.Sc.(2017) in texture analysis of natural images andPh.D. (2021) in Alzheimer’s disease prognosis usingimage retrieval and AI were earned at the sameuniversity. He is currently a Postdoctoral Fellow at University of Calgary in Alberta

Murilo Costa de Barros, University of Campinas

Murilo C Barros holds a bachelor’s degree in Building Construction, defending in his last year the work X-band electromagnetic waves in ceramic blocks at University of Campinas. He holds a M.Sc.
in Information and Communication Systems, in the topic of predicting Tourette Syndrome using traditional machine learning techniques. He is currently a Ph.D. candidate at the University of Campinas on CNN applied to the study of Tourette Syndrome, with an internship at National Taiwan University
(Taiwan).

Abhijot Singh Sidhu, University of Calgary

Abhijot S Sidhu holds a Bachelor of Health Sciences Degree (Honours) from the University of Calgary, with emphasis in Biomedical Science and Nanoscience. He obtained his M.Sc. in Biomedical Engineering by understanding physiological brain changing in healthy normal aging using functional Magnetic Resonance images. He is pursuing a PhD in Biomedical Engineering in Dr. Richard Frayne’s
lab.

David Gobbi, University of Calgary

David G Gobbi is the lead software architect at the Calgary Image Processing and Analysis Centre (CIPAC - University of Calgary). Dr. Gobbi holds a BSc, Physics (Honours) at University of British Columbia, a M.Sc. in Physics at Carleton University, and a Ph.D. in Medical Biophysics at University of Western Ontario. He is also an open-source software advocate and a contributor to the Visualization Toolkit.

Cheryl McCreary, University of Calgary

Cheryl R McCreary has a extensive background in adopting imaging techniques to tissues, animal models and people. Among these techniques, some include imaging of white matter to evaluate myelin in a murine model of multiple sclerosis, and MR imaging biomarkers of neurodegeneration associated with cerebral small vessel disease and aging. Dr. McCreary has been working under Drs. Smith and Frayne as an imaging research scientist and MR research manager, developing novel, non-invasive MRI markers.

Feryal Saad, University of Calgary

Feryal Saad earned her medical school degree in the Damascus university and pursued training in radiology at the University of Henri-Poincare and Nancy University hospitals (France). She is a former director and consultant radiologist at Safita Medical Centre - a polyclinic in radiology, cardiology, and digestive specialties. She worked as a consultant radiologist at the Dammam Central Hospital, and  Dammam Medical Complex Tower in Saudi Arabia  and was also a member of the radiology residency
training program.

Richard Camicioli, University of Alberta

Richard Camicioli worked in engineering and medicinal chemistry prior to obtaining his MD, CM, from  McGill University where he also completed a neurologic residency. He perfomed a fellowship training in geriatric neurology at Oregon Health and Sciences University (1994). Dr. Camicioli joined the University of Alberta as an associate professor (2000) and became full professor (2008). Dr. Camicioli’s research interests include cognitive dysfunction and motor dysfunction, especially gait disorders in aging and dementia.

Eric Smith, University of Calgary

Eric E Smith is Professor of Radiology, Neurology, and Community Health Sciences at the University of Calgary - Canada, and holder of the Katthy Taylor Chair in Vascular Dementia. He is the Medical Director of the Cognitive Neurosciences Clinic and a member of the Calgary Stroke Program. He runs  the Clinical and Research Fellowship program in Cognitive Neurosciences. Dr. Smith graduated from McGill University, trained in Neurology in teaching hospitals of Harvard Medical School, and was  Assistant Professor of Neurology at Harvard University, joining University of Calgary in 2008.

Marco Carvalho, University of Campinas

Marco AG Carvalho holds a B.Sc. in Electrical Engineering (Universidade Federal do Rio Grande do Norte, Brazil, 1994), a M.Sc. in image processing at the School of Electrical and Computer Engineering (University of Campinas, Brazil, 1997) and a Ph.D. degree in image processing at School of Electrical and Computer Engineering (University of Campinas, Brazil, 2004). His main contributions are in areas of image processing, architectural segmentation, and image feature understanding.

Richard Frayne, University of Calgary

Richard Frayne is a tenure Professor at University of Calgary (departments of Radiology and  Clinical Neuroscience), and the Deputy Director of the Hotchkiss Brain Institute (HBI). In addition, Dr. Frayne is an associate member of the Libin Cardiovascular Institute, at the Cumming School of Medicine. He manages the Vascular Imaging Laboratory group. He is a former Centre’s Scientific Director (2010-7). During 2003-13, he was the chair at Canada Research in Image Science, as well as held the Hopewell Professorship in Brain Imaging during 2010-24.

References

J. M. Wardlaw, M. C. V. Hern´andez, and S. Mu˜noz-Maniega, “What are white matter hyperintensities made of?” Journal of the American Heart Association, vol. 4, no. 6, p. e001140, 2015. [Online]. Available:

https://doi.org/10.1161/JAHA.114.001140

C. E. Bauer, V. Zachariou, E. Seago, and B. T. Gold, “White

matter hyperintensity volume and location: Associations with wm

microstructure, brain iron, and cerebral perfusion,” Frontiers in

Aging Neuroscience, vol. 13, 2021. [Online]. Available: https:

//doi.org/10.3389/fnagi.2021.617947

J. M. Wardlaw, E. E. Smith, G. J. Biessels, C. Cordonnier, F. Fazekas,

R. Frayne, R. I. Lindley, J. T. O’Brien, F. Barkhof, O. R. Benavente,

S. E. Black, C. Brayne, M. Breteler, H. Chabriat, C. DeCarli,

F.-E. de Leeuw, F. Doubal, M. Duering, N. C. Fox, S. Greenberg,

V. Hachinski, I. Kilimann, V. Mok, R. van Oostenbrugge, L. Pantoni,

O. Speck, B. C. M. Stephan, S. Teipel, A. Viswanathan, D. Werring,

C. Chen, C. Smith, M. van Buchem, B. Norrving, P. B. Gorelick,

and M. Dichgans, “Neuroimaging standards for research into small

vessel disease and its contribution to ageing and neurodegeneration,”

The Lancet Neurology, vol. 12, no. 8, pp. 822–838, 2013. [Online].

Available: https://doi.org/10.1016/S1474-4422(13)70124-8

C. Haffner, R. Malik, and M. Dichgans, “Genetic factors in cerebral

small vessel disease and their impact on stroke and dementia,” Journal

of Cerebral Blood Flow & Metabolism, vol. 36, no. 1, pp. 158–171,

[Online]. Available: https://doi.org/10.1038/jcbfm.2015.71

K. T. N. Duarte, A. S. Sidhu, M. C. Barros, D. G. Gobbi, C. R.

McCreary, F. Saad, R. Camicioli, E. E. Smith, M. P. Bento, and

R. Frayne, “Multi-stage semi-supervised learning enhances white matter

hyperintensity segmentation,” Frontiers in Computational Neuroscience,

vol. 18, 2024. [Online]. Available: https://doi.org/10.3389/fncom.2024.

K. Duarte, P. V. de Paiva, P. Martins, and M. Carvalho, “Predicting

the early stages of the Alzheimer’s disease via combined brain

multi-projections and small datasets,” in Proceedings of the 14th

International Joint Conference on Computer Vision, Imaging and

Computer Graphics Theory and Applications. SCITEPRESS -

Science and Technology Publications, 2019. [Online]. Available:

http://doi.org/10.5220/0007404705530560

G. Tosto, M. E. Zimmerman, J. L. Hamilton, O. T. Carmichael,

and A. M. Brickman, “The effect of white matter hyperintensities

on neurodegeneration in mild cognitive impairment,” Alzheimer’s and

Dementia, vol. 11, no. 12, pp. 1510–1519, Jun. 2015. [Online].

Available: https://doi.org/10.1016/j.jalz.2015.05.014

S. Lee, F. Viqar, M. E. Zimmerman, A. Narkhede, G. Tosto, T. L.

Benzinger, D. S. Marcus, A. M. Fagan, A. Goate, N. C. Fox, N. J.

Cairns, D. M. Holtzman, V. Buckles, B. Ghetti, and E. M. et al,

“White matter hyperintensities are a core feature of alzheimer’s disease:

Evidence from the dominantly inherited Alzheimer network,” Annals of

Neurology, vol. 79, no. 6, pp. 929–939, Apr. 2016. [Online]. Available:

https://doi.org/10.1002/ana.24647

E. E. Smith, M. O’Donnell, G. Dagenais, S. A. Lear, A. Wielgosz,

M. Sharma, P. Poirier, G. Stotts, S. E. Black, S. Strother, M. D.

Noseworthy, O. Benavente, J. Modi, M. Goyal, S. Batool, K. Sanchez,

V. Hill, C. R. McCreary, R. Frayne, S. Islam, J. DeJesus, S. Rangarajan,

K. Teo, S. Yusuf, and on behalf of the PURE Investigators, “Early

cerebral small vessel disease and brain volume, cognition, and gait,”

Annals of Neurology, vol. 77, no. 2, pp. 251–261, 2015. [Online].

Available: https://doi.org/10.1002/ana.24320

E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional networks

for semantic segmentation,” IEEE Transactions on Pattern Analysis

and Machine Intelligence, vol. 39, no. 4, pp. 640–651, 2017. [Online].

Available: https://doi.org/10.1109/TPAMI.2016.2572683

N. Siddique, S. Paheding, C. P. Elkin, and V. Devabhaktuni, “U-Net and

its variants for medical image segmentation: A review of theory and

applications,” IEEE Access, vol. 9, pp. 82 031–82 057, 2021. [Online].

Available: https://doi.org/10.1109/ACCESS.2021.3086020

O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional

networks for biomedical image segmentation,” in Medical Image

Computing and Computer-Assisted Intervention – MICCAI 2015,

N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, Eds.

Cham: Springer International Publishing, 2015, pp. 234–241. [Online].

Available: https://doi.org/10.48550/arXiv.1505.04597

B. Chen, Y. Liu, Z. Zhang, G. Lu, and A. W. K. Kong,

“Transattunet: Multi-level attention-guided U-Net with transformer

for medical image segmentation,” 2022. [Online]. Available: https:

//doi.org/10.1109/TETCI.2023.3309626

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N.

Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” 2017.

[Online]. Available: https://doi.org/10.48550/arXiv.1706.03762

R. Heinen, M. D. Steenwijk, F. Barkhof, J. M. Biesbroek, W. M.

van der Flier, H. J. Kuijf, N. D. Prins, H. Vrenken, G. J. Biessels,

J. de Bresser, and TRACE-VCI study group, “Performance of five

automated white matter hyperintensity segmentation methods in a

multicenter dataset,” Scientific Reports, vol. 9, no. 1, 2019. [Online].

Available: https://doi.org/10.1038/s41598-019-52966-0

W. Zhu, H. Huang, Y. Zhou, F. Shi, H. Shen, R. Chen, R. Hua,

W. Wang, S. Xu, and X. Luo, “Automatic segmentation of white

matter hyperintensities in routine clinical brain MRI by 2D VB-Net:

A large-scale study,” Frontiers in Aging Neuroscience, vol. 14, 2022.

[Online]. Available: https://doi.org/10.3389/fnagi.2022.915009

S. Liu, X. Wu, S. He, X. Song, F. Shang, and X. Zhao, “Identification

of white matter lesions in patients with acute ischemic lesions using

U-Net,” Frontiers in Neurology, vol. 11, 2020. [Online]. Available:

https://doi.org/10.3389/fneur.2020.01008

V. Sundaresan, G. Zamboni, P. M. Rothwell, M. Jenkinson,

and L. Griffanti, “Triplanar ensemble U-Net model for white

matter hyperintensities segmentation on MR images,” Medical Image

Analysis, vol. 73, p. 102184, 2021. [Online]. Available: https:

//doi.org/10.1016/j.media.2021.102184

G. Park, J. Hong, B. A. Duffy, J.-M. Lee, and H. Kim, “White matter

hyperintensities segmentation using the ensemble U-Net with multiscale

highlighting foregrounds,” NeuroImage, vol. 237, p. 118140, 2021.

[Online]. Available: https://doi.org/10.1016/j.neuroimage.2021.118140

S. Lee, Z. Rieu, R. E. Kim, M. Lee, K. Yen, J. Yong, and D. Kim,

“Automatic segmentation of white matter hyperintensities in T2-FLAIR

with AQUA: A comparative validation study against conventional

methods,” Brain Research Bulletin, vol. 205, p. 110825, 2023.

[Online]. Available: https://www.sciencedirect.com/science/article/pii/

S0361923023002502

H. J. Kuijf, A. Casamitjana, D. L. Collins, M. Dadar, A. Georgiou, and

et al., “Standardized assessment of automatic segmentation of white

matter hyperintensities and results of the WMH segmentation challenge,”

IEEE Transactions on Medical Imaging, vol. 38, pp. 2556–2568, 2019.

[Online]. Available: https://doi.org/10.1109/tmi.2019.2905770

K. T. Duarte, D. G. Gobbi, A. S. Sidhu, C. R. McCreary, F. Saad,

R. Camicioli, E. E. Smith, and R. Frayne, “Segmenting white matter

hyperintensities in brain magnetic resonance images using convolution

neural networks,” Pattern Recognition Letters, vol. 175, pp. 90–94,

[Online]. Available: https://doi.org/10.1016/j.patrec.2023.07.014

K. T. Duarte, D. G. Gobbi, A. S. Sidhu, C. R. McCreary,

F. Saad, N. Das, E. E. Smith, and R. Frayne, “Segmenting white

matter hyperintensity in Alzheimer’s disease using U-Net cnns,”

in 2022 35th SIBGRAPI Conference on Graphics, Patterns and

Images (SIBGRAPI), vol. 1, 2022, pp. 109–114. [Online]. Available:

https://doi.org/10.1109/SIBGRAPI55357.2022.9991752

C. R. McCreary, M. Salluzzi, L. B. Andersen, D. Gobbi, L. Lauzon,

F. Saad, E. E. Smith, and R. Frayne, “Calgary Normative Study:

Design of a prospective longitudinal study to characterise potential

quantitative MR biomarkers of neurodegeneration over the adult

lifespan,” BMJ Open, vol. 10, no. 8, 2020. [Online]. Available:

https://doi.org/10.1136/bmjopen-2020-038120

S. Peca, C. R. McCreary, E. Donaldson, G. Kumarpillai, N. Shobha, and

et al, “Neurovascular decoupling is associated with severity of cerebral

amyloid angiopathy,” Neurology, vol. 81, no. 19, pp. 1659–1665, 2013.

[Online]. Available: https://doi.org/10.1212/01.wnl.0000435291.49598.

M. S. Albert, S. T. DeKosky, D. Dickson, B. Dubois, H. H. Feldman,

N. C. Fox, A. Gamst, D. M. Holtzman, W. J. Jagust, R. C. Petersen,

P. J. Snyder, M. C. Carrillo, B. Thies, and C. H. Phelps, “The

diagnosis of mild cognitive impairment due to Alzheimer’s disease:

Recommendations from the National Institute on Aging-Alzheimer’s

Association workgroups on diagnostic guidelines for Alzheimer’s

disease,” Alzheimer’s & Dementia, vol. 7, no. 3, pp. 270–279, 2011.

[Online]. Available: https://doi.org/10.1016/j.jalz.2011.03.008

A. Subotic, C. McCreary, F. Saad, A. Nguyen, A. Alvarez-Veronesi,

and et al, “Cortical thickness and its association with clinical cognitive

and neuroimaging markers in cerebral amyloid angiopathy,” Journal of

Alzheimer’s Disease, vol. 81, pp. 1–9, 05 2021. [Online]. Available:

https://doi.org/10.3233/JAD-210138

D. Gobbi, Q. Lu, R. Frayne, and M. Salluzzi, “Cerebra-WML: A rapid

workflow for quantification of white matter hyperintensities,” Canadian

Stroke Congress, vol. 40, pp. E128–E129, 2012. [Online]. Available:

https://www.researchgate.net/publication/278294844 Cerebra-WML a

rapid workflow for quantification of white matter hyperintensities

J. C. Kosior, S. Idris, D. Dowlatshahi, M. Alzawahmah, M. Eesa, and

et al, “Quantomo: Validation of a computer-assisted methodology for

the volumetric analysis of intracerebral haemorrhage,” International

Journal of Stroke, vol. 6, no. 4, pp. 302–305, 2011. [Online]. Available:

https://doi.org/10.1111/j.1747-4949.2010.00579.x

N. J. Tustison, B. B. Avants, P. A. Cook, Y. Zheng, A. Egan, P. A.

Yushkevich, and J. C. Gee, “N4ITK: improved N3 bias correction,”

IEEE Trans. Medical Imaging, vol. 29, no. 6, pp. 1310–1320, 2010.

[Online]. Available: https://doi.org/10.1109/TMI.2010.2046908

K. T. N. Duarte, M. C. de Barros, D. G. Gobbi, A. S. Sidhu, M. A. G.

de Carvalho, and R. Frayne, “Changes in 3D radiomic texture descriptors

in Alzheimer’s disease stages,” in 18th International Symposium on

Medical Information Processing and Analysis, J. Brieva, P. Guevara,

N. Lepore, M. G. Linguraru, L. Rittner, and E. R. C. M.D., Eds., vol.

, International Society for Optics and Photonics. SPIE, 2023, p.

S. [Online]. Available: https://doi.org/10.1117/12.2670246

K. T. N. Duarte, D. G. Gobbi, R. Frayne, and M. A. G.

de Carvalho, “Detecting Alzheimer’s disease based on structural region

analysis using a 3D shape descriptor,” in 2020 33rd SIBGRAPI

Conference on Graphics, Patterns and Images (SIBGRAPI), 2020, pp.

–187. [Online]. Available: https://doi.org/10.1109/SIBGRAPI51738.

00032

R. Guerrero, C. Qin, O. Oktay, C. Bowles, L. Chen, and et al.,

“White matter hyperintensity and stroke lesion segmentation and

differentiation using convolutional neural networks,” NeuroImage:

Clinical, vol. 17, pp. 918–934, 2018. [Online]. Available: https:

//doi.org/10.1016/j.nicl.2017.12.022

O. Oktay, J. Schlemper, L. L. Folgoc, M. Lee, M. Heinrich, K. Misawa,

K. Mori, S. McDonagh, N. Y. Hammerla, B. Kainz, B. Glocker, and

D. Rueckert, “Attention U-Net: Learning where to look for the pancreas,”

[Online]. Available: https://doi.org/10.48550/arXiv.1804.03999

K. Simonyan and A. Zisserman, “Very deep convolutional networks for

large-scale image recognition,” arXiv 1409.1556, 09 2014. [Online].

Available: https://doi.org/10.48550/arXiv.1409.1556

Z. Zhou, M. M. Rahman Siddiquee, N. Tajbakhsh, and J. Liang,

“Unet++: A nested U-Net architecture for medical image segmentation,”

in Deep Learning in Medical Image Analysis and Multimodal Learning

for Clinical Decision Support, D. Stoyanov, Z. Taylor, G. Carneiro,

T. Syeda-Mahmood, A. Martel, L. Maier-Hein, J. M. R. Tavares,

A. Bradley, J. P. Papa, V. Belagiannis, J. C. Nascimento, Z. Lu,

S. Conjeti, M. Moradi, H. Greenspan, and A. Madabhushi, Eds. Cham:

Springer International Publishing, 2018, pp. 3–11. [Online]. Available:

https://doi.org/10.48550/arXiv.1807.10165

H. Huang, L. Lin, R. Tong, H. Hu, Q. Zhang, Y. Iwamoto,

X. Han, Y.-W. Chen, and J. Wu, “Unet 3+: A full-scale connected

UNet for medical image segmentation,” 2020. [Online]. Available:

https://doi.org/10.48550/arXiv.2004.08790

Downloads

Published

2025-06-26

How to Cite

Tartarotti Nepomuceno Duarte, K., Costa de Barros, M., Singh Sidhu, A., Gobbi, D., McCreary, C. ., Saad, F. ., Camicioli, R. ., Smith, E., Carvalho, M., & Frayne, R. (2025). Attention Blocks Improve White Matter Hyperintensity Semantic Segmentation using U-Nets. IEEE Latin America Transactions, 23(8), 646–661. Retrieved from https://latamt.ieeer9.org/index.php/transactions/article/view/9615

Issue

Vol. 23 No. 8 (2025): Special Issue on Technology Megatrends

Section

Computing