Cardiac Magnetic Resonance Imaging Focus Generative Adversarial Network Segmentation
DOI:
https://doi.org/10.29304/jqcsm.2023.15.31291Keywords:
Cardiac imaging,, Generative adversarial networks(GAN),, Cardiac Segmentation,, image Segmentation by GANsAbstract
Cardiovascular function analysis is crucial for illness diagnosis, risk assessment, and therapy selection in clinical cardiology. Doctors may identify cardiac disorders such as right ventricular failure, hypertrophic cardiomyopathy, and dilated cardiomyopathy using a variety of imaging modalities that allow them to spot pathological alterations. The optimum course of therapy may be chosen more quickly thanks to accurate automation of the relevant duties. Artificial neural networks and deep learning are the foundation of generative adversarial networks (GANs), which are methods for creating synthetic pictures. The potential capacity of the GANs to solve problems has attracted interest in addition to their inherent flexibility and the adaptability of deep learning, on which they are founded. This survey aims to examine the significance of medical imaging in the study and diagnosis of cardiac disease. Demonstrate the widespread adoption of GAN Network approaches in the field of magnetic resonance imaging (MRI) medical image analysis; Explain the recent segmentation application of generative adversarial networks. GANs in cardiovascular imaging additionally Identify the hurdles to the effective application of the GAN Network to medical imaging tasks and highlight particular contributions that address or get around these problems
Downloads
References
T. McInerney, D. Terzopoulos,” Deformable models in medical image analysis: a survey”, Med Image Anal, Volume1, Pages 91-108(1996),https://doi.org/10.1016/S1361-8415(96)80007-7.
S. Hussain, I. Mubeen, N. Ullah, S. Shahab, et al., “Modern Diagnostic Imaging Technique Applications and Risk Factors in the Medical Field: A Review”, BioMed Research International, vol. 2022, 19 pages (2022). https://doi.org/10.1155/2022/5164970
A. Iqbal, M. Sharif,” MDA-Net: Multiscale dual attention based network for breast lesion segmentation using ultrasound images,” Journal of King Saud University –Computer and Information Sciences, Volume 34, Issue 9, Pages 7283-7299(2022). https://doi.org/10.1016/j.jksuci.2021.10.002
G. Litjens, T. Kooi, B. Bejnordi, A. Arindra, F. Ciompi etal.,” A survey on deep learning in medical image analysis, Med Image Anal,” 42:60-88(2017), doi: 10.1016/j.media.07.005.
D. R. Sarvamangala and R. V. Kulkarni, “Convolutional neural networks in medical image understanding: a survey,” Evolutionary Intelligence volume 15, pages 1–22 (2022).
A. Singhal, M. Phogat, D. Kumar, A. Kumar, M. Dahiya, V.K. Hrivastava, “Materials Today: Proceedings, “Study of deep learning techniques for medical image analysis, Volume 56, Pages 209-214(2022).
I. Arel, D. C. Rose and T. P. Karnowski, "Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier]," in IEEE Computational Intelligence Magazine, vol. 5, no.4, pp.13-18, Nov.2010, doi:10.1109/MCI.2010.938364.
X . Yi, E . Walia, P . Babyn,” Generative adversarial network in medical imaging: a review,” Med Image Anal, Volume 58,( 2019) https://doi.org/10.1016/j.media.2019.101552
S. Kazeminia, C. Baur, A. Kuijper, et al, “GANs for medical image analysis, “Artif Intel Med, Volume 109, 101938, (2020). https://doi.org/10.1016/j.artmed.2020.101938
V . Sorin, Y . Barash, E . Konen, E . Klang,” Creating artificial images for radiology applications using generative adversarial networks (GANs)—a systematic review, “Acad Radiol, Volume 27, ISSUE 8, P1175-1185 (2020) https://doi.org/10.1016/j.acra.2019.12.024.
P.Kumar MR, P. Jayagopal,” Generative adversarial networks: a survey on applications and challenges, “International,10,pages1–24 (2021)https://doi.org/10.1007/s13735-020-00196-w
S. Standring, Gray's Anatomy E-book: The Anatomical Basis of Clinical Practice, Elsevier Health Sciences, (2015).
L. H. Opie, Heart Physiology: From Cell to Circulation, 4th Edition Lippincott Williams &Wilkins, (2004).
M. P. Nash and P. J. Hunter, “Computational Mechanics of the Heart,” Journal of Elasticity and the Physical Science of Solids volume 61, pages113–141 (2000)
S. Conolly, A. Macovski, J. Pauly, J. Schenck, K. K. Kwong, D. A. Chesler, X. P. Hu, W. Chen, M. Patel, and K. Ugurbil, “Magnetic Resonance Imaging,” In Medical Devices and Systems, pages 243–282(2006). CRC Press.
R. W. Brown, E. M. Haacke, Y. C. N. Cheng, M. R, Thompson, and R. Venkatesan, Magnetic Resonance Imaging: Physical Principles and Sequence Design. John Wiley & Sons, (2014).
E. Castillo, J. AC. Lima, and D. A. Bluemke,” Regional Myocardial Function: Advances in MR Imaging and Analysis,” Radiographics, 23(suppl 1): S127–S140(2003).
A. Graaf, P. Bhagirath, S. Ghoerbien, and M. G¨otte, Cardiac Magnetic Resonance Imaging: Artefacts for Clinicians, Netherlands Heart Journal,22(12):542–549(2014).
M. A. Saad, A. C. Bovik, and C. Charrier, “Blind Image Quality Assessment: ANatural Scene Statistics Approach in the DCT Domain, “IEEE Transactions on Image Processing, 21(8):3339–3352(2012).
M. Motwani, D. Dey, D. Berman, et al., “Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: a 5-year multicenter prospective registry analysis,” Eur Heart, J;38:500–7, [PubMed: 27252451], (2017).
R. Nakanishi, D. Dey, F. Commandeur, et al.,” Machine learning in predicting coronary heart disease and cardiovascular disease events: results from the Multi-Ethnic Study of Atherosclerosis (MESA) (abstract),” J Am Coll Cardiol,71: A1483, (2018).
D.Dey, M. Zamudio, A. Schuhbaeck, et al.,” Relationship between quantitative adverse plaque features from coronary computed tomography angiography and downstream impaired myocardial flow reserve by 13N-ammonia positron emission tomography: a pilot study,” Circ Cardiovasc Imaging, 8:3255–65(2015).
S. Gaur, K. ovrehus, D. Dey, et al.,” Coronary plaque quantification and fractional flow reserve by coronary computed tomography angiography identify ischemia-causing lesions,” Eur Heart J,37:1220–7(2016), [PubMed: 26763790]
D.Dey, S. Gaur, K. Ovrehus, et al.,” Integrated prediction of lesion-specific ischaemia from quantitative coronary CT angiography using machine learning: a multicentre study, “Eur Radiol,28:2655–64, (2018). [PubMed: 29352380]
R. Nakanishi, S. Sankaran, L. Grady, et al.,” Automated estimation of image quality for coronary computed tomographic angiography using machine learning,” Eur Radiol,1–9(2018).
R. Stebbing, A. Namburete, R. Upton, P. Leeson, J. Noble,” Data-driven shape parameterization for segmentation of the right ventricle from 3Dþt echocardiography,” Med Image Anal. 21:29–39(2018), [PubMed: 25577559].
A. Krizhevsky, I. Sutskever, G. Hinton, “Image-Net classification with deep convolutional neural networks,” Advances in neural information processing systems,1097–105, (2012)
J. Wolterink, T. Leiner, B.de Vos, W.van Hamersvelt, M. Viergever, I. Isgum,” Automatic coronary artery calcium scoring in cardiac CT angiography using paired convolutional neural networks,” Med Image Anal.,21:30022–6, (2016).
C. HZY, J. Park, P. Heng, S. Zhou, “Iterative multi-domain regularized deep learning for anatomical structure detection and segmentation from ultrasound images,” Med Image Compute Assist Interv,487–95, (2016).
H. Yang, J. Sun, H. Li, L. Wang, Z. Xu,” Deep fusion net for multi-atlas segmentation: Application to cardiac MR images, “Med Image Comput Assist Interv,521–8, (2016).
Tan LK, McLaughlin RA, Lim E, Abdul Aziz YF, Liew YM.,” Fully automated segmentation of the left ventricle in cine cardiac MRI using neural network regression”, J Magn Reson Imaging, 48:140–52, (2018). [PubMed: 29316024]
J. Betancur, F. Commandeur, M. Motlagh, T. Sharir, A. Einstein, S. Bokhari, M. Fish, et al.,” Deep learning for prediction of obstructive disease from fast myocardial perfusion SPECT: a multicenter study,” J Am Coll Cardiol Img, 11(11):1654-1663(2018)
A. Bhan, P. Mangipudi, A. Goyal, “Deep Learning Approach for Automatic Segmentation and Functional Assessment of LV in Cardiac MRI,” Electronics, 11(21):3594, (2022).
I. Goodfellow, J. Pouget, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio,” Generative Adversarial Nets, In Conference on Neural Information Processing Systems, “pages 2672–2680(2014).
A. Odena, C. Olah, and J. Shlens,” Conditional Image Synthesis with Auxiliary Classifier GANs,” In Proceedings of the International Conference on Machine Learning, volume 70, pages 2642–2651(2017), JMLR. org.
M. Mirza and S. Osindero, “Conditional generative adversarial nets,” 6 Nov 2014, https://arxiv.org/abs/1411.1784.
M. Wiechowski, K. Godlewski, B. Sawicki, et al.,” Monte Carlo Tree Search: a review of recent modifications and applications,” Artif Intell Rev 56, 2497–2562 (2023). https://doi.org/10.1007/s10462-022-10228-y
C. Chen, C. Qin, H. Qiu, G. Tarroni, J. Duan, W. Bai, D. Rueckert,” Deep Learning for Cardiac Image Segmentation: A Review,” Frontiers in Cardiovascular Medicine, Volume 7, 7: 25(2020), doi: 10.3389/fcvm.2020.00025.
A. Radford, L. Metz, and S. Chintala,” Unsupervised representation learning with deep convolutional generative adversarial networks,” conference paper at ICLR 19 November( 2015), https://arxiv.org/abs/1511. 06434.
T. Karras, S. Laine, T. Aila, “A style-based generator architecture for generative adversarial networks,” In Proceedings of the conference on Computer Vision and Pattern Recognition, 4401–4410(2019).
M. Arjovsky, S. Chintala, L. Bottou, “Wasserstein generative adversarial networks, “In Proceedings of the 34th International Conference on Machine Learning,” ICML 2017, Sydney, NSW, Australia, of Proceedings of Machine Learning Research, volume 70, pages 214–223(2017). PMLR.
L. Roberts, Machine Perception of 5ree-Dimensional Solids, IEEE, New York, NY, USA, (1963).
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324(1998).
D. Rui, G. Guo, X. Yan, B. Chen, Z. Liu, and X. He,” BiGAN: collaborative filtering with bidirectional generative adversarial networks,” in Proceedings of the 2020 SIAM International Conference on Data Mining, pp. 82–90(2020), Cincinnati, OH, USA.
J. Zhu, T. Park, P. Isola, and A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” arXiv preprint arXiv:1703.10593, (2017), https://arxiv.org/abs/1703.10593.
C. Ledig, L. 0eis, F. Huszar, et al., “Photo-realistic single image super-resolution using a generative adversarial network,” arXiv:1609.04802v5 [cs.CV] ,( 2017), https://arxiv.org/abs/1609.04802.
A. Odena, C. Olah, and J. Shlens,” Conditional image synthesis with auxiliary classifier GANs,” (2016), https://arxiv.org/ abs/1610.09585
E. Denton, S. Chintala, a. szlam, and R. Fergus,” Deep generative image models using a Laplacian pyramid of adversarial networks,” in Advances in Neural Information Processing Systems, vol. 28, pp. 1486–1494(2015), Curran Associates, Inc., Red Hook, NY, USA.
H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena,” Self-attention generative adversarial networks,” in Proceedings of the International Conference on Machine Learning; PMLR, pp. 7354–7363(2019), Long Beach, CA, USA.
D. Im, C. Kim, H. Jiang, R. Memisevic,” Generating images with recurrent adversarial networks,” (2016), https:// arxiv.org/abs/1602.05110.
A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow, B. Frey, “Adversarial autoencoders,” (2016), https://arxiv.org/abs/ 1511.05644.
P. Isola, J. Y. Zhu, T. Zhou, A. Efros,” Image-to-image translation with conditional adversarial networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134(2017), Honolulu, HI, USA.
I. Oksuz, J. Clough, W. Bai, B. Ruijsink, E. Antón, G. Cruz, C. Prieto, A. King, J. Schnabel, “High-quality segmentation of low-quality cardiac MR images using k-space artifact correction,” Proceedings of Machine Learning Research,102:380–389(2019), https://openreview.net/group?id=MIDL.io/2019/Conference.
H.Zhang, X. Cao, L.Xu, L.Qi,” Conditional convolution generative adversarial network for Bi-ventricle segmentation in cardiac MR images,” ACM Int Conf Proc Ser. https://doi.org/10.1145/3364836.3364860.
W. Yan, Y. Wang, S. Gu, L. Huang, F. Yan, L. Xia, Q. Tao, The Domain Shift Problem of Medical Image Segmentation and Vendor-Adaptation by Unet-GAN, in book: Medical Image Computing and Computer Assisted Intervention – MICCAI 2019, (2019).
A. Chartsias, T. Joyce, G. Papanastasiou, S. Semple, M. Williams, D. Newby, R. Dharmakumar, A. Sotirios. “Tsaftaris, Factorised, spatial representation learning: application in semi-supervised myocardial segmentation,” In arXiv. Springer International Publishing, pp 490–498(2018),https://doi.org/10.1007/978-3-030-00934-2_55
M.Rezaei, H.Yang, S.Meinel, “Whole heart and great vessel segmentation with context-aware of generative adversarial networks,” Inform Aktuell, volume 11, pp 353–358(2018), https://doi.org/10.1007/978-3-662-56537-7_89
S. Dong, G. Luo, K. Wang, S. Cao, A. Mercado, O. Shmuilovich, H. Zhang, S. Li,” VoxelAtlasGAN: 3D left ventricle segmentation on echocardiography with atlas guided generation and voxel-to-voxel discrimination,” 21st International Conference, Granada, Spain, September 2018, 1806.03619 (2018), Proceedings, Part IV.
Y. Skandarani, N. Painchaud, P. Jodoin, A. Lalande,” On the effectiveness of GAN generated cardiac MRIs for segmentation,” Medical Imaging with Deep Learning,1–4(2020)
J. Ossenberg, V. Grau,” Conditional generative adversarial networks for the prediction of cardiac contraction from individual frames,” In Springer, pp 109–118(2019).
A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow, B. Frey,” Adversarial autoencoders, “(2016), https://arxiv.org/abs/1511.05644.
Z. Zhuang, P. Jin, A. Joseph, Y. Yuan, S. Zhuang,” Interactive echocardiography translation using few-shot GAN transfer learning,” Comput Math Methods Med,2020.
C. Xu, L. Xu, P. Ohorodnyk, M. Roth, B. Chen, S. Li,” Contrast agent-free synthesis and segmentation of ischemic heart disease images using progressive sequential causal GANs,” Med ImageAnal 2020,62:101668(2020). https://doi.org/10.1016/j.media.2020.101668
W. Yan, Y. Wang, S. Gu, L. Huang, F. Yan, L. Xia, Q. Tao, “The domain shift problem of medical image segmentation and vendor-adaptation by Unet-GAN,” arXiv 2019,1:623–631(2019),https://arxiv.org/ftp/arxiv/papers/1910/1910.13681.pdf
C. Xu, L. Xu, P. Ohorodnyk, M. Roth, B. Chen, S.Li,” Contrast agent-free synthesis and segmentation of ischemic heart disease images using progressive sequential causal GANs, “Med. Image Anal. 62:101668(2020), doi: 10.1016/j.media.2020.101668
C. Xu, L. Xu, G. Brahm, H. Zhang, S. Li, “MuTGAN: simultaneous segmentation and quantification of myocardial infarction without contrast agents via joint adversarial learning,” in Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. MICCAI (2018).
K. Le, Z. Lou, W. Huo, X. Tian, “Auto Whole Heart Segmentation from CT images Using an Improved Unet-GAN,” Journal of Physics: Conference Series 1769 (2021) 012016 IOP Publishing doi:10.1088/1742-6596/1769/1/012016.
C. Decourta, L. Duongb,” Semi-supervised generative adversarial networks for the segmentation of the left ventricle in pediatric MRI,” Elsevier, Volume 123, 103884(2020), https://doi.org/10.1016/j.compbiomed.2020.103884
Y. Zhang, L. Yang, J. Chen, M. Fredericksen, D. P. Hughes, D. Z. Chen,” Deep Adversarial Networks for Biomedical Image Segmentation Utilizing Unannotated Images,” in M. Descoteaux, Medical Image Computing and Computer Assisted Intervention MICCAI 2017, Springer International Publishing, Cham, pp. 408–416(2017).
C. Chen, W. Bai, R. Davies, A. Bhuva, C. Manisty, J. Moon, et al.,” Improving the generalizability of convolutional neural network-based segmentation on CMR images,” Front. Cardiovasc. Med, 30 June 2020, Volume 7. https://doi.org/10.3389/fcvm.2020.00105
A. Andreopoulos, J. Tsotsos, “Efficient and generalizable statistical models of shape and appearance for analysis of cardiac MRI,” Med Image Anal. 12:335–57(2008). doi: 10.1016/j.media.2007.12.003. Data source: http://www.cse.yorku.ca/~mridataset/
P. Radau, Y. Lu, G. Connelly, A. Dick, G. Wright,” Evaluation framework for algorithms segmenting short axis cardiac MRI,” MIDAS J. 10:16:48(2009). Available online at: http://hdl.handle.net/10380/3070.
A. Suinesiaputra, B. Cowan, A. Al-Agamy, M. Elattar, N. Ayache, A. Fahmy, et al.,” A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images, “Med Image Anal. 18:50–62(2014), doi: 10.1016/j.media.2013.09.001.
C.Petitjean, M. Zuluaga, W. Bai, J. Dacher, D. Grosgeorge, J. Caudron,” Right ventricle segmentation from cardiac MRI: a collation study, “Med Image Anal1,9:187–202(2015).
R. Karim, R. Housden, M. Balasubramaniam, Z. Chen, D. Perry, A. Uddin, Y. Al-Beyatti, E. Palkhi et, “Evaluation of current algorithms for segmentation of scar tissue from late gadolinium enhancement cardiovascular magnetic resonance of the left atrium, “an open-access grand challenge. J Cardiovasc Magn Reson. 15:105(2015).
R. Karim, P. Bhagirath, P.t Claus, R. Housden, Z. Chen, Z. Karimaghaloo, H. Sohn et al,” Evaluation of state-of-the-art segmentation algorithms for left ventricle infarct from late gadolinium enhancement MR images,” Med Image Anal. 30:95-107(2016).
C. Gomez, A. Geers, J. Peters, J. Weese, K. Pinto, R. Karim, M. Ammar, A. Daoudi, et al, “Benchmark for algorithms segmenting the left atrium from 3D CT and MRI datasets,” IEEE Trans Med Imaging. 34(7):1460-1473(2015), doi: 10.1109/TMI.2015.2398818.
D. Pace, A. Dalca, T. Geva, A. Powell, M. Moghari, P. Golland, “Interactive whole-heart segmentation in congenital heart disease,” Med Image Comput Comput Assist Interv. 9351:80–8(2015). doi: 10.1007/978-3-319-24574-4_10.
O. Bernard, A. Lalande, C. Zotti, F. Cervenansky, X. Yang, P. Heng, I. Cetin, K. Lekadir, O. Camara et al,” Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved?” IEEE TransMed Imaging. 37:2514–25(2018). doi: 10.1109/TMI.2018.2837502
J. Zhao, Z. Xiong, Left Atrial Segmentation Challenge Dataset (2018). available online at: http://atriaseg2018.cardiacatlas.org/.
X.Zhuang, L. Li, C. Payer, D. Stern, M. Urschler, M. Heinrich, et al.,” Evaluation of algorithms for multi-modality whole heart segmentation: an open-access grand challenge,” Med Image Anal. 58:101537(2019). doi: 10.1016/j.media.2019.101537.
M. Schaap, T. Coert, T. Walsum, A. Giessen, A. Weustink, etal.,” Standardized evaluation methodology and reference database for evaluating coronary artery centerline extraction algorithms,” Med Image Anal. 13:701–14(2009). doi: 10.1016/j.media.2009.06.003.
H. Kirişli, M. Schaap, C. Metz, A. Dharampal, W. Meijboom, S. Papadopoulou, A. Dedic, K. Nieman, M. Graaf, M. Meijs, M. Cramer etal.,” Standardized evaluation framework for evaluating coronary artery stenosis detection, stenosis quantification, and lumen segmentation algorithms in computed tomography angiography,” Med Image Anal. 17:859–76(2013). doi: 0.1016/j.media.2013.05.007.
O. Bernard, J. Bosch, B. Heyde, M. Alessandrini, D. Barbosa, S. Camarasu, F. Cervenansky etal.,” Standardized evaluation system for left ventricular segmentation algorithms in 3D echocardiography,” IEEE Trans Med Imaging. 35:967–77(2016). doi: 10.1109/TMI.2015.2503890
S. Leclerc, E. Smistad, J. Pedrosa, A. Ostvik, F. Cervenansky, F. Espinosa, T. Espeland, et.,” Deep learning for segmentation using an open large-scale dataset in 2D echocardiography”, IEEE Trans Med Imaging. 38:2198–210(2019). doi: 10.1109/TMI.2019.29 00516.
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Maysaa Abd Ulkareem Naser , Abbas H. Hassin Al-Asadi
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
