Download PDF
Semin Musculoskelet Radiol 2020; 24(04): 451-459
DOI: 10.1055/s-0040-1709482
Review Article

Improving Quantitative Magnetic Resonance Imaging Using Deep Learning

Fang Liu 
1   Department of Radiology, Gordon Center for Medical Imaging, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts
› Author Affiliations
› Further Information
Also available at
    Permissions and Reprints

Abstract

Deep learning methods have shown promising results for accelerating quantitative musculoskeletal (MSK) magnetic resonance imaging (MRI) for T2 and T1ρ relaxometry. These methods have been shown to improve musculoskeletal tissue segmentation on parametric maps, allowing efficient and accurate T2 and T1ρ relaxometry analysis for monitoring and predicting MSK diseases. Deep learning methods have shown promising results for disease detection on quantitative MRI with diagnostic performance superior to conventional machine-learning methods for identifying knee osteoarthritis.

Keywords

deep learning - magnetic resonance imaging - quantitative imaging - rapid imaging - relaxometry

Financial Disclosure

Fang Liu receives funding support from National Institutes of Health grants NIH P41-EB022544 and R01-CA165221.




Publication History

Article published online:
29 September 2020

© 2020. Thieme. All rights reserved.

Thieme Medical Publishers
333 Seventh Avenue, New York, NY 10001, USA.

 

  • References

  • 1 LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521 (7553): 436-444
    CrossrefPubMedGoogle Scholar
  • 2 Wernick MN, Yang Y, Brankov JG, Yourganov G, Strother SC. Machine learning in medical imaging. IEEE Signal Process Mag 2010; 27 (04) 25-38
    CrossrefPubMedGoogle Scholar
  • 3 Giger ML. Machine learning in medical imaging. J Am Coll Radiol 2018; 15 (3 Pt B): 512-520
    CrossrefPubMedGoogle Scholar
  • 4 Erickson BJ, Korfiatis P, Akkus Z, Kline TL. Machine learning for medical imaging. Radiographics 2017; 37 (02) 505-515
    CrossrefPubMedGoogle Scholar
  • 5 Abadi M, Agarwal A, Barham P. , et al. TensorFlow: large-scale machine learning on heterogeneous distributed systems. Available at: https://arxiv.org/pdf/1603.04467.pdf . Accessed May 20, 2020
    PubMedGoogle Scholar
  • 6 Litjens G, Kooi T, Bejnordi BE. , et al. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42: 60-88
    CrossrefPubMedGoogle Scholar
  • 7 Shen D, Wu G, Suk H-I. Deep learning in medical image analysis. Annu Rev Biomed Eng 2017; 19: 221-248
    CrossrefPubMedGoogle Scholar
  • 8 Suzuki K. Overview of deep learning in medical imaging. Radiological Phys Technol 2017; 10 (03) 257-273
    CrossrefPubMedGoogle Scholar
  • 9 Liu F, Zhou Z, Jang H, Samsonov A, Zhao G, Kijowski R. Deep convolutional neural network and 3D deformable approach for tissue segmentation in musculoskeletal magnetic resonance imaging. Magn Reson Med 2018; 79 (04) 2379-2391
    CrossrefPubMedGoogle Scholar
  • 10 Zhou Z, Zhao G, Kijowski R, Liu F. Deep convolutional neural network for segmentation of knee joint anatomy. Magn Reson Med 2018; 80 (06) 2759-2770
    CrossrefPubMedGoogle Scholar
  • 11 Norman B, Pedoia V, Majumdar S. Use of 2D U-Net convolutional neural networks for automated cartilage and meniscus segmentation of knee MR imaging data to determine relaxometry and morphometry. Radiology 2018; 288 (01) 177-185
    CrossrefPubMedGoogle Scholar
  • 12 Liu F. SUSAN: Segment unannotated image structure using adversarial network. Magn Reson Med 2019; 81 (05) 3330-3345
    CrossrefPubMedGoogle Scholar
  • 13 Byra M, Wu M, Zhang X. , et al. Knee menisci segmentation and relaxometry of 3D ultrashort echo time cones MR imaging using attention U-Net with transfer learning. Magn Reson Med 2020; 83 (03) 1109-1122
    CrossrefPubMedGoogle Scholar
  • 14 Deniz CM, Xiang S, Hallyburton RS. , et al. Segmentation of the proximal femur from MR images using deep convolutional neural networks. Sci Rep 2018; 8 (01) 16485
    CrossrefPubMedGoogle Scholar
  • 15 Gaj S, Yang M, Nakamura K, Li X. Automated cartilage and meniscus segmentation of knee MRI with conditional generative adversarial networks. Magn Reson Med 2019 ; December 2 (Epub ahead of print)
    PubMedGoogle Scholar
  • 16 Liu F, Samsonov A, Chen L, Kijowski R, Feng L. SANTIS: Sampling-Augmented Neural neTwork with Incoherent Structure for MR image reconstruction. Magn Reson Med 2019; 82 (05) 1890-1904
    CrossrefPubMedGoogle Scholar
  • 17 Hammernik K, Klatzer T, Kobler E. , et al. Learning a variational network for reconstruction of accelerated MRI data. Magn Reson Med 2018; 79 (06) 3055-3071
    CrossrefPubMedGoogle Scholar
  • 18 Liu F, Feng L, Kijowski R. MANTIS: Model-Augmented Neural neTwork with Incoherent k-space Sampling for efficient MR parameter mapping. Magn Reson Med 2019; 82 (01) 174-188
    CrossrefPubMedGoogle Scholar
  • 19 Kijowski R, Liu F, Caliva F, Pedoia V. Deep learning for lesion detection, progression, and prediction of musculoskeletal disease. J Magn Reson Imaging 2019 ; November 25 (Epub ahead of print)
    PubMedGoogle Scholar
  • 20 Li X, Benjamin Ma C, Link TM. , et al. In vivo T(1rho) and T(2) mapping of articular cartilage in osteoarthritis of the knee using 3 T MRI. Osteoarthritis Cartilage 2007; 15 (07) 789-797
    CrossrefPubMedGoogle Scholar
  • 21 Zarins ZA, Bolbos RI, Pialat JB. , et al. Cartilage and meniscus assessment using T1rho and T2 measurements in healthy subjects and patients with osteoarthritis. Osteoarthritis Cartilage 2010; 18 (11) 1408-1416
    CrossrefPubMedGoogle Scholar
  • 22 Menezes NM, Gray ML, Hartke JR, Burstein D. T2 and T1rho MRI in articular cartilage systems. Magn Reson Med 2004; 51 (03) 503-509
    CrossrefPubMedGoogle Scholar
  • 23 Duvvuri U, Reddy R, Patel SD, Kaufman JH, Kneeland JB, Leigh JS. T1rho-relaxation in articular cartilage: effects of enzymatic degradation. Magn Reson Med 1997; 38 (06) 863-867
    CrossrefPubMedGoogle Scholar
  • 24 Liu F, Choi KW, Samsonov A. , et al. Articular cartilage of the human knee joint: In vivo multicomponent T2 analysis at 3.0 T. Radiology 2015; 277 (02) 477-488
    CrossrefPubMedGoogle Scholar
  • 25 Eckstein F, Burstein D, Link TM. Quantitative MRI of cartilage and bone: degenerative changes in osteoarthritis. NMR Biomed 2006; 19 (07) 822-854
    CrossrefPubMedGoogle Scholar
  • 26 Liu F, Chaudhary R, Hurley SA. , et al. Rapid multicomponent T2 analysis of the articular cartilage of the human knee joint at 3.0T. J Magn Reson Imaging 2014; 39 (05) 1191-1197
    CrossrefPubMedGoogle Scholar
  • 27 Kim M, Chan Q, Anthony MP, Cheung KMC, Samartzis D, Khong PL. Assessment of glycosaminoglycan distribution in human lumbar intervertebral discs using chemical exchange saturation transfer at 3 T: feasibility and initial experience. NMR Biomed 2011; 24 (09) 1137-1144
    CrossrefPubMedGoogle Scholar
  • 28 Du J, Carl M, Bydder M, Takahashi A, Chung CB, Bydder GM. Qualitative and quantitative ultrashort echo time (UTE) imaging of cortical bone. J Magn Reson 2010; 207 (02) 304-311
    CrossrefPubMedGoogle Scholar
  • 29 Bae WC, Chen PC, Chung CB, Masuda K, D'Lima D, Du J. Quantitative ultrashort echo time (UTE) MRI of human cortical bone: correlation with porosity and biomechanical properties. J Bone Miner Res 2012; 27 (04) 848-857
    CrossrefPubMedGoogle Scholar
  • 30 Liu F, Kijowski R. Deep learning in musculoskeletal imaging. Clin. Radiol. 2019; 1: 83-94
    CrossrefPubMedGoogle Scholar
  • 31 Griswold MA, Jakob PM, Heidemann RM. , et al. Generalized autocalibrating partially parallel acquisitions (GRAPPA). Magn Reson Med 2002; 47 (06) 1202-1210
    CrossrefPubMedGoogle Scholar
  • 32 Sodickson DK, Manning WJ. Simultaneous acquisition of spatial harmonics (SMASH): fast imaging with radiofrequency coil arrays. Magn Reson Med 1997; 38 (04) 591-603
    CrossrefPubMedGoogle Scholar
  • 33 Jakob PM, Griswold MA, Edelman RR, Sodickson DK. AUTO-SMASH: a self-calibrating technique for SMASH imaging. SiMultaneous Acquisition of Spatial Harmonics. MAGMA 1998; 7 (01) 42-54
    CrossrefPubMedGoogle Scholar
  • 34 Samsonov AA, Kholmovski EG, Parker DL, Johnson CR. POCSENSE: POCS-based reconstruction for sensitivity encoded magnetic resonance imaging. Magn Reson Med 2004; 52 (06) 1397-1406
    CrossrefPubMedGoogle Scholar
  • 35 Lustig M, Donoho D, Pauly JM. Sparse MRI: The application of compressed sensing for rapid MR imaging. Magn Reson Med 2007; 58 (06) 1182-1195
    CrossrefPubMedGoogle Scholar
  • 36 Lustig M, Pauly JM. SPIRiT: Iterative self-consistent parallel imaging reconstruction from arbitrary k-space. Magn Reson Med 2010; 64 (02) 457-471
    CrossrefPubMedGoogle Scholar
  • 37 Kijowski R, Rosas H, Samsonov A, King K, Peters R, Liu F. Knee imaging: Rapid three-dimensional fast spin-echo using compressed sensing. J Magn Reson Imaging 2017; 45 (06) 1712-1722
    CrossrefPubMedGoogle Scholar
  • 38 Zibetti MVW, Sharafi A, Otazo R, Regatte RR. Compressed sensing acceleration of biexponential 3D-T1ρ relaxation mapping of knee cartilage. Magn Reson Med 2019; 81 (02) 863-880
    CrossrefPubMedGoogle Scholar
  • 39 Bratke G, Rau R, Weiss K. , et al. Accelerated MRI of the lumbar spine using compressed sensing: quality and efficiency. J Magn Reson Imaging 2019; 49 (07) e164-e175
    CrossrefPubMedGoogle Scholar
  • 40 Yi J, Lee YH, Hahn S, Albakheet SS, Song HT, Suh JS. Fast isotropic volumetric magnetic resonance imaging of the ankle: acceleration of the three-dimensional fast spin echo sequence using compressed sensing combined with parallel imaging. Eur J Radiol 2019; 112: 52-58
    CrossrefPubMedGoogle Scholar
  • 41 Chaudhari AS, Kogan F, Pedoia V, Majumdar S, Gold GE, Hargreaves BA. Rapid knee MRI acquisition and analysis techniques for imaging osteoarthritis. J Magn Reson Imaging 2019 ; November 21 (Epub ahead of print)
    PubMedGoogle Scholar
  • 42 Garwood ER, Recht MP, White LM. Advanced imaging techniques in the knee: benefits and limitations of new rapid acquisition strategies for routine knee MRI. AJR Am J Roentgenol 2017; 209 (03) 552-560
    CrossrefPubMedGoogle Scholar
  • 43 Li H, Yang M, Kim J. , et al. Ultra-fast simultaneous T1rho and T2 mapping using deep learning. Paper presented at: ISMRM Annual Meeting; August 8–13, 2020
    PubMedGoogle Scholar
  • 44 Zibetti MVW, Sharafi A, Hammernik K, Knoll F, Regatte RR. Comparing learned variational networks and compressed sensing for T1rho mapping of knee cartilage. Paper presented at: ISMRM Workshop on Machine Learning, Part II; May 18, 2018; Washington, DC
    PubMedGoogle Scholar
  • 45 Zibetti MVW, Sharafi A, Hammernik K, Knoll F, Regatte RR. Variational Networks for Accelerating Biexponential 3D–T1rho Mapping of Knee Cartilage. Paper presented at: ISMRM Annual Meeting; August 8–13, 2020
    PubMedGoogle Scholar
  • 46 Chaudhari AS, Fang Z, Kogan F. , et al. Super-resolution musculoskeletal MRI using deep learning. Magn Reson Med 2018; 80 (05) 2139-2154
    CrossrefPubMedGoogle Scholar
  • 47 Chaudhari AS, Stevens KJ, Wood JP. , et al. Utility of deep learning super-resolution in the context of osteoarthritis MRI biomarkers. J Magn Reson Imaging 2020; 51 (03) 768-779
    CrossrefPubMedGoogle Scholar
  • 48 Chaudhari AS, Black MS, Eijgenraam S. , et al. Five-minute knee MRI for simultaneous morphometry and T2 relaxometry of cartilage and meniscus and for semiquantitative radiological assessment using double-echo in steady-state at 3T. J Magn Reson Imaging 2018; 47 (05) 1328-1341
    CrossrefPubMedGoogle Scholar
  • 49 Pedoia V, Majumdar S, Link TM. Segmentation of joint and musculoskeletal tissue in the study of arthritis. MAGMA 2016; 29 (02) 207-221
    CrossrefPubMedGoogle Scholar
  • 50 Badrinarayanan V, Kendall A, Cipolla R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. Available at: https://arxiv.org/pdf/1511.00561.pdf
    PubMedGoogle Scholar
  • 51 Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF. , eds. Medical Image Computing and Computer-Assisted Intervention. MICCAI 2015: 18th International Conference, Munich, Germany, October 5–9, 2015. Proceedings, Part III. Cham, Switzerland: Springer; 2015: 234-241 DOI: 10.1007/978-3-319-24574-4_28
    CrossrefGoogle Scholar
  • 52 Kellgren JH, Lawrence JS. Radiological assessment of osteo-arthrosis. Ann Rheum Dis 1957; 16 (04) 494-502
    Google Scholar
  • 53 Liu F, Zhou Z, Samsonov A. , et al. Deep learning approach for evaluating knee MR images: achieving high diagnostic performance for cartilage lesion detection. Radiology 2018; 289 (01) 160-169
    CrossrefPubMedGoogle Scholar
  • 54 Liu F, Guan B, Zhou Z. , et al. Fully automated diagnosis of anterior cruciate ligament tears on knee MR images by using deep learning. Radiol Artif Intell 2019; 1 (03) 180091
    CrossrefPubMedGoogle Scholar
  • 55 Pedoia V, Norman B, Mehany SN, Bucknor MD, Link TM, Majumdar S. 3D convolutional neural networks for detection and severity staging of meniscus and PFJ cartilage morphological degenerative changes in osteoarthritis and anterior cruciate ligament subjects. J Magn Reson Imaging 2019; 49 (02) 400-410
    CrossrefPubMedGoogle Scholar
  • 56 Roblot V, Giret Y, Bou Antoun M. , et al. Artificial intelligence to diagnose meniscus tears on MRI. Diagn Interv Imaging 2019; 100 (04) 243-249
    CrossrefPubMedGoogle Scholar
  • 57 Bien N, Rajpurkar P, Ball RL. , et al. Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of MRNet. PLOS Med 2018; 15: e1002699
    CrossrefPubMedGoogle Scholar
  • 58 Pedoia V, Lee J, Norman B, Link TM, Majumdar S. Diagnosing osteoarthritis from T2 maps using deep learning: an analysis of the entire Osteoarthritis Initiative baseline cohort. Osteoarthritis Cartilage 2019; 27 (07) 1002-1010
    CrossrefPubMedGoogle Scholar
  • 59 Huang G, Liu Z, van der Maaten L, Weinberger KQ. Densely connected convolutional networks. 2016 ; ArXiv.org/abs/1608.06993
    PubMedGoogle Scholar
  • 60 Pedoia V, Li X, Su F, Calixto N, Majumdar S. Fully automatic analysis of the knee articular cartilage T1ρ relaxation time using voxel-based relaxometry. J Magn Reson Imaging 2016; 43 (04) 970-980
    CrossrefPubMedGoogle Scholar