Artificial Intelligence Explained for Nonexperts

Abstract

Artificial intelligence (AI) has made stunning progress in the last decade, made possible largely due to the advances in training deep neural networks with large data sets. Many of these solutions, initially developed for natural images, speech, or text, are now becoming successful in medical imaging. In this article we briefly summarize in an accessible way the current state of the field of AI. Furthermore, we highlight the most promising approaches and describe the current challenges that will need to be solved to enable broad deployment of AI in clinical practice.

Keywords

deep learning - artificial intelligence - review - medical imaging

Financial Disclosure

We acknowledge support from the National Institutes of Health under grants R01-EB024532, P41-EB017183, and R21 EB027241.

• References

• 1 Bishop CM. Pattern Recognition and Machine Learning. New York, NY: Springer; 2006
• 2 Hastie T, Tibshirani R, Friedman J, Franklin J. The elements of statistical learning: data mining, inference and prediction. Math Intell 2005; 27 (02) 83-85
• 3 Goodfellow I, Bengio Y, Courville A. Deep Learning. Cambridge, MA: MIT Press; 2016
• 4 Zhang R, Isola P, Efros AA. Colorful image colorization. Computer Vision – ECCV 2016. Available at: https://arxiv.org/abs/1603.08511 . Accessed January 8, 2020
• 5 Noroozi M, Favaro P. Unsupervised learning of visual representations by solving jigsaw puzzles. Computer Vision – ECCV 2016. Available at: https://arxiv.org/abs/1603.09246 . Accessed January 8, 2020
• 6 Agrawal P, Carreira J, Malik J. Learning to see by moving. Proceedings of the IEEE International Conference on Computer Vision. Available at: https://arxiv.org/abs/1505.01596 . Accessed January 8, 2020
• 7 Devlin J, Chang M-W, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. arXiv [csCL]. October 2018. Available at: http://arxiv.org/abs/1810.04805 . Accessed January 8, 2020
• 8 Lample G, Conneau A. Cross-lingual language model pretraining. arXiv [csCL]. January 2019. Available at: http://arxiv.org/abs/1901.07291 . Accessed January 8, 2020
• 9 Wang A, Singh A, Michael J, Hill F, Levy O, Bowman S. GLUE: a multi-task benchmark and analysis platform for natural language understanding. arXiv [csCL]. April 2018. Available at: http://arxiv.org/abs/1804.07461 . Accessed January 8, 2020
• 10 Silver D, Schrittwieser J, Simonyan K. , et al. Mastering the game of Go without human knowledge. Nature 2017; 550 (7676): 354-359
• 11 Brown N, Sandholm T. Superhuman AI for multiplayer poker. Science 2019; 365 (6456): 885-890
• 12 Mnih V, Kavukcuoglu K, Silver D. , et al. Playing Atari with deep reinforcement learning. arXiv [csLG]. December 2013. Available at: http://arxiv.org/abs/1312.5602 . Accessed January 8, 2020
• 13 Arulkumaran K, Cully A, Togelius J. AlphaStar: an evolutionary computation perspective. arXiv [csNE]. February 2019. Available at: http://arxiv.org/abs/1902.01724 . Accessed January 8, 2020
• 14 Zhu B, Liu J, Koonjoo N, Rosen B, Rosen MS. AUTOmated pulse SEQuence generation (AUTOSEQ) and neural network decoding for fast quantitative MR parameter measurement using continuous and simultaneous RF transmit and receive. ISMRM Annual Meeting & Exhibition. Vol 1090. 2019. Available at: http://www.enc-conference.org/portals/0/Abstracts2019/ENC20198520.4608VER.2.pdf . Accessed January 8, 2020
• 15 Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition. Available at: http://www.image-net.org/papers/imagenet_cvpr09.pdf . Accessed January 8, 2020
• 16 Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ. , eds. Advances in Neural Information Processing Systems 25. Red Hook, NY: Curran Associates; 2012: 1097-1105
• 17 Raina R, Madhavan A, Ng AY. Large-scale deep unsupervised learning using graphics processors. In: Proceedings of the 26th Annual International Conference on Machine Learning. ICML '09. New York, NY: ACM; 2009: 873-880
• 18 Jia Y, Shelhamer E, Donahue J. , et al. Caffe: Convolutional Architecture for Fast Feature Embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia. MM '14. New York, NY: ACM; 2014: 675-678
• 19 Collobert R, Bengio S, Mariéthoz J. Torch: a modular machine learning software library. Available at: https://infoscience.epfl.ch/record/82802/files/rr02-46.pdf . Accessed January 8, 2020
• 20 Bergstra J, Breuleux O, Bastien F. , et al. Theano: a CPU and GPU math expression compiler. In: Proceedings of the Python for Scientific Computing Conference (SciPy). Vol 4. Austin, TX; 2010. Available at: https://conference.scipy.org/scipy2010/slides/james_bergstra_theano.pdf . Accessed January 8, 2020
• 21 Abadi M, Agarwal A, Barham P. , et al. TensorFlow: Large-scale machine learning on heterogeneous distributed systems. arXiv [csDC]. March 2016. Available at: http://arxiv.org/abs/1603.04467 . Accessed January 8, 2020
• 22 Paszke A, Gross S, Chintala S. , et al. Automatic differentiation in PyTorch. October 2017. Available at: https://openreview.net/pdf?id=BJJsrmfCZ . Accessed October 14, 2019
• 23 LeCun Y, Boser BE, Denker JS. , et al. Handwritten digit recognition with a back-propagation network. In: Touretzky DS. , ed. Advances in Neural Information Processing Systems 2. San Francisco, CA: Morgan-Kaufmann; 1990: 396-404
• 24 LeCun Y, Boser B, Denker JS. , et al. Backpropagation applied to handwritten zip code recognition. Neural Comput 1989; 1 (04) 541-551
• 25 Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv [csCV]. September 2014. Available at: http://arxiv.org/abs/1409.1556 . Accessed January 8, 2020
• 26 Szegedy C, Liu W, Jia Y. , et al. Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2015: 1-9
• 27 Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, LeCun Y. OverFeat: Integrated recognition, localization and detection using convolutional networks. arXiv [csCV]. December 2013. Available at: http://arxiv.org/abs/1312.6229 . Accessed January 8, 2020
• 28 He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. arXiv [csCV]. December 2015. Available at: http://arxiv.org/abs/1512.03385 . Accessed January 8, 2020
• 29 Huang G, Liu Z, van der Maaten L, Weinberger KQ. Densely connected convolutional networks. arXiv [csCV]. August 2016. Available at: http://arxiv.org/abs/1608.06993 . Accessed January 8, 2020
• 30 He K, Zhang X, Ren S, Sun J. Identity mappings in deep residual networks. Computer Vision – ECCV 2016. Available at: https://arxiv.org/abs/1603.05027 . Accessed January 8, 2020
• 31 Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. arXiv [csCV]. May 2015. Available at: http://arxiv.org/abs/1505.04597 . Accessed January 8, 2020
• 32 Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2015: 3431-3440
• 33 Hammernik K, Klatzer T, Kobler E. , et al. Learning a variational network for reconstruction of accelerated MRI data. Magn Reson Med 2018; 79 (06) 3055-3071
• 34 Knoll F, Hammernik K, Kobler E, Pock T, Recht MP, Sodickson DK. Assessment of the generalization of learned image reconstruction and the potential for transfer learning. Magn Reson Med 2019; 81 (01) 116-128
• 35 Kobler E, Muckley M, Chen B. , et al. Variational deep learning for low-dose computed tomography. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2018: 6687-6691
• 36 Lustig M, Donoho D, Pauly JM. Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn Reson Med 2007; 58 (06) 1182-1195
• 37 Aggarwal HK, Mani MP, Jacob M. MoDL: Model-Based Deep Learning architecture for inverse problems. IEEE Trans Med Imaging 2019; 38 (02) 394-405
• 38 Schlemper J, Caballero J, Hajnal JV, Price AN, Rueckert D. A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans Med Imaging 2018; 37 (02) 491-503
• 39 Qin C, Schlemper J, Caballero J, Price AN, Hajnal JV, Rueckert D. Convolutional recurrent neural networks for dynamic MR image reconstruction. IEEE Trans Med Imaging 2019; 38 (01) 280-290
• 40 Zhu B, Liu JZ, Cauley SF, Rosen BR, Rosen MS. Image reconstruction by domain-transform manifold learning. Nature 2018; 555 (7697): 487-492
• 41 Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 2014; 15: 1929-1958
• 42 Nair V, Hinton GE. Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10); 2010: 807-814
• 43 Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016: 2818-2826
• 44 Hochreiter S. The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int J Uncertain Fuzziness Knowl Based Syst 1998; 06 (02) 107-116
• 45 Srivastava RK, Greff K, Schmidhuber J. Highway networks. arXiv [csLG]. May 2015. Available at: http://arxiv.org/abs/1505.00387 . Accessed January 8, 2020
• 46 Yang J, Parikh D, Batra D. Joint unsupervised learning of deep representations and image clusters. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016: 5147-5156
• 47 Jayaraman D, Grauman K. Slow and steady feature analysis: higher order temporal coherence in video. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016. Doi: 10.1109/cvpr.2016.418
• 48 Doersch C, Gupta A, Efros AA. Unsupervised visual representation learning by context prediction. In: Proceedings of the IEEE International Conference on Computer Vision; 2015: 1422-1430
• 49 Wen J, Thibeau-Sutre E, Samper-Gonzalez J. , et al. Convolutional neural networks for classification of Alzheimer's disease: overview and reproducible evaluation. arXiv [csLG]. April 2019. Available at: http://arxiv.org/abs/1904.07773 . Accessed January 8, 2020
• 50 Guo C, Pleiss G, Sun Y, Weinberger KQ. On calibration of modern neural networks. arXiv [csLG]. June 2017. Available at: http://arxiv.org/abs/1706.04599 . Accessed January 8, 2020
• 51 Raghu M, Blumer K, Sayres R. , et al. Direct uncertainty prediction for medical second opinions. arXiv [csLG]. July 2018. Available at: http://arxiv.org/abs/1807.01771 . Accessed January 8, 2020
• 52 Lipton ZC. The myth of model interpretability. Available at: https://arxiv.org/pdf/1606.03490.pdf . Accessed January 8, 2020
• 53 Adebayo J, Gilmer J, Muelly M, Goodfellow I, Hardt M, Kim B. Sanity checks for saliency maps. In: Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R. , eds. Advances in Neural Information Processing Systems 31. Red Hook, NY: Curran Associates; 2018: 9505-9515
• 54 Zeiler MD, Fergus R. Visualizing and understanding convolutional networks. Computer Vision – ECCV 2014. Available at: https://arxiv.org/abs/1311.2901 . Accessed January 8, 2020
• 55 Szegedy C, Zaremba W, Sutskever I. , et al. Intriguing properties of neural networks. arXiv [csCV]. December 2013. Available at: http://arxiv.org/abs/1312.6199 . Accessed January 8, 2020
• 56 Olah C, Mordvintsev A, Schubert L. Feature visualization. Distill 2017; 2 (11) e7
• 57 Springenberg JT, Dosovitskiy A, Brox T, Riedmiller M. Striving for simplicity: the all convolutional net. arXiv [csLG]. December 2014. Available at: http://arxiv.org/abs/1412.6806 . Accessed January 8, 2020
• 58 Sundararajan M, Taly A, Yan Q. Axiomatic attribution for deep networks. arXiv [csLG]. March 2017. Available at: http://arxiv.org/abs/1703.01365 . Accessed January 8, 2020
• 59 Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A. Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016: 2921-2929
• 60 Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. arXiv [csCL]. September 2014. Available at: http://arxiv.org/abs/1409.0473 . Accessed January 8, 2020
• 61 Xu K, Ba J, Kiros R. , et al. Show, attend and tell: neural image caption generation with visual attention. arXiv [csLG]. February 2015. Available at: http://arxiv.org/abs/1502.03044 . Accessed January 8, 2020
• 62 Theis L, van den Oord A, Bethge M. A note on the evaluation of generative models. arXiv [statML]. November 2015. Available at: http://arxiv.org/abs/1511.01844 . Accessed January 8, 2020
• 63 Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann EK. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study. PLoS Med 2018; 15 (11) e1002683
• 64 Subbaswamy A, Schulam P, Saria S. Preventing failures due to dataset shift: learning predictive models that transport. arXiv [statML]. December 2018. Available at: http://arxiv.org/abs/1812.04597 . Accessed January 8, 2020
• 65 Gyftopoulos S, Lin D, Knoll F, Doshi AM, Rodrigues TC, Recht MP. Artificial intelligence in musculoskeletal imaging: current status and future directions. AJR Am J Roentgenol 2019; 213 (03) 506-513
• 66 Hirschmann A, Cyriac J, Stieltjes B, Kober T, Richiardi J, Omoumi P. Artificial intelligence in musculoskeletal imaging: review of current literature, challenges, and trends. Semin Musculoskelet Radiol 2019; 23 (03) 304-311
• 67 Burns JE, Yao J, Summers RM. Artificial intelligence in musculoskeletal imaging: a paradigm shift. J Bone Miner Res 2019 ; August 9 (Epub ahead of print)
• 68 LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521 (7553): 436-444
• 69 Riley P. Three pitfalls to avoid in machine learning. Nature 2019; 572 (7767): 27-29
• 70 Wiens J, Saria S, Sendak M. , et al. Do no harm: a roadmap for responsible machine learning for health care. Nat Med 2019; 25 (09) 1337-1340
• 71 Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2017: 2097-2106
• 72 Irvin J, Rajpurkar P, Ko M. , et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. arXiv [preprint arXiv:1901 07031]. 2019 . Available at: https://arxiv.org/abs/1901.07031 . Accessed January 8, 2020
• 73 Johnson AEW, Pollard TJ, Berkowitz SJ. , et al. MIMIC-CXR: a large publicly available database of labeled chest radiographs. arXiv [csCV]. January 2019. Available at: http://arxiv.org/abs/1901.07042 . Accessed January 8, 2020
• 74 Rajpurkar P, Irvin J, Bagul A. , et al. MURA: large dataset for abnormality detection in musculoskeletal radiographs. arXiv [physics.med-ph]. December 2017. Available at: http://arxiv.org/abs/1712.06957 . Accessed January 8, 2020
• 75 Zbontar J, Knoll F, Sriram A. , et al. fastMRI: an open dataset and benchmarks for accelerated MRI. arXiv [csCV]. November 2018. Available at: http://arxiv.org/abs/1811.08839 . Accessed January 8, 2020