Key words
radiomics - artificial intelligence - deep learning - precision medicine - molecular imaging
Introduction
Evidence-based medicine is defined as “the conscientious, explicit, judicious, and reasonable use of modern, best evidence in making decisions about the care of individual patients” [1]. It was introduced to educate physicians about standardized, science-guided, traceable but dynamically evolving patient care [2]. It comes close to the definition of “precision medicine”, where patient cohorts are subclassified according to their disease characteristics but do not automatically require personalization of treatment, the latter being difficult to realize in clinical practice.
Therapeutics targeting very specific molecular characteristics of diseases are an integral element of evidence-based medicine. However, increasing specificity of therapeutics also requires higher granularity in patient selection, resulting in an increasing demand for diagnostic tools that provide information beyond morphology. In detail, the diagnostic method should provide information that directly or indirectly describes the pathophysiology, the therapeutic target, or the dominant response mechanism to the therapeutic drug.
There are two main strategies as to how this can be achieved by imaging: First, one can try to identify one or a few key features of the pathology or the mechanism of action of the drug and visualize them with specific diagnostic probes ([Fig. 1A]). About 20 years ago, this led to the rise of molecular imaging, facilitated by the increasing availability of high-sensitivity imaging modalities such as PET, SPECT, and optical imaging, as well as advances in probe development [3]
[4]. Although the proof of concept was frequently provided, only a few molecular imaging probes finally entered clinical practice. This can be attributed to many reasons, such as high costs, low revenues for probes applied in small patient subpopulations, lack of superiority over established imaging methods, and competition with in vitro omics analyses.
Fig. 1 Routes to realize evidence-based therapy management by either molecular imaging A or radiomic/multi-omic strategies B. A: The molecular imaging approach starts with intense research on a molecular biomarker that is accessible to drugs and highly specific for the related medical issue. Then, a binding ligand is developed that can either be used diagnostically or therapeutically. Here, the diagnostic agent can be used as a companion diagnostic agent to stratify treatment with the therapeutic one. Alternatively, both diagnostic and therapeutic properties can also be realized in the same molecule, resulting in a theranostic agent. B: The alternative approach is to use data derived from multiple diagnostic interventions to perform clustered analyses and to derive patterns that support therapy management. For this purpose, quantitative features need to be extracted from histopathological, radiological/nuclear medicine image data. Machine and deep learning are used to analyze the features together with the other data and to generate algorithms that can be applied for decision support. However, as these algorithms do not indicate causalities, their careful validation is highly important. Much research is currently spent on making them explainable, linking the results to disease pathophysiology, and using them to build digital disease models.
At the same time, triggered by the improvements in data storage, processing, and image analysis, a new field of research emerged, which is called radiomics [5]
[6]. In principle, the intention of radiomics is very similar to molecular imaging but the approach is different. Here, multiple features are evaluated from – mostly – routine clinical images, and cluster analysis is performed using classical machine learning or advanced artificial imaging (AI) tools ([Fig. 1B]). The resulting feature panels or pattern can be indicative for the pathological process, although the pathophysiological meaning of the individual feature is often not known.
This article discusses the complementarity and competitiveness of both approaches and predicts a picture of future evidence-based therapy management. It also highlights the disease-dependent challenges on the engineering of future imaging devices, probes, analysis algorithms, and databases.
Is molecular imaging still important?
Is molecular imaging still important?
The popularity of scientific fields typically shows a waveform that follows the so-called Gartner Hype Cycle [7]. Initially, there are many new ideas, and expectations are high. Furthermore, scientists tend to oversell their findings and present unrealistic translational perspectives to acquire grant money and further their careers and the field. Following initial disappointment, the community tends to doubt the value of the field and its popularity decreases. However, some robust and meaningful approaches typically survive leading to a maturation of the field, and often a re-increase in popularity is observed when translational successes can be reported.
Furthermore, in molecular imaging, many translational successes are not classified as molecular imaging and thus are not recognized as such. However, there is no doubt that molecular imaging already plays a crucial role in evidence-based therapy control and that it will even become more important in the future [8]. [Table 1] provides an overview of clinically approved molecular imaging agents. Certainly, the greatest successes were achieved in the field of nuclear medicine, i. e., PET and SPECT, due to their high sensitivity to probes and the ability for clinical translation. Additionally, it is noteworthy that also in MRI for liver imaging a molecularly targeted probe (Gd-EOB DTPA) is meanwhile the clinical standard [9]. Translational successes were also obtained in ultrasound where a first angiogenesis-targeted probe is currently under clinical investigation [10]. Nevertheless, molecular imaging applications are always competing with other imaging approaches and need to prove their superiority and added value with respect to therapy management. Intense discussions with clinicians are required to identify the ideal applications. In this context, imaging of chronic kidney disease (CKD) using the elastin-targeted MRI agent ESMA, which was originally introduced by Rene Botnar and co-workers to assess arteriosclerosis [11], is a good example [12]: No imaging method or serum analysis can reliably distinguish between kidney regeneration and chronification of the disease after an acute event. Thus, many patients remain untreated in the early but still treatable stage of disease. Using various preclinical models, molecular imaging of elastin was demonstrated to provide this decisive information about CKD chronification [12] and thus, molecular imaging of the extracellular matrix may address an urgent medical need. Theranostics is another field where molecular imaging meets evidence-based therapy control. In particular, in the field of antibody-based therapeutics, multiple clinical theranostic studies are ongoing that have been summarized by Moek et al. [13]. Currently, most clinically applied theranostic agents are radiopharmaceuticals that can be equipped with diagnostic and therapeutic radioisotopes [14]. Depending on the radioisotope they can then be used for patient selection, therapy monitoring, and therapy. Alternatively, therapy control can be performed with companion diagnostic agents. These agents are able to predict or monitor the success of one or a group of pharmaceuticals. For example, Ramananthan and co-workers used superparamagnetic iron oxide nanoparticles (SPION) to predict the “enhanced permeability and retention” (EPR)-based accumulation of nanomedicines in tumors and showed that high SPION accumulation correlated with therapy response [15]. Despite this promising result, however, the clinical success of the approach will depend on whether the same information can be obtained from standard contrast-enhanced MRI scans, radiomics, or histological analyses.
Table 1
Overview of clinically approved molecular imaging agents, their targets, and indications (*academic approval only at a few facilities).
Probe
|
Target
|
Indication
|
Status
|
Positron Emission Tomography (PET)
|
11C Choline
|
Choline kinase (Enzyme)
|
Suspected prostate cancer recurrence with elevated blood levels of prostate-specific antigen (PSA); differential diagnosis of brain tumors, lung cancer, esophageal cancer
|
FDA*
|
64Cu Dotatate, 64Cu oxodotreotide
|
Somatostatin receptors
|
Somatostatin receptor positive neuroendocrine tumors
|
FDA
|
18F Florbetaben
|
Amyloid beta
|
Alzheimerʼs disease and other cognitive impairments
|
FDA
|
18F Florbetapir
|
Amyloid beta
|
Alzheimerʼs disease and other cognitive impairments
|
FDA
|
18F Flortaucipir/AV1451
|
Aggregated Tau protein (neurofibrillary tangles; NFT)
|
Alzheimerʼs disease
|
FDA
|
18F Fluciclovine
|
Enters cells through energy-independent L-type amino acid transporter system (LAT) and is metabolized since it is an AA analogue
|
Prostate cancer diagnosis based on elevated PSA levels
|
FDA
|
18F Piflufolastat
|
Prostate-specific membrane antigen (PSMA)
|
Suspected metastasis and recurrence of prostate cancer
|
FDA
|
18F FDG Fluorodeoxyglucose
|
Glucose metabolism
|
Oncology: Hodgkin’s, non-Hodgkin lymphoma, colorectal cancer, melanoma, lung cancer, head and neck tumors, laryngeal cancer, esophageal cancer, cervical cancer, breast cancer, malignant ovarian cancer, chronic lymphocytic leukemia.
Assessing myocardial hibernation.
Identification of foci of epileptic seizures in the brain
|
FDA
|
18F Fluoroestradiol
|
Estrogen receptor (ER)
|
Recurrent or metastatic breast cancer
|
FDA
|
18F Flutemetamol
|
Amyloid beta
|
Alzheimer‘s disease and other cognitive impairments
|
FDA
|
68Ga Dotatate
|
Somatostatin receptors
|
Neuroendocrine tumors
|
FDA
|
68Ga Dotatoc
|
Somatostatin receptors
|
Neuroendocrine tumors
|
FDA
|
68Ga PSMA-11
|
PSMA
|
Prostate cancer recurrence and metastasis
|
FDA*
|
89Zr Panitumumab
|
Epidermal growth factor receptor EGFR
|
Colorectal cancer
|
IND
|
Single Photon Emission Tomography (SPECT)
|
123I Ioflupane
|
Presynaptic dopamine transporters (DAT)
|
Suspected Parkinson’s syndromes. Differentiation between Parkinson’s and other tremor causes
|
FDA
|
Magnetic Resonance Imaging (MRI)
|
Gadoxetic acid, Gd-EOB-DTPA
|
OATP receptors
|
Liver imaging
|
FDA
|
Ultrasound
|
BR55
|
VEGFR2
|
Imaging of tumor angiogenesis (prostate, breast, ovarian cancers)
|
IND
|
Optical Invivo Imaging
(only dyes approved; reported are examples for clinical studies using the dyes as imaging tag)
|
FITC (Fluorescein isothiocyanate)
|
Folate receptor
TNF-alpha
|
Interoperative ovarian and breast cancer imaging. Rheumatic diseases (e. g., psoriasis)
|
FDA
|
IRDye 800CW
|
EGFR
VEGF
|
Intraoperative imaging of pancreatic and lung cancer. Imaging of tumor angiogenesis (e. g., in colon carcinomas/adenomas)
|
FDA
|
Scintigraphy
|
111 In Pentetreotide
|
Somatostatin receptors
|
Primary and metastatic neuroendocrine tumors
|
FDA
|
Why is radiomics suddenly emerging?
Why is radiomics suddenly emerging?
Feature-based image analysis has been around for decades. Furthermore, most machine learning algorithms as well as convolutional neural networks (CNN) that are applied in radiomics were developed a long time ago. Major reasons for the rise of radiomics are digitalization in medicine, the higher quality of images, the systematic collection of image data, and the increasing awareness that a single imaging biomarker often sufficiently describes disease processes ([Table 2]). Furthermore, budgetary pressure in the health systems promotes maximum exploitation of currently applied imaging methods over the costly development of new imaging modalities and probes. Finally, clinical decision making becomes increasingly complex and demands the integrated evaluation of various diagnostic features deriving from different disciplines [16]. As humans can simultaneously consider only a handful of parameters, computer-aided decision support is in high demand.
Table 2
Examples of commercially approved image analysis tools.
Company name
|
Application
|
Approved
|
Webpage
|
Arterys
|
Cardio, lung, liver, breast
|
FDA
|
https://www.arterys.com/
|
GE Health Edison
|
Breast
|
FDA
|
https://www.gehealthcare.com/products/edison
|
Hermes Medical Solutions
|
Onco, neuro, cardio, pulmonary,
|
FDA
|
https://www.hermesmedicalsolutions.com/
|
Intrasense Myrian
|
Cardio, neuro, breast
|
FDA
|
https://www.intrasense.fr/
|
MeVis Medical Solutions AG
|
Breast, lung, neuro, prostate, bowel, liver
|
EN ISO, FDA
|
https://www.mevis.de/
|
Oncoradiomics SA – RadiomiX
|
Onco
|
EN ISO
|
https://radiomics.bio
|
Philips Healthcare – IntelliSpace Portal
|
Cardio, onco, neuro, pulmonary
|
FDA
|
https://www.philips.com/
|
Siemens Healthineers
|
Cardio, onco, neuro, breast, pulmonary
|
FDA
|
https://www.siemens-healthineers.com/
|
The authors distinguish three major approaches for how computer-aided decision support can be realized ([Fig. 2]):
Fig. 2 Three main approaches to how computer-aided decision support can be realized A. Data-driven approaches rely on extracting features from input data (e. g., imaging features in the case of radiomics), while model-based approaches use the data as input to an analytical (causal) model or to train a machine learning model, resulting in the targeted biomarker vector. Hybrid approaches are about to combine these two approaches by getting the best of both approaches. Different architectures for analyzing and storing data and methods B with their advantages (+) and disadvantages (–). Data augmentation using Generative Adversarial Networks (GANs) to generate arbitrary syntactic data from a learned data distribution C. The shown synthetic images were generated using the GAN from [37]. The left part shows the basic architecture of a GAN, which consists of a generative model that generates the synthetic (fake or synthetic) data, while the discriminator is trained to distinguish fake from real data. Both networks are trained in a concurrent manner. Further details can be found in [37].
1. Data-driven: This is the classical radiomics approach where multiple features are extracted from the images, clustered, ranked and correlated with disease or therapy response characteristics. As these image features can be handled as any other diagnostic data, integrated analysis with clinical findings and parameters from blood and urine can be realized [17]. This approach is suited to build decision trees and to detect dependencies between parameters but hardly provides causalities.
Furthermore, it is noteworthy that not every data analysis approach fits for every data collection and CNN. While CNNs enable unsupervised learning and, thus, the identification of novel classifications within a patient population, they depend on considerably large data collectives and represent a black box. To identify decision hierarchies in smaller data collections, classical machine learning approaches are often preferable. In this context, gradient tree boosting (GTB) appears particularly attractive, as decisions are fully retrievable and it even works with incomplete and considerably heterogeneous data [18]. GTB belongs to the class of supervised learning methods and is based on the combination of a set of decision trees. Here, boosting relies on an iterative approach where more decision trees are added to improve the prediction of the GTB. GTB has recently been applied in several areas, such as computer-aided diagnosis of lung nodules [19] and real-time reconstruction of the location of the gamma-crystal interaction of PET detectors [20].
2. Model-based: The aim of this approach is to display the pathological process in a mathematical model [21], ideally resulting in a “digital twin”. The advantage over the data-driven approach is its mechanistic nature. As it provides causalities, drug responses can be simulated, and even new treatments can be tested in silico. The integration of imaging into the models can be used to identify biomarkers predicting ideal patient cohorts or indicating therapy response. However, a precondition for these model-based approaches (also known as “systems medicine”) is profound knowledge of the pathophysiological regulations and their direct impact on the images. As this is often not sufficiently given, the process often fails, although it reflects the most ideal solution. Particularly in biology, many pathophysiological processes are not tightly regulated and are influenced by multiple co-factors, many of which are even unknown. In contrast, the use of model-based approaches for image acquisition, data reconstruction and analysis is very widespread. For example, a very successful class of model-based image reconstruction methods is using the theory of compressed sensing (CS) [22]
[23]
[24]. In CS, the regularization model is based on the assumption that the image information is sparse in certain regions. Lustig et al. presented several applications of CS with a particular focus on rapid MRI imaging [22]. Furthermore, model-based iterative image reconstruction is very often used in the field of CT, PET, and SPECT image reconstruction, in which physical models of the scanners are being used [25]
[26]. Salomon et al. used a scanner model and the singles and coincidence data to achieve relative scanner normalization [27]. Furthermore, 4 D lung or heart motion models of the patient for motion-compensated reconstruction are also very common [25].
Another area in which model-based methods are intensively used is the entire domain of super-resolution image reconstruction. Here, super-resolution ultrasound is an excellent example of model-based data reconstruction [28]. For example, in “motion model ultrasound localization microscopy”, a Markov Chain Monte Carlo Data Association Algorithms is applied to assess – based on the enhancement of voxels – the probability of motion of a microbubble within an ultrasound image over time. The resulting tracks can be visualized in much higher resolution than provided by the ultrasound transducer. Furthermore, blood velocities and flow direction can be assessed in individual microvessels and relative blood volume determined without the need for complex pharmacological models [29]. The multiple parameters derived from such super-resolution techniques can then be fed into data-driven or model-based analyses to improve disease characterization and more sensitively assess therapy responses.
Another example of model-based data analysis was provided by Gremse and coworkers [30]. Here, it was the aim to automatically detect arterial stenosis on CT and MR angiography datasets. The approach included a 3 D reconstruction of the vasculature. Then, a virtual ball was sent through the vessels that adapted its size to the vessel lumen. If the size of the ball decreases, there is usually a bifurcation. However, if it decreases and subsequently re-increases there is usually a stenosis. The suspicious areas can then automatically be indicated to the physician for further assessment.
3. Hybrid modeling: Hybrid modeling is currently hardly used in imaging. However, it is the logical next step considering the strengths and weaknesses of the two approaches mentioned above [31]
[32]. The basis is a mathematical (disease) model, which can be simple but should be able to iteratively grow. The model is additionally fed by algorithms provided by a data-driven approach, e. g., a CNN. Thus, the “descriptive” data and correlative findings are continuously transferred into causalities and, thus, become more and more explainable [33]
[34]. An appealing overview of strategies and methods to reveal hybrid models across resolution scales and different data types has been presented by Herrgårdh and coworkers with a focus on stroke care [32]. However, the strategies that are presented and discussed could easily be adopted to other disease areas.
What are the current and future data storage and sharing architectures?
What are the current and future data storage and sharing architectures?
For the future development of evidence-based therapy management, the architecture of data processing and data storage is a crucial factor. Here, the authors distinguish three processing architectures ([Fig. 2B]):
1. Cloud computing is an architecture that provides on-demand availability of computing power and storage resources, without active management by the user [35]. The core idea of cloud computing is resource sharing, typically used as a “pay-as-you-go” model. Through the shared use of computing and storage resources, cloud computing is one of the most flexible solutions. One disadvantage is that the patient data to be processed must be transferred to the cloud and thus data security and data protection must be guaranteed.
2. A Data Warehouse is a system used for reporting and data analysis. These systems are centralized repositories for integrated data from one or more different sources [36]. The types of data to be stored (images, omics, etc.) are ordered and integrated in a clearer way. The disadvantage is that on the one hand the patient data is copied into the data warehouse, and on the other hand these solutions are typically not offered across company boundaries, which can limit their adoption in the medical field.
3. Distributed analytics is an architecture in which algorithms, rather than data, are shared [37]. This avoids the sharing of sensitive patient data. Thus, this last architecture is already suitable for multi-center studies. A necessary prerequisite for distributed analytics is that the data has to be available in a standardized form. Since the algorithm and not the data is shared, the intellectual property is usually shared as well, which requires a cultural change.
For the two first architectures, Cloud Computing and Data Warehousing, the handling of patient-related data is very critical. In addition, training future AI methods will require far more data than are easily available. Therefore, the generation of synthetic data has recently established itself as a new field of research. A very promising approach is to use the aforementioned GANs [38], which can generate an infinite amount of synthetic data or images that closely resemble real data from a learned data distribution ([Fig. 2C]).
Conclusion
As the information content of medical images is currently not fully exploited in diagnostic ratings, radiomics provides an important chance to improve diagnostic accuracy, and to elucidate new imaging biomarkers that can be used for evidence-based therapy management. It may also support the cross-disciplinary and integrated use of various diagnostic data. However, this does not mean that radiomics will replace molecular imaging and theranostics. This is particularly true for the latter where only the “diagnostic aspect” can be replaced. In addition, as theranostic drugs provide direct feedback on drug accumulation and performance, other diagnostic methods may not be able to provide higher diagnostic accuracy, but rather may complement the diagnostic information.
Furthermore, important features of the disease or information about therapeutic targets may often not be available from routine images. These information deficits need to be determined and molecular imaging probes need to be developed to fill these gaps. Then, molecular imaging data may become part of the radiomic analysis, leading to a merge of both approaches. Furthermore, radiomics, which currently provides patterns and correlations rather than causalities, needs to evolve, which may effectively work via hybrid modeling, finally aiming at the generation of digital twins [31]. The latter could then be used to develop and refine personalized treatment schemes in silico with a much lower risk of failure. However, to reach these goals, new concepts for data sharing need to be developed, and there must be an openness to collaborative data use. Furthermore, for evidence therapy management, engineering of devices, probes, image analysis, and data storage should be tightly coordinated on each other and focused on the particular application.
Thus, in summary the following take home messages can be formulated:
-
Molecular imaging and radiomics are both providing valuable disease biomarkers that potentially complement each other.
-
Integrated diagnostics based on data-driven, model-based, and hybrid modeling approaches will allow pathophysiological conclusions with high precision.
-
Disease-tailored refinements of devices, molecular probes, image analysis methods and databases are prerequisites for future evidence-based therapy management.