Keywords
retina - optical coherence tomography - artificial intelligence - retinal imaging
Introduction
The number of patients exhibiting retinal diseases is steadily increasing. Retinal diseases are also the most common cause of visual impairment and loss of vision amongst older people in industrialised nations [1]. This has led to rapid development in AI (artificial intelligence) algorithms towards automatically analysing the immense amounts of data collected every day in clinical routine. Patients with age-related macular degeneration (AMD) and diabetic retinopathy (DR) in particular require regular monitoring to detect complications requiring treatment at an early stage for conditions that may otherwise lead to potentially irreversible vision loss such as neovascular AMD (nAMD) and diabetic macular oedema (DME) [2]. Apart from that, new treatment options and imaging techniques are constantly leading to new developments and breakthroughs in ophthalmology. The introduction of intravitreal anti-vascular endothelial growth
factor (VEGF) injections for treating nAMD and DME have contributed to a substantial decrease in legal blindness [3], [4]. The number of patients receiving regular intravitreal injections and follow-up visits is set to increase with the approval of new treatments in the USA for geographic atrophy (GA), the dry late form of AMD [5], [6]. High-resolution imaging methods such as optical coherence tomography (OCT) provide new and detailed insights into disease processes as well as mechanisms of action in new forms of treatment [7]. Ophthalmologists cannot detect most of these subclinical biomarkers in patients without additional image analysis. AI-based algorithms are able to evaluate large amounts of data accurately, objectively, and extremely rapidly, thus significantly reducing workloads and allowing more efficient and individual disease
management for ophthalmologists treating patients in clinical routine as well as in the healthcare system in general.
Artificial Intelligence and Deep Learning
Artificial Intelligence and Deep Learning
AI is a branch of computer science aimed at training human-like cognitive abilities in machines or computer systems [8]. AI-based models have mainly become based on machine learning, a subfield of AI. Machine learning (ML) is a process where algorithms learn principles and structures independently from given data rather than from preset definitions by human specialists. However, the ML principle relies on feature engineering, that is, predefined “features” or biomarkers such as central retinal thickness in an OCT volume. The computer can then find solutions and self-improve without being specifically programmed to do so. The “learned” solution approaches can then be applied to new unknown contexts for more efficient data analysis [8]. Machine learning has since developed into artificial neural networks (ANNs), which mimic neural networks in the human brain and therefore also human learning behaviour. ANNs consist of
interconnected artificial neurons arranged in several layers. The information fed into the network is processed, sorted, and refined for final analysis across several layers. ANNs are especially well-suited in classification or diagnostic systems such as in screening for diabetic retinopathy based on biomarkers in colour fundus photography. ANNs have developed into deep neural networks (DNNs) finally forming a new subfield – deep learning (DL) [9]. These DNNs consist of several intermediate layers and fewer artificial neurons, thus increasing their efficiency. A relevant benefit from DNNs is the continuous increase in performance with the size of the training dataset. Hardware development in computer processing power has also led to more rapid data processing by DNNs, which has since exceeded classical ML. DNN models are able to extract “features” from the data autonomously without human specialists giving them predefined definitions ([Fig. 1]). DNNs with specific convolutional neural network (CNN) architectures have emerged as the most capable of image analysis and mimic the structure of the human visual cortex. Specifically, CNNs are able to recognise visual patterns and identify objects in images. DL-based algorithms need large annotated datasets for training; this improves precision, but interpretability is more challenging in these models. It is possible to train ML-based models with smaller annotated datasets, which leads to greater transparency and interpretability [8].
Fig. 1 Diagram showing classical machine learning and deep learning using diabetic retinopathy screening as an example. Machine learning algorithms need features – predefined biomarkers – specified by human specialists, whereas deep learning identifies and classifies these features autonomously.(Source: Schmidt-Erfurth U, Sadeghipour A, Gerendas BS et al. Artificial intelligence in retina. Prog Retin Eye Res 2018; 67: 1 – 29. doi:10.1016/j.preteyeres.2018.07.004 [rerif])
Ophthalmology is especially suited for developing and applying novel AI-based models due to the increasing number of non-invasive high-resolution imaging techniques, especially on the retina. AI as used in retinal health can be divided into the following categories: Screening, disease stage classification, segmentation and quantification of lesions, disease progression prediction, and clinical decision support systems (CDSSs).
Diabetic Retinopathy
Almost 500 million people across the world suffer from diabetes, a number expected to rise to 600 million by 2040. One-third of those affected exhibit diabetic retinopathy, around 10% of which are at risk of serious loss of vision. Diabetic retinopathy is asymptomatic in its early stages, but still requires regular monitoring to detect later stages or complications that may lead to vision loss [10]. Vision loss is often too late to restore once sustained due to conditions such as tractional changes in the retina. The increasing number of patients requiring regular check-ups places a heavy burden on the healthcare system in time and expense, resulting in limited therapeutic impact in most cases. AI-supported models may relieve the burden on ophthalmologists and the entire healthcare system while also providing efficient support for screening and monitoring in clinical routine, especially in identifying patients in need of treatment.
Diabetic retinopathy: Screening
Developed in 2016, the IDx-DR system for screening diabetic retinopathy from colour fundus photographs (CFP) was one of the first medical AI applications ([Fig. 2]). The principle is based on DL and showed a sensitivity and specificity of 87% and 91%, respectively, in the validation study. The US Food and Drug Administration (FDA) approved the system as the first screening method for the presence of diabetic retinopathy based on these results [11]. The system does not differentiate absence of diabetic retinopathy from presence of diabetic retinopathy in its mild form as this stage does not require immediate treatment. Consultation with an ophthalmologist will be recommended if the device generates a positive result, that is, moderate or severe diabetic retinopathy. EyeArt is another FDA-approved system already in clinical use for screening diabetic retinopathy [12]; this system has shown
similar sensitivity and specificity as the IDx-DR system. In contrast, ophthalmologists show a sensitivity and specificity of 73% and 91%, respectively, in diagnosing diabetic retinopathy [13]. AI-based models for screening diabetic retinopathy have already come into clinical use with results on par with, or even surpassing, clinical specialists.
Fig. 2 Diagram showing diabetic retinopathy screening using the IDx-DR system.(Source: Schmidt-Erfurth U, Sadeghipour A, Gerendas BS et al. Artificial intelligence in retina. Prog Retin Eye Res 2018; 67: 1 – 29. doi:10.1016/j.preteyeres.2018.07.004 [rerif])
Diabetic retinopathy: Staging
The International Clinical Diabetic Retinopathy Disease Severity Scale classifies diabetic retinopathy into non-proliferative diabetic retinopathy (NPDR) and proliferative diabetic retinopathy (PDR) with NPDR further classified into mild, moderate, and severe [14]. Potential complications include diabetic macular oedema (DME), vitreomacular traction, epiretinal membrane formation, and retinal detachment. AI-based algorithms can also assist in distinguishing and classifying different stages in these cases. Two exemplary studies have investigated algorithms for automatic diabetic retinopathy staging using large datasets. One study using two datasets with 8,788 and 1,745 CFP images reported a sensitivity of 90% and 87% with a specificity of 98% in detecting moderate diabetic retinopathy or worse, a sensitivity of 84% and 88% with a specificity of 98% and 99% for severe diabetic retinopathy or worse, and a sensitivity of 91% and 90% with a specificity
of 98% and 99% for DME alone [15]. Another study using almost half a million CFP images achieved a sensitivity/specificity of 91%/92% and 100%/91% in detecting clinically relevant diabetic retinopathy and vision-threatening diabetic retinopathy [16].
Diabetic retinopathy: Prediction
We have investigated for OCT criteria affecting treatment outcomes in DME based on information from 629 eyes from the Protocol-T study and AI-based analysis and found a prominent correlation between intraretinal cysts and outcomes. In particular, resorption of intraretinal cysts after the first injection gave the greatest predictive value. However, it was not possible to give a reliable prediction for central visual acuity [17].
Age-related Macular Degeneration
Age-related Macular Degeneration
AMD is a chronic progressive disease that leads to progressive loss of retinal tissue and sensory cells, and therefore also to loss of vision. Around 196 million people are affected across the world with prevalence expected to increase to 288 million by 2040 due to increasing population age [18]. The risk of developing AMD increases exponentially with age. AMD is the most common cause of age-related blindness in the industrialised world. One in four people over sixty develop AMD [19]. AMD is distinguished into early, intermediate, and late forms depending on the size of the extracellular deposits or drusen. Late AMD is distinguished into two forms, neovascular AMD (nAMD) with macular neovascularisation (MNV) and geographic atrophy (GA) [20]. Injections using anti-VEGF drugs are used to treat nAMD, but no GA treatment has yet been approved in Europe. In contrast, the FDA has already issued full
approval for the first two drugs as intravitreal complement inhibitors for GA treatment [5], [6].
Age-related macular degeneration: Screening
AMD is usually asymptomatic in its early stages, but late forms of AMD lead to rapid and sometimes irreversible vision loss if left untreated. AI-based methods based on CFP for automatic AMD detection have shown a sensitivity/specificity of 93%/89% as examined on 72,610 images, and an accuracy of 90% from 130,000 images compared to human specialists [21]. However, the role of CFP in screening for AMD is still confined to its early stages, unlike diabetic retinopathy. Late stages exhibit subclinical parameters that cannot be assessed reliably using CFP. In particular, a reliable assessment of exudation in nAMD or thinning in the outer retinal layers as early atrophic changes is not possible using CFP. OCT imaging plays an important role here, as it can faithfully display subclinical changes at high resolution. A study based on a large dataset comprising 3,265 OCT volumes used in screening for AMD achieved a sensitivity and specificity of 93%
compared to ophthalmologists [22]. Early forms of GA are especially suitable for screening as they initially occur outside the fovea, with many patients only showing symptoms once the fovea or its immediate surroundings have been reached. Major multicentre projects are currently investigating the possibility of AI-based screening at population level, while also involving other professional groups such as pharmacists and orthoptists.
Age-related macular degeneration: Staging
Different stages of AMD have different implications in clinical routine and treatment. In such cases, AI-based algorithms can also help in the clinical setting and in rapid analysis of large databases for research. Results from OCT volumes from 367 individuals showed a sensitivity and specificity of 98% and 91% for classification into no AMD, early AMD, intermediate AMD, GA and nAMD compared to examination by ophthalmologists [22]. A DL-based algorithm has been developed and validated for automatic AMD classification into four stages – no AMD, intermediate AMD, nAMD, and GA – using real-world data on 3,765 OCT volumes from 1,849 eyes in the UK Biobank. The algorithm achieved an area under the curve (AUC) value of 0.94 [23]. This emphasises the potential of AI-based models in rapidly and reliably analysing large databases for both clinical routine and retrospective research on disease progression and conversion.
Age-related macular degeneration: Prediction
Predicting conversion from early or intermediate AMD to late AMD as nAMD or GA is especially relevant in a clinical setting. Treatment for nAMD should begin as early as possible after diagnosis to prevent potential complications such as bleeding or fibrosis that may lead to irreversible loss of vision. The FDA has already approved two drugs for GA treatment, which could see approval in Europe later this year [5], [6]. Treatment for GA should also begin as soon as possible to preserve central vision for as long as possible before the onset of subfoveal atrophy. An ML-based predictive model for automatic prediction of conversion to late AMD was tested on OCT data from 495 patients, achieving an accuracy of 68% for predicting MNV and 80% for predicting GA [24]. Various characteristic pathomechanisms were identified using an AI-based analysis of different biomarkers such as drusen, hyperreflective
foci, pseudodrusen, and segmentation of the retinal layers. Drusen volume played the main role in predicting conversion to MNV, whereas hyperreflective foci and thinning in the outer neurosensory layers were shown to be especially relevant as biomarkers for conversion to GA [24]. This demonstrated the ability of AI-based algorithms to provide new insights into disease progression.
Quantifying different biomarkers is one of the main applications of AI in clinical routine and in clinical studies towards improving disease activity assessment and treatment response. Quantifying the various fluid compartments plays an especially important role in nAMD; these fluid compartments include intraretinal fluid (IRF), subretinal fluid (SRF), and sub-RPE fluid (RPE: retinal pigment epithelium) for pigment epithelial detachment (PED) [25]. Automatically quantifying these biomarkers showed IRF in particular to predict a more adverse prognosis whereas SRF could be tolerated to some extent [26]. Ophthalmologists are not able to quantify fluid volumes at nanolitre amounts. AI-based models could provide support in objective and personalised therapy decisions in such cases by providing precise and reproducible values ([Fig. 3]) [27].
Fig. 3 Segmentation of different biomarkers on CFP and OCT images. Automatic segmentation is compared to manual segmentation by human specialists and lies within the range of interreader variability.(Source: Schmidt-Erfurth U, Sadeghipour A, Gerendas BS et al. Artificial intelligence in retina. Prog Retin Eye Res 2018; 67: 1 – 29. doi:10.1016/j.preteyeres.2018.07.004 [rerif])
GA has been undergoing a major paradigm shift since the FDA approved the first two drugs for treating GA, which may also see approval in Europe later this year. Results from pivotal studies based on fundus autofluorescence (FAF) measuring GA progression have shown significantly slower growth in treated patients compared to untreated patients [5], [6]. Using FAF to measure GA is characterised by the absence of fluorophoric metabolites such as lipofuscin due to missing RPE [28]. Post-hoc analysis using AI-based models for automatic segmentation of RPE and photoreceptors measured on the ellipsoid zone (EZ) has shown reliable visualisation of treatment effects at RPE level using OCT [29], [30]. Automatic segmentation of EZ loss also revealed the treatment effect to be more pronounced at EZ level, and therefore at the neurosensory layer rather
than at RPE level. Identifying a pathognomonic pattern in GA progression was also possible using automatic segmentation and precise evaluation of OCT volumes. EZ loss was shown to exceed RPE loss and precede the progression of RPE loss using AI-assisted analysis of OCT volumes from Phase III OAKS and DERBY study data. EC loss also affects treatment response. Data from the OAKS/DERBY trial, the largest successful Phase III study on GA treatment to date, also showed significantly faster growth and better treatment response in patients with high EZ-RPE loss ratios compared to patients with low ratios [30].
Several biomarkers have been identified as associated with rapid growth in GA progression assessment; these include: Pseudodrusen, hyperreflective foci, specific FAF patterns in the junctional zone, and multifocal extrafoveal lesions. However, these biomarkers have only been studied at population level so far; assessing individual risk of rapid growth towards predicting local lesion spread has remained a challenge. Pioneering studies have shown AI-based models to predict GA growth and localisation in en face images from a single OCT volume [31], [32], [33]. AI-supported analysis provides groundbreaking insights into the relevant mechanisms of GA progression and treatment response; these cannot be reliably assessed using FAF, nor are human specialists able to detect and quantify them at clinical examination.
AI-based models have also been investigated for their ability to predict treatment frequency and outcomes from anti-VEGF injections for nAMD. Selecting an appropriate treatment regime and interval remains a challenge in nAMD treatment. Compared to clinical trials, real-world results have shown undertreatment and more adverse outcomes for vision in treatment with anti-VEGF injections; this is likely due to late diagnosis or treatment delays on recurring exudation requiring treatment [27]. Pioneering studies have investigated the potential of AI-based algorithms to predict treatment needs [34], [35]. One model has been trained to distinguish a priori between low, moderate, and high therapy needs using on OCT volumes from 317 patients in a PRN regime (PRN: pro re nata). The algorithm achieved an AUC of 70% to 77%, a 50% performance increase compared to retina specialists [36]. An AI-based algorithm was also able to distinguish between interval extension suitability and unsuitability in patients on a treat-and-extend (T&E) regime [37].
Functional response to anti-VEGF injection is subject to wide interindividual variability, making it difficult for experienced ophthalmologists to predict. Even so, predicting treatment response based on central visual acuity may lead to higher patient compliance and motivation for ophthalmologists. Apart from that, factoring in a poor functional response prognosis may save important human and cost-intensive resources. A DL-based algorithm applied to 270 previously untreated patients randomised to receive ranibizumab in a T&E regime achieved an accuracy level of 0.87 in predicting central visual acuity. IRF and SRF volumes after the first injection have emerged as the most relevant biomarkers for predicting central visual acuity [38].
Other Retinal Diseases and Systemic Comorbidities
Other Retinal Diseases and Systemic Comorbidities
Other areas where AI has been applied to retinal diseases include screening and prognostics for retinopathy of prematurity [39], glaucoma [40], vascular occlusion [41], and hereditary retinal diseases [42]. Algorithms can also be applied in general screening between healthy individuals and individuals with retinal disease [43]. However, the potential areas of AI application go beyond ophthalmology; morphology of the retina and retinal vascularisation can also provide indications as to systemic risk factors and comorbidities [44]. AI-based algorithms can recognise patterns from huge amounts of data in determining age and gender based on CFP or OCT volume [45], [46]. One recent study used CFP to predict cardiovascular risk factors such as nicotine
consumption, body mass index, and HbA1c values using DL-based models [44]. The future will also see comorbidities such as Alzheimerʼs disease or multiple sclerosis being detected by applying AI-based analysis to CFP or OCT scans.
Approved MDR-certified Algorithms
Approved MDR-certified Algorithms
Using AI-based algorithms in clinical routine will require a stringent regulatory approval process beforehand. The Vienna Fluid Monitor and GA Monitor (RetInSight, Vienna, Austria) developed at the Medical University of Vienna ([Fig. 4]) are two MDR-certified algorithms for use in retinal diseases; MDR refers to the European Medical Device Regulation [47].
Fig. 4 Fluid Monitor (left) and GA Monitor (right) reports. These reports quantify different fluid compartments in nanolitres, RPE and photoreceptor segmentation in mm², and corresponding colour images on OCT scans showing precise future progression.(Source: https://retinsight.com/; Stand: 26.08.2024 [rerif])
Vienna Fluid Monitor was developed for automatic segmentation and quantification of IRF, SRF, and PED; the algorithm has been validated on both study data and data from clinical routine. Different fluid compartments are shown in colour on the OCT scans with volumes given in nanolitres. Fluid dynamics can be observed over time, and the algorithm can distinguish between active and inactive fluid, such as in degenerative cysts.
GA Monitor is designed for automatic segmentation and quantification of RPE loss as well as EZ loss representing photoreceptor status. This allows rapid and accurate quantification of GA growth and classification of disease activity using the EZ-RPE loss ratio.
Both algorithms use Heyex 2.6 software in its current version for the Heidelberg Spectralis device; the algorithms are to be approved for each of the other OCT device manufacturers. The system takes a matter of minutes to generate a report for the ophthalmologist showing the quantified fluid volumes in nanolitres or atrophy segmentations in mm2 including corresponding images. Tools such as Fluid Monitor (FM) and GA Monitor (GM) generate complete reports from the cloud at the click of a mouse with en face lesion localisation and new quantifications, that is, any increase or decrease of activity, from a standard OCT device (Spectralis) for the ophthalmologist to make a precise diagnosis and decide on suitable treatment there and then at patient consultation. The FDA currently considers EC loss to be the primary relevant biomarker for assessing photoreceptor status in GA. This biomarker provides a reliable measure of activity and therapeutic response for GA management
in clinical routine similar to the “fluid” biomarker in nAMD and is set to play a crucial role in recommending GA treatment for ophthalmological patients.
Other MDR-based CE-certified systems for retinal disease management using OCT comprise RetinAI Discovery (image and data management platform, RetinAI Medical AG) [48], iPredict (screening for AMD, DR, glaucoma using OCT and CFP, Ihealthservices Inc.) [49], and RetinaLyze (screening for DR, AMD, and glaucoma, RetinaLyze System A/S) [50]. RetInSight GA Monitor is the only MDR-certified algorithm to display photoreceptor loss as measured by EZ in addition to RPE loss.
Continuous development and integration of AI-based models in clinical routine will enable efficient, accurate, and objective processing for the huge amount of retinal imaging information available. These models provide new insights into pathomechanisms and treatment response in clinical studies while also enabling thorough and methodical evaluation of large databases. AI systems used as clinical decision support tools allow objective, reproducible, and most importantly, individual treatment decisions, initiating a paradigm shift in precision healthcare in ophthalmology.
