Keywords
retina - optical coherence tomography - artificial intelligence - retinal imaging
Introduction
The number of patients exhibiting retinal diseases is steadily increasing. Retinal
diseases are also the most common cause of visual impairment and loss of vision amongst
older people in industrialised nations [1]. This has led to rapid development in AI (artificial intelligence) algorithms towards
automatically analysing the immense amounts of data collected every day in clinical
routine. Patients with age-related macular degeneration (AMD) and diabetic retinopathy
(DR) in particular require regular monitoring to detect complications requiring treatment
at an early stage for conditions that may otherwise lead to potentially irreversible
vision loss such as neovascular AMD (nAMD) and diabetic macular oedema (DME) [2]. Apart from that, new treatment options and imaging techniques are constantly leading
to new developments and breakthroughs in ophthalmology. The introduction of intravitreal
anti-vascular endothelial growth
factor (VEGF) injections for treating nAMD and DME have contributed to a substantial
decrease in legal blindness [3], [4]. The number of patients receiving regular intravitreal injections and follow-up
visits is set to increase with the approval of new treatments in the USA for geographic
atrophy (GA), the dry late form of AMD [5], [6]. High-resolution imaging methods such as optical coherence tomography (OCT) provide
new and detailed insights into disease processes as well as mechanisms of action in
new forms of treatment [7]. Ophthalmologists cannot detect most of these subclinical biomarkers in patients
without additional image analysis. AI-based algorithms are able to evaluate large
amounts of data accurately, objectively, and extremely rapidly, thus significantly
reducing workloads and allowing more efficient and individual disease
management for ophthalmologists treating patients in clinical routine as well as in
the healthcare system in general.
Artificial Intelligence and Deep Learning
Artificial Intelligence and Deep Learning
AI is a branch of computer science aimed at training human-like cognitive abilities
in machines or computer systems [8]. AI-based models have mainly become based on machine learning, a subfield of AI.
Machine learning (ML) is a process where algorithms learn principles and structures
independently from given data rather than from preset definitions by human specialists.
However, the ML principle relies on feature engineering, that is, predefined “features”
or biomarkers such as central retinal thickness in an OCT volume. The computer can
then find solutions and self-improve without being specifically programmed to do so.
The “learned” solution approaches can then be applied to new unknown contexts for
more efficient data analysis [8]. Machine learning has since developed into artificial neural networks (ANNs), which
mimic neural networks in the human brain and therefore also human learning behaviour.
ANNs consist of
interconnected artificial neurons arranged in several layers. The information fed
into the network is processed, sorted, and refined for final analysis across several
layers. ANNs are especially well-suited in classification or diagnostic systems such
as in screening for diabetic retinopathy based on biomarkers in colour fundus photography.
ANNs have developed into deep neural networks (DNNs) finally forming a new subfield
– deep learning (DL) [9]. These DNNs consist of several intermediate layers and fewer artificial neurons,
thus increasing their efficiency. A relevant benefit from DNNs is the continuous increase
in performance with the size of the training dataset. Hardware development in computer
processing power has also led to more rapid data processing by DNNs, which has since
exceeded classical ML. DNN models are able to extract “features” from the data autonomously
without human specialists giving them predefined definitions ([Fig. 1]). DNNs with specific convolutional neural network (CNN) architectures have emerged
as the most capable of image analysis and mimic the structure of the human visual
cortex. Specifically, CNNs are able to recognise visual patterns and identify objects
in images. DL-based algorithms need large annotated datasets for training; this improves
precision, but interpretability is more challenging in these models. It is possible
to train ML-based models with smaller annotated datasets, which leads to greater transparency
and interpretability [8].
Fig. 1 Diagram showing classical machine learning and deep learning using diabetic retinopathy
screening as an example. Machine learning algorithms need features – predefined biomarkers
– specified by human specialists, whereas deep learning identifies and classifies
these features autonomously.(Source: Schmidt-Erfurth U, Sadeghipour A, Gerendas BS
et al. Artificial intelligence in retina. Prog Retin Eye Res 2018; 67: 1 – 29. doi:10.1016/j.preteyeres.2018.07.004 [rerif])
Ophthalmology is especially suited for developing and applying novel AI-based models
due to the increasing number of non-invasive high-resolution imaging techniques, especially
on the retina. AI as used in retinal health can be divided into the following categories:
Screening, disease stage classification, segmentation and quantification of lesions,
disease progression prediction, and clinical decision support systems (CDSSs).
Diabetic Retinopathy
Almost 500 million people across the world suffer from diabetes, a number expected
to rise to 600 million by 2040. One-third of those affected exhibit diabetic retinopathy,
around 10% of which are at risk of serious loss of vision. Diabetic retinopathy is
asymptomatic in its early stages, but still requires regular monitoring to detect
later stages or complications that may lead to vision loss [10]. Vision loss is often too late to restore once sustained due to conditions such
as tractional changes in the retina. The increasing number of patients requiring regular
check-ups places a heavy burden on the healthcare system in time and expense, resulting
in limited therapeutic impact in most cases. AI-supported models may relieve the burden
on ophthalmologists and the entire healthcare system while also providing efficient
support for screening and monitoring in clinical routine, especially in identifying
patients in need of treatment.
Diabetic retinopathy: Screening
Developed in 2016, the IDx-DR system for screening diabetic retinopathy from colour
fundus photographs (CFP) was one of the first medical AI applications ([Fig. 2]). The principle is based on DL and showed a sensitivity and specificity of 87% and
91%, respectively, in the validation study. The US Food and Drug Administration (FDA)
approved the system as the first screening method for the presence of diabetic retinopathy
based on these results [11]. The system does not differentiate absence of diabetic retinopathy from presence
of diabetic retinopathy in its mild form as this stage does not require immediate
treatment. Consultation with an ophthalmologist will be recommended if the device
generates a positive result, that is, moderate or severe diabetic retinopathy. EyeArt
is another FDA-approved system already in clinical use for screening diabetic retinopathy
[12]; this system has shown
similar sensitivity and specificity as the IDx-DR system. In contrast, ophthalmologists
show a sensitivity and specificity of 73% and 91%, respectively, in diagnosing diabetic
retinopathy [13]. AI-based models for screening diabetic retinopathy have already come into clinical
use with results on par with, or even surpassing, clinical specialists.
Fig. 2 Diagram showing diabetic retinopathy screening using the IDx-DR system.(Source: Schmidt-Erfurth
U, Sadeghipour A, Gerendas BS et al. Artificial intelligence in retina. Prog Retin
Eye Res 2018; 67: 1 – 29. doi:10.1016/j.preteyeres.2018.07.004 [rerif])
Diabetic retinopathy: Staging
The International Clinical Diabetic Retinopathy Disease Severity Scale classifies
diabetic retinopathy into non-proliferative diabetic retinopathy (NPDR) and proliferative
diabetic retinopathy (PDR) with NPDR further classified into mild, moderate, and severe
[14]. Potential complications include diabetic macular oedema (DME), vitreomacular traction,
epiretinal membrane formation, and retinal detachment. AI-based algorithms can also
assist in distinguishing and classifying different stages in these cases. Two exemplary
studies have investigated algorithms for automatic diabetic retinopathy staging using
large datasets. One study using two datasets with 8,788 and 1,745 CFP images reported
a sensitivity of 90% and 87% with a specificity of 98% in detecting moderate diabetic
retinopathy or worse, a sensitivity of 84% and 88% with a specificity of 98% and 99%
for severe diabetic retinopathy or worse, and a sensitivity of 91% and 90% with a
specificity
of 98% and 99% for DME alone [15]. Another study using almost half a million CFP images achieved a sensitivity/specificity
of 91%/92% and 100%/91% in detecting clinically relevant diabetic retinopathy and
vision-threatening diabetic retinopathy [16].
Diabetic retinopathy: Prediction
We have investigated for OCT criteria affecting treatment outcomes in DME based on
information from 629 eyes from the Protocol-T study and AI-based analysis and found
a prominent correlation between intraretinal cysts and outcomes. In particular, resorption
of intraretinal cysts after the first injection gave the greatest predictive value.
However, it was not possible to give a reliable prediction for central visual acuity
[17].
Age-related Macular Degeneration
Age-related Macular Degeneration
AMD is a chronic progressive disease that leads to progressive loss of retinal tissue
and sensory cells, and therefore also to loss of vision. Around 196 million people
are affected across the world with prevalence expected to increase to 288 million
by 2040 due to increasing population age [18]. The risk of developing AMD increases exponentially with age. AMD is the most common
cause of age-related blindness in the industrialised world. One in four people over
sixty develop AMD [19]. AMD is distinguished into early, intermediate, and late forms depending on the
size of the extracellular deposits or drusen. Late AMD is distinguished into two forms,
neovascular AMD (nAMD) with macular neovascularisation (MNV) and geographic atrophy
(GA) [20]. Injections using anti-VEGF drugs are used to treat nAMD, but no GA treatment has
yet been approved in Europe. In contrast, the FDA has already issued full
approval for the first two drugs as intravitreal complement inhibitors for GA treatment
[5], [6].
Age-related macular degeneration: Screening
AMD is usually asymptomatic in its early stages, but late forms of AMD lead to rapid
and sometimes irreversible vision loss if left untreated. AI-based methods based on
CFP for automatic AMD detection have shown a sensitivity/specificity of 93%/89% as
examined on 72,610 images, and an accuracy of 90% from 130,000 images compared to
human specialists [21]. However, the role of CFP in screening for AMD is still confined to its early stages,
unlike diabetic retinopathy. Late stages exhibit subclinical parameters that cannot
be assessed reliably using CFP. In particular, a reliable assessment of exudation
in nAMD or thinning in the outer retinal layers as early atrophic changes is not possible
using CFP. OCT imaging plays an important role here, as it can faithfully display
subclinical changes at high resolution. A study based on a large dataset comprising
3,265 OCT volumes used in screening for AMD achieved a sensitivity and specificity
of 93%
compared to ophthalmologists [22]. Early forms of GA are especially suitable for screening as they initially occur
outside the fovea, with many patients only showing symptoms once the fovea or its
immediate surroundings have been reached. Major multicentre projects are currently
investigating the possibility of AI-based screening at population level, while also
involving other professional groups such as pharmacists and orthoptists.
Age-related macular degeneration: Staging
Different stages of AMD have different implications in clinical routine and treatment.
In such cases, AI-based algorithms can also help in the clinical setting and in rapid
analysis of large databases for research. Results from OCT volumes from 367 individuals
showed a sensitivity and specificity of 98% and 91% for classification into no AMD,
early AMD, intermediate AMD, GA and nAMD compared to examination by ophthalmologists
[22]. A DL-based algorithm has been developed and validated for automatic AMD classification
into four stages – no AMD, intermediate AMD, nAMD, and GA – using real-world data
on 3,765 OCT volumes from 1,849 eyes in the UK Biobank. The algorithm achieved an
area under the curve (AUC) value of 0.94 [23]. This emphasises the potential of AI-based models in rapidly and reliably analysing
large databases for both clinical routine and retrospective research on disease progression
and conversion.
Age-related macular degeneration: Prediction
Predicting conversion from early or intermediate AMD to late AMD as nAMD or GA is
especially relevant in a clinical setting. Treatment for nAMD should begin as early
as possible after diagnosis to prevent potential complications such as bleeding or
fibrosis that may lead to irreversible loss of vision. The FDA has already approved
two drugs for GA treatment, which could see approval in Europe later this year [5], [6]. Treatment for GA should also begin as soon as possible to preserve central vision
for as long as possible before the onset of subfoveal atrophy. An ML-based predictive
model for automatic prediction of conversion to late AMD was tested on OCT data from
495 patients, achieving an accuracy of 68% for predicting MNV and 80% for predicting
GA [24]. Various characteristic pathomechanisms were identified using an AI-based analysis
of different biomarkers such as drusen, hyperreflective
foci, pseudodrusen, and segmentation of the retinal layers. Drusen volume played the
main role in predicting conversion to MNV, whereas hyperreflective foci and thinning
in the outer neurosensory layers were shown to be especially relevant as biomarkers
for conversion to GA [24]. This demonstrated the ability of AI-based algorithms to provide new insights into
disease progression.
Quantifying different biomarkers is one of the main applications of AI in clinical
routine and in clinical studies towards improving disease activity assessment and
treatment response. Quantifying the various fluid compartments plays an especially
important role in nAMD; these fluid compartments include intraretinal fluid (IRF),
subretinal fluid (SRF), and sub-RPE fluid (RPE: retinal pigment epithelium) for pigment
epithelial detachment (PED) [25]. Automatically quantifying these biomarkers showed IRF in particular to predict
a more adverse prognosis whereas SRF could be tolerated to some extent [26]. Ophthalmologists are not able to quantify fluid volumes at nanolitre amounts. AI-based
models could provide support in objective and personalised therapy decisions in such
cases by providing precise and reproducible values ([Fig. 3]) [27].
Fig. 3 Segmentation of different biomarkers on CFP and OCT images. Automatic segmentation
is compared to manual segmentation by human specialists and lies within the range
of interreader variability.(Source: Schmidt-Erfurth U, Sadeghipour A, Gerendas BS
et al. Artificial intelligence in retina. Prog Retin Eye Res 2018; 67: 1 – 29. doi:10.1016/j.preteyeres.2018.07.004 [rerif])
GA has been undergoing a major paradigm shift since the FDA approved the first two
drugs for treating GA, which may also see approval in Europe later this year. Results
from pivotal studies based on fundus autofluorescence (FAF) measuring GA progression
have shown significantly slower growth in treated patients compared to untreated patients
[5], [6]. Using FAF to measure GA is characterised by the absence of fluorophoric metabolites
such as lipofuscin due to missing RPE [28]. Post-hoc analysis using AI-based models for automatic segmentation of RPE and photoreceptors
measured on the ellipsoid zone (EZ) has shown reliable visualisation of treatment
effects at RPE level using OCT [29], [30]. Automatic segmentation of EZ loss also revealed the treatment effect to be more
pronounced at EZ level, and therefore at the neurosensory layer rather
than at RPE level. Identifying a pathognomonic pattern in GA progression was also
possible using automatic segmentation and precise evaluation of OCT volumes. EZ loss
was shown to exceed RPE loss and precede the progression of RPE loss using AI-assisted
analysis of OCT volumes from Phase III OAKS and DERBY study data. EC loss also affects
treatment response. Data from the OAKS/DERBY trial, the largest successful Phase III
study on GA treatment to date, also showed significantly faster growth and better
treatment response in patients with high EZ-RPE loss ratios compared to patients with
low ratios [30].
Several biomarkers have been identified as associated with rapid growth in GA progression
assessment; these include: Pseudodrusen, hyperreflective foci, specific FAF patterns
in the junctional zone, and multifocal extrafoveal lesions. However, these biomarkers
have only been studied at population level so far; assessing individual risk of rapid
growth towards predicting local lesion spread has remained a challenge. Pioneering
studies have shown AI-based models to predict GA growth and localisation in en face
images from a single OCT volume [31], [32], [33]. AI-supported analysis provides groundbreaking insights into the relevant mechanisms
of GA progression and treatment response; these cannot be reliably assessed using
FAF, nor are human specialists able to detect and quantify them at clinical examination.
AI-based models have also been investigated for their ability to predict treatment
frequency and outcomes from anti-VEGF injections for nAMD. Selecting an appropriate
treatment regime and interval remains a challenge in nAMD treatment. Compared to clinical
trials, real-world results have shown undertreatment and more adverse outcomes for
vision in treatment with anti-VEGF injections; this is likely due to late diagnosis
or treatment delays on recurring exudation requiring treatment [27]. Pioneering studies have investigated the potential of AI-based algorithms to predict
treatment needs [34], [35]. One model has been trained to distinguish a priori between low, moderate, and high
therapy needs using on OCT volumes from 317 patients in a PRN regime (PRN: pro re
nata). The algorithm achieved an AUC of 70% to 77%, a 50% performance increase compared
to retina specialists [36]. An AI-based algorithm was also able to distinguish between interval extension suitability
and unsuitability in patients on a treat-and-extend (T&E) regime [37].
Functional response to anti-VEGF injection is subject to wide interindividual variability,
making it difficult for experienced ophthalmologists to predict. Even so, predicting
treatment response based on central visual acuity may lead to higher patient compliance
and motivation for ophthalmologists. Apart from that, factoring in a poor functional
response prognosis may save important human and cost-intensive resources. A DL-based
algorithm applied to 270 previously untreated patients randomised to receive ranibizumab
in a T&E regime achieved an accuracy level of 0.87 in predicting central visual acuity.
IRF and SRF volumes after the first injection have emerged as the most relevant biomarkers
for predicting central visual acuity [38].
Other Retinal Diseases and Systemic Comorbidities
Other Retinal Diseases and Systemic Comorbidities
Other areas where AI has been applied to retinal diseases include screening and prognostics
for retinopathy of prematurity [39], glaucoma [40], vascular occlusion [41], and hereditary retinal diseases [42]. Algorithms can also be applied in general screening between healthy individuals
and individuals with retinal disease [43]. However, the potential areas of AI application go beyond ophthalmology; morphology
of the retina and retinal vascularisation can also provide indications as to systemic
risk factors and comorbidities [44]. AI-based algorithms can recognise patterns from huge amounts of data in determining
age and gender based on CFP or OCT volume [45], [46]. One recent study used CFP to predict cardiovascular risk factors such as nicotine
consumption, body mass index, and HbA1c values using DL-based models [44]. The future will also see comorbidities such as Alzheimerʼs disease or multiple
sclerosis being detected by applying AI-based analysis to CFP or OCT scans.
Approved MDR-certified Algorithms
Approved MDR-certified Algorithms
Using AI-based algorithms in clinical routine will require a stringent regulatory
approval process beforehand. The Vienna Fluid Monitor and GA Monitor (RetInSight,
Vienna, Austria) developed at the Medical University of Vienna ([Fig. 4]) are two MDR-certified algorithms for use in retinal diseases; MDR refers to the
European Medical Device Regulation [47].
Fig. 4 Fluid Monitor (left) and GA Monitor (right) reports. These reports quantify different
fluid compartments in nanolitres, RPE and photoreceptor segmentation in mm², and corresponding
colour images on OCT scans showing precise future progression.(Source: https://retinsight.com/; Stand: 26.08.2024 [rerif])
Vienna Fluid Monitor was developed for automatic segmentation and quantification of
IRF, SRF, and PED; the algorithm has been validated on both study data and data from
clinical routine. Different fluid compartments are shown in colour on the OCT scans
with volumes given in nanolitres. Fluid dynamics can be observed over time, and the
algorithm can distinguish between active and inactive fluid, such as in degenerative
cysts.
GA Monitor is designed for automatic segmentation and quantification of RPE loss as
well as EZ loss representing photoreceptor status. This allows rapid and accurate
quantification of GA growth and classification of disease activity using the EZ-RPE
loss ratio.
Both algorithms use Heyex 2.6 software in its current version for the Heidelberg Spectralis
device; the algorithms are to be approved for each of the other OCT device manufacturers.
The system takes a matter of minutes to generate a report for the ophthalmologist
showing the quantified fluid volumes in nanolitres or atrophy segmentations in mm2 including corresponding images. Tools such as Fluid Monitor (FM) and GA Monitor (GM)
generate complete reports from the cloud at the click of a mouse with en face lesion
localisation and new quantifications, that is, any increase or decrease of activity,
from a standard OCT device (Spectralis) for the ophthalmologist to make a precise
diagnosis and decide on suitable treatment there and then at patient consultation.
The FDA currently considers EC loss to be the primary relevant biomarker for assessing
photoreceptor status in GA. This biomarker provides a reliable measure of activity
and therapeutic response for GA management
in clinical routine similar to the “fluid” biomarker in nAMD and is set to play a
crucial role in recommending GA treatment for ophthalmological patients.
Other MDR-based CE-certified systems for retinal disease management using OCT comprise
RetinAI Discovery (image and data management platform, RetinAI Medical AG) [48], iPredict (screening for AMD, DR, glaucoma using OCT and CFP, Ihealthservices Inc.)
[49], and RetinaLyze (screening for DR, AMD, and glaucoma, RetinaLyze System A/S) [50]. RetInSight GA Monitor is the only MDR-certified algorithm to display photoreceptor
loss as measured by EZ in addition to RPE loss.
Continuous development and integration of AI-based models in clinical routine will
enable efficient, accurate, and objective processing for the huge amount of retinal
imaging information available. These models provide new insights into pathomechanisms
and treatment response in clinical studies while also enabling thorough and methodical
evaluation of large databases. AI systems used as clinical decision support tools
allow objective, reproducible, and most importantly, individual treatment decisions,
initiating a paradigm shift in precision healthcare in ophthalmology.
