1 Introduction
The emergence of the global pandemic of coronavirus COVID-19 dominated much of the
health informatics and medical research landscape during 2020. Hence it is appropriate
that this end-of-year review of recent developments in medical knowledge management
focuses on the pandemic.
The pandemic has highlighted the clear need for informatics to support management
and synthesis of health information at the global scale and pace in the face of a
rapidly spreading infection. However, it has also highlighted the presence of severe
limitations in our ability to share, integrate, and analyse data at this scale.
To address those limitations, we propose that the model of a “global learning health
system” (gLHS) can be deployed. The concept is of a learning health system (LHS) [[1]
[2]], expanded to a global scale but with the singular focus on the viral disease. Indeed,
the international effort to quickly gather and share knowledge for clinical diagnosis,
management, and treatment of COVID-19 can be seen as an exemplar of a gLHS, albeit
not yet fully realised or effective.
The key elements of a LHS, including the core information cycles that characterise
it, were observed throughout the interactions of the global scientific community.
Information flowed from practice (what was being done on the ground to manage COVID-19
patients) to data (what was captured about those patients and their clinical characteristics
or response to interventions) to knowledge (about disease characteristics and trends,
what care approaches worked, and what did not, based on analysis and modeling of the
data) and rapid implementation for practice again. Furthermore, as required by a LHS,
information technology infrastructure played a critical role in enabling these information
flows.
In the wake of the previous SARS and Ebola virus epidemics, it was already argued
that unified frameworks supporting clinical and biological data integration were critical
to support evidence generation in a pandemic [[3]] and that information technology was needed for knowledge management [[4]]. Broader adoption of electronic health records has facilitated evidence generation,
including in observational studies and in traditional randomised controlled trials,
through their use to identify eligible patients, support data collection, and monitor
outcomes [[5]]. Electronic sharing of patient-level data from trials facilitates re-analysis of
outcomes and fosters reproducibility and trust in findings [[6]]. But, our public health and clinical responses during COVID-19 demanded much more
sophisticated strategies for rapid information synthesis and knowledge management
than was available, which we argue an effective gLHS would facilitate.
The rapid spread of COVID-19 internationally and its immediate impact on the global
economy has led to much more widespread appreciation of the need to coordinate pandemic
research, including substantially increased scientific globalism [[7]] – international research collaboration – and sharing of patient-level clinical
data [[6]]. However, despite a few shining examples of rapid deployment of multi-site clinical
trials [[8]
[9]
[10]
[11]], clinical research for COVID-19 has been highly fragmented [[12]], with a particular dearth of meaningful evidence in the area of non-drug interventions
with public policy impacts [[13]].
This survey will show that although there is still work to be done, the pandemic has
illustrated that many of the elements required for a global learning health system
are in place, critically including the human motivation to achieve it.
The framework of the gLHS that we present is not only a useful way of characterising
how knowledge evolved during the pandemic under the strong impetus to support knowledge-informed
clinical care and public health response to COVID-19, but also provides an architecture
for a system that we can invest in to ensure robust knowledge evolution for ongoing
global health needs into the future.
2 A Global Learning COVID-19 System
In this section, we introduce the framework for ‘learning’ that we will adopt in our
analysis of the evolution of knowledge during the first year of the COVID-19 crisis
(Section 2.1). We then present a model of the collective activity toward learning
about COVID-19, viewed as a global-scale learning health system (Section 2.2), and
illustrating the elements of the gLHS that emerged as observed from the literature.
2.1 Framework for Conceptualising Learning in a Crisis
To frame the evolution of knowledge in the COVID-19 crisis, we follow a recent proposal
by Tovstiga and Tovstiga (2020) to adopt a classical four-quadrant ‘conscious-competence’
conceptual framework from learning theory [[14]]. The model is presented in [Figure 1], illustrating the stages of a learning trajectory, from unconscious ignorance of
lack of knowledge, to deeply embedded knowledge.
-
Quadrant 1: Zone of uncertainty, including lack of clarity about a topic
-
Quadrant 2: Zone of learning, where the value of knowledge on the topic is recognised and sought.
Questions play a crucial role in this zone;
-
Quadrant 3: Zone of actionable knowledge, where learning is consolidated and integrated with
existing knowledge;
-
Quadrant 4: Zone of embedded understanding, enabling intuitive action. Knowledge in this zone
is often not fully recognised.
Fig. 1 The ‘conscious-competence’ matrix of learning, based on a model originally attributed
to Broadwell (1969) [[83]]; adapted from [[14]].
While typically applied in the context of an individual learner, the Tovstigas argue
that the framework effectively reflects the general knowledge evolution process in
the context of the COVID-19 crisis. They further suggest that it is useful for structuring
and understanding the learning trajectory with respect to the crisis, demonstrating
the various phases through an analysis of information about COVID-19 communicated
through news reports.
Their analysis identifies data analytics and scientific knowledge sharing as key drivers
of the learning trajectory in COVID-19, citing efforts such as the World Health Organization's
creation of a global COVID-19 clinical information platform based on a standard case
report form[1] requesting data on specific detailed clinical and demographic parameters on COVID-19
positive patients. This underscores the important role of data standards in supporting
the learning needed to manage the pandemic. These are also key elements of the gLHS
and highlights the relevance of the model.
2.2 Modelling the COVID-19 Knowledge Ecosystem Through the Learning Health System
Building on this framework for conceptualising learning, we propose that the core
model of a LHS [[1]
[2]] can be applied to characterise the rapid evolution of knowledge that has occurred
during the COVID-19 pandemic. In this model, data and analytics over clinical practice
data from patient care drives learning of new knowledge that can be implemented to
improve clinical practice, leading to continuous improvement. A number of critical
knowledge management and information technology elements can be identified as key
enablers of this learning process, supporting the significant human efforts that catalysed
and provided the appropriate socio-technical conditions for the learning cycle.
Our proposed model for the COVID-19 gLHS is shown in [Figure 2]. The impetus for the learning process in COVID-19 arose from a knowledge gap, the
gap between purposeful action grounded in knowledge (Q3) that exists for routine clinical
care and the uncertainty surrounding diagnosis and management of the novel virus in
affected patients (Q1). This then triggered a broad effort to gather data to fill
the gap, primarily taking advantage of data collected through electronic health record
systems. Data collection and integration, facilitated through electronic data sharing,
then enabled learning (Q2), actioned through observational analysis, clinical trials,
predictive modelling, and other research leveraging data, with the objective of turning
data into information. Publications summarising these studies served as the key vehicle
for sharing research results, but challenges in finding and interpreting papers –
particularly in the face of a flurry of research activity – resulted in remaining
uncertainty in the knowledge that existed (Q4). Translation of new knowledge into
practice (Q3) required evidence synthesis approaches such as systematic reviews, critically
involving searching (information retrieval), screening, appraisal, and meta-analysis
of research publications. Key conclusions were rapidly shared via actively maintained,
living guidelines [[15]] and platforms [[16]] or tools [[17]] for making available clinical decision support knowledge artifacts.
Fig. 2 Abstraction of the structure of the global learning system in place for COVID-19.
Knowledge management activities are overlaid onto the Learning Health System model
(core LHS cycle figure adapted from [[84]]), and related to the learning framework of ‘conscious-competence’ presented in
[Figure 1].
2.2.1 Electronic Data Capture and Data Sharing
Electronic health records (EHRs) are a key resource in the learning health system,
as they provide the data that is used to drive learning from practice. For COVID-19,
EHRs were analysed to characterise early cases of the infection in Wuhan, China [[18]
[19]], to provide important information related to the efficacy of symptom-based screening
[[20]], and to collect data on patients prospectively after enrolment in a trial [[21]]. In the UK, the OpenSafely Platform[2] was used to identify factors associated with COVID-19 deaths through analysis of
the primary care records of over 17 million patients [[22]], facilitated through the use of a single EHR system (TPP SystemOne) by general
practice surgeries covering approximately 40% of the UK population. Similarly, a study
of risk factors associated with death due to COVID-19 [[23]] was made possible by the use of a single integrated EHR system across many sites,
and the Quick COVID-19 Severity Index was developed with data from a single health
system with nine Emergency Departments [[24]]. A highly-cited study demonstrating lack of effectiveness of hydroxychloroquine
treatment was conducted using data extracted directly from the New York-Presbyterian
/ Columbia University Irving Medical Center EHR [[25]].
Leveraging distributed EHRs across national and international boundaries through collaborative
consortia and clinical data networks [[26]], several large-scale studies were undertaken to characterise COVID-19 patients
in relation to similar disease groups [[27]], to understand the trajectory of the disease [[28]], and to study interaction of the disease with patient medications [[29]]. Such studies were enabled through the adoption of common data models to harmonise
data, prominently the Observational Health Data Sciences and Informatics (OHDSI) Observational
Medical Outcomes Partnership (OMOP) Common Data Model (CDM) [[30]]; partners in the OHDSI network contribute statistical results based on federated
querying of local data sources represented with the CDM.
The Columbia Open Health Data for COVID-19 Research (COHD-COVID) data set [[31]] takes a different approach, also based on the OMOP CDM. They make publicly available
prevalence data for conditions, drugs, procedures and their co-occurrences, calculated
from their EHR, thereby sharing aggregate counts rather than patient-level data. Coupled
with slight perturbation and suppression of rare concepts, this eliminates privacy
concerns while still enabling comparative data analysis.
Absent adoption of common data models and data sharing platforms for EHR data, research
across multiple sites required manual extraction of defined clinical data elements
from EHRs submitted to a central study team. This was done in the UK for a pediatric
study related to COVID-19 [[32]] and for a study of factors associated with coronavirus death in the US [[33]]. In China, targeted national data reporting to a central research team supported
observational studies [[34]
[35]]. Data harmonisation and cleaning in these cases relied on expert review of data
submissions. For example, to support the development of a machine learning-based predictive
model for COVID-19 associated mortality based on EHR data from five New York hospitals,
expert mappings and data harmonisation were performed by a multi-disciplinary team
of clinicians [[36]].
Other approaches to collecting electronic clinical data were also utilised, including
deployment of clinical natural language processing to rapidly identify and characterise
patients relevant to COVID clinical questions [[37]]. Registries were established to collect disease-specific data items utilising case
report forms (CRFs). The most sophisticated of these used electronic CRFs mapped to
common data standards and submitted to a central database, such as the VIRUS-COVID-19
registry[3] [[38]
[39]] based on REDCap [[40]], which coordinated data entered in over twenty sites. Electronic surveys to gather
data directly from patients[4] were introduced to allow for collection of data from non-hospital settings [[41]].
Despite these successful demonstrations of the use of EHR and registry data to study
COVID-19, a number of challenges remain, summarised effectively in Madhavan et al. [[26]]. EHRs are not primarily designed to support coordinated research and public health
response, and their use in this context placed substantial strain on informatics and
data science teams at hospitals. Even basic conversion of spreadsheet-based systems
for research data collection to formal database structures can prove a challenge where
local differences exist in how fields are interpreted or used [[4]]. Infrastructure for individual-level storage and exchange of data for research
purposes is required, as well as commitment to common data models, terminologies,
and data interchange standards [[42]]. Rapid adaptation of standard clinical vocabularies such as ICD, LOINC, and SNOMED-CT
to include relevant new vocabulary is needed [[43]
[44]]. Governance and ethics factors must also be crucially addressed. The National COVID
Cohort Collaborative[5] (N3C) in the US aims to address many of these points, utilising the OMOP CDM to
bring together data from disparate sources, and aiming to facilitate record-level
analysis of COVID-19 patients (and matched controls) in a secure environment [[45]]. Tremendous progress has been made in the rush to address the pandemic, but there
is still much work to be done before truly international-scale data can be efficiently
and effectively brought together.
2.2.2 Data Analytics and Modelling
With the availability of large-scale and complex data about COVID-19 came the need
to analyse and model it. Advanced computational methods including machine learning,
natural language processing and other artificial intelligence (AI) methods can play
key roles [[46]], and indeed significantly contributed to detecting the COVID-19 outbreak, diagnosing
the disease, and predicting outcomes [[47]]. Models are critical to inform decision making [[48]], supporting prediction and simulation of outcomes under varying conditions or patient
characteristics.
Several of the EHR-based studies cited above utilise machine learning over clinical
variables [21]
[36]], while more traditional statistical or epidemiological modelling is typically employed
for observational studies. Imaging analysis models have also been adapted to COVID-19
from models for related diseases such as pneumonia [[49]], facilitated by public sharing of COVID-19 images with the AI community[6]
[7]
[8] [[50]].
The challenges faced in building sufficiently large data sets has meant that the modelling
of COVID-19 has resulted in high risk of bias and poor external validation [[51]
[52]
[53]]. Additionally, the inherent nature of observational EHR-based studies, lacking
controlled cohort selection, may lead to unreliable results due to confounding [[5]] and a risk of case contamination due to ambiguous cohort definitions [[54]].
2.2.3 COVID-19 Information Retrieval and Synthesis
The amount of COVID-19 research output has been remarkable; based on the LitCovid
index of this research at the US National Library of Medicine[9] [[55]], over 75,000 COVID-19-related publications were added to the PubMed literature
repository between January and late November 2020, at a steady pace of approximately
2000 articles per week (see [Figure 3]). A review of the literature focused on clinical presentation and management of
COVID-19 as of June 15, 2020, prioritised for general medicine readers, identified
over 100 relevant articles [[56]]. Over 7000 COVID-19 clinical trials are registered in the World Health Organization's
International Clinical Trials Registry Platform[10], which plays a key role in identifying research gaps [[57]]. Nearly 3000 systematic reviews related to COVID-19 are currently catalogued in
the Living OVerview of Evidence (L·OVE) platform[11] [[58]].
Fig. 3 Weekly publications in 2020 related to COVID-19, as indexed in the LitCovid collection
of PubMed [[55]].
This scientific knowledge is also broadly accessible; over 75% of this research is
available in open access publications, an unprecedented proportion, more than double
the rate for publications generally during 2015-2019 and for other topics in 2019-2020
[[7]].
However, the accumulation of this body of evidence about one disease in such a short
time is also overwhelming. A key challenge in COVID-19 knowledge management lay in
navigating this massive quantity of research evidence to support diagnosis, treatment,
and public policies, as well as molecular information about the virus. The sheer volume
of the research – published in natural language texts that must be read and interpreted
– requires significant effort to translate into knowledge. Studies must be synthesised
and evaluated, and broader conclusions drawn from comparing multiple studies examining
the same question.
Therefore, many systems based on information retrieval or text mining were created
in response to this challenge, including our COVID-SEE Scientific Evidence Explorer
system [[59]]; more are reviewed in [[60]
[61]
[62]]. An important resource in these efforts was the COVID-19 Open Research Dataset
(CORD-19) which compiled a significant collection of literature for both COVID-19
and related coronaviruses into a single, downloadable resource [[63]].
Leveraging such tools, community-based approaches to collect, curate, and model knowledge
rapidly emerged for COVID-19. Groups began working together to review the literature
and build living evidence guidelines that were updated as new information was made
available [[15]]. Utilising systematic review automation technologies, complete reviews could be
undertaken in a matter of weeks [[64]].
However, in the rapidly changing information space of COVID-19, the rush to explore
and share research outcomes also resulted in poor study designs, poor research reporting,
and lack of coordination and redundancy in research activities [[13]]. Coupled with the data biases noted above, this creates new problems – wasted effort,
increased review and quality appraisal work, and uncertainty about key diagnostic,
prognostic, and treatment decisions. The gLHS, effectively implemented, could provide
the coordination and feedback mechanisms needed to address these problems.
2.2.4 Knowledge Dissemination
Knowledge has been recognised as strategically important for managing pandemics [[48]] and it plays a central role in our learning-based model. As knowledge is acquired
through learning, it must be shared in order to have impact. While publications serve
a key role in disseminating knowledge, alone they are insufficient and ambiguous to
guide practice. Social media have been used effectively for knowledge dissemination
during COVID-19 [[65]], but this focuses on transferring knowledge between individuals.
Knowledge management implemented through information technology can improve information
sharing and coordination [[4]]. Several key elements for knowledge management in pandemics have been identified
[[66]]:
-
Shared knowledge spaces utilising consistent vocabulary.
-
Formal representations of knowledge.
-
Enabling reusable knowledge.
-
Empowering human collaboration through knowledge sharing.
All of these elements were adopted to one degree or another during the COVID-19 pandemic,
through the scientific globalism that emerged; in consortia like the N3C [[45]], working groups like those organised through the Research Data Alliance [[42]], and informal data and clinical networks.
Knowledge has been disseminated through numerous mechanisms, including online platforms
such as the Australian National COVID-19 Clinical Taskforce[12], Registry sites [[39]] and through clinical decision support tools such as the Magic Evidence Ecosystem
Foundation (MAGIC) MAGICApp [[17]].
The push for formal representation of knowledge, including computational and executable
models that can be integrated into health information systems to enable application
of knowledge to practice [[67]
[68]], has gained momentum during COVID-19. The COVID-19 Disease Map project [[69]
[70]] captured and made available molecular interaction information for the SARS-CoV-2
virus, based on manual curation, supported by weekly videoconferences. Knowledge graphs
are also being used to support representation and integration of the variety of biomedical
data related to COVID-19 [[71]], including via text mining [[72]]. Through the adoption of standardised ontology identifiers, comparable data in
different resources can be linked together for analysis, and combined in different
ways for different tasks.
Much of this work has been based on automated or semi-automated analysis of literature.
Direct generation of computable evidence from structured clinical trial registries
has also been proposed [[73]
[74]], which would shortcut the need for literature-based synthesis.
3 Discussion
Friedman and colleagues have stated [[75]]:
“A national-scale LHS will have to be understood and designed as such a cyber-social
ecosystem: a large-scale, decentralized, human-intensive, cyber-catalyzed and cyber-supported information processing system. The
system as a whole—not just the digital infrastructure, but also networks of people
and institutions—will have to be understood not just as users of a technological infrastructure,
but also as parts of the information system itself.”
Extending this to a global-scale LHS demands an even broader view of the relevant
ecosystem. The scale is larger, even more decentralised, and crosses a more diverse
set of legal jurisdictions. It is apparent that the people and institutions play a
critical part of the information system, to make possible the required data sharing
– including tackling legal barriers and leading ethical discussions around data sharing
– as well as by supporting effective communication of knowledge.
It has been observed that we entered the pandemic without a functioning LHS [[76]]. The authors ask [[76]]:
“We have the motivation. We have the vision. We have the technology. We have a roadmap.
What are the barriers?”
answering:
“The issue is culture. We need to treat medical data as a public good.”
They further point to the ethics framework of Faden et al. [[77]] that identifies the dual obligations of health professionals to learn and implement,
and patients to participate in the learning system by contributing their data.
Initiatives such as the US N3C are making important strides towards realising a gLHS.
We do appear to have the motivation, the vision, and the technology. A recent review
of the use of digital technologies during COVID-19 highlights how far we have come
in leveraging technology for the pandemic response [[78]]. What is required to achieve an ongoing gLHS is a commitment to the vision, coupled
with rigorous data governance and legal and regulatory frameworks that safeguard patient
privacy while supporting the learning knowledge ecosystem.
4 Conclusions
The model we have proposed is strongly aligned with the Agency for Healthcare Research
and Quality evidence-based Care Transformation Support (ACTS) Knowledge Ecosystem
initiative referred to as the ‘ACTS COVID-19 Evidence to Guidance to Action Collaborative’[13], which aims to continually enhance patient care throughout the pandemic, as the
evidence base evolves. This Collaborative emphasises development of digital infrastructure
to support the Knowledge Ecosystem, a cycle of Action-Data-Evidence-Guidance that
mirrors the LHS cycle. It has further been active in developing groups such as COKA,
the COVID-19 Knowledge Accelerator Initiative[14] [[74]], a response focused on COVID-19 to the call by Dunn and Bourgeois [[73]] to aim for computable knowledge synthesis and representation, through the use of
standards such as EBMonFHIR[15] [[79]] and CPGonFHIR[16], or rule formalisms for computational clinical guideline specification [[80]].
The vision pursued in these initiatives is still under active development, and has
required a vast community of clinicians, researchers, informaticians, developers,
industry and government representatives, and beyond coming together with the common
objective of addressing the technical, policy or legal, and cultural hurdles to enable
more effective management of the COVID-19 pandemic. It has been argued that infrastructure
is currently sorely lacking in most public health organisations to realise this vision
effectively or efficiently [[81]]. There are still many unanswered questions about how to overcome bias and determine
causality through real-world data [[24]
[82]]. As we have shown, many of the learning and knowledge sharing activities in the
context of the pandemic have been limited to very human-intensive approaches.
However, the core gLHS framework is in place, technology has been harnessed in many
ways to share data and knowledge at a pace that arguably outstripped the spread of
the virus, the requirements for information technology systems to support data and
knowledge exchange are increasingly being clarified, and the initial steps toward
achieving the vision have been made. This is entirely thanks to a tremendous response
by the scientific community with a shared objective of improving outcomes for patients.
Successful examples of large-scale, truly international data sharing and research
collaborations now exist [[28]]. Both the need for and the value of continued work towards a healthcare system
enabled through data and information technologies – a system that can be achieved
through the gLHS – are now obvious.
Continued efforts towards achieving a robust gLHS are important, not only to allow
us to respond to this pandemic and to prepare us to respond to the next pandemic,
but to support continuous improvements in how we care for human health. We now know
that we can do this.
