Keywords
electronic health records - computerized medical record - electronic medical records
- meaningful use - primary care - scoping review
Background and Significance
Background and Significance
Electronic health record (EHR) systems have the potential to significantly impact
the practice of primary care by improving quality of care, reducing costs, and improving
health outcomes; certain countries have provided funding/incentives to encourage their
widespread adoption and use.[1]
[2] The most notable of such national programs is the United States' provision of “Meaningful
Use” incentives, supported by the HITECH (Health Information Technology for Economic
and Clinical Health) Act of 2009.[3] Canada has also created national initiatives to promote adoption.[4] Health information technology adoption rates among primary care physicians in countries
such as the United Kingdom, Australia, and New Zealand are high (at, or above 95%).[5] As of 2015, 73% of Canadian family physicians had adopted an EHR.[6] However, adoption rates vary greatly between provinces,[7] and rates of multifunctional EHR use are much lower.[8]
There are many reported benefits of EHRs over traditional article records, namely,
increased workflow efficiency, error reduction, improved patient safety, and better
communication between providers.[9]
[10] While some studies report generally positive improvements in care with EHR use,[11]
[12] others are mixed,[13]
[14] and some show negative effects on quality of care.[15]
[16] These mixed results may be due to inconsistent use of this technology in clinical
practice.[17]
[18]
[19] There are indications that when more mature, or advanced EHR use exists,[18] including, for example, better integration with workflow,[20] positive effects may be realized. Therefore, it is important to be able to assess
EHR use effectively, so that the link between use and outcomes can be further explored.
Current approaches to assessing the degree to which clinicians use their EHRs in day-to-day
practice focus primarily on the use/nonuse of component parts of the EHR—i.e., functions
of the EHR such as ordering laboratory tests. Therefore, unidimensional (for example,
asking if the EHR is used yes or no), and binary measures of EHR use (for example,
those that simply ask if an EHR function is used – yes or no) may be inadequate to
evaluate actual use; positive outcomes associated with EHRs may be better gauged when
multiple dimensions of EHR use are taken into account.
It is thus important to develop a comprehensive, multidimensional set of measures
for evaluating EHR use. While Schoen et al define multifunctional EHR use as “Uses electronic medical records and at least two electronic functions
in each of the following four domains: generating patient information, generating
panel information, order entry management, and routine clinical decision support,”
we propose that assessments of use can take into account more than just the use of
multiple EHR functions.[5] Therefore, multidimensional assessments of electronic medical record (EMR) use could
go beyond these traditional measures to include, taking into account the way that
the EHR is used during patient encounters, exploring the way that different primary
care practitioners use the EHR, understanding how the EHR is being used by a team
of practitioners versus an individual, and evaluating degrees of function use. As
a first step, we need to understand how primary care EHR use has been measured.
In Canada, while EHR use measures exist,[21]
[22] there is currently no nationally agreed upon framework for EHR use in the primary
care setting. There are many provincially based incentive programs for EHR adoption,
which employ differing frameworks for measuring use.[2] While approaches to measuring EHR use exist for other settings,[23]
[24] there is no universally accepted, standard way EHR use is assessed across all health
care settings. There are some unique aspects of primary care, for example, that it
is defined by first contact care provided on a longitudinal basis to a population
of patients attached to a particular practice, where the practitioner is often the
coordinator of care across settings, that make primary care EHRs and the data within
them, different from medical records used in other settings. There is a body of literature
which we seek to explore in this article, which focuses on measuring EHR use in the
primary care setting. To date, there are no sources which comprehensively describe
the sets of primary care EHR use measures reported in the literature. We focused on
primary care EHRs with a view to define the review according to our interests in the
type of EHR used in primary care practice, as well as the importance and scope of
the setting in terms of the health care system.[25]
[26] Electronic records systems in health care are named inconsistently in the literature
as EMRs[2] and EHRs.[3]
[11] For the purposes of this study, we use the term EHR, defined as computer-based systems
that record and store a patient's health and medical information such as medical history,
encounter notes, prescriptions, and laboratory reports. EHR systems also allow electronic
sharing of such records with authorized health care providers.[27]
Objective
This article conducts a scoping review of the literature to identify and classify
measures of primary care EHR use with a focus on the Canadian context. In this context,
measures are tools or instruments by which clinician use of EHRs in practice can be
assessed.
Methods
This scoping review was performed following the methodological framework of Arksey
and O'Malley[28] and Levac et al.[29] Scoping reviews “map rapidly key concepts underpinning a research area and the main
sources and types of evidence available […] especially where an area is complex or
has not been reviewed comprehensively before.”[28] The review followed six stages: identifying the research question; identifying relevant
studies; study selection; charting data; collating, summarizing, and reporting results;
and stakeholder consultations.
Identifying Relevant Studies
A preliminary search was performed on the following literature databases: PubMed,
Cochrane, CINAHL, Web of Science, and Proquest in December 2015. No date restrictions
were defined. Search queries included keywords relating to EHRs, primary care, and
measuring use. If available (PubMed, Cochrane, and CINAHL), indexed subject headings
were also used. A supplementary search was conducted in PubMed using the National
Library of Medicine's custom query for EHRs.[30] A research librarian from The University of Western Ontario was consulted as we
developed our search strategy. Once preliminary results were obtained, abstracts of
15 randomly selected articles were evaluated to ensure that the topic of interest
was effectively captured. Search queries were refined based on this evaluation (see
Appendix A).
Study Selection
A total of 514 results were obtained from the combined searches (see [Fig. 1]). After the removal of 106 duplicates, 408 articles remained. Five additional texts
identified based on prior knowledge of one author (A.L.T.) were added for a total
of 413 unique articles, which were independently reviewed by A.L.T. and M.Z.H. Abstracts
were included if: (1) the article included the primary care setting, (2) there was
a focus on health information technology, and (3) the study focus/objective was to
measure EHR use (please see Appendix A for keywords and subject headings we used in our search that correspond to each of
these concepts). The latter criterion reflects our interest in measures of actual
EHR use versus the simple availability of functions. Articles were excluded if the
abstract could not be obtained. Following independent review of the abstracts, we
resolved inclusion/exclusion disagreements in person by consensus. Of 413 articles,
84 had abstracts that met the review criteria.
Fig. 1 Study selection.
Full texts of the 84 articles were obtained, and evaluated independently by A.L.T.
and M.Z.H. Full texts were included if they: (1) explicitly included primary care
practices, and (2) focused on applying a multidimensional set of measures for EHR
use. Conflicts were reconciled in-person. As we were interested in measures that assessed
overall EHR use as a more realistic reflection of day-to-day use, we excluded studies
assessing only one EHR function. Of the 84 articles, 56 did not meet the review criteria.
One article was excluded as the full text could not be accessed.
As a supplement, we also selected four articles identified through a gray literature
search. The Web sites of governmental and professional organizations relating to eHealth/health
information technology in Canada, the United States, the United Kingdom, Australia,
and New Zealand were manually searched (see [Appendix B]). The countries were chosen because they were English-speaking and had high levels
of EHR adoption.[5] Types of literature excluded were: newsletters, press releases, commentaries, newspaper
and magazine articles, and annual reports. The inclusion criteria for the gray literature
were the same as that for full text articles.
Reference lists of included texts were manually screened for additional relevant citations.
Six additional articles were identified. In total, 37 articles were included for review
([Fig. 1]).
Data Extraction
A preliminary data charting form was developed and tested independently by A.L.T.
and M.Z.H. on five randomly selected articles. Feedback from testing was used to refine
the form (see [Appendix C]), after which it was applied by M.Z.H. on the remaining articles. The following
information was recorded, if available: author affiliation, country of origin, type
of study, rationale of study, setting, unit of analysis, basis for measure development,
and whether the measure applied had an overall classification of levels of use. If
the article measured use of specific EHR functions, all functions evaluated and degrees
of use were recorded and organized into categories (see [Table 1]).
Appendix C
Charting Strategy
|
General characteristics
|
|
• Citation
|
|
• Author affiliation
|
|
• Country of origin
|
|
• Type of study
|
|
• Setting
|
|
• Unit of analysis
|
|
• Rationale of study
|
|
Measure characteristics
|
|
• Data collection method
|
|
• Nature of the measure
|
|
• Rational for developing measures
|
|
• Basis for measure development
|
|
• Functionalities measured (If applicable)
|
|
o Degree of use for each functionality measured
|
|
• Degrees of overall use measured
|
Table 1
Functions measured in review studies
|
Reference
|
Administrative
|
Health information
|
Order generation
|
Information exchange
|
Decision support and care management
|
Patient support
|
Public health
|
|
Belz (2015)[31]
|
X
|
X
|
|
X
|
X
|
X
|
|
|
Bowes (2010)[32]
|
X
|
X
|
|
|
X
|
|
|
|
COACH (2013)[62]
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
Djalali et al (2015)[64]
|
X
|
X
|
X
|
X
|
|
|
|
|
Dombrosky (2014)[63]
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
Felt-Lisk et al (2010)[56]
|
|
X
|
X
|
X
|
X
|
|
|
|
Goetz Goldberg et al (2012)[33]
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
Gordon et al (2015)[34]
|
X
|
X
|
X
|
X
|
X
|
X
|
|
|
Hogan and Kissam (2010)[35]
|
|
X
|
X
|
X
|
X
|
X
|
X
|
|
Hsiao et al (2013)[36]
|
|
X
|
X
|
X
|
X
|
X
|
|
|
Lanham et al (2014)[37]
|
X
|
|
|
X
|
X
|
X
|
|
|
Lanham et al (2012)[55]
|
X
|
|
|
X
|
X
|
X
|
|
|
Li (2011)[38]
|
|
X
|
X
|
X
|
X
|
X
|
X
|
|
Makam et al (2013)[39]
|
|
X
|
X
|
|
X
|
X
|
|
|
McClellan et al (2013)[40]
|
|
X
|
|
|
X
|
|
|
|
Meehan et al (2014)[41]
|
X
|
|
|
|
X
|
X
|
|
|
Miller et al (2004)[52]
|
X
|
X
|
X
|
X
|
X
|
X
|
|
|
OntarioMD (2015)[21]
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
Ornstein et al (2015)[42]
|
|
X
|
|
|
X
|
X
|
X
|
|
Paré et al (2015)[57]
|
X
|
X
|
X
|
X
|
X
|
|
X
|
|
Pfoh et al (2012)[43]
|
X
|
X
|
X
|
X
|
X
|
|
X
|
|
Poon et al (2010)[44]
|
|
X
|
X
|
X
|
X
|
|
|
|
Price et al (2011)[22]
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
Price et al (2013)[60]
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
Rao et al (2011)[50]
|
|
X
|
X
|
X
|
X
|
|
|
|
Raymond et al (2015)[58]
|
X
|
X
|
X
|
X
|
X
|
|
X
|
|
Riddell et al (2014)[45]
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
Rimmer et al (2014)[61]
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
Shin and Sharac (2013)[46]
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
Simon et al (2008)[51]
|
|
X
|
X
|
X
|
X
|
|
X
|
|
Singh et al (2012)[54]
|
|
X
|
X
|
X
|
X
|
|
|
|
Tagg (2015)[59]
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
Wilcox et al (2008)[53]
|
X
|
X
|
X
|
X
|
X
|
X
|
|
|
Wright et al (2014)[47]
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
Xiao et al (2012)[48]
|
X
|
X
|
X
|
X
|
X
|
X
|
|
|
Zhou et al (2009)[49]
|
|
X
|
X
|
X
|
X
|
|
|
|
Total = 36
|
24
(67%)
|
33
(92%)
|
29
(81%)
|
31
(86%)
|
35
(97%)
|
24
(67%)
|
18
(50%)
|
Note: Not included is Gillies,[65] in which the stages of use described were broad and specific functions were not
listed.
Functional categories were defined as follows:
Administration: Functions used outside clinical practice which do not directly pertain
to health information (e.g., billing, scheduling).
Health Information: Viewing and recording patient information (e.g., medical history,
visit notes, laboratory results).
Order Generation: Generating prescription, laboratory, imaging, and consultation orders.
Information Exchange: Electronically sending/receiving data with other health care
organizations.
Decision Support and Care Management: Functions which enhance or process information
to facilitate care and care management (e.g., drug or laboratory-related alerts, care
reminders, viewing clinical guidelines).
Patient Support: Use of patient portals, patient education materials.
Public Health: Submitting data to patient registries, public health reporting.
Expert Consultation
In June and July 2016, we conducted four semistructured interviews with seven stakeholders
(two of the interviews were conducted with a group of participants) who had EHR expertise.
Participants included one clinician researcher, one provincial-level clinician knowledge
user, and five knowledge users, all from Canada. In keeping with scoping review methodology,[29] prior to the interviews we circulated an executive summary of the review to participants.
In the interviews, we asked participants questions focused on their experience with
EHR use, their views on the findings of the scoping review, and their ideas regarding
important considerations for measuring EHR use in Canada. The interviews were conducted
by A.L.T. and M.Z.H. (see [Appendix D]). Each interview was recorded and notes were taken; A.L.T. and M.Z.H. met after
each interview to debrief and discuss the results. Ethics approval was received from
the Health Sciences Research Ethics Board at The University of Western Ontario.
Appendix D
Interview Guide
|
1. Characteristics of the participant
|
|
2. Could you share your views, in general, about measuring EHR use?
|
|
3. Could you share your thoughts about the preliminary findings of the scoping review?
|
|
4. What do you think is important to measure in terms of EHR use?
|
|
5. What do you think are important considerations for applying measures of EHR use
in a Canadian context?
|
|
6. In what direction should further action be taken in regard to measuring EHR use
in Canada?
|
Results
Study Characteristics
Of the 37 articles included, 26 reported on studies that were conducted in the United
States.[31]
[32]
[33]
[34]
[35]
[36]
[37]
[38]
[39]
[40]
[41]
[42]
[43]
[44]
[45]
[46]
[47]
[48]
[49]
[50]
[51]
[52]
[53]
[54]
[55]
[56] Nine articles were Canadian,[21]
[22]
[57]
[58]
[59]
[60]
[61]
[62]
[63] one was from Switzerland,[64] and one was from the United Kingdom.[65] The dates of publication ranged from 2000 to 2015. An increasing trend in articles
published per year was noted ([Fig. 2]). Most studies were cross-sectional (n = 25). Six longitudinal studies evaluated changes in EHR use over a period of time.[22]
[34]
[36]
[41]
[56]
[61] One text was a discussion article presenting a model for measuring information maturity,[65] and one article featured a comparison of two national surveys.[38]
Fig. 2 Publication year of studies included in the review.
Setting
Thirteen studies were specific to primary care. Ten studies did not specify a setting,
but included primary care physicians or practices in their analyses. Some specified
a particular category of health organization, such as patient-centered medical homes
(n = 2),[31]
[46] small- to medium-sized practices (n = 4),[33]
[40]
[45]
[56] and rural primary care practices (n = 2).[22]
[54] The perspectives of the studies reviewed varied in whether they measured EHR use
at the level of the practice/clinic (n = 16) or at the level of the individual physician (n = 15). One study was conducted at the level of physician–nurse teams (n = 1).[55]
Data Collection Method
Five studies used a mixed-methods approach to collect EHR use data.[33]
[37]
[46]
[52]
[55] Quantitative data collection methods were most frequently used, and included surveys
(n = 21) administered by mail, phone, or the Internet. Five studies used electronic
data extraction or reporting.[45]
[46]
[47]
[53]
[63] Electronic reporting includes the analysis of Meaningful Use attestation reports,
which were required as of 2011 in the United States. Qualitative data collection methods,
including in-person interviews (n = 8),[22]
[33]
[37]
[52]
[55]
[56]
[60]
[61] and onsite observation (n = 5),[33]
[37]
[41]
[52]
[55] were used less often.
Measuring EHR Use
Measuring use through assessing the utilization of individual EHR functions was the
most prevalent metric encountered; however, there were a subset of studies which also
included other factors involved in EHR use.
Use of EHR Functions
With the exception of Gillies, all studies measured use of specific EHR functions
(n = 36). The functions fell into the seven broad functional categories of administration
(n = 24), health information (n = 33), order generation (n = 29), information exchange (n = 31), decision support and care management (n = 35), patient support (n = 24), and public health (n = 18) (see [Table 1]). The specific functions that were encountered most frequently in each category
are shown in [Table 2].
Table 2
Top two measured functions within each category
|
Function
|
Number of articles
|
|
Administration
|
|
|
Performance reporting
|
12
|
|
Privacy and security
|
9
|
|
Health information
|
|
|
Viewing/recording medication lists
|
25
|
|
Viewing/recording laboratory results
|
24
|
|
Order generation
|
|
|
Prescription generation
|
18
|
|
Laboratory order generation
|
14
|
|
Information exchange
|
|
|
Electronic transmission of health information to other health providers
|
15
|
|
Electronic transmission of prescriptions to pharmacies
|
14
|
|
Decision support
|
|
|
Use of drug-related alerts
|
14
|
|
Use of care reminders
|
13
|
|
Patient support
|
|
|
Generation of patient education materials
|
11
|
|
Use of patient portals
|
9
|
|
Public health
|
|
|
Use of patient registries
|
8
|
|
Public health reporting
|
7
|
Metrics for degree of use of specific functionalities were most often binary, distinguishing
between use and nonuse of functions (n = 15). Likert scales (e.g., “do not use,” “use sometimes,” “use all of the time”)
to determine use frequency were also used (n = 7),[34]
[42]
[44]
[48]
[49]
[50]
[54] and ranged from 3 to 5 points.
The basis for the creation of the list of EHR functions for evaluation was often not
stated. Only a few studies stated that a list of EHR functions was generated via a
review of literature (n = 3).[57]
[58]
[64] Two articles used a set of functionalities that were determined by an expert panel.[35]
[54] The list of core EHR functions described by the Institute of Medicine was referenced
by four studies as a basis for developing the lists used.[22]
[49]
[59]
[60]
Other Measures of EHR Use
Other measures of EHR use reported included Makam et al,[39] where methods used by primary care physicians to record encounter notes within the
EHR (dictation, free text, prepopulated dot phrases, electronic templates) were analyzed.
This study also recorded the amount of time that physicians spent on clinical documentation
in the EHR outside of their scheduled clinic time. The measurement framework used
in Lanham et al[55] and Lanham et al[37] took into account the frequency that EHR use behaviors changed when new features
were introduced.
Frameworks of Overall Measures of EHR Use
[Fig. 3] shows our characterization of measures of EHR use. Almost half of the articles (n = 18) measured the use of each EHR function individually. In contrast, we identified
19 studies that categorized use of various EHR functions into maturity stages that
characterized overall levels of use. Within the 19 studies, 8 distinct frameworks
were used. Many were models of general EHR maturity within a practice, and included
measures of use. In general, functions directly related to the clinical environment
(note-taking, order generation) were at the lowest levels of use, followed by more
complex local functions (administration, decision support, patient support), and finally,
health-system integration functions (information exchange, public health) at the highest
level. The measures of use were categorized based on their origins from either professional/government
organizations, or individual authors (see [Table 3]).
Table 3
Summary of frameworks for overall EHR use
|
Developed by
|
Name
|
Articles used
|
Levels of use categorized
|
Functionality covered
|
|
Centers for Medicare and Medicaid Services—United States
|
Meaningful Use (Stage 1)
|
Hsiao et al (2013),[36]
Meehan et al (2014),[41]
Riddell et al (2014),[45]
Shin and Sharac (2013),[46]
Wright et al (2014),[47]
Xiao et al (2012)[48]
|
Meaningful Use status achieved if eligible provider meets all 15 core objectives and
any 5 menu objectives
|
Core measures
- Uses CPOE for medication orders
- Implements drug to drug and drug allergy interaction checks
- Generates and transmits permissible prescriptions electronically (eRx)
- Records patient demographics
- Maintains an up-to-date problem list of current and active diagnoses
- Maintains active medication list
- Maintains active medication allergy list
- Records and charts changes in vital signs
- Records smoking status patients age 13+ 10.
- Implements one clinical decision support rule
- Reports ambulatory clinical quality measures
- Provides patients with an electronic copy of their health information
- Provides clinical summaries for patients for each office visit
- Exchanges key clinical information among providers of care
- Protects electronic health information
Menu measures
- Submits electronic data to immunization registries
- Submits syndromic surveillance data to public health agencies
- Implements drug formulary checks
- Incorporates clinical laboratory test results as structured data
- Generates lists of patients by specific conditions for QI, outreach
- Sends reminders to patients for preventive/follow-up care
- Provides patients with timely electronic access to their health information
- Identifies and provides patient-specific education resources if appropriate
- Performs medication reconciliation at relevant transfers of care
- Provides summary of care record for each transition of care or referral
|
|
COACH—Canada
|
Canadian EMR Adoption and Maturity Model
|
COACH (2013),[62]
Dombrosky (2014),[63]
OntarioMD (2015)[21]
|
Level 0: Paper-Based
Level 1: Basic Electronic Record Keeping
Level 2: Clinical Processes
Level 3: Advanced Disease Management
Level 4: Integrated Care
Level 5: Population Impact
|
Practice management
- Appointment scheduling
- Practice billing
- Communication and coordination
- Business continuity and planning
Information management
- Registration information
- Encounter documentation
- Data quality management
- Nomenclature consistency
- Document management (scanning)
- Privacy and security
Patient results management
- Laboratory results
- Diagnostic image reports
- Hospital summary information
- Referrals and consults tracking (physician)
- Referrals and consults tracking (specialist)
Diagnosis support
- Patient assessment tools
- Preventative/follow-up care
- Evidence-based resources
Treatment planning support
- Care planning and coordination
- Medication management
- Complex care/chronic disease management
Patient engagement and communication
- Patient education
- Self-care/comanagement
Evaluation and monitoring
- Health quality indicator
- Public health reporting
Health outcomes (jurisdictional)
|
|
eHealth Observatory—Canada
|
EMR Adoption Model
|
Price et al (2011),[22]
Price et al (2013),[60]
Tagg (2015)[59]
|
Stage 0: Traditional paper-based practice
Stage 1: Electronic reference material
Stage 2: Partial use of computers at point of care for recording patient information
Stage 3: Computer has replaced patient chart
Stage 4: Advanced decision support
Stage 5: Fully interconnected EMR
|
- Health information
- Laboratory management
- Diagnostic imaging
- Prescription management
- Referrals
- Decisions support
- Electronic communication
- Patient support
- Administrative processes
- Practice reporting
|
|
HIMSS Analytics
|
Ambulatory EMR Adoption Model
|
Belz (2015)[31]
|
Stages 0–7
|
Stage 1:
- Desktop access to clinical information
- Unstructured data
- Multiple data sources
- Infraoffice/informal messaging
Stage 2:
- Desktop access to clinical information
- Orders and results
- Multiple data sources
- Infraoffice/informal messaging
Stage 3:
- Electronic messaging
- Clinical documentation
- Clinical decision support
Stage 4:
- CPOE
- Use of structured data
- Internal and external data sharing
Stage5:
- Personal health record connection
- Patient portal
Stage 6
- Clinical decision support
- Advanced decision support
- Proactive care management
- Structured messaging
Stage 7:
- Health information exchange
- EHR connectivity
- Business and clinical intelligence
|
|
Physician Information Technology Office—British Columbia
|
Clinical Value Model
|
Rimmer et al (2014)[61]
|
Level 0: Front Office Administration
Level 2: EMR Basics
Level 3: Full EMR (considered baseline)
Level 4: Proactive Care/Data-driven Practice
Level 5: Community Shared Care
|
- Billing
- Scheduling
- Receive labs reports
- Record-keeping capabilities
- Structured data entry
- Registries
- Recalls
- Reminders
- Templates/flowsheets
- Guideline-informed care
Data transfer between GP and specialists and other care providers
|
|
Lanham et al (2012)
|
EHR Use Categories
|
Lanham et al (2012),[55]
Lanham et al (2014)[37]
|
1. Low
2. Medium
3. High
|
- Reports, flow sheets, tracking, and trending features
- EHR-generated patient literature
- Macros/quick text feature
- EHR-enabled communication within practice
- EHR-enabled communication outside practice
- EHR-enabled communication with pharmacies
- Frequency to change EHR use with introduction of new features
|
|
Miller et al (2004)
|
Physician EMR User Types
|
Miller et al (2004),[52]
Wilcox et al (2008)[53]
|
1. Viewer
2. Basic User
3. Striver
4. Arriver
5. System Changer
|
- Viewing (chart notes, problem and medication lists, discharge summaries, laboratory
results
- Order entry (prescriptions, laboratory tests, radiology studies, and referrals)
- Documentation (visit notes, other information)
- Care planning and management
- Patient-directed (patient education materials, patient viewing of EMR data, secure
email with care providers)
- Billing and other administrative (insurance eligibility, visit level coding, referral
management, and tracking)
- Messaging (email or other messaging system within and outside practice)
- Performance reporting (to internal and external audiences)
|
|
Gillies (2000)
|
General Practice Information Maturity Model
|
Gillies (2000)[65]
|
Level 0: Paper-based
Level 1: Computer system used by practice staff
Level 2: Computer system used by practice staff and PHC team
Level 3: Limited use of read codes
Level 4: Use of coding policies and clinical protocols
Level 5: Completely paperless
|
- NA
|
Abbreviations: CPOE, computerized physician order entry; EHR, electronic health record;
EMR, electronic medical record; GP, general practitioner; PHC, primary health care.
Fig. 3 Typology of measures of electronic medical record (EMR) use.
Frameworks by Professional/Government Organizations
Centers for Medicare and Medicaid Services—United States
The Meaningful Use program was created through the 2009 HITECH act in the United States,
and is implemented by the Centers for Medicare and Medicaid Services (CMS). The program
consists of progressive stages of EHR use, each with requirements that change as EHR
adoption matures. Six articles measured EHR use by the Meaningful Use Stage 1 criteria.[36]
[41]
[45]
[46]
[47]
[48] No article measured use by the Meaningful Use Stage 2 criteria (announced in 2012).
COACH—Canada
A white article by COACH (Canada's Health Informatics Association), described the
Canadian EMR Adoption and Maturity Model (CEMRAMM), that was developed via a compilation
of EHR adoption models used in four Canadian jurisdictions (Alberta, British Columbia,
Manitoba, and Ontario), and proposed as a common model.[62] Six levels exist within this model, from 0 (article-based) to 5 (population impact).
The CEMRAMM is also used in the Saskatchewan EMR Program,[63] and is nearly identical to the EMR Maturity Model used by the Ontario organization,
OntarioMD.[21]
eHealth Observatory—University of Victoria, Canada
Price et al[22] and Price et al[60] used the EMR Adoption Model developed at the eHealth Observatory at the University
of Victoria, Canada. The framework defines six stages of EHR use which were made to
be consistent with that of the Health Information Management Systems Society (HIMSS)
EMR Adoption Model (EMRAM). Each stage is defined by the use of specific EHR functional
categories, which were adapted from the Institute of Medicine's eight key capabilities
of EHR systems. The eHealth Observatory EMRAM was also used by Tagg[59] in a survey of EHR use by family practices in Alberta.
Physician Information Technology Office (PITO)—British Columbia, Canada
The Clinical Value Model is a framework developed by the PITO program in British Columbia,
which had the goal of supporting province-wide adoption of EHRs. This model consists
of five stages, and was used in Rimmer et al.[61]
HIMSS Analytics
The Ambulatory EMRAM (A-EMRAM) developed by HIMSS Analytics consists of eight stages
of EHR Adoption for practices, from stage 0 (article chart based) to stage 7 (fully
integrated and health information exchange-capable EHR). An evaluation of the stages
of EHR adoption of patient-centered medical homes in the United States was conducted
by Belz using this model.
Frameworks by Individual Authors
Lanham et al
Lanham et al[55] described three (low, medium, and high) categories of EHR use, depending on the
degree of EHR feature use, as well as the level of EHR-enabled communication with
others both within and outside the clinic. Also measured was the frequency at which
physicians changed their usage habits when new features were introduced in the EHR.
This model was subsequently used in Lanham et al.[37]
Miller et al
In a study of physicians in solo and small practices, Miller et al[52] categorized five progressive EHR user types based on variety and degree of feature
use: Viewer, Basic User, Striver, Arriver, and System Changer. Wilcox et al[53] adopted this model to categorize the EHR usage patterns of a larger sample of 3,348
physicians in the Intermountain Healthcare organization.
General Practice Information Maturity Model—United Kingdom
The General Practice Information Maturity Model (GPIMM) is described by Gillies.[65] This model outlines six levels (0–5) of information maturity in a general practice,
from article-based to articleless. It takes into account the presence of EHR coding
features, as well as which staff members in the practice use them.
Feedback from Stakeholder Consultations
There were two main areas of feedback from participants regarding the scoping review.
First, participants in the stakeholder consultation phase of the scoping review identified
resources which added to the currency of this review's content. The majority of the
new resources that were identified were incorporated into the background section of
this article. Second, participants offered new insights and perspectives regarding
measuring EHR use, particularly in Canada.
In terms of the scoping review findings, participants noted the small number of studies
overall, reflecting the early stage of this body of literature. The dominance of studies
from the United States was discussed, as was the need for caution regarding the applicability
of these studies in Canada's much different health care context. Participants echoed
many of the points raised in the review, including: the importance of making a distinction
between simple adoption, and full multifunctional EHR use; the fact that there are
disparate existing EHR assessment tools, and that there is a need for a consensus
across Canada on how to measure EHR use in a standard way; as well as the importance
of measuring capabilities of EHR systems. There were strong views expressed about
the need for ongoing nuanced measurement of EHR use (for example, determining what
kind of EHR use would be needed for different patient groups, creating measures sensitive
to different primary care contexts), and that these measurements should be linked
to initiatives to advance EHR use among practitioners. Participants suggested that
multiple initiatives could improve EHR use—including incentives in the primary care
context, as well as quality improvement activities. The need to identify solutions
to improving EHR use, potentially through research, was voiced.
Sharing the summary of the scoping review with stakeholders afforded the authors the
opportunity to conduct the first steps of knowledge translation. In turn, the participants
shared new resources with the authors and discussed important areas for future research.
Taken together, the consultation process served to enhance the scoping review itself,
while also affording participants and authors the opportunity for knowledge transfer
and exchange.
Discussion
In the past, primary care EHR adoption in Canada lagged behind other countries such
as the United Kingdom and the Netherlands. More recently, however, adoption levels
among primary care practitioners have increased significantly, though there is variation
among provinces.[7] A variety of support programs among the provinces encouraged adoption.[7] However, overall there is limited use of advanced features in the EHR among Canadian
primary care practitioners.[6] Understanding advanced EHR use is a complex task, and currently there is a lack
of information in the Canadian context in this regard.[57] Given the rising use of EHRs, and the link between advanced use and potential benefits
to patient care,[18]
[20] it is important that we are able to measure EHR use effectively. Given the nature
of the Canadian primary care system (e.g., that it is: recognized as the foundation
of the system; the first point of contact; where care is coordinated; and where the
majority of care is provided),[25]
[66]
[67] it is important that measures of EHR use are applicable in this context. Thus, we
set out to explore existing EHR use measures in the literature within this review.
This review found a relatively small body of literature on measuring EHR use in primary
care. The volume of this literature is developing, and the rate of publication of
articles on this topic is increasing as the number of EHR users grows. The current
literature focuses on assessing the use of specific functions within the EHR system.
Reviewed articles were relatively evenly divided between the practice and the physician-level
perspective. Incentives programs such as Meaningful Use, as well as provincial programs
in Canada, give incentives to eligible physicians, and develop measures intended for
physician-level use. However, most of the eight frameworks identified in this review
can be applied to both perspectives. For example, of the six articles which used the
Meaningful Use framework, four were based on a practice-level perspective, and two
were physician-level.
Within the articles in this review, surveys were the most widely used method for collecting
data on EHR use. For quantitative studies, surveys are appropriate as they can facilitate
larger sample sizes. However, survey studies can be subject to bias because they are
based on self-report. Further inaccuracies can occur when a single physician is chosen
to report on usage data for the entire practice (for example, Singh et al).[51] Direct observation and semistructured interviews can provide a more detailed evaluation
of use, but are also more time-intensive. Most notably, this review highlights the
potential of electronic reporting to evaluate use. Two articles used EHR audit data
to determine levels of use. Three studies relied on Meaningful Use attestation reports,
which are also reported electronically through the EHR system. Electronic reporting
presents an objective and efficient method to evaluate use; however, the capability
to do so must be present in the EHR software. Ongoing certification of EHR systems
and vendors can ensure the consistency of functions available.[68]
All measures encountered were focused on the use of different EHR functions. The types
of functions measured varied between frameworks, with functions in public health,
patient support, and administration being the least prevalent. This may be because
these functions were perceived to have lesser value to clinical practice, and represent
an extension of basic EHR functionality. The literature also shows that other measures
of use exist, such as the change in use patterns in response to the addition of new
EHR features. These measures should be taken into consideration when evaluating EHR
use in the future. A recent development by HIMSS Analytics is the “Continuity of Care
Maturity Model,”[69] which outlines the stages of interoperability, information exchange, and care coordination
beyond the stages of the HIMSS A-EMRAM. The “Continuity of Care Maturity Model” gauges
the next steps toward an interconnected network of care, after the widespread adoption
and advanced use of EHRs is achieved.
This review also highlighted the importance of characterizing the overall level of
EHR use of a practice or physician. Frameworks such as the HIMSS EMRAM and COACH's
EMR Adoption and Maturity Model take a more user-oriented approach which measures
overall use by successive levels. The implementation of adoption levels within this
type of framework are an improvement over simpler frameworks that categorize advanced
EHR use as a binary feature, for example, the CMS Meaningful Use requirements. A stepwise
progression from basic to advanced use gives users a roadmap to follow, which can
help to alleviate major barriers to the use of EHRs, including a lack of computer
literacy in users,[7] and the perceived complexity of the EHR system.[70] Canadian provinces from which frameworks for EHR use were found in the review (BC,
ON, SK) are among the provinces with the highest EHR adoption rates.[7]
Another important consideration when measuring EHR use is the limitation of the EHR
system itself. The degree of function use is limited by what functions exist in the
system. As such, use should be evaluated in the context of EHR capability. Only a
small number of studies in the review, such as Poon et al[44] and Gordon et al,[34] evaluated both the availability and use of functions. Current EHR product certification
programs can ensure that systems adopted by health care providers can support advanced
use. Use can also be limited by the infrastructure of the health care system within
which the practice or physician is located. Jurisdictions that lack a network for
sharing patient information limit the health information exchange capabilities of
EHR users. Of note, the relationship between EHR use, EHR capability, and health care
infrastructure is considered within the eHealth Observatory's EMR Adoption Framework.[22]
Measuring EHR use has important implications in five main areas. The first is that
individual users and teams could use an EHR use measure to assess their own level
of the use of this technology, potentially supporting a stepwise progression in improvements
in use. This could also enable comparisons among groups of users. Second, the motivation
to use an EHR is tied to its value in clinical practice.[71] We know that a link exists between levels of EHR use and the potential benefits
that may be realized in terms of care and patient outcomes.[18]
[20] Better measurement of EHR use could further establish the link between types of
use and the outcomes that may be achieved, thus emphasizing the value of advanced
EHR in practice. Third, understanding EHR use more comprehensively could allow the
development of tailored resources for training or the creation of unique support programs
among different jurisdictions. Fourth, exploring EHR use may lead to improved knowledge
of the factors which impact a practitioner's ability to achieve advanced or improved
use.[57] Finally, assessments of patterns and levels of EHR use among practitioners could
inform the refining or creation of EHR software products that are better aligned with
actual clinical practice.[72]
Future Research
While there is mixed evidence of the impact of EHR use on clinical practice,[73] early research points to the importance of more mature, or advanced EHR use,[18] including integration with workflow,[20] in achieving positive effects. Further research could explore not only the impact
of advanced EHR use on clinical practice and patient outcomes, but also the link between
the use of specific EHR functions and their potential effects. With growing initiatives
to assess the impact and uptake of EHRs, EHR research relating to care outcomes, barriers
to use, and user satisfaction should apply a more nuanced view of EHR use. This more
nuanced picture of use could capture the way a clinician uses the EHR during a patient
encounter, for example, not just simply determining if an individual has used an EHR
function or not. New research should therefore continue to explore aspects of EHR
use by practitioners that go beyond assessing the use of individual software functions.
Possibilities include exploring best practices in EHR use when interacting with patients,
analyzing individual's use patterns, and examining changes in use behavior over time.
Many unanswered questions remain, such as the appropriate use of EHR measures given
different settings and systems, and ultimately, how best to help practitioners advance
their use of EHRs.
Limitations
Additional studies relating to measuring EHR use in Canada have been released following
completion of this review.[74]
[75] There may be studies missed by the manual gray literature search. Finally, the categorization
of functions was created by A.L.T. and M.Z.H.; other categorizations of EHR functions
exist.[22]
Conclusion
With Canada and other select countries reaching high levels of EHR adoption, it is
important to understand how these systems are being used in primary care. This scoping
review explored the body of literature available regarding measuring the use of EHRs
within primary care. We found that this literature is still in its early stages, but
rapidly developing. Although certain frameworks were recurrent, many researchers are
creating de novo measures. Therefore, there is a need to create more common, widely
used, and validated framework for measuring EHR use. Creating a unified, multidimensional
use assessment framework, and applying it consistently across jurisdictions, would
have several benefits including: facilitating comparisons of levels of EHR use across
primary care settings, supporting the identification of factors associated with higher
and lower levels of EHR use, and limiting confusion about what framework to apply,
thus simplifying the process of evaluating use.
This would also address the limitation of current frameworks which are mostly only
applied in a limited capacity, and focus almost exclusively on use of EHR functions.
Existing frameworks also do not consider the context of EHR system capability, which
limits the degree of possible use. Most of the literature is focused on measuring
use of specific EHR functions, and there is more work to be done in exploring other
aspects of EHR use, such as how this technology is used during patient encounters,
team versus individual use of the EHR, and the way different primary care practitioners
use the EHR. We recommend that a nuanced view of EHR adoption be used in future studies,
taking into account the variety of ways this technology is used in practice.
Clinical Relevance Statement
Clinical Relevance Statement
As adoption of EHRs in select countries increases, focus has shifted to measuring
the use of these systems. By compiling and classifying current EHR use metrics, this
review highlights the facets and nuances that must be considered when measuring EHR
use. An understanding of these existing measures is essential to the development of
unified and comprehensive EHR use metrics, which can facilitate the optimal use of
EHRs by health care practitioners and organizations.
Multiple Choice Question
Within the current literature on that measures the use of EHR functions, which category
of functionality is most often neglected?
-
Order Generation
-
Public Health
-
Administration
-
Patient Support
Correct Answer: The correct answer is b. Our review found that Public Health EHR functions were the
most neglected, with only 18 of the 36 articles which measured use of EHR functions
including this category (see [Table 1]).
Appendix A Database Search Queries
Appendix A Database Search Queries
PubMed
(“general practice”[All Fields] OR “general practitioners”[All Fields] OR “general
practitioner”[All Fields] OR “family physicians”[All Fields] OR “family physician”[All
Fields] OR “family practice”[All Fields] OR “primary health care”[All Fields] OR “primary
care”[All Fields] OR “primary healthcare”[All Fields] OR “internal medicine”[All Fields]
OR “internist”[All Fields] OR “paediatrics”[All Fields] OR “pediatrics”[All Fields]
OR “paediatrician”[All Fields] OR “pediatrician”[All Fields] OR Ambulatory[All Fields]
OR “general practice”[MeSH Terms] OR “general practitioners”[MeSH Terms] OR “physicians,
family”[MeSH Terms] OR “family practice”[MeSH Terms] OR “primary health care”[MeSH
Terms] OR “primary health care”[MeSH Terms] OR “internal medicine”[MeSH Terms] OR
“pediatrics”[MeSH Terms])
AND
(“computerised medical records”[All Fields] OR “computerised health records”[All Fields]
OR “computerised patient records”[All Fields] OR “computerized medical records”[All
Fields] OR “computerized health records”[All Fields] OR “computerized patient records”[All
Fields] OR “electronic medical records”[All Fields] OR “electronic health records”[All
Fields] OR “electronic patient records”[All Fields] OR “medical records systems, computerized”[MeSH
Terms] OR “electronic health records”[MeSH Terms])
AND
(“Meaningful Use”[All Fields] OR “Advanced Use”[All Fields] OR “Maturity Model”[All
Fields] OR “Adoption Model”[All Fields] OR “Clinical Value” OR “Clinical Quality”[All
Fields] OR “CQM”[All Fields] OR “CQMs”[All Fields] OR “Multifunctional Electronic”[All
Fields] OR “meaningful use”[MeSH Terms])
(“general practice”[All Fields] OR “general practitioners”[All Fields] OR “general
practitioner”[All Fields] OR “family physicians”[All Fields] OR “family physician”[All
Fields] OR “family practice”[All Fields] OR “primary health care”[All Fields] OR “primary
care”[All Fields] OR “primary healthcare”[All Fields] OR “internal medicine”[All Fields]
OR “internist”[All Fields] OR “paediatrics”[All Fields] OR “pediatrics”[All Fields]
OR “paediatrician”[All Fields] OR “pediatrician”[All Fields] OR Ambulatory[All Fields]
OR “general practice”[MeSH Terms] OR “general practitioners”[MeSH Terms] OR “physicians,
family”[MeSH Terms] OR “family practice”[MeSH Terms] OR “primary health care”[MeSH
Terms] OR “primary health care”[MeSH Terms] OR “internal medicine”[MeSH Terms] OR
“pediatrics”[MeSH Terms])
AND
((health information exchange[tw] OR hie[tw] OR rhio[tw] OR regional health information
organization[tw] OR hl7[tw] OR health level seven[tw] OR “unified medical language
system”[MeSH Major Topic] OR umls[tw] OR loinc[tw] OR rxnorm[tw] OR snomed[tw] OR
icd9 cm[ti] OR icd 9 cm[ti] OR icd10[ti] OR icd 10[ti] OR metathesaurus[tw] OR patient
card[tw] OR patient cards[tw] OR health card[tw] OR health cards[tw] OR electronic
health data[tw] OR personal health data[tw] OR personal health record[tw] OR personal
health records[tw] OR “health records, personal”[MeSH Major Topic] OR “health records,
personal”[MeSH Major Topic] OR ehealth[tw] OR e-health[tw] OR “medical informatics
applications”[MeSH Terms] OR “medical informatics applications”[MeSH Terms] OR “medical
records systems, computerized”[MeSH Terms] OR “medical records systems, computerized”[MeSH
Terms] OR computerized patient medical records[tw] OR automated medical record system[tw]
OR automated medical record systems[tw] OR automated medical records system[tw] OR
automated medical records systems[tw] OR computerized medical record[tw] OR computerized
medical records[tw] OR computerized patient records[tw] OR computerized patient record[tw]
OR computerized patient medical record[tw] OR electronic health record[tw] OR electronic
health records[tw] OR “electronic health records”[MeSH Major Topic] OR “electronic
health records”[MeSH Major Topic] OR electronic patient record[tw] OR electronic patient
records[tw] OR electronic medical record[tw] OR electronic medical records[tw] OR
electronic healthcare records[tw] OR electronic healthcare record[tw] OR electronic
health care record[tw] OR electronic health care records[tw] OR “archives”[MeSH Major
Topic] OR ehr[tw] OR ehrs[tw] OR phr[tw] OR phrs[tw] OR emr[tw] OR emr[tw] OR “health
information systems”[MeSH Major Topic]) AND (medical record[ti] OR “medical records”[MeSH
Terms] OR medical records[ti] OR patient record[ti] OR patient records[ti] OR patient
health record[ti] OR patient health records[ti] OR “patient identification systems”[MeSH
Terms] OR “patient identification systems”[MeSH Terms] OR “patient outcome assessment”[MeSH
Major Topic] OR “patient discharge summaries”[MeSH Major Topic] OR healthcare record[ti]
OR healthcare records[ti] OR health care record[ti] OR health care records[ti] OR
health record[ti] OR health records[ti] OR hospital information system[tw] OR hospital
information systems[tw] OR umae[ti] OR “attitude to computers”[MeSH Terms] OR medical
informatics[ti])) OR ((“medical records systems, computerized”[MeSH Major Topic] OR
“medical records systems, computerized”[MeSH Terms] OR computerized patient medical
record[tw] OR computerized patient medical records[tw] OR automated medical record
system[tw] OR automated medical record systems[tw] OR automated medical records system[tw]
OR automated medical records systems[tw] OR computerized medical record[tw] OR computerized
medical records[tw] OR computerized patient records[tw] OR computerized patient record[tw]
OR electronic health record[tw] OR electronic health records[tw] OR electronic patient
record[tw] OR electronic patient records[tw] OR electronic medical record[tw] OR electronic
medical records[tw] OR electronic healthcare records[tw] OR electronic healthcare
record[tw] OR electronic health care record[tw] OR electronic health care records[tw]
OR “unified medical language system”[MeSH Major Topic] OR unified medical language
system[tw] OR umls[tw] OR loinc[tw] OR rxnorm[tw] OR snomed[tw] OR icd9 cm[ti] OR
icd 9 cm[ti] OR icd10[ti] OR icd 10[ti] OR Metathesaurus[tw] OR ehr[tw] OR ehrs[tw]
OR phr[tw] OR phrs[tw] OR emr[tw] OR emrs[tw] OR meaningful use[tiab] OR meaningful
use[tw] OR “meaningful use”[MeSH Major Topic]) AND (“J AHIMA”[Journal] OR “J Am Med
Inform Assoc”[Journal] OR “AMIA Annu Symp Proc”[Journal] OR “Health Data Manag”[Journal]
OR “Int J Med Inform”[Journal] OR “Yearb Med Inform”[Journal] OR “Telemed J E Health”[Journal]
OR “Stud Health Technol Inform”[Journal]))
AND
(“Meaningful Use”[All Fields] OR “Advanced Use”[All Fields] OR “Maturity Model”[All
Fields] OR “Adoption Model”[All Fields] OR “Clinical Value” OR “Clinical Quality”[All
Fields] OR “CQM”[All Fields] OR “CQMs”[All Fields] OR “Multifunctional Electronic”[All
Fields] OR “meaningful use”[MeSH Terms])
Cochrane
(“general practice”[All Fields] OR “general practitioners”[All Fields] OR “general
practitioner”[All Fields] OR “family physicians”[All Fields] OR “family physician”[All
Fields] OR “family practice”[All Fields] OR “primary health care”[All Fields] OR “primary
care”[All Fields] OR “primary healthcare”[All Fields] OR “internal medicine”[All Fields]
OR “internist”[All Fields] OR “paediatrics”[All Fields] OR “pediatrics”[All Fields]
OR “paediatrician”[All Fields] OR “pediatrician”[All Fields] OR Ambulatory[All Fields]
OR “general practice”[MeSH Terms] OR “general practitioners”[MeSH Terms] OR “physicians,
family”[MeSH Terms] OR “family practice”[MeSH Terms] OR “primary health care”[MeSH
Terms] OR “primary health care”[MeSH Terms] OR “internal medicine”[MeSH Terms] OR
“pediatrics”[MeSH Terms])
AND
(“computerised medical records”[All Fields] OR “computerised health records”[All Fields]
OR “computerised patient records”[All Fields] OR “computerized medical records”[All
Fields] OR “computerized health records”[All Fields] OR “computerized patient records”[All
Fields] OR “electronic medical records”[All Fields] OR “electronic health records”[All
Fields] OR “electronic patient records”[All Fields] OR “medical records systems, computerized”[MeSH
Terms] OR “electronic health records”[MeSH Terms])
AND
(“Meaningful Use”[All Fields] OR “Advanced Use”[All Fields] OR “Maturity Model”[All
Fields] OR “Adoption Model”[All Fields] OR “Clinical Value” OR “Clinical Quality”[All
Fields] OR “CQM”[All Fields] OR “CQMs”[All Fields] OR “Multifunctional Electronic”[All
Fields] OR “meaningful use”[MeSH Terms])
CINAHL
(“General Practice” OR “General Practitioner” OR “Family Physician” OR “Family Practice”
OR “Family Practitioner” OR “Primary Health Care” OR “Primary Care” OR “Primary Healthcare”
OR “Internal Medicine” OR “Internist” OR “paediatrics” OR “pediatrics” OR “paediatrician”
OR “pediatrician” OR “Ambulatory” OR (MH “Physicians, Family”) OR (MH “Primary Health
Care”) OR (MH “Internal Medicine”) OR (MH “Pediatricians”) OR (MH “Pediatrics”))
AND
(“computerised medical records” OR “computerised health records” OR “computerised
patient records” OR “computerized medical records” OR “computerized health records”
OR “computerized patient records” OR “electronic medical records” OR “electronic health
records” OR “electronic patient records” OR (MH “Computerized Patient Record”) OR
(MH “Patient Record Systems”) OR (MH “Clinical Information Systems”))
AND
(“Meaningful Use” OR “Advanced Use” OR “Maturity Model” OR “Adoption Model” OR “Clinical
Value” OR “Clinical Quality” OR “CQM” OR “CQMs” OR “Multifunctional Electronic” OR
(MH “Meaningful Use”))
Web of Science
(“General Practi*” OR “Family Physician*” OR “Family Practi*” OR “Primary Health Care”
OR “Primary Care” OR “Primary Healthcare” OR “Internal Medicine” OR “Internist*” OR
“paediatric*” OR “pediatric*” OR “Ambulatory”)
AND
(“Medical Record* System*” OR “Health Record* System*” OR “Patient Record* System*”
OR “Computeri?ed Patient Record*” OR “Computeri?ed Medical Record*” OR “Computeri?ed
Health Record*” OR “Electronic Patient Record*” OR “Electronic Medical Record*” OR
“Electronic Health Record*”)
AND
(“Meaningful Use” OR “Advanced Use” OR “Maturity Model” OR “Adoption Model” OR “Clinical
Value” OR “Clinical Quality” OR “CQM” OR “CQMs” OR “Multifunctional Electronic”)
Proquest Theses and Dissertations
(“General Practi*” OR “Family Physician*” OR “Family Practi*” OR “Primary Health Care”
OR “Primary Care” OR “Primary Healthcare” OR “Internal Medicine” OR “Internist*” OR
“paediatric*” OR “pediatric*” OR “Ambulatory”)
AND
(“Medical Record* System*” OR “Health Record* System*” OR “Patient Record* System*”
OR “Computeri?ed Patient Record*” OR “Computeri?ed Medical Record*” OR “Computeri?ed
Health Record*” OR “Electronic Patient Record*” OR “Electronic Medical Record*” OR
“Electronic Health Record*”)
AND
(“Meaningful Use” OR “Advanced Use” OR “Maturity Model” OR “Adoption Model” OR “Clinical
Value” OR “Clinical Quality” OR “CQM” OR “CQMs” OR “Multifunctional Electronic”)
