Quantifying Electronic Health Record Data Quality in Telehealth and Office-Based Diabetes Care

Abstract

Objective Data derived from the electronic health record (EHR) are commonly reused for quality improvement, clinical decision-making, and empirical research despite having data quality challenges. Research highlighting EHR data quality concerns has largely been examined and identified during traditional in-person visits. To understand variations in data quality among patients managing type 2 diabetes mellitus (T2DM) with and without a history of telehealth visits, we examined three EHR data quality dimensions: timeliness, completeness, and information density.

Methods We used EHR data (2016–2021) from a local enterprise data warehouse to quantify timeliness, completeness, and information density for diagnostic and laboratory test data. Means and chi-squared significance tests were computed to compare data quality dimensions between patients with and without a history of telehealth use.

Results Mean timeliness or T2DM measurement age for the study sample was 77.8 days (95% confidence interval [CI], 39.6–116.4). Mean completeness for the sample was 0.891 (95% CI, 0.868–0.914). The mean information density score was 0.787 (95% CI, 0.747–0.827). EHR data for patients managing T2DM with a history of telehealth use were timelier (73.3 vs. 79.8 days), and measurements were more uniform across visits (0.795 vs. 0.784) based on information density scores, compared with patients with no history of telehealth use.

Conclusion Overall, EHR data for patients managing T2DM with a history of telehealth visits were generally timelier and measurements were more uniform across visits than for patients with no history of telehealth visits. Chronic disease care relies on comprehensive patient data collected via hybrid care delivery models and includes important domains for continued data quality assessments prior to secondary reuse purposes.

Keywords

electronic health record - data quality - telehealth - chronic disease

Background and Significance

Electronic health records (EHRs) are often a source of secondary data for use in clinical and health services research.[1] [2] Nevertheless, such data may face numerous data quality challenges[3] from manual entry, lags between the patient visits and actual documentation, differential documentation guidelines, and changes in standards.[4] [5] As a result, EHR data are commonly discordant and incomplete.[6] [7] Extant research on EHR data quality is based primarily on traditional in-person, face-to-face visits between patients and providers. However, the extent to which EHR data quality is associated with other types of clinical visits is not well known.[6] [8] [9]

Specifically, the proportion of remote or telehealth visits has steadily increased over the past 20 years[10] [11] [12] and became even more common during the coronavirus disease 2019 (COVID-19) pandemic.[13] [14] While limited, evidence suggests the use of telehealth may have further effects on EHR data quality.[15] [16] For example, patients have expressed concerns about the accuracy and quality of data gathered during the encounter.[15] Likewise, providers and researchers note that telehealth visits may not include sufficient diagnostic data for providers to effectively manage patient symptoms.[16] Additionally, prior attempts to leverage data collected via telehealth for decision support systems were unsuccessful due to data quality issues.[16] Namely, the authors have found that the quality of pulse oximetry and blood pressure data affects the performance of decision support systems. In general, examining whether and to what extent new health care tools have improved or worsened EHR data quality is an understudied phenomenon.[17] Thus, a better understanding of different care modalities' effects on data quality is needed.

Type 2 diabetes mellitus (T2DM) is an appropriate focal condition for such a comparison. First, opportunities for comparisons exist as T2DM self-management, nutrition consultations, and wellness programs are amenable to telehealth.[18] [19] Second, T2DM is associated with a wide array of EHR data elements and types: laboratory measurements, medication regimens, self-management education, and nutrition consultations.[20] Patients routinely interact with multispecialty team-based clinicians during care management and care coordination as the standard of medical care for patients managing T2DM. Data collection across care settings and providers may increase data incompleteness because of a failure to consistently record relevant T2DM measurements.[21]

Objective

This study compared EHR data quality among patients managing T2DM with and without a history of telehealth use. We assessed structured EHR data quality in terms of timeliness, completeness, and patient information density. Care delivery has undergone a transition from primarily in-person visits to hybrid approaches that integrate telehealth and face-to-face encounters. This study examines the extent to which EHR data quality might vary among care delivery formats as the demand for complementary telehealth care continues to grow.

Methods

Study Sample

The study sample included patients with T2DM aged ≥18 years who were seen between 2016 and 2021 at two health systems in central Indiana. The sample was limited to outpatient encounters for patients who received a T2DM diagnosis between their index and most recent encounter date. We defined the index visit as the first unique T2DM outpatient encounter during the study period. The study period includes uniform telehealth adoption and use among both health systems which began in 2020.

Data

The primary dataset used in the current study was derived from each health system's enterprise data warehouse (EDW) and the Indiana Network for Patient Care (INPC). The INPC was established in 1994 as a repository for cross-institutional patient data including EHR data.[22] We obtained data representing patient demographics and clinical diagnoses inputted by treating clinicians regardless of specialty.[23]

Telehealth Status

Telehealth status was defined as any encounter where remote use of audio or video services was provided and indicated by common procedural terminology (CPT) code modifiers. Patients who had more than one initial telehealth visit were determined to have a history of telehealth visits during the study period. Because of the potential selection bias associated with the use of telehealth, we frequency-matched controls on age, sex, and total visit count.

Clinical Visit Type

We identified patients with a history of telehealth use as any outpatient visit linkable to patients by common identifiers and coded as having used audio or video technologies during a visit. Patients with no history of telehealth use were identified where there was no available indication of telehealth use or virtual care technologies present during or as a result of a visit.

Electronic Health Record Data Quality Dimensions

We quantified dimensions of timeliness, completeness, and information density to assess the quality of EHR data for patients managing T2DM with and without a history of telehealth use. Each dimension of quality was computed for data representing the following patient characteristics and relevant T2DM structured data elements and measurements: body mass index, serum creatinine, glycated hemoglobin A1c, cholesterol, blood pressure, and smoking status.

Timeliness

Prior research defined timeliness as data elements that represent a patient's health state at a desired time of interest and accounts for the age of the observed attribute.[24] [25] We operationalized timeliness as the age of T2DM measurements at the most recent encounter. Specifically, we determined the age of measurements as the number of days between a patient encounter and the most recent measurement dates.[26] Timeliness was measured at the encounter level for each patient in the study.

Completeness

Completeness is defined as the presence of a measurement in the EHR.[24] We quantified completeness using elements from the structured EHR data. T2DM measurements were identified and flagged as complete where a laboratory value or indicator was available at its first indication for sequential patient encounters. Completeness values ranged from 0 to 1 where a higher score indicated that data were more complete.

Information Density Score

Chronic disease care relies on repeated health care interactions and measurements of multiple clinical indicators. Information density scores are measures of completeness that account for the irregular nature of patient measurements taken over time.[27] [28] I or information density is the average amount of information each observation provides for a patient observed n times. An information density score is a number between 0 and 1 where higher scores indicate that patient measurements were more equally distributed across patient visits. The equation for information density can be found in [Supplementary Appendix A] (available in the online version).

Analysis

Frequencies, percentages, and means described EHR data quality dimensions by patient characteristics and over time. We compared data quality dimensions by patient telehealth status using χ² statistical tests to examine the equality of means and relationships between patient and encounter characteristics.

Results

Our final matched study population included 5,027 patients. Most clinical visits were for patients who were documented as African American (52.1%) and female (66.1%) ([Table 1]). The mean patient age in the sample was approximately 57.6 years.

Table 1 Patient and encounter descriptive characteristics by history of telehealth use

		History of patient telehealth use by demographic characteristics
	Total, n (%)	History of telehealth use, n (%)	No history of telehealth use, n (%)	p-Value
Patient sex				0.412
Male	1,705 (33.9)	638 (33.2)	1,067 (34.4)
Female	3,322 (66.1)	1,284 (66.8)	2,038 (65.6)
Patient race				<0.001
White	1,480 (29.4)	620 (32.3)	820 (27.7)
Black	2,617 (52.1)	1,029 (53.5)	1,588 (51.1)
Hispanic	579 (11.5)	161 (8.4)	418 (13.5)
Asian	48 (0.9)	16 (0.8)	32 (1.0)
American Indiana/Alaska Native	5 (0.1)	19 (0.9)	5 (0.2)
Native Hawaiian/Other Pacific Islander	59 (1.2)	35 (1.8)	40 (1.3)
More than one race	143 (2.8)	42 (2.2)	54 (1.7)
Age (y)				<0.001
18–25	66 (1.3)	28 (1.5)	38 (1.2)
26–45	781 (15.5)	284 (14.8)	497 (16.0)
46–55	1130 (22.5)	421 (21.9)	709 (22.8)
56–65	1,592 (31.7)	561 (29.2)	1,031 (33.2)
66–75	966 (19.2)	425 (22.1)	541 (17.4)
≥75	492 (9.8)	102 (10.6)	289 (9.3)
Charlson's comorbidity score				<0.001
0	549 (10.9)	188 (9.8)	361 (11.6)
1	1,919 (38.2)	674 (35.1)	1,245 (40.1)
2	1,554 (30.9)	644 (33.5)	910 (29.3)
3	726 (14.4)	305 (15.9)	421 (13.6)
4	221 (4.4)	86 (4.5)	135 (5.4)
5	48 (0.9)	20 (1.0)	28 (0.9)
6	10 (0.2)	5 (0.3)	5 (0.2)
Insurance				<0.001
Commercial	574 (11.4)	140 (7.3)	434 (13.9)
Medicare	2,756 (54.8)	1,245 (64.8)	1,511 (48.7)
Medicaid	1,395 (27.6)	449 (23.4)	946 (30.5)
Self-pay	205 (4.1)	47 (2.5)	158 (5.1)
Other	96 (1.9)	41 (2.1)	55 (1.8)
COVID-19				0.359
Pre-COVID	4,787 (95.2)	1,823 (94.9)	2,964 (95.5)
Post-COVID	240 (4.8)	99 (5.2)	141 (4.5)
Year				0.002
2016	2,120 (42.2)	857 (44.6)	1,263 (40.7)
2017	1,414 (28.1)	516 (26.9)	898 (28.9)
2018	628 (12.5)	219 (11.4)	409 (13.2)
2019	518 (10.3)	181 (9.4)	337 (10.9)
2020	335 (6.7)	147 (7.7)	188 (6.1)
2021	12 (0.2)	2 (0.1)	10 (0.32)

Abbreviation: COVID-19, coronavirus disease 2019.

Telehealth Use

Nearly 40% of patients in this sample had a history of telehealth visits ([Table 1]). Most telehealth visits constituted patients who were female, non-Hispanic Black, middle-aged (56–65), had Charlson scores of 1, and Medicare beneficiaries ([Table 1]).

Electronic Health Record Data Timeliness

In terms of timeliness, across all patient encounters and data elements, the average data element was 77.8 (95% confidence interval [CI], 39.6–116.4) days old. On average, data were timelier, that is, fewer days between EHR data attribute updates, for males, patients who were documented as non-Hispanic white, elderly patients, patients with severe comorbidities, patients on Medicare, and during pre-COVID-19 years ([Table 2]). There were differences in timeliness among some patient characteristics. For example, non-Hispanic white patients had lower timeliness (in days) compared with the remaining race categories. Results from computing Hinrich's timeliness quotient are available in [Supplementary Appendix B] (available in the online version).

Table 2 Patient and encounter characteristics by timeliness and completeness dimensions for T2DM EHR data

	Timeliness, completeness, and information density score for T2DM EHR data
	Timeliness (in days)[a]		Completeness[b]		Sperrin's I (information density score)[b]
	Mean	95% CI	Mean	95% CI	Mean	95% CI
Telehealth use
No	79.8	(62.9–96.7)	0.892	(0.872–0.912)	0.784	(0.767–0.801)
Yes	73.3	(58.1–88.5)	0.892	(0.889–0.895)	0.795	(0.763–0.827)
Patient sex
Male	62.8	(43.8–81.8)	0.896	(0.884–0.908)	0.784	(0.766–0.802)
Female	101	(83.4–118.6)	0.892	(0.881–0.903)	0.79)	(0.773–0.807)
Patient race/ethnicity
White	71.7	(52.7–90.7)	0.889	(0.866–0.912)	0.790	(0.786–0.794)
Black	77.1	(60.1–94.1)	0.890	(0.864–0.916)	0.792	(0.749–0.835)
Hispanic	88.8	(715.-106.1)	0.903	(0.891–0.915)	0.775	(0.754–0.796)
Asian	99.3	(80.6–118)	0.891	(0.887–0.895)	0.807	(0.795–0.819)
American Indian/Alaska Native	85	(66.7–103.3)	0.882	(0.873–0.891)	0.736	(0.725–0.747)
Native Hawaiian/Other Pacific Islander	83.5	(64.2–102.8)	0.906	(0.905–0.907)	0.768	(0.754–0.782)
More than one race	73.3	(57.6–89)	0.906	(0.874–0.938)	0.751	(0.651–0.851)
Age (y)
18–25	86.4	(63.5–109.3)	0.887	(0.883–0.891)	0.775	(0.757–0.793)
26–45	88.3	(74.5–102.1)	0.891	(0.848–0.934)	0.776	(0.759–0.793)
46–55	78.8	(57.1–100.5)	0.892	(0.871–0.913)	0.785	(0.753–0.817)
56–65	77.5	(55.9–99.1)	0.891	(0.879–0.903)	0.789	(0.772–0.806)
66–75	71.2	(48.3–94.1)	0.892	(0.881–0.903)	0.795	(0.773–0.817)
≥75	66.7	(49–84.4)	0.895	(0.881–0.909)	0.798	(0.756–0.840)
Charlson's comorbidity score
0	87.9	(68.1–107.7)	0.876	(0.854–0.898)	0.798	(0.772–0.824)
1	92.3	(73.6–111)	0.890	(0.878–0.902)	0.786	(0.774–0.798)
2	72.2	(52.6–91.8)	0.895	(0.877–0.913)	0.781	(0.777–0.785)
3	53.4	(31.6–75.2)	0.897	(0.880–0.914)	0.793	(0.784–0.802)
4	45.9	(27.1–64.7)	0.904	(0.872–0.936)	0.804	(0.803–0.805)
5	37.7	(20.1–55.3)	0.893	(0.876–0.910)	0.825	(0.813–0.837)
6	42.2	(20.5–63.9)	0.900	(0.878–0.922)	0.806	(0.788–0.824)
Insurance
Commercial	86.3	(69–103.6)	0.886	(0.868–0.904)	0.776	(0.763–0.789)
Medicare	72	(51.5–92.5)	0.892	(0.875–0.909)	0.796	(0.779–0.813)
Medicaid	80.9	(60.9–100.9)	0.893	(0.861–0.925)	0.779	(0.762–0.796)
Self-pay	96	(76.9–115.1)	0.896	(0.879–0.913)	0.788	(0.766–0.810)
Other	84.7	(64.9–104.5)	0.899	(0.877–0.921)	0.762	(0.729–0.795)
COVID-19
Pre-COVID	72.2	(52.4–92)	0.893	(0.873–0.913)	0.786	(0.759–0.813)
Post-COVID	143	(129.1–156.9)	0.862	(0.859–0.865)	0.822	(0.819–0.825)
Year
2016	69.7	(49.5–89.9)	0.896	(0.894–0.898)	0.799	(0.796–0.802)
2017	94.8	(79.4–110.2)	0.894	(0.882–0.906)	0.782	(0.764–0.800)
2018	78.7	(60–97.4)	0.896	(0.889–0.903)	0.772	(0.755–0.789)
2019	75.3	(61.3–89.3)	0.884	(0.880–0.888)	0.766	(0.734–0.798)
2020	54.5	38.6–70.4	0.867	(0.858–0.876)	0.803	(0.788–0.818)
2021	15.4	(3.3–27.1)	0.826	(0.825–0.827)	0.905	(0.901–0.909)

Abbreviations: CI, confidence interval; COVID, coronavirus disease; EHR, electronic health record; T2DM, type 2 diabetes mellitus.

^a Timeliness is measured at the encounter level. Lower timeliness, in days, indicates higher levels of data quality.

^b Completeness is measured at the patient level. Higher levels of completeness indicate that more data are available for each patient.

In general, T2DM measurements were timelier among patients who had a history of telehealth use compared with patients with no history of telehealth use ([Table 3]). That is, for all T2DM clinical measurements examined in this study, the average number of days between EHR data attribute updates was shorter for patients with a history of telehealth use.

Table 3 Data quality dimensions for T2DM EHR data by patients' use of telehealth

	EHR data quality dimensions for T2DM EHR data
	History of telehealth use		No history of telehealth use
	Mean	95% CI	Mean	95% CI
Timeliness (in days)[a]
Body mass index	50.6	(12.6–88.6)	61.2	(13.2–109.2)
Blood pressure	42.6	(18.6–66.6)	52.1	(30.2–74)
HbA1c	324	(241.9–406.1)	412	(324.9–499.1)
Cholesterol	326	(255.4–396.6)	432	(366.2–497.8)
Serum creatinine	123	(82–164)	135	(99–171)
Smoking status	29	(7.1–50.9)	31.9	(3.8–60)
Completeness[b]
Body mass index	0.991	(0.990–0.092)	0.990	(0.983–0.997)
Blood pressure	0.993	(0.990–0.996)	0.993	(0.989–0.997)
HbA1c	0.886	(0.799–0.973)	0.836	(0.834–0.838)
Cholesterol	0.903	(0.889–0.917)	0.814	(0.727–0.901)
Serum creatinine	0.989	(0.987–0.991)	0.983	(0.969–0.997)
Smoking status	0.993	(0.988–0.998)	0.994	(0.992–0.996)
Sperrin's I (information density score)[b]
Body mass index	0.831	(0.820–0.842)	0.806	(0.794–0.818)
Blood pressure	0.820	(0.788–0.852)	0.866	(0.863–0.869)
HbA1c	0.881	(0.864–0.898)	0.827	(0.826–0.828)
Cholesterol	0.822	(0.768–0.876)	0.838	(0.830–0.846)
Serum creatinine	0.791	(0.766–0.816)	0.814	(0.795–0.833)
Smoking status	0.889	(0.887–0.891)	0.880	(0.873–0.887)

Abbreviations: CI, confidence interval; EHR, electronic health record; T2DM, type 2 diabetes mellitus.

^a Timeliness is measured at the encounter level. Lower timeliness, in days, indicates higher levels of data quality.

^b Completeness is measured at the patient level. Higher levels of completeness indicate that more data are available for each patient.

Electronic Health Record Data Completeness

The mean completeness for all relevant patient data was 0.891 (95% CI, 0.868–0.914) indicating moderate-to-high completeness for data recorded in T2DM measurements ([Table 2]). Completeness scores were, on average, higher among patients with severe comorbidities and patients who were seen in years preceding the COVID-19 pandemic ([Table 2]). These scores for T2DM measurements were generally higher among patients who had a history of telehealth use compared with patients with no history of telehealth use ([Table 3]).

Electronic Health Record Data Information Density

The mean information density score for T2DM EHR data was 0.787 (95% CI, 0.747–0.82). This demonstrates that patient visits and relevant T2DM measurements were somewhat uniform during the period in which they were identified in the EHR dataset. Information density scores for EHR data were higher among females, patients who were documented as Asian, older patients, and Medicare beneficiaries ([Table 1]). T2DM measurements became more uniform across patient visits in the year following the start of the COVID-19 pandemic. Information density scores, which account for the irregular nature of T2DM physiological measurements, were, on average higher for patients who had a history of telehealth use. However, blood pressure, cholesterol, and serum creatinine information density scores were higher among patients who did not have a history of telehealth use compared with patients who used telehealth during the study period.

Discussion

EHR data quality for adult patients managing T2DM was generally similar or better among patients with a history of telehealth use than for those with no history of telehealth use. However, differential data quality among care formats may be due to encounter, patient, and organizational factors not observed in this study. Quantifying the quality of EHR data across modalities and patient populations is critical for health care organizations and researchers as the delivery of care evolves in the United States.[29] While our results show that patients with a history of telehealth tended toward higher quality data, dimensions across both patient groups had room for improvement. For example, clinical guidelines recommend measuring HbA1c at least twice a year, but the average HbA1c measure was nearly a year old.[22] Results for some patient laboratory tests and measurements may fall within a clinically acceptable shelf life that enables actionable decision-making.[9] [20] For other measurements, options for improving data collection and documentation exist. For example, patient portals that allow for patient input and self-reporting may help bridge incomplete or erroneous data.[30] [31] [32] Similarly, patient portals give patients opportunities to view, verify, and potentially correct information.[32] Likewise, organizations could consider the use of remote monitoring tools. The improved accuracy of blood glucose monitors enables remote data sharing and access.[19] During patient encounters, the use of built-in structured forms and support personnel, like scribes, has been linked to reductions in workflow disruptions, improvements in documentation completion time, and efficient and high-quality documentation.[33]

Associations between telehealth visits and data quality for the most relevant clinical measurements indicate that there may be unobserved clinical efficiencies in care delivery using digital health tools. Our results are reassuring given the increase in telehealth use nationwide. Past research has indicated that patients who actively use digital health tools like patient portals and telehealth are more engaged and perceptive to errors in their own information.[31] [32] The relationship between telehealth visits and data quality may also be driven mostly by educated middle-aged patients who had previous experience using video visits for their medical care.[34] An increasing number of payers and policy-makers had already been encouraging the wider use of digital health tools and services for chronic disease management.[35] The Centers for Medicare and Medicaid Services finalized two policy changes for remote therapeutic and physiologic monitoring services extending beyond the Public Health Emergency Declaration.[36] These new rules seek to improve care delivery, cost management, and health outcomes for chronic disease patients who rely on remote monitoring as part of disease management.[7] As noted above, opportunities and strategies exist to improve data quality for T2DM patients, but these findings at least suggest telehealth visits may not be detrimental to EHR data quality.

Our results reflect existing literature that has examined the utility of patient EHR data and the extent to which it is actionable to inform care decisions.[16] [17] [21] Data in this study were subject to some preprocessing before extraction as a means of auditing its quality for secondary reuse. However, findings suggest these data still maintained varying levels of completeness and timeliness that were likely present at the source origin. The shelf life and availability of patient data are critical characteristics that have implications for shared clinical decision-making. We posit that the differences in EHR data quality among patients with and without a history of telehealth use may be due to unobserved patient characteristics. Additionally, providers have been granted more flexibility in closing patient records to focus on quality care delivery.[37] Conversely, timely telehealth payment in the wake of new reimbursement regulations necessitated prompt, complete, and accurate documentation.[38]

Overall, the levels of data quality in this study population were mostly consistent with the literature examining gaps in complete data for patients managing diabetes.[23] This study did not determine the reasons for less than timely and incomplete data, but determinants of poor EHR data quality in chronic disease care domains can include disparate standards, incomplete documentation, spelling and coding errors, noncompliant data protocols, and errors in extraction, to name a few.[21] [39] [40] [41] Direct comparisons with other studies on timeliness and completeness are challenging which warranted the use simpler metrics. While timeliness and completeness are well-defined data quality constructs,[26] [27] [28] [29] many studies of EHR data quality do not always account for the longitudinal nature of patient clinical interactions.[42] We adapted our approach to examine the completeness of relevant chronic disease and demographic data elements longitudinally to examine the full historical account of data at the patient level.[28]

Additionally, this study highlighted additional notable differences in data quality. EHR data for underrepresented racial minority patients were marginally less timely than for non-Hispanic white patients. The magnitude of these differences may reflect distributions in patient characteristics that were not examined in this study. Given our sample is majority African American/Black and T2DM affects a large share of African American/Black patients in the United States, the effects of incomplete and less timely data on care quality and care coordination should be explored further.[43] In addition, this study illustrates the very disruptive nature of the COVID-19 pandemic on telehealth use for chronic disease care[44] and EHR data quality. EHR data completeness scores decreased during the pandemic period. This indicates future EHR-based research may have to account for lower quality data during this period.

Limitations

This study has several limitations. These findings solely included structured EHR data elements from an EDW. Data from the EDW may be subject to some biases due to their preprocessing policies. However, the extent to which data were timely and complete was consistent with prior work that analyzed completeness and timeliness.[21] [26] It is possible that relevant data were documented in clinical notes or other text documents and, therefore, would have been available at either encounter type. While potentially available for clinical care, any text-based data would be less accessible for secondary uses. Due to the COVID-19 pandemic, timeliness measures may have been affected by delays in care experienced by all primary care settings. This study did not assess whether observed levels of data quality were sufficient for effective decision-making and disease management. The levels of data quality may not be generalizable to other settings with different documentation practices, scheduling practices, and workflows. The intensity of telehealth use was not quantified in this study which warrants additional exploration of a potential dose–response relationship among frequent telehealth users. Lastly, these findings on EHR data quality are limited in terms of the generalizability of the measurements and the patient population.

Conclusion

EHR data for patients managing T2DM with a history of telehealth use were generally more timely and more complete than data for patients with no history of telehealth use. Differences between visit types and across patient characteristics may limit care coordination and secondary data uses. Improvements to data collection will be needed to improve overall quality as care delivery models incorporate both in-person and virtual formats.

Clinical Relevance Statement

Type 2 diabetes requires frequent laboratory tests and other measurements that support clinical decision-making. Tests and measurements must be timely and maintain levels of completeness and uniformity that are actionable to improve care quality and coordination for patients managing chronic disease. New approaches to data collection, including remote and continuous patient monitoring, will produce additional measurements that may not require clinician intervention. These data derived via digital health and telehealth technologies will be subject to varying levels of quality which necessitates quantification to determine reliability and accuracy in supporting critical clinical decision-making.

Multiple Choice Questions

1. Authors of this study quantified which of the following electronic health record (EHR) data quality dimensions?

   • Concordance, accuracy, plausibility

   • Completeness, timeliness, information density (correct response)

   • Correctness, concordance, and timeliness

   • Plausibility, completeness, concordance

   Correct Answer: The correct answer is option b. Authors quantified completeness, timeliness, and information density using existing approaches derived from the scholarly literature. Based on prior research, completeness, timeliness, and information density are considered dimensions of data quality. Completeness, timeliness, and information density are data dimensions critical to examining the overall quality of EHR data documented on behalf of patients managing chronic disease. In this study, completeness is quantified as any identifiable type 2 diabetes measurement that was complete at its first indication for a sequence of patient encounters. Timeliness was quantified using a simple metric that computes the age or interval between a patient encounter and the most recent measurement date. Lastly, information density was quantified as the average amount of information each observation provides for a patient observed n times. Researchers have developed metrics to quantify other dimensions of data quality, including concordance, plausibility, correctness, and accuracy; however, those were not the focus of our analyses. Importantly, some data quality dimensions use similar measurement and quantification approaches.

2. Examining electronic health record (EHR) data for patients managing type 2 diabetes provides data elements on which of the following:

   • Laboratory tests, physiological measurements, and behavioral factors (correct response)

   • Social services and tax information

   • Administrative data

   • Patient experience indicators

   Correct Answer: The correct answer is option a. The laboratory and physiological measurements selected in this study include body mass index, serum creatinine, glycated hemoglobin A1c, cholesterol, and blood pressure. These laboratory tests and physiological measurements are important indicators of type 2 diabetes management. Smoking status is an important behavioral factor that exacerbates chronic diseases including type 2 diabetes. Other data elements, like social services data, tax information, administrative data, and patient experience indicators are important in examining patients' social determinants of health and health holistically; however, these data are typically not found in the EHR.

Conflict of Interest

None declared.

Protection of Human and Animal Subjects

This study was approved by the Indiana University Institutional Review Board (IRB) as exempt research. This research analyzed secondary electronic health record (EHR) data and did not involve Human Subjects.

• References

• 1 Meystre SM, Lovis C, Bürkle T, Tognola G, Budrionis A, Lehmann CU. Clinical data reuse or secondary use: current status and potential future progress. Yearb Med Inform 2017; 26 (01) 38-52
• 2 Hersh WR. Adding value to the electronic health record through secondary use of data for quality assurance, research, and surveillance. Am J Manag Care 2007; 13 (6 Part 1): 277-278
• 3 Capobianco E. Data-driven clinical decision processes: it's time. J Transl Med 2019; 17 (01) 44
• 4 Khairat S, Coleman C, Ottmar P, Jayachander DI, Bice T, Carson SS. Association of electronic health record use with physician fatigue and efficiency. JAMA Netw Open 2020; 3 (06) e207385
• 5 Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc 2020; 27 (04) 531-538
• 6 Botsis T, Hartvigsen G, Chen F, Weng C. Secondary use of EHR: data quality issues and informatics opportunities. Summit Translat Bioinforma 2010; 2010: 1-5
• 7 Faulconer ER, de Lusignan S. An eight-step method for assessing diagnostic data quality in practice: chronic obstructive pulmonary disease as an exemplar. Inform Prim Care 2004; 12 (04) 243-254
• 8 Singh H, Meyer AND, Thomas EJ. The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations. BMJ Qual Saf 2014; 23 (09) 727-731
• 9 Marani H, Halperin IJ, Jamieson T, Mukerji G. Quality gaps of electronic health records in diabetes care. Can J Diabetes 2020; 44 (04) 350-355
• 10 Adler-Milstein J, Kvedar J, Bates DW. Telehealth among US hospitals: several factors, including state reimbursement and licensure policies, influence adoption. Health Aff (Millwood) 2014; 33 (02) 207-215
• 11 Neufeld JD, Doarn CR. Telemedicine spending by Medicare: a snapshot from 2012. Telemed J E Health 2015; 21 (08) 686-693
• 12 Menachemi N, Burke DE, Ayers DJ. Factors affecting the adoption of telemedicine–a multiple adopter perspective. J Med Syst 2004; 28 (06) 617-632
• 13 Hollander JE, Carr BG. Virtually perfect? Telemedicine for Covid-19. N Engl J Med 2020; 382 (18) 1679-1681
• 14 Opportunities and Barriers for Telemedicine in the U.S. During the COVID-19 Emergency and Beyond. KFF. Published May 11, 2020. Accessed July 10, 2020 at: https://www.kff.org/womens-health-policy/issue-brief/opportunities-and-barriers-for-telemedicine-in-the-u-s-during-the-covid-19-emergency-and-beyond/
• 15 Gordon HS, Solanki P, Bokhour BG, Gopal RK. “I'm not feeling like I'm part of the conversation” patients' perspectives on communicating in clinical video telehealth visits. J Gen Intern Med 2020; 35 (06) 1751-1758
• 16 Mohktar MS, Sukor JA, Redmond SJ, Basilakis J, Lovell NH. Effect of home telehealth data quality on decision support system performance. Procedia Comput Sci 2015; 64: 352-359
• 17 Freckmann G, Pleus S, Grady M, Setford S, Levy B. Measures of accuracy for continuous glucose monitoring and blood glucose monitoring devices. J Diabetes Sci Technol 2019; 13 (03) 575-583
• 18 Izquierdo RE, Knudson PE, Meyer S, Kearns J, Ploutz-Snyder R, Weinstock RS. A comparison of diabetes education administered through telemedicine versus in person. Diabetes Care 2003; 26 (04) 1002-1007
• 19 Carter EL, Nunlee-Bland G, Callender C. A patient-centric, provider-assisted diabetes telehealth self-management intervention for urban minorities. Perspect Health Inf Manag 2011; 8: 1b
• 20 American Diabetes Association. Standards of medical care for patients with diabetes mellitus. Diabetes Care 2003; 26 (Suppl 1): S33-S50
• 21 Goudswaard AN, Lam K, Stolk RP, Rutten GEHM. Quality of recording of data from patients with type 2 diabetes is not a valid indicator of quality of care. A cross-sectional study. Fam Pract 2003; 20 (02) 173-177
• 22 McDonald CJ, Overhage JM, Barnes M. et al; INPC Management Committee. The Indiana network for patient care: a working local health information infrastructure. An example of a working infrastructure collaboration that links data from five health systems and hundreds of millions of entries. Health Aff (Millwood) 2005; 24 (05) 1214-1220
• 23 Sholle ET, Pinheiro LC, Adekkanattu P. et al. Underserved populations with missing race ethnicity data differ significantly from those with structured race/ethnicity documentation. J Am Med Inform Assoc 2019; 26 (8-9): 722-729
• 24 Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Inform Assoc 2013; 20 (01) 144-151
• 25 Heinrich B, Kaiser M. HOW TO MEASURE DATA QUALITY? A METRIC BASED APPROACH. Twenty Eighth International Conference on Information Systems, Published online 2007. Accessed October 28, 2022 at: https://epub.uni-regensburg.de/23633/1/heinrich.pdf
• 26 Hills RA, Revere D, Altamore R, Abernethy NF, Lober WB. Timeliness and data element completeness of immunization data in Washington State in 2010: a comparison of data exchange methods. AMIA Annu Symp Proc 2012; 2012: 340-349
• 27 Sperrin M, Thew S, Weatherall J, Dixon W, Buchan I. Quantifying the longitudinal value of healthcare record collections for pharmacoepidemiology. AMIA Annu Symp Proc 2011; 2011: 1318-1325
• 28 Weiskopf NG, Hripcsak G, Swaminathan S, Weng C. Defining and measuring completeness of electronic health records for secondary use. J Biomed Inform 2013; 46 (05) 830-836
• 29 O'Connor PJ. Organizing diabetes care: identify, monitor, prioritize, intensify. Diabetes Care 2001; 24 (09) 1515-1516
• 30 Ross SE, Haverhals LM, Main DS, Bull SS, Pratte K, Lin CT. Adoption and use of an online patient portal for diabetes (Diabetes-STAR). AMIA Annu Symp Proc 2006; 2006: 1080
• 31 Genes N, Violante S, Cetrangol C, Rogers L, Schadt EE, Chan YY. From smartphone to EHR: a case report on integrating patient-generated health data. NPJ Digit Med 2018; 1: 23
• 32 Bell SK, Delbanco T, Elmore JG. et al. Frequency and types of patient-reported errors in electronic health record ambulatory care notes. JAMA Netw Open 2020; 3 (06) e205867-e205867
• 33 Aziz F, Talhelm L, Keefer J, Krawiec C. Vascular surgery residents spend one fifth of their time on electronic health records after duty hours. J Vasc Surg 2019; 69 (05) 1574-1579
• 34 Predmore ZS, Roth E, Breslau J, Fischer SH, Uscher-Pines L. Assessment of patient preferences for Telehealth in post-COVID-19 pandemic health care. JAMA Netw Open 2021; 4 (12) e2136405-e2136405
• 35 Final Policy, payment, and quality provisions changes to the Medicare physician fee schedule for calendar year 2021. Accessed November 6, 2021 at: https://www.cms.gov/newsroom/fact-sheets/final-policy-payment-and-quality-provisions-changes-medicare-physician-fee-schedule-calendar-year-1
• 36 Medicare Program. CY 2022 Payment Policies under the Physician Fee Schedule and Other Changes to Part B Payment Policies; Medicare Shared Savings Program Requirements; Provider Enrollment Regulation Updates; Provider and Supplier Prepayment and Post-Payment Medical Review Requirements.;1–1747. Accessed October 28, 2022 at: https://public-inspection.federalregister.gov/2021-14973.pdf
• 37 Centers for Medicare and Medicaid Services. Physicians and Other Clinicians: CMS Flexibilities to Fight COVID-19; August 18, 2022.
• 38 Centers for Medicare and Medicaid Services. C2C Telehealth Provider Toolkit.; August 26, 2022.
• 39 Foley & Lardner LLP. CMS Proposes New Remote Therapeutic Monitoring Codes: What You Need to Know. Published July 15, 2021. Accessed December 23, 2021 at: https://www.foley.com/en/insights/publications/2021/07/cms-new-remote-therapeutic-monitoring-codes
• 40 Redman TC. Measuring data accuracy: a framework and review. In: Information Quality. England, U.K.: Routledge; 2014: 33-48
• 41 Michalakidis G, Kumarapeli P, Ring A, van Vlymen J, Krause P, de Lusignan S. A system for solution-orientated reporting of errors associated with the extraction of routinely collected clinical data for research and quality improvement. Stud Health Technol Inform 2010; 160 (pt. 1): 724-728
• 42 Mitsch C, Huber P, Kriechbaum K. et al. eHealth 2015 special issue: impact of electronic health records on the completeness of clinical documentation generated during diabetic retinopathy consultations. Appl Clin Inform 2015; 6 (03) 478-487
• 43 Rodríguez JE, Campbell KM. Racial and ethnic disparities in prevalence and care of patients with type 2 diabetes. Clin Diabetes 2017; 35 (01) 66-70
• 44 Bitar H, Alismail S. The role of eHealth, telehealth, and telemedicine for chronic disease patients during COVID-19 pandemic: a rapid systematic review. Digit Health 2021; 7: 20 552076211009396

Publication History

Received: 23 June 2022

Accepted: 11 October 2022

Article published online:
14 December 2022