Health IT Usability Focus Section: Adapting EHR-Based Medication Instructions to Comply with Plain Language Guidance—A Randomized Experiment

Abstract

Objective Patient instructions are generally written by clinicians. However, clinician-centered language is challenging for patients to understand; in the case of pediatric medication instructions, consequences can be serious. Using examples of clinician-written medication instructions from an electronic health record, we conducted an experiment to determine whether parental misinterpretations would be reduced by instructions that followed best practices for plain language.

Methods We selected examples of dosing instructions from after-visit summaries in a commercial electronic health record. A demographically diverse sample of parents and adult caregivers was recruited from an online panel to participate in an English-language experiment, in which they received a comprehension questionnaire with either original after-visit summary instructions or instructions revised to comply with federal and other sources of plain-language guidance.

Results Nine-hundred and fifty-one respondents completed the experiment; 50% were women, the mean age was 36 years, and 38% had less than a 4-year college education. The revisions were associated with an 8 percentage point increase in correct answers overall (from 55% to 63%, p < 0.001), although revisions were not equally effective for all instructions. Health literacy and health numeracy were strong and independent predictors of comprehension. Overall, mistakes on comprehension questions were common, with respondents missing an average of 41% (6.1 of 15) of questions.

Conclusion In this experimental study, a relatively simple intervention of revising text was associated with a modest reduction in frequency of misinterpretations of medication instructions. As a supplement to more intensive high-touch interventions, revising electronic health record output to replace complex language with patient-centered language in an automated fashion is a potentially scalable solution that could reduce medication administration errors by parents.

Keywords

health literacy - pediatrics - drug overdose - medication errors - survey and questionnaires

Background and Significance

Individualized medication instructions are typically written by physicians or pharmacists and presented to the patient in the form of an electronic after-visit summary, discharge instruction, or prescription printed or delivered via an electronic patient portal. Unfortunately, it is well-established that patients frequently misunderstand clinician language, even when clinicians try to be simple. For example, one study showed that nearly 40% of a sample of clinic outpatients could not correctly operationalize the seemingly simple instruction “twice a day.”[1] Comprehension problems are more frequent among those with low health literacy and health numeracy.[2] [3] [4] [5] [6] [7] An estimated 14% of U.S. adults[8] and up to 29% of parents[6] have limited health literacy, defined as the skills and knowledge needed to obtain, understand, and apply information to their own health and medical decisions.[9] [10] [11] [12] Low health numeracy is even more prevalent and is common at even the highest literacy levels.[13] [14] Health numeracy is the skill set needed to apply quantitative information to health, including information about risks, times and dates, and quantities.[13] [15] [16]

Particularly in the case of pediatric medications, the consequences can be serious or even fatal, as children are uniquely vulnerable to adverse events from medication errors.[17] [18] More than 70,000 children present to U.S. emergency departments each year with accidental overdoses,[19] most caused by parent or caregiver administration errors.[2] [20] [21] [22] [23] [24] No comparable national-level data are available about underdosing, but this has also been found to be a common problem in small-scale studies.[2] [22] [25] [26] In administering children's medications, parents must operationalize instructions about dose, route, and frequency, as well as additional issues unique to children such as weight-based dosing, age-based thresholds and contraindications, and liquid medications.[2] [22] [25] [26]

One way to prevent medication errors could be to redesign the product that facilitates the errors, in this case, the medication instructions that parents frequently misunderstand. This perspective comes from human factors engineering, the practice of optimizing human well-being and system performance by making products, tasks, and systems compatible with the needs, abilities, and limitations of people, as well as from usability engineering, the subset of human factors engineering that focuses on making electronic systems easy to learn, satisfying to use, and matched to user needs, capabilities, and goals.[27] [28] [29] In fact, a large body of research already shows that patients, even those with low literacy or numeracy, can effectively manage medication administration when supported with evidence-based materials.[30] [31] Examples include plain-language instructions,[1] [3] as well as more complex materials such as visual schedules displaying pills on a calendar,[32] [33] “pictograph” illustrations,[1] [7] [34] [35] and marked pillboxes and preloaded syringes.[31] [35] [36] [37] In other words, the comprehension errors arise not solely from the patient's literacy but rather from a system that fails to match communication modality to patient capabilities.[13] [15] [38] [39]

To date, many of the communication strategies found to be efficacious in controlled research situations have not been implemented widely in practice because they are labor-intensive and difficult to implement consistently.[40] [41] An intervention with modest efficacy that reaches large numbers of patients reproducibly at low cost and effort may have a large public health impact.[42] [43] [44] In fact, the net effect may be larger than the effect of a highly efficacious intervention that is challenging to implement and so reaches only small numbers.[42] [43] [44]

We therefore examined whether electronic health records (EHRs) could be targeted for low-effort, scalable, usability interventions that would improve comprehension of medication instructions.

Objective

Our objective was to assess whether easy-to-implement wording changes based on best practices for plain language would improve comprehension of common medication instructions, using a randomized experiment. The intervention involved wording changes with limited visual illustrations, because this approach could be easily implemented and automated in existing commercial EHRs. Our secondary objectives were (1) to assess the effects of literacy, numeracy, and demographics on comprehension, and (2) to assess the effect of the intervention on the subsets of wrong answers associated with overdoses and underdoses.

Methods

Experimental Design

In this between-subject experiment, participants were randomly assigned to see the original wording (usual care) or the revisions (intervention; [Table 1]), and then to answer 15 multiple-choice comprehension questions (see Appendix).

Table 1 Original and revised instructions

Original wording (usual care)			Revised wording (intervention)
Primary research question: Electronic health record instructions
2 pills every 12 h			Give 2 pills in the morning, and 2 pills in the evening
Administer 5 drops into both ears 2 times daily			In the morning: 5 drops in right ear and 5 drops in left ear In the evening: 5 drops in right ear and 5 drops in left ear
10 mL by mouth every 4 h			Give 10 mL at all of these times  • Morning  • Noon-time  • Late afternoon  • Bedtime  • Midnight  • 4 a.m.
1 pill per day for 3 d, switch to 1 pill every other day for 4 d			Monday morning	Tuesday morning	Wednesday morning	Thursday morning	Friday morning	Saturday morning	Sunday morning
			1 pill	1 pill	1 pill	No pill	1 pill	No pill	1 pill
Apply topically 4 times daily until rash is gone and for 3 additional days			Until rash is gone: Spread the cream on skin every morning, noon, afternoon, and evening After rash is gone: Keep using the cream in the same way for 3 more days
Give every 4 h as needed for fever/pain, max 5 doses			Give this medicine if your child has fever or pain. Wait at least 4 h. Does the child still have fever or pain? If so, give medicine again. Don't give it more than 5 times in one day
Give on an empty stomach			Give 30 min before the child eats breakfast
2 drops each nostril ×3–4 per day as needed			If the child's nose is stuffy, give 2 drops in the right nostril and 2 drops in the left nostril How often per day: up to 4 times
For very high fevers, alternate between tylenol and ibuprofen every 3 h			If your child has a fever of 104 degrees or higher:  • Start with a dose of tylenol  • 3 h later, check to see if the fever is still high. If so, give a dose of ibuprofen  • 3 h later, check to see if the fever is still high. If so, give a dose of tylenol You can switch between the medicines in this way until the fever comes down
Secondary research question: Over-the-counter medication instructions
1 unpacked level scoop (8.5 g) per 2 fl oz			To make		Use this much water	Mix in this much powder
			2 ounces		2 ounces	1 scoop
			4 ounces		4 ounces	2 scoops
			6 ounces		6 ounces	3 scoops
Weight (lb)	Age (mo)	Dose (mL)	If you know your child's weight, use this information:  • under 18 pounds, give 1.25 mL  • 18 pounds or more, give 1.875 mL If you do not know your child's weight, use this information:  • less than 12 mo old, give 1.25 mL  • 12 mo or older, give 1.875 mL But if your child is under 6 mo old, ask a doctor whether you should use this medicine.
–	Under 6 mo	Ask a doctor
12–17 lbs	6–11 mo	1.25 mL
18–23 lbs	12–23 mo	1.875 mL

Setting and Sample

Survey Sampling International (SSI; www.surveysampling.com) is an international survey panel and market research firm widely used in online surveys. We contracted with SSI to recruit a sample. The inclusion criteria were that participants had to be: (1) U.S. adults with primary caregiving responsibilities for at least one child under the age of 18, and (2) comfortable completing a questionnaire in English. We also specified to SSI that a minimum of 30% of the sample should have less than a college education. SSI has a large existing online panel of registered individuals who have agreed to be contacted for online surveys and questionnaires to be entered into drawings to earn modest incentives of their choice (which might include money, airline miles, etc.). These registered individuals have all completed extensive demographic questionnaires. Using our eligibility criteria, SSI disseminated the recruitment announcement to individuals who met our criteria, while continuously monitoring the education levels of those who agreed to participate. Recruitment was closed early for the higher education category and extended for the lower education category to ensure the 30% representation we had specified.

Questionnaire Development

We selected examples of common dosing instructions from the after-visit summary of a commercial EHR (Epic Systems, Verona, Wisconsin, United States). Instructions in our institution's after-visit summary already take a step toward patient-centeredness by replacing acronyms and abbreviations (e.g., a provider instruction of “2× per day” is automatically replaced with “2 times a day”).

For exploratory purposes, we also added two questions from common over-the-counter pediatric pain relievers and formula, plus an instruction to alternate between acetaminophen and ibuprofen for fever reduction. (Although this practice is not encouraged at our institution in light of an American Academy of Pediatrics white paper,[45] we included it because patients elsewhere may encounter it.)

To create the revised versions of the instructions, we selected a package of five plain-language principles that were well supported by evidence (although not always in the context of medication instructions; see references) and that appeared to be easily applicable to informatics practice (e.g., could potentially be accomplished through automatic term substitutions rather than major redesign). We drew from www.plainlanguage.gov and other resources such as Shoemaker et al[46] and the references listed below:

1. Avoid unfamiliar terms, jargon, and abbreviations. This involved substituting familiar terms for unfamiliar ones where possible,[47] [48] [49] [50] defining medical terms parenthetically when they could not be replaced with more familiar terms,[46] [50] and inserting explanations of selected concepts that require medical knowledge (e.g., for “on an empty stomach,” explaining how long before and after the meal the child would have an empty stomach).[46]

2. Perform computations for the reader.[13] [46] Instead of instructing the reader to administer medications every X hours, administration times were phrased in terms of recurring events such as morning and evening or mealtimes.[1] [50] [51]

3. Avoid complex sentences containing multiple instructions or steps.[46] [49] This involved putting one instruction or step per bullet or sentence,[46] separating multiple “if-then” or “when” conditions into a separate instruction,[49] and keeping the “if” or “when” statement short and placing it after the verb.[49]

4. Use active voice and address the reader.[46]

5. Use illustrations if they reinforce or explain the text.[46] [49] [52] [53]

We opted not to apply formulas to reduce the “grade level” of the text; revisions to reduce the grade level would have involved replacing long words with short ones and breaking long sentences into shorter ones.[54] [55] This decision was made on basis of research suggesting that word familiarity was more important to comprehension than word length,[56] and that the contextual explanations needed to help novices understand health-related text often make text longer.[57] [58]

Covariates

For all participants, health literacy was assessed with the three-item Chew scale.[59] [60] Following Wallace et al,[61] we assigned 1 point to each question for which the participant answered that they had any difficulty or required help, to produce a health literacy score of 0 (adequate), 1 (marginal), or 2 or 3 (inadequate). Health numeracy was assessed with the 8-item Short Numeracy Understanding in Medicine Instrument (S-NUMi).[62] [63] Following Schapira, a health numeracy score of 7 or higher was classified as high numeracy, 4 to 6 as adequate numeracy, and 3 or less as low numeracy.

In addition, participants were asked about their personal experience of administering pediatric medications, their medical training or experience, and personal and family demographics.

Analytic Methods

The primary outcome was number of correct answers to the comprehension questions. A secondary outcome was incorrect responses that would have led to overdoses or underdoses. To establish this, each of the incorrect response options for each question was classified a priori as a potential overdose or underdose by a pharmacist collaborator (A.S.), employing professional judgment and reference works as appropriate.

Student's t-tests and analysis of variance for continuous variables and chi-squared tests for categorical ones were used to assess comparability of the two arms as well as bivariate associations with the primary outcome (correct answers to the comprehension questions). Descriptive statistics and tables were computed in IBM SPSS Statistics version 24 (Armonk, New York, United States). Variables significant at 0.05 on bivariate analysis were considered for multivariate regression models to assess the secondary outcomes of the effects of literacy, numeracy, and demographics on comprehension. Multivariate models were constructed in SAS (version 9.4, Cary, North Carolina, United States) using proc GLM applying forward and backward selection to drop nonsignificant variables, and proc GLMselect using stepwise selection on the basis of the Akaike information criterion.

Human Subjects Research Approval

The Weill Cornell Institutional Review Board determined that the project was exempt because no personally identifying information was collected from the participants.

Results

A total of 1,012 individuals started the questionnaire, and 952 (94.1%) completed it (with equal dropout rates in the two arms, p = 0.23). One response was eliminated for an apparent data quality problem (self-reported age over 90 years) leaving a final sample size of 951.

As shown in [Table 2], half the sample were women, the mean age was ∼37 years, 83% were white, 38% had educational attainment of less than a bachelor's degree, and ∼12% had health insurance provided by Medicaid (the U.S. public insurance program for low-income individuals and families). One quarter reported that they had some form of medical experience or training. Inclusion criteria included having primary caregiving responsibilities for a child under the age of 18, but some participants also reported having children older than 18 years. Participant characteristics were well balanced across the two arms with the exception of age: respondents in the control (usual care) arm were an average of ∼1 year older than respondents in the intervention arm.

Table 2 Participant demographic and knowledge/skills characteristics

Characteristics	Whole group	Version 1 (usual care)	Version 2 (intervention)	p-Value
N	951	480	471
Demographic questions
Women, N (%)	475 (49.9)	240 (50.0)	235 (49.9)	0.97
Mean (SD) age	36.5 (9.5)	37.1 (9.7)	35.8 (9.4)	0.03
Race (multiple-choice possible), N (%)
White or Caucasian	789 (82.8)	394 (81.9)	395 (83.7)	0.47
Black or African-American	91 (9.5)	45 (9.4)	46 (9.7)	0.84
Asian	46 (4.8)	28 (5.8)	18 (3.8)	0.15
American Indian or Alaska Native	20 (2.1)	11 (2.3)	9 (1.9)	0.68
Native Hawaiian or Pacific Islander	4 (0.4)	1 (0.2)	3 (0.6)	0.37
Other	27 (2.8)	14 (2.9)	13 (2.8)	0.88
Prefer not to say	7 (0.7)	2 (0.4)	5 (1.1)	0.25
Hispanic/Latino, N (%)				>0.99
Hispanic/Latino	112 (11.8)	57 (11.9)	55 (11.7)
Not Hispanic/Latino	823 (86.4)	415 (86.3)	408 (86.4)
Prefer not to say	16 (1.7)	8 (1.7)	8 (1.7)
Education, N (%)				0.55
Did not complete high school	16 (1.7)	8 (1.7)	8 (1.7)
Completed high school	194 (20.4)	87 (18.1)	107 (22.7)
Completed a 2-y college degree	151 (15.9)	82 (17.1)	69 (14.6)
Completed a 4-y college degree	290 (30.5)	146 (30.4)	144 (30.6)
Completed graduate degree	297 (31.2)	155 (32.3)	142 (30.1)
Prefer not to say	3 (0.3)	2 (0.4)	1 (0.2)
Household income, N (%)				0.59
0–$24,999	89 (9.3)	45 (9.4)	44 (9.3)
$25,000–$49,999	159 (16.7)	75 (15.6)	84 (17.8)
$50,000–$74,999	214 (22.5)	102 (21.2)	112 (23.7)
$75,000–$99,999	231 (24.2)	126 (26.2)	105 (22.2)
$100,000–$124,999	136 (14.3)	66 (13.7)	70 (14.8)
$125,000 or more	122 (12.8)	66 (13.7)	56 (11.9)
Insurance coverage, N (%)				0.12
None	38 (4.0)	18 (3.7)	20 (4.2)
Private	564 (59.2)	292 (60.7)	272 (57.6)
Medicare managed care	42 (4.4)	28 (5.8)	14 (3.0)
Medicare	113 (11.9)	50 (10.4)	63 (13.3)
Medicaid managed care	21 (2.2)	8 (1.7)	13 (2.8)
Medicaid	90 (9.4)	50 (10.4)	40 (8.5)
Both Medicare and Medicaid	41 (4.3)	18 (3.7)	23 (4.9)
Don't know	16 (1.7)	5 (1.0)	11 (2.3)
Other	26 (2.7)	11 (2.3)	15 (3.2)
Children's insurance, N (%)				0.77
None	31 (3.3)	14 (2.9)	17 (3.6)
Private	601 (63.1)	310 (64.4)	291 (61.7)
Medicaid managed care	80 (8.4)	44 (9.1)	36 (7.8)
Medicaid	190 (19.9)	89 (18.5)	101 (21.4)
Don't know	17 (1.8)	8 (1.7)	9 (1.9)
Other	32 (3.4)	15 (3.1)	17 (3.6)
Number of children
Overall	1,879	977	902
Mean (SD)	2.1 (1.1)	2.1 (1.2)	2.0 (1.1)	0.11
Had at least one child <2 y	172	90	82	0.72
Had at least one child 2–5 y	401	196	205	0.57
Had at least one child 6–11 y	579	300	279	0.54
Had at least one child 12–18 y	520	275	245	0.43
Had at least one child >18 y	207	116	91	0.49
Caregiver only; no children	39 (4.1%)	19 (4.0%)	20 (4.2%)
Knowledge and skills questions
How confident are you that you know your youngest child's current weight?				0.83
Very	501 (52.7)	255 (53.1)	246 (52.2)
Somewhat	386 (40.6)	191 (39.8)	195 (41.4)
Not at all	64 (6.7)	34 (7.1)	30 (6.4)
Has “medical training or experience”	240 (25.2)	124 (25.8)	116 (24.6)	0.67
Has given children medications often/very often in last 3 mo	251 (25.2)	121 (25.1)	120 (25.5)	0.14
Has used kitchen spoons or other nonstandard dosing for children	313 (32.9)	171 (35.6)	142 (30.2)	0.07
Health literacy, N (%)				0.17
Adequate	542 (57.0)	258 (53.8)	262 (55.6)
Marginal	203 (21.3)	104 (21.7)	99 (21.0)
Inadequate	206 (21.7)	96 (20.0)	110 (23.4)
Health numeracy, N (%)				0.38
High	235 (24.7)	126 (26.3)	109 (23.1)
Adequate	430 (45.2)	207 (43.1)	223 (47.3)
Low	286 (30.1)	147 (30.6)	139 (29.5)

Abbreviation: SD, standard deviation.

Note: Medicaid and Medicaid managed care are public insurance programs in the United States available only to low-income individuals and families. Medicare and Medicare managed care are public insurance programs in the United States available to all individuals over age 65 as well as to younger people with certain severe disabilities or chronic kidney failure.

As shown in [Table 3], the revised instructions were associated with a 1.1-point absolute or 13.3% relative improvement in the medication comprehension score (from 8.3 to 9.4 correct out of 15 questions; p < 0.01). In secondary outcomes, the intervention reduced the likelihood of responses that would have led to medication underdoses but not likelihood of selecting responses associated with overdoses ([Table 3], rows 2 and 3). For the individual comprehension questions, the revised instructions significantly improved the likelihood of selecting the correct answer for seven of the questions, significantly reduced the likelihood of selecting the correct answer for two questions, and made no difference for the remaining six questions ([Table 3]).

Table 3 Medication instruction comprehension results

	Whole group	Version 1 (usual care)	Version 2 (intervention)	p-Value
Mean (SD) correct answers	8.9 (3.8)	8.3 (3.8)	9.4 (3.7)	<0.01a
Mean (SD) answer with risk of overdose	2.7 (1.8)	2.6 (1.8)	2.7 (1.8)	0.68
Mean (SD) answer with risk of underdose	2.7 (2.5)	3.2 (2.5)	2.1 (2.4)	<0.01a
Correct answers
Question 1 (how many pills in one day), N (%)	650 (68.2)	276 (57.4)	374 (79.2)	<0.01a
Question 2 (how many pills in the morning), N (%)	701 (73.6)	303 (63.0)	398 (84.3)	<0.01a
Question 3 (how many drops in one day), N (%)	496 (52.0)	192 (39.9)	304 (64.4)	<0.01a
Question 4 (how many drops in the morning), N (%)	569 (59.7)	244 (50.7)	325 (68.9)	<0.01a
Question 5 (how many doses in one day), N (%)	626 (65.7)	308 (64.0)	318 (67.4)	0.28
Question 6 (how many pills in one week), N (%)	629 (66.0)	314 (65.3)	315 (66.7)	0.63
Question 7 (how much powder to make 6 ounces of formula), N (%)	705 (74.0)	328 (68.2)	377 (79.9)	<0.01a
Question 8 (what is the last day to use the cream), N (%)	126 (13.2)	60 (12.5)	66 (14.0)	0.49
Question 9 (should you give another dose now), N (%)	497 (52.2)	258 (53.6)	239 (50.6)	0.35
Question 10 (how much medicine for a 6-mo-old), N (%)	478 (50.2)	219 (45.5)	259 (54.9)	< 0.01a
Question 11 (when should your child eat breakfast), N (%)	505 (53.0)	291 (60.5)	214 (45.3)	<0.01a,b
Question 12 (how many drops right now), N (%)	502 (52.7)	253 (52.6)	249 (52.8)	0.96
Question 13 (how many times can you give the drops in a day), N (%)	638 (66.9)	268 (55.7)	370 (78.4)	<0.01a
Question 14 (alternating medicines: which medicine now), N (%)	698 (73.2)	344 (71.5)	354 (75.0)	0.22
Question 15 (alternating medicines: which medicine at 5 p.m.), N (%)	607 (63.7)	349 (72.6)	258 (54.7)	<0.01a,b

Abbreviation: SD, standard deviation.

Note: Findings marked with ^a are statistically significant at 0.05 in the hypothesized direction. Findings marked with ^b are statistically significant but in the direction opposite to the hypothesized direction.

Bivariate analyses showed that health numeracy was a very strong predictor of comprehension. Mean comprehension scores for individuals with high, adequate, and low numeracy were 12.0, 9.8, and 5.0, respectively. Health literacy was also a strong predictor, with mean scores for individuals with adequate, marginal, and inadequate literacy of 10.1, 8.0, and 6.4, respectively.

When we controlled for health numeracy and literacy in the multivariable analyses, none of the following demographic variables was statistically significant and therefore were dropped from the final model: parental age, race, ethnicity, education, household income, insurance status, being a self-reported frequent administrator of pediatric medications, and having medical training/experience. The final model (model R ² = 0.562) demonstrated that being a woman, having higher health numeracy, having higher health literacy, and receiving the revised instructions were all significantly and independently associated with improvements in comprehension score ranging from 0.4 points to ∼1 point ([Table 4]). Interaction terms between version and numeracy and version and literacy were not statistically significant, suggesting the effect of the revision was comparable in all literacy and numeracy levels.

Table 4 Multivariate regression predictors of instruction comprehension

Parameter	Estimate	Standard error	t Value	Pr > |t|
Intercept	2.65	0.27	9.67	<0.0001
Received Version 2 (intervention)	1.07	0.16	6.50	<0.0001
Female	1.06	0.17	6.17	<0.0001
Health numeracy score	1.14	0.04	26.31	<0.0001
Health literacy category	0.41	0.10	4.14	<0.0001

Note: The estimates in this linear model represent estimated changes (improvements) in total comprehension score associated with each variable. Variables not included in the final model after forward/backward stepwise selection were: parental age, race, ethnicity, education, household income, insurance status, being a self-reported frequent administrator of pediatric medications, and having medical training/experience.

Discussion

Several plain-language revisions have been individually demonstrated to improve comprehension. In the current experiment, we demonstrated that a package of five of these revisions employed together was associated with improved comprehension of common medication instructions. In addition, the experiment showed that health literacy and health numeracy were both independently associated with comprehension, and that the effect size associated with the plain-language revisions did not differ by literacy level or numeracy level. The package of revisions was associated with fewer wrong answers that would have led to underdoses but not overdoses. Although the total comprehension score was higher with the revisions, there was no effect for a subset of six questions, and comprehension was lower for a subset of two questions.

Our findings were consistent with many other studies showing that health literacy was associated with misunderstanding of instructions[1] [2] [3] [6] [64] and that text revisions can assist in interpretation.[65] [66] In addition, however, we demonstrated that health numeracy was independently related to comprehension, and that numeracy accounted for significantly more of the variability in performance than literacy did. Furthermore, other demographic variables were not associated with comprehension in models that controlled for health literacy and numeracy (with the exception of gender). We conclude that poor health numeracy may be an underrecognized predictor of poor comprehension and may in fact account for many of the previously observed demographic predictors of medication misinterpretations.[13] Numeracy is a particularly important factor given that low numeracy is more prevalent than low literacy and is found even among people with adequate literacy.[10] [67] [68] Of the individuals in our study with adequate health literacy, 15% had low health numeracy.

In addition to these primary and secondary findings, it is noteworthy that medication instructions disseminated to patients via commercial EHR technology were frequently misunderstood by a diverse sample of parents of all literacy and numeracy levels. For example, many parents did not appear to understand that instructions including the word “max” indicated an upper maximum threshold for the medication. No individual instruction was well understood by more than 72% of respondents. Consequently, it is possible that these instructions pose a safety threat to pediatric patients. Furthermore, only 53% of the parents were confident that they knew the current weight of their youngest child, and 33% reported using kitchen utensils or other nonstandard devices for pediatric medications. Such basic information would be needed for self-administered weight-based dosing (such as might be common with over-the-counter medications), and about basic safety precautions about measuring pediatric medications. Our sample reported a very high rate of using nonstandard devices for pediatric medications, a practice that has previously been shown to increase the rate of medication errors.[22] Other studies have found that between 6 and 23% of parents used or described using nonstandard instruments.[5] [22] [61]

The effect size associated with this intervention was modest. However, as a supplement to more intensive high-touch interventions, we propose that revising EHR output to replace complex language with simpler language is a potentially scalable solution that could reduce medication administration errors by parents. Some of these highly effective high-touch interventions include providing patients with customized medication administration tools,[36] [37] consultations with dedicated medication nurses,[69] and illustrated or diagrammed instructions.[30] [32] [33] [52] [70]

Although the revised instructions were associated with comprehension scores ∼8 percentage points higher overall (from 55 to 63%), the effect size was not equal for each instruction. The revisions were associated with a significantly higher rate of correct answers for seven of the component questions and a significantly significant lower rate of correct answers for two of the component questions.

Limitations of the Study

This study should be interpreted in light of several limitations. Testing five plain-language revisions at once means that conclusions can be drawn only about the package of all five, not about the relative efficacy of each type of revision. The online-only format, although it produced a demographically diverse population with a range of education and literacy levels, probably excluded individuals in the very lowest computer literacy and/or literacy categories. It is possible, therefore, that real-world comprehension is actually worse than what we found in our sample. Also, many of the medication instructions used in this study were drawn from an EHR system that had already undergone in-house customization to replace abbreviations (e.g., “2 × ” was already automatically replaced with “2 times”). It is possible that the effect of our intervention would be even greater if the revised instructions were compared with the prescribing provider's original instructions.

The study was administered in English only. Black and Hispanic patients were underrepresented (9.5% of our sample was black compared with ∼13% of the U.S. population, and 11.8% were Hispanic compared with ∼17% of the U.S. population). A large number of respondents reported some medical experience or training. We specifically developed the inclusion criteria to include any adult caregiver of a child, and the demographics suggest the possibility that the sample may have included grandparents. However, we did not collect data that would have allowed us to break down the results by whether the respondent was a parent or another type of a caregiver. Because we were testing a hypothesis about application of plain-language guidelines, we did not pretest the questionnaire for optimization before the survey, which may have contributed to the situations in which revised instructions reduced comprehension. A final limitation is that we used hypothetical questions only; generalizability to actual medication administration is not known. However, it seems possible that parents of sick children in reality might perform worse than they did in this relatively low-stress questionnaire study.

Conclusion

Misinterpretations of pediatric medication instructions commonly provided to patients are frequent. Simple language revisions, most of which could be implemented in the EHR without the need for additional formatting or graphics, were associated with reduced frequency of misinterpretations overall, although not for every instruction. Revising EHR output through automated substitutions could be a scalable solution that would reduce the number of parents who misinterpret pediatric medication instructions. However, instructions were still subject to misinterpretation even after the revision, so this approach should be considered a low-effort supplement to more intensive high-touch interventions that could reduce parental medication administration errors. Furthermore, additional usability and literacy testing will be required to fully develop such an intervention and test it in practice. Future testing would be most beneficial if it focused specifically on the most vulnerable groups such as individuals with limited health literacy or English proficiency.

Clinical Relevance Statement

Patients derive important information from EHR-generated documents. A relatively simple intervention of replacing complex text with patient-friendly text reduced misinterpretations that would be likely to lead to medication mistakes. The intervention tested here could be automated with relatively simple phrase substitution.

Multiple Choice Questions

1. Limited health numeracy, which is more prevalent than limited health literacy, is best defined as poor ability to:

   • Conduct statistical analysis of health data

   • Apply quantitative information to health decisions

   • Interpret peer-reviewed medical journal articles

   • Apply written and oral information to health decisions

   Correct answer: The correct answer is B, apply quantitative information to health decisions. Health numeracy is the set of skills and knowledge that patients need to read, understand, and apply quantitative information to personal health decisions. Health numeracy is not used to describe the more advanced set of quantitative skills used by physicians and scientists to analyze data or stay informed about the peer-reviewed literature. Health literacy similarly describes a broad set of skills and knowledge that patients need to read, understand, and apply written, textual, and oral information to their health; some authors consider health numeracy to be a subset of health literacy.

2. Patients with limited health literacy have difficulty comprehending medication instructions. Examples of interventions that have been associated with significant improvements in comprehension by low-literacy populations include:

   • Medication instructions revised in plain language

   • Health literacy coursework for patients

   • Training on effective prescription writing for physicians

   • Fully automated consumer-friendly translations of instructions

   Correct answer: The correct answer is A, medication instructions revised in plain language. This study adds new evidence to an already extensive literature on revising medication instructions to improve patient comprehension. Health literacy coursework, and effective prescription writing training, both appear to be useful ideas but have not been demonstrated to improve medication comprehension by patients. Also, no fully automated method has yet been demonstrated to improve patient comprehension of medication instructions.

Appendix

Conflict of Interest

None.

Acknowledgments

The researchers thank Sana Ali for assisting with the questionnaire programming.

Protection of Human and Animal Subjects

The study was reviewed by the Weill Cornell Institutional Review Board and was considered exempt because no personally identifying information was collected from the participants.

Funding

This study was supported by a pilot grant from the Department of Healthcare Policy & Research. Dr. Ancker is supported by K01 HS 021531 from the Agency for Healthcare Research and Quality.

• References

• 1 Davis TC, Federman AD, Bass III PF. , et al. Improving patient understanding of prescription drug label instructions. J Gen Intern Med 2009; 24 (01) 57-62
• 2 Lokker N, Sanders L, Perrin EM. , et al. Parental misinterpretations of over-the-counter pediatric cough and cold medication labels. Pediatrics 2009; 123 (06) 1464-1471
• 3 Bailey SC, Pandit AU, Yin S. , et al. Predictors of misunderstanding pediatric liquid medication instructions. Fam Med 2009; 41 (10) 715-721
• 4 Waldrop-Valverde D, Osborn CY, Rodriguez A, Rothman RL, Kumar M, Jones DL. Numeracy skills explain racial differences in HIV medication management. AIDS Behav 2010; 14 (04) 799-806
• 5 Yin HS, Dreyer BP, Foltin G, van Schaick L, Mendelsohn AL. Association of low caregiver health literacy with reported use of nonstandardized dosing instruments and lack of knowledge of weight-based dosing. Ambul Pediatr 2007; 7 (04) 292-298
• 6 Yin HS, Johnson M, Mendelsohn AL, Abrams MA, Sanders LM, Dreyer BP. The health literacy of parents in the United States: a nationally representative study. Pediatrics 2009; 124 (Suppl. 03) S289-S298
• 7 Yin HS, Mendelsohn AL, Wolf MS. , et al. Parents' medication administration errors: role of dosing instruments and health literacy. Arch Pediatr Adolesc Med 2010; 164 (02) 181-186
• 8 Kutner M, Greenberg E, Jin Y, Paulsen C, White S. The Health Literacy of America's Adults: Results from the 2003 National Assessment of Adult Literacy. Washington, DC: National Center for Health Statistics; 2006. . Contract No.: NCES 2006–483
• 9 McCray AT. Promoting health literacy. J Am Med Inform Assoc 2005; 12 (02) 152-163
• 10 Kutner M, Greenberg E, Baer J. A First Look at the Literacy of America's Adults in the 21st Century: National Assessment of Adult Literacy. Washington, DC: National Center for Education Statistics; 2005
• 11 Rudd R, Kirsch I, Yamamoto K. Literacy and Health in America. Princeton, NJ: Center for Global Assessment Policy Information Center Research and Development Educational Testing Service; 2004
• 12 Nielsen-Bohlman L, Panzer AM, Kindig DA. , eds. Committee on Health Literacy. Health Literacy: A Prescription to End Confusion. Washington, DC: Institute of Medicine, The National Academies Press; 2004
• 13 Ancker JS, Kaufman D. Rethinking health numeracy: a multidisciplinary literature review. J Am Med Inform Assoc 2007; 14 (06) 713-721
• 14 Lipkus IM, Samsa G, Rimer BK. General performance on a numeracy scale among highly educated samples. Med Decis Making 2001; 21 (01) 37-44
• 15 Ancker JS. Addressing health literacy and numeracy through systems approaches. In: Patel VL, Arocha JF, Ancker JS. , eds. Cognitive Informatics in Health and Biomedicine: Understanding and Modeling Health Behaviors. Cham: Springer International Publishing; 2017: 237-251
• 16 Golbeck AL, Ahlers-Schmidt CR, Paschal AM, Dismuke SE. A definition and operational framework for health numeracy. Am J Prev Med 2005; 29 (04) 375-376
• 17 Zandieh SO, Goldmann DA, Keohane CA, Yoon C, Bates DW, Kaushal R. Risk factors in preventable adverse drug events in pediatric outpatients. J Pediatr 2008; 152 (02) 225-231
• 18 Walsh KE, Kaushal R, Chessare JB. How to avoid paediatric medication errors: a user's guide to the literature. Arch Dis Child 2005; 90 (07) 698-702
• 19 Schillie SF, Shehab N, Thomas KE, Budnitz DS. Medication overdoses leading to emergency department visits among children. Am J Prev Med 2009; 37 (03) 181-187
• 20 Kang AM, Brooks DE. US poison control center calls for infants 6 months of age and younger. Pediatrics 2016; 137 (02) e20151865
• 21 Li SF, Lacher B, Crain EF. Acetaminophen and ibuprofen dosing by parents. Pediatr Emerg Care 2000; 16 (06) 394-397
• 22 Yin HS, Dreyer BP, Ugboaja DC. , et al. Unit of measurement used and parent medication dosing errors. Pediatrics 2014; 134 (02) e354-e361
• 23 Mehndiratta S. Strategies to reduce medication errors in pediatric ambulatory settings. J Postgrad Med 2012; 58 (01) 47-53
• 24 Johnson KB, Butta JK, Donohue PK, Glenn DJ, Holtzman NA. Discharging patients with prescriptions instead of medications: sequelae in a teaching hospital. Pediatrics 1996; 97 (04) 481-485
• 25 Simon HK, Weinkle DA. Over-the-counter medications. Do parents give what they intend to give?. Arch Pediatr Adolesc Med 1997; 151 (07) 654-656
• 26 Patel VL, Branch T, Arocha JF. Errors in interpreting quantities as procedures: the case of pharmaceutical labels. Int J Med Inform 2002; 65 (03) 193-211
• 27 Carayon P, Schoofs Hundt A, Karsh B-T. , et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care 2006; 15 (Suppl. 01) i50-i58
• 28 Carayon P. , ed. Human Factors and Ergonomics in Healthcare and Patient Safety. 2nd ed. Boca Raton, FL: CRC Press, Taylor & Francis Group; 2012
• 29 Rosson MB, Carroll JM. Usability Engineering: Scenario-based Development of Human-Computer Interaction. San Francisco: Morgan Kaufmann; 2002
• 30 Moore JO, Boyer EW, Safren S. , et al. Designing interventions to overcome poor numeracy and improve medication adherence in chronic illness, including HIV/AIDS. J Med Toxicol 2011; 7 (02) 133-138
• 31 Fogarty L, Roter D, Larson S, Burke J, Gillespie J, Levy R. Patient adherence to HIV medication regimens: a review of published and abstract reports. Patient Educ Couns 2002; 46 (02) 93-108
• 32 Machtinger EL, Wang F, Chen LL, Rodriguez M, Wu S, Schillinger D. A visual medication schedule to improve anticoagulation control: a randomized, controlled trial. Jt Comm J Qual Patient Saf 2007; 33 (10) 625-635
• 33 Kripalani S, Schmotzer B, Jacobson TA. Improving Medication Adherence through Graphically Enhanced Interventions in Coronary Heart Disease (IMAGE-CHD): a randomized controlled trial. J Gen Intern Med 2012; 27 (12) 1609-1617
• 34 Bailey SC, Wolf MS, Lopez A. , et al. Expanding the Universal Medication Schedule: a patient-centred approach. BMJ Open 2014; 4 (01) e003699
• 35 Yin HS, Dreyer BP, Moreira HA. , et al. Liquid medication dosing errors in children: role of provider counseling strategies. Acad Pediatr 2014; 14 (03) 262-270
• 36 McMahon SR, Rimsza ME, Bay RC. Parents can dose liquid medication accurately. Pediatrics 1997; 100 (3 Pt 1): 330-333
• 37 Frush KS, Luo X, Hutchinson P, Higgins JN. Evaluation of a method to reduce over-the-counter medication dosing error. Arch Pediatr Adolesc Med 2004; 158 (07) 620-624
• 38 Baker DW. The meaning and the measure of health literacy. J Gen Intern Med 2006; 21 (08) 878-883
• 39 Rudd RE. Improving Americans' health literacy. N Engl J Med 2010; 363 (24) 2283-2285
• 40 Osterberg L, Blaschke T. Adherence to medication. N Engl J Med 2005; 353 (05) 487-497
• 41 Nieuwlaat R, Wilczynski N, Navarro T. , et al. Interventions for enhancing medication adherence. Cochrane Database Syst Rev 2014; (11) CD000011
• 42 Workgroup to Evaluate and Enhance the Reach and Dissemination of Health Promotion Interventions. RE-AIM. Available at: http://re-aimorg . Accessed June 12, 2017
• 43 Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999; 89 (09) 1322-1327
• 44 Bakken S, Ruland CM. Translating clinical informatics interventions into routine clinical care: how can the RE-AIM framework help?. J Am Med Inform Assoc 2009; 16 (06) 889-897
• 45 Sullivan JE, Farrar HC. ; Section on Clinical Pharmacology and Therapeutics; Committee on Drugs. Fever and antipyretic use in children. Pediatrics 2011; 127 (03) 580-587
• 46 Shoemaker SJ, Wolf MS, Brach C. Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information. Patient Educ Couns 2014; 96 (03) 395-403
• 47 Leroy G, Endicott JE, Mouradi O, Kauchak D, Just ML. Improving perceived and actual text difficulty for health information consumers using semi-automated methods. AMIA Annu Symp Proc 2012; 2012: 522-531
• 48 Julian-Reynier C, Welkenhuysen M, Hagoel L, Decruyenaere M, Hopwood P. ; CRISCOM Working Group. Risk communication strategies: state of the art and effectiveness in the context of cancer genetic services. Eur J Hum Genet 2003; 11 (10) 725-736
• 49 www.plainlanguage.gov
• 50 Bailey SC, Hasnain-Wynia R, Chen AH. , et al. Developing multilingual prescription instructions for patients with limited English proficiency. J Health Care Poor Underserved 2012; 23 (01) 81-87
• 51 Bailey SC, Sarkar U, Chen AH, Schillinger D, Wolf MS. Evaluation of language concordant, patient-centered drug label instructions. J Gen Intern Med 2012; 27 (12) 1707-1713
• 52 Yin HS, Mendelsohn AL, Fierman A, van Schaick L, Bazan IS, Dreyer BP. Use of a pictographic diagram to decrease parent dosing errors with infant acetaminophen: a health literacy perspective. Acad Pediatr 2011; 11 (01) 50-57
• 53 Ancker JS, Senathirajah Y, Kukafka R, Starren JB. Design features of graphs in health risk communication: a systematic review. J Am Med Inform Assoc 2006; 13 (06) 608-618
• 54 Kincaid J, Fishburne R, Rogers R, Chissom B. Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy enlisted personnel. Research Branch Report 8–75. Memphis: Naval Air Station; 1975
• 55 McLaughlin GH. SMOG grading: a new readability formula. J Read 1969; 12 (08) 639-646
• 56 Leroy G, Kauchak D. The effect of word familiarity on actual and perceived text difficulty. J Am Med Inform Assoc 2014; 21 (e1): e169-e172
• 57 Zarcadoolas C. The simplicity complex: exploring simplified health messages in a complex world. Health Promot Int 2011; 26 (03) 338-350
• 58 Zarcadoolas C, Pleasant A, Greer DS. Understanding health literacy: an expanded model. Health Promot Int 2005; 20 (02) 195-203
• 59 Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Fam Med 2004; 36 (08) 588-594
• 60 Morris NS, MacLean CD, Chew LD, Littenberg B. The Single Item Literacy Screener: evaluation of a brief instrument to identify limited reading ability. BMC Fam Pract 2006; 7 (21) 21
• 61 Wallace LS, Keenum AJ, DeVoe JE, Bolon SK, Hansen JS. Women's understanding of different dosing instructions for a liquid pediatric medication. J Pediatr Health Care 2012; 26 (06) 443-450
• 62 Schapira MM, Walker CM, Miller T. , et al. Development and validation of the numeracy understanding in Medicine Instrument short form. J Health Commun 2014; 19 (Suppl. 02) 240-253
• 63 Schapira MM, Walker CM, Cappaert KJ. , et al. The numeracy understanding in medicine instrument: a measure of health numeracy developed using item response theory. Med Decis Making 2012; 32 (06) 851-865
• 64 Davis TC, Wolf MS, Bass III PF. , et al. Literacy and misunderstanding prescription drug labels. Ann Intern Med 2006; 145 (12) 887-894
• 65 Wolf MS, Davis TC, Curtis LM. , et al. A patient-centered prescription drug label to promote appropriate medication use and adherence. J Gen Intern Med 2016; 31 (12) 1482-1489
• 66 Wolf MS, Bailey SC, Serper M. , et al. Comparative effectiveness of patient-centered strategies to improve FDA medication guides. Med Care 2014; 52 (09) 781-789
• 67 Rampey BD, Finnegan R, Goodman M. , et al. Skills of US Unemployed, Young, and Older Adults in Sharper Focus: Results from the Program for the International Assessment of Adult Competencies (PIAAC) 2012/2014. Washington, DC: National Center for Education Statistics, U.S. Department of Education; 2016. . Contract No.: NCES 2016–039
• 68 Ancker JS, Weber EU, Kukafka R. Effects of game-like interactive graphics on risk perceptions and decisions. Med Decis Making 2011; 31 (01) 130-142
• 69 Greengold NL, Shane R, Schneider P. , et al. The impact of dedicated medication nurses on the medication administration error rate: a randomized controlled trial. Arch Intern Med 2003; 163 (19) 2359-2367
• 70 Yin HS, Dreyer BP, van Schaick L, Foltin GL, Dinglas C, Mendelsohn AL. Randomized controlled trial of a pictogram-based intervention to reduce liquid medication dosing errors and improve adherence among caregivers of young children. Arch Pediatr Adolesc Med 2008; 162 (09) 814-822