Using Electronic Health Records to Identify Adverse Drug Events in Ambulatory Care: A Systematic Review

Abstract

Objective We identified the methods used and determined the roles of electronic health records (EHRs) in detecting and assessing adverse drug events (ADEs) in the ambulatory setting.

Methods We performed a systematic literature review by searching PubMed and Google Scholar for studies on ADEs detected in the ambulatory setting involving any EHR use published before June 2017. We extracted study characteristics from included studies related to ADE detection methods for analysis.

Results We identified 30 studies that evaluated ADEs in an ambulatory setting with an EHR. In 27 studies, EHRs were used only as the data source for ADE identification. In two studies, the EHR was used as both a data source and to deliver decision support to providers during order entry. In one study, the EHR was a source of data and generated patient safety reports that researchers used in the process of identifying ADEs. Methods of identification included manual chart review by trained nurses, pharmacists, and/or physicians; prescription review; computer monitors; electronic triggers; International Classification of Diseases codes; natural language processing of clinical notes; and patient phone calls and surveys. Seven studies provided examples of search phrases, laboratory values, and rules used to identify ADEs.

Conclusion The majority of studies examined used EHRs as sources of data for ADE detection. This retrospective approach is appropriate to measure incidence rates of ADEs but not adequate to detect preventable ADEs before patient harm occurs. New methods involving computer monitors and electronic triggers will enable researchers to catch preventable ADEs and take corrective action.

Keywords

review - drug-related side effects and adverse reactions - adverse drug reaction reporting systems - medication errors - medical records systems - computerized - ambulatory care

Background and Significance

Adverse drug events (ADEs) are anticipated and unanticipated side effects of taking certain medications.[1] These events occur both in the inpatient and outpatient settings, often leading to patient injury or death. However, previous research has focused on occurrences of ADEs in the inpatient setting. More recent work has characterized rates in the ambulatory setting to be estimated at anywhere between 3 and 38%.[2] [3] [4] [5] [6]

Reporting on the rates of occurrences has evolved with the implementation of electronic health records (EHRs) and the integration of computerized provider order entry (CPOE) with clinical decision support. Support for CPOE and its usefulness in reducing ADEs has been established in the inpatient setting.[7] Previously described ADE detection methods rely on the “gold standard” of manual chart review by physicians and other trained health professionals of medical notes, laboratory results, and changes made to prescriptions. This process takes considerable time and resources to perform on a large scale. Many ADEs that occur in the outpatient setting require further means of identification, especially if patients do not seek treatment for their symptoms. One approach to measuring ADEs that occur in the outpatient setting relies on patient surveys.[8] Recent studies have created tools such as electronic triggers and automated computer monitors to assist in detecting ADEs.[9] [10] [11] [12] These methods either use the EHR as a data source or have incorporated those tools as built-in functionality.

More than 4 billion prescriptions are filled per year in the outpatient setting.[2] For this reason, there is a need to examine the outpatient setting where ADEs are often difficult to detect due to underreporting and lack of treatment. Previous reviews have assessed overall rates of occurrence, but none have performed a detailed comparison on the methods of identification, especially in recent years, during which EHRs may have changed the way ADEs are detected. In this study, we aimed to identify the methods used and determine the roles of EHRs in detecting and assessing ADEs in the ambulatory setting through systematic review of published literature.

Methods

We searched the PubMed database and Google Scholar for studies published before June 2017 on ADEs detected in the ambulatory setting involving some degree of use of an EHR. We also used references cited as an additional means of identifying studies. The Medical Subject Headings terms searched in PubMed were (medication errors or adverse drug reaction reporting systems or drug therapy/adverse effects or drug-related side effects and adverse reactions or iatrogenic disease/drug therapy) and (emergency medical services or primary health care or patient admission or hospitalization or outpatients or ambulatory care or ambulatory care facilities or physicians, family or family practice) and (medical records systems, computerized or medication systems or software or ambulatory care information systems or drug therapy, computer-assisted or medical order entry systems or decision support systems, clinical). The keywords searched in Google Scholar were (adverse drug event OR medication error) AND (ambulatory OR outpatient OR primary care) AND (electronic health record OR electronic medical record). We limited the Google Scholar search to the top 100 results.

All abstracts were independently screened by two reviewers; abstracts determined to be potentially relevant by either reviewer were included for full-text analysis. Both reviewers then independently reviewed the full-text articles for inclusion in the final analysis. Differences were discussed and reconciled between the authors. We included peer-reviewed articles published in English from any country as long as the study aimed to measure in the incidence of ADEs in an ambulatory setting and incorporated use of an EHR. Studies were excluded if they did not measure ADEs in an ambulatory setting, measured ADEs but without use of an EHR, were systematic reviews or meta-analyses, were not complete with data and results, or not accessible as full text. From the full-text articles we manually extracted: study setting, study design, sample size, follow-up time, ADE detection methods, EHR role, ADE definitions, ADE prevalence, and limitations.

Results

We screened a total of 2,280 studies and reviewed 57 full-text articles. We included 30 studies in our final analysis. We excluded 2,250 studies that occurred in an inpatient setting, measured ADEs but without use of an EHR, were systematic reviews or meta-analyses, were not complete with data and results, or not accessible as full text ([Fig. 1]). All included studies are listed in [Supplementary Table S1] (available in the online version).

Fig. 1 Flow diagram of included studies.

Eighteen studies were conducted in the United States, 1 in Australia, 3 in Canada, 2 in France, 1 in Germany, 1 in New Zealand, 2 in Spain, 1 in Sweden, and 1 in Taiwan. Five studies were conducted in hospitals, 24 in ambulatory care, and 1 took place at both settings. Four studies were randomized controlled trials, 9 were prospective, and 17 were retrospective studies.

[Table 1] provides a summary of EHR use across included studies. In 27 studies, the EHR served as a data source for indications that an ADE had occurred. In two studies, the EHR was not only a source of data, but it also included clinical decision support that appeared while providers were ordering medications via CPOE. In the Genco et al 2016 study, the EHR was used as a source of data and had a role in analysis, as reports generated by the system were used by researchers to identify ADEs.[13]

Table 1 Summary of EHR use for included studies

Role of EHR	Number of studies	Citations
Source of data only	27	[2] [9] [14] [15] [16] [17] [18] [19] [21] [23] [24] [26] [27] [29] [30] [31] [32] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45]
Source of data, decision support	2	[20] [25]
Source of data, role in analysis	1	[13]

Abbreviation: EHR, electronic health record.

All included studies used a combination of methods to identify and characterize ADEs in the ambulatory setting. Methods included manual chart review by trained nurses, pharmacists, and/or physicians; prescription review; computer monitors; electronic triggers; International Classification of Diseases (ICD) codes; natural language processing of clinical notes; and patient phone calls and surveys. Two studies report using return visit to the emergency department (ED) or admission from the ED within 24 hours as their ADE trigger.[14] [15] None of the studies provided a comprehensive list of search phrases, laboratory values, or logic rules. Five studies utilized the Naranjo algorithm to determine the likelihood the ADE occurred as a result of a drug rather than other factors.[13] [16] [17] [18] [19] Two used the Beers Criteria in studies involving adults aged 65 years or older.[20] [21]

The most common role the EHR played was to act as a source of data for researchers to use to identify ADEs. EHRs took this passive role in 27 included studies. Instead of performing paper chart review, researchers searched through electronic charts for indicators of an ADE. Chart review was conducted by a combination of trained abstractors, research nurses, physicians, pharmacists, and toxicologists. In one study, the case file was reviewed by senior ED nurses, and if it was not rejected by them, the file was then passed over for review by emergency physicians.[15] In a French study, a committee of clinical pharmacologists, internists, and general practitioners conducted the case review.[22] In all cases, chart review was maintained as the “gold standard” of ADE identification, even in studies where a computer monitor or electronic trigger was used.[16]

Studies included varied approaches to searching the EHR for signs of an ADE. Laboratory values, clinical notes, and ICD codes were commonly used. Cantor et al searched free-text notes for trigger phrases indicative of ADEs.[23] Brenner et al instead identified six laboratory values, international normalized ratio > 5, serum creatinine > 2.5, blood urea nitrogen > 60, alanine aminotransferase > 84, aspartate aminotransferase > 80, and thyroid-stimulating hormone undetectable while on levothyroxine, which were used to determine at what stage of the medication process the ADE occurred.[9] [10] These laboratory values were adapted from a more comprehensive tool developed by Singh et al.[10] Only the laboratory values were used because they were shown to have a high positive predictive value, and researchers were able to extract the rest of the data associated with those values. Gandhi et al developed more sophisticated tools to search through laboratory values, medication lists, and applied logical rules to decide whether a potential ADE was present. This study also created a search monitor that worked off a set of rules to search the free-text electronic notes for symptom words that may have signaled an ADE.[24] A similar search tool was created by Honigman et al that looked through ICD-9 codes, allergy rules, computer event monitoring rules, and an automated chart review utilizing text searching of the EHR.[16]

In three studies, EHR use was more innovative.[13] [20] [25] In two of these studies, CPOE provided decision support in the form of alerts and pop-up notifications.[20] [25] In the Terrell et al study, clinical decision support was provided in a randomized controlled trial for nine medications that were determined to represent 80% of potentially inappropriate medications prescribed to seniors in the ED. The proportion of ED discharges that resulted in a potentially inappropriate medicated was 3.9% for the control group and 2.6% for the intervention group.[20] In the second study that included EHR as decision support, researchers only examined physicians' responses to dose-range alerts, but did not measure incidence of ADE occurrence due to prescription errors.[25] The third study by Genco et al utilized an EHR system that created data-based reports on patient safety, which were then included in the review process of identifying ADEs.[13]

Commonly described limitations across studies were primarily related to the dependence on EHRs as a true source of information. A lack of standard documentation practices between physicians and across practices would not be representative of the observed rate of ADE occurrences; missing relevant information in patient charts may have led to misclassification, and errors not documented in the EHR may not have been identified, leading to an underreporting of ADEs.[23] [26] [27] [28] [29] In the case of Brenner et al looking at laboratory value triggers, it was unclear from EHR documentation whether errors occurred due to a lack of monitoring or appeared in the course of adhering to recommended medication monitoring.[9] Study designs often focused on catching prescribing error and would miss preventable ADEs, such as those due to wrong patients receiving medications, an incorrect diagnosis causing the wrong medication to be prescribed, or a drug–drug interaction with other home medications.[17] [30] [31]

Studies also acknowledged that bias might have been introduced in cases where researchers and reviewing physicians were not blinded to the purpose of the study. Researchers might have been more careful in their reviews or excluded high-risk patients from the study, and providers may have been more careful when writing prescriptions.[25] [31] [32] In addition, Abramson et al noted that ADEs were best measured by chart review in combination with patient interviews or surveys, an approach that was not used in many of the included studies.[18]

Discussion

We identified 30 studies that met review specifications. We included studies that utilized EHRs in the study methods of identifying ADEs in the ambulatory setting. The majority of studies examined used EHRs as sources of data for chart review, replacing traditional paper chart review with electronic laboratory values and visit notes. Select studies highlight how researchers created computerized monitors and search tools that were able to comb through electronic patient data to identify ADEs.

EHR use has increased in recent years with the advancement of health information technology and Meaningful Use directives.[33] ADE research has spanned decades, and while prior studies have examined ADEs in the ambulatory setting, none have reported in detail the methods of identifying ADEs in the ambulatory setting with a focus on EHRs. Thomsen et al conducted a systematic review on incidences of ADEs in ambulatory care in 2007. While similar study search criteria were used, the previous review focused on characteristics of ADEs and not methods of identification, as in this study.[34] The ambulatory setting of our review highlights the unique challenge of outpatient identification of ADEs as compared with previous inpatient studies. Patients do not have close contact with their physicians, unlike in a hospital setting where physicians examine patients on a daily basis. They are responsible for obtaining and administering their own medications and do not maintain as thorough records as hospitals do, thus limiting the efficacy of retrospective chart review.[35] Examining methods of ADE identification is necessary to ensure researchers are obtaining the truest measure of ADEs in the ambulatory setting and that the methods can be reproduced in future research.

In this review, we found that EHRs were primarily used as sources of data for ADE detection. The majority of studies reviewed utilized a retrospective approach, which was useful to measure incidence rates of ADEs but not to catch preventable ADEs. Studies that created electronic tools capable of searching through the EHR to detect trigger phrases or laboratory values show promise of a transition away from complete manual chart review for the detection of ADEs. Manual chart review limits researchers and physicians to detecting ADEs after they have already happened. With computer monitors and electronic triggers searching the EHR in real time, health care providers may be able to identify preventable ADEs and take corrective action before patient harm occurs.

Future research is needed to measure consistency of documentation in various ambulatory settings, from large outpatient clinics to small primary care facilities. If there is a large disparity in patient data being documented, then study results will not be representative of the true measure of ambulatory ADEs.

This systematic review has some limitations. We searched the publicly available databases PubMed and Google Scholar, thus potentially excluding literature published in other sources. Inherent publication bias limited the number of articles available for review. Fewer studies were available in published literature on ADEs in the ambulatory setting. Because we were only interested in studies using EHRs, it limited us to looking at more recent studies, where EHR use became more prevalent. Due to the cost of EHRs, studies were most likely conducted at facilities that had the resources to afford an EHR, so results may not be generalizable.

Beyond the limitations of this review, EHRs are also limited by the information entered into them; a lack of standard documentation requirements may lead to incomplete charts that hinder ADE detection and monitoring. Medication reconciliation is another important step that needs to be completed during patient visits to ensure the med list is up-to-date in the system. Together with physician compliance, enhanced research tools that work with EHRs will enable researchers to better measure, characterize, and detect ADEs in the ambulatory setting.

Clinical Relevance Statement

ADEs are directly tied to patient outcomes and patient safety. Research in this area would benefit patients by improving ADE detection, providers by increasing their quality of care, and hospitals by decreasing ADE rates.

Multiple Choice Questions

1. What is the current most common use of EHRs in the detection of ADEs by researchers?

   • Decision support.

   • Role in analysis.

   • Source of data.

   • Options a and c.

   • Options b and c.

   Correct Answer: The correct answer is option c. As discussed in the review, the majority of studies selected used EHRs as sources of data in the process of detecting ADEs. In place of paper chart review, electronic review of data was done via EHR search.

2. What is the “gold standard” method of ADE detection?

   • Computerized trigger based on laboratory values.

   • Chart review.

   • Pharmacist review.

   • Patient description of symptoms.

   • Keyword detection in encounter notes.

   Correct Answer: The correct answer is option b. The “gold standard” is manual chart review, the method used by researchers before the advent of EHRs.

Conflict of Interest

None declared.

Protection of Human and Animal Subjects

No human and/or animal subjects were used in this review.

• References

• 1 Zafar A, Hickner J, Pace W, Tierney W. An adverse drug event and medication error reporting system for ambulatory care (MEADERS). AMIA Annu Symp Proc 2008; 2008: 839-843
• 2 Overhage JM, Gandhi TK, Hope C. , et al. Ambulatory computerized prescribing and preventable adverse drug events. J Patient Saf 2016; 12 (02) 69-74
• 3 Gandhi TK, Weingart SN, Borus J. , et al. Adverse drug events in ambulatory care. N Engl J Med 2003; 348 (16) 1556-1564
• 4 Gurwitz JH, Field TS, Harrold LR. , et al. Incidence and preventability of adverse drug events among older persons in the ambulatory setting. JAMA 2003; 289 (09) 1107-1116
• 5 Hanlon JT, Schmader KE, Koronkowski MJ. , et al. Adverse drug events in high risk older outpatients. J Am Geriatr Soc 1997; 45 (08) 945-948
• 6 Hutchinson TA, Flegel KM, Kramer MS, Leduc DG, Kong HH. Frequency, severity and risk factors for adverse drug reactions in adult out-patients: a prospective study. J Chronic Dis 1986; 39 (07) 533-542
• 7 Bates DW. Using information technology to reduce rates of medication errors in hospitals. BMJ 2000; 320 (7237): 788-791
• 8 Roulet L, Asseray N, Dary M, Chiffoleau A, Potel G, Ballereau F. Implementing a clinical pharmacy survey of adverse drug events in a French emergency department. Int J Clin Pharm 2012; 34 (06) 902-910
• 9 Brenner S, Detz A, López A, Horton C, Sarkar U. Signal and noise: applying a laboratory trigger tool to identify adverse drug events among primary care patients. BMJ Qual Saf 2012; 21 (08) 670-675
• 10 Singh R, McLean-Plunckett EA, Kee R. , et al. Experience with a trigger tool for identifying adverse drug events among older adults in ambulatory primary care. Qual Saf Health Care 2009; 18 (03) 199-204
• 11 Kane-Gill SL, MacLasco AM, Saul MI. , et al. Use of text searching for trigger words in medical records to identify adverse drug reactions within an intensive care unit discharge summary. Appl Clin Inform 2016; 7 (03) 660-671
• 12 Boyce RD, Jao J, Miller T, Kane-Gill SL. Automated screening of emergency department notes for drug-associated bleeding adverse events occurring in older adults. Appl Clin Inform 2017; 8 (04) 1022-1030
• 13 Genco EK, Forster JE, Flaten H. , et al. Clinically inconsequential alerts: the characteristics of opioid drug alerts and their utility in preventing adverse drug events in the emergency department. Ann Emerg Med 2016; 67 (02) 240-248
• 14 Zhang E, Hung S-C, Wu C-H, Chen L-L, Tsai M-T, Lee W-H. Adverse event and error of unexpected life-threatening events within 24hours of ED admission. Am J Emerg Med 2017; 35 (03) 479-483
• 15 Calder L, Pozgay A, Riff S. , et al. Adverse events in patients with return emergency department visits. BMJ Qual Saf 2015; 24 (02) 142-148
• 16 Honigman B, Lee J, Rothschild J. , et al. Using computerized data to identify adverse drug events in outpatients. J Am Med Inform Assoc 2001; 8 (03) 254-266
• 17 Abramson EL, Barrón Y, Quaresimo J, Kaushal R. Electronic prescribing within an electronic health record reduces ambulatory prescribing errors. Jt Comm J Qual Patient Saf 2011; 37 (10) 470-478
• 18 Abramson EL, Pfoh ER, Barrón Y, Quaresimo J, Kaushal R. The effects of electronic prescribing by community-based providers on ambulatory medication safety. Jt Comm J Qual Patient Saf 2013; 39 (12) 545-552
• 19 Sevilla-Sanchez D, Molist-Brunet N, Amblàs-Novellas J, Roura-Poch P, Espaulella-Panicot J, Codina-Jané C. Adverse drug events in patients with advanced chronic conditions who have a prognosis of limited life expectancy at hospital admission. Eur J Clin Pharmacol 2017; 73 (01) 79-89
• 20 Terrell KM, Perkins AJ, Dexter PR, Hui SL, Callahan CM, Miller DK. Computerized decision support to reduce potentially inappropriate prescribing to older emergency department patients: a randomized, controlled trial. J Am Geriatr Soc 2009; 57 (08) 1388-1394
• 21 Buck MD, Atreja A, Brunker CP. , et al. Potentially inappropriate medication prescribing in outpatient practices: prevalence and patient characteristics based on electronic health records. Am J Geriatr Pharmacother 2009; 7 (02) 84-92
• 22 Bénard-Laribière A, Miremont-Salamé G, Pérault-Pochat M-C, Noize P, Haramburu F. ; EMIR Study Group on behalf of the French Network of pharmacovigilance centres. Incidence of hospital admissions due to adverse drug reactions in France: the EMIR study. Fundam Clin Pharmacol 2015; 29 (01) 106-111
• 23 Cantor MN, Feldman HJ, Triola MM. Using trigger phrases to detect adverse drug reactions in ambulatory care notes. Qual Saf Health Care 2007; 16 (02) 132-134
• 24 Gandhi TK, Seger AC, Overhage JM. , et al. Outpatient adverse drug events identified by screening electronic health records. J Patient Saf 2010; 6 (02) 91-96
• 25 Sethuraman U, Kannikeswaran N, Murray KP, Zidan MA, Chamberlain JM. Prescription errors before and after introduction of electronic medication alert system in a pediatric emergency department. Acad Emerg Med 2015; 22 (06) 714-719
• 26 Hanlon JT, Sloane RJ, Pieper CF, Schmader KE. Association of adverse drug reactions with drug-drug and drug-disease interactions in frail older outpatients. Age Ageing 2011; 40 (02) 274-277
• 27 Tomlin A, Reith D, Dovey S, Tilyard M. Methods for retrospective detection of drug safety signals and adverse events in electronic general practice records. Drug Saf 2012; 35 (09) 733-743
• 28 Gonzales R, Anderer T, McCulloch CE. , et al. A cluster randomized trial of decision support strategies for reducing antibiotic use in acute bronchitis. JAMA Intern Med 2013; 173 (04) 267-273
• 29 Walsh KE, Dodd KS, Seetharaman K. , et al. Medication errors among adults and children with cancer in the outpatient setting. J Clin Oncol 2009; 27 (06) 891-896
• 30 Nanji KC, Rothschild JM, Salzberg C. , et al. Errors associated with outpatient computerized prescribing systems. J Am Med Inform Assoc 2011; 18 (06) 767-773
• 31 Gandhi TK, Weingart SN, Seger AC. , et al. Outpatient prescribing errors and the impact of computerized prescribing. J Gen Intern Med 2005; 20 (09) 837-841
• 32 Beaudoin FL, Merchant RC, Janicki A, McKaig DM, Babu KM. Preventing iatrogenic overdose: a review of in-emergency department opioid-related adverse drug events and medication errors. Ann Emerg Med 2015; 65 (04) 423-431
• 33 Henry J, Pylypchuk Y, Searcy T, Patel V. . Adoption of Electronic Health Record Systems among U.S. Non-Federal Acute Care Hospitals: 2008–2015. ONC Data Brief, no. 35. Office of the National Coordinator for Health Information Technology: Washington DC; 2016
• 34 Thomsen LA, Winterstein AG, Søndergaard B, Haugbølle LS, Melander A. Systematic review of the incidence and characteristics of preventable adverse drug events in ambulatory care. Ann Pharmacother 2007; 41 (09) 1411-1426
• 35 O'Neil AC, Petersen LA, Cook EF, Bates DW, Lee TH, Brennan TA. Physician reporting compared with medical-record review to identify adverse medical events. Ann Intern Med 1993; 119 (05) 370-376
• 36 Elliott RA, Lee CY, Beanland C, Vakil K, Goeman D. Medicines management, medication errors and adverse medication events in older people referred to a community nursing service: a retrospective observational study. Drugs Real World Outcomes 2016; 3: 13-24
• 37 Forster AJ, Erlanger TE, Jennings A. , et al. Effectiveness of a computerized drug-monitoring program to detect and prevent adverse drug events and medication non-adherence in outpatient ambulatory care: study protocol of a randomized controlled trial. Trials 2015; 8 (16) 2
• 38 Dainty KN, Adhikari NKJ, Kiss A, Quan S, Zwarenstein M. Electronic prescribing in an ambulatory care setting: a cluster randomized trial. J Eval Clin Pract 2012; 18 (04) 761-767
• 39 Olivier P, Bertrand L, Tubery M, Lauque D, Montastruc J-L, Lapeyre-Mestre M. Hospitalizations because of adverse drug reactions in elderly patients admitted through the emergency department: a prospective survey. Drugs Aging 2009; 26 (06) 475-482
• 40 Bénard-Laribière A, Miremont-Salamé G, Pérault-Pochat M-C, Noize P, Haramburu F. ; the EMIR Study Group on behalf of the French network of pharmacovigilance centres. Incidence of hospital admissions due to adverse drug reactions in France: the EMIR study. Fundam Clin Pharmacol 2015; 29 (01) 106-111
• 41 Sönnichsen A, Trampisch US, Rieckert A. , et al. Polypharmacy in chronic diseases-Reduction of Inappropriate Medication and Adverse drug events in older populations by electronic Decision Support (PRIMA-eDS): study protocol for a randomized controlled trial. Trials 2016; 17: 57
• 42 González-Rubio F, Calderón-Larrañaga A, Poblador-Plou B, Navarro-Pemán C, López-Cabañas A, Prados-Torres A. Underreporting of recognized adverse drug reactions by primary care physicians: an exploratory study. Pharmacoepidemiol Drug Saf 2011; 20 (12) 1287-1294
• 43 Hedna K, Hakkarainen KM, Gyllensten H, Jönsson AK, Petzold M, Hägg S. Potentially inappropriate prescribing and adverse drug reactions in the elderly: a population-based study. Eur J Clin Pharmacol 2015; 71 (12) 1525-1533
• 44 Armor BL, Wight AJ, Carter SM. Evaluation of adverse drug events and medication discrepancies in transitions of care between hospital discharge and primary care follow-up. J Pharm Pract 2016; 29 (02) 132-137
• 45 Lopez-Picazo JJ, Ruiz JC, Sanchez JF, Ariza A, Aguilera B. A randomized trial of the effectiveness and efficiency of interventions to reduce potential drug interactions in primary care. Am J Med Qual 2011; 26 (02) 145-153