Health System Implementation of a Tobacco Quitline eReferral

Abstract

Background Proactive referrals through electronic orders (eReferrals) can increase patient connection with tobacco quitlines. More information is needed on “real-world” implementation of electronic health record tools to promote tobacco cessation while minimizing provider burden.

Objectives This paper examines the health system implementation of an eReferral to a tobacco quitline without best practice alerts in primary care, specialty, and hospital settings in an academic health system.

Methods This is a prospective implementation study of a health system tobacco eReferral to a state quitline that was completed with an approach to minimize provider cognitive burden. Data are drawn from electronic health record data at University of California, Davis Health Systems (March 2013–February 2016).

Results Over 3 years, 16,083 encounters with smokers resulted in 1,137 eReferral orders (7.1%). Treatment reach was 1.6% for quitline services and 2.3% for outpatient group classes. While the group classes were offered to outpatient smokers, the eReferral order was included in an outpatient order set and eventually an automated inpatient discharge order set; no provider alerts were implemented. Referrals were sustained and doubled after inpatient order set implementation. Among all first time eReferral patients, 12.2% had a 6 to 12 month follow-up visit at which they were documented as nonsmoking.

Conclusion This study demonstrates a quitline eReferral order can be successfully implemented and sustained with minimal promotion, without provider alerts and in conjunction with group classes. Reach and effectiveness were similar to previously described literature.

Keywords

specific condition: tobacco use cessation - electronic health record - implementation science - preventive medicine - adoption

Background and Significance

Referrals to tobacco cessation resources are an important prevention and treatment tool, part of the “5 A's” or ask–advise–refer approaches to assist smokers.[1] [2] However, resources like group classes or tobacco treatment specialists may not be feasible for all health care systems.[1] [3] Tobacco quitlines are an effective resource available nationwide that can improve long-term cessation rates.[4] [5] [6] Electronic health records (EHRs) are a burgeoning tool for providers to connect patients to tobacco cessation resources.[7] [8] Proactive referrals to a quitline using an EHR (hereafter “eReferrals”) may increase a patient's connection and enrollment into quitline services by 13-fold.[9] In “bidirectional” eReferrals, the outcomes of the quitline are reported back to the health system's ordering provider and have higher referral rates than traditional fax-based referrals.[10] [11] [12]

Most eReferral studies relied on team-based models and are focused on either inpatient or outpatient implementations rather than changes across a health care system or in specialty settings.[9] [13] [14] [15] [16] Specialty services like oncology are important settings for promoting tobacco cessation.[17] When implementing new preventive strategies, best practice alerts (BPAs) are often recommended,[18] yet these may cause provider “alert fatigue.”[19] More information is needed on “real-world” implementation of EHR tools to promote tobacco cessation in diverse clinical settings while minimizing provider burden.[7] [8] [20] [21] Uses of the EHR to promote tobacco cessation resources in the absence of tobacco treatment specialists or team-based care may inform future implementation of tobacco-cessation programs.[22] [23]

In 2013, California's first bidirectional quitline eReferral was launched between University of California, Davis Health Systems (UCD), and the California Smokers' Helpline (Helpline). The eReferral was implemented without a team-based model and without BPAs. The purpose of this study is to examine the “real-world” implementation of California's first bidirectional eReferral system.

Methods

Study Setting and Participants

This prospective study examines the 2013 implementation of a Helpline eReferral order at UCD. UCD is an academic medical center with almost 1 million annual encounters in the primary care (14 offices at the time in 10 area communities), specialty care, emergency, and inpatient settings with approximately 1,600 active clinicians at the time of this study. This project was considered exempt by the UCD Institutional Review Board as it did not involve human patients' research.

Description of UCD eReferral to Helpline

UCD has used Epic Systems Corporation software (“Epic”) for its EHR since 2006. There was no prior integrated UCD referral system. Prior to this study, primary care medical assistants were trained on meaningful use requirements to document smoking status for patients in the EHR. The eReferral order is an outpatient order available across all clinics and the medical center at hospital discharge. The eReferral order workflow follows a bidirectional pathway. Once a patient verbally consents, the provider electronically places an eReferral order which takes about 1 minute or less. The patient's information is automatically sent to the Helpline through a bidirectional interface using standard Health Level Seven (HL7) V.2 messages. The outcomes of the Helpline's encounter with the patient are sent back to the ordering provider's inbox and can be reviewed within the patient's medical record in the EHR. The Helpline contacts the patient within 1 to 2 days, conducts an intake questionnaire, and offers counseling services. All California Helpline users are eligible for counseling services.

Data Sources

Data sources included UCD EHR and Helpline caller data for orders placed between March 2013 and February 2016. During the study-time period, 16,083 patient encounters included a diagnostic code of current tobacco use, 1,974 were during an inpatient admission, and 14,109 were during an outpatient encounter. The Helpline provided monthly eReferral aggregate contact and service outcome reports. Mortality was not assessed; patients who received an eReferral were included in analyses without regard to vital status.

As seen in [Fig. 1], the UCD EHR data had 1,198 eReferrals, 49 orders which were not included because they were canceled, leaving 1,149 eReferrals. The Helpline reported 1,137 total UCD eReferrals during the same time period. The Helpline caller data were used to describe how many patients were contacted. The source of the discrepancy of 12 eReferrals is unknown but may reflect user error.

Fig. 1 First eReferral order count (March 2013–February 2016).

Study Design and Measures

The Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) conceptual framework[23] [24] was used to evaluate the implementation of the UCD eReferral system. RE-AIM consists of the following five components and assesses implementation from both a patient and system perspective: (1) reach—the proportion of the target population who participate in an intervention, (2) effectiveness—the effect of an intervention on outcomes of interest, (3) adoption—the proportion of those who implement the intervention on an individual and setting level, (4) implementation—whether the intervention was delivered as intended, and (5) maintenance—the institutionalization of the intervention. In the results, implementation is presented first to provide context for the other components.

Four measures were used to describe reach of the eReferral, as defined by the North American Quitline Consortium[25] as follows: 1) contact rate—the proportion who were successfully contacted by the Helpline, among those referred; (2) rate of accepting services—proportion who accepted any services (e.g., intake and mailed educational booklet) from the Helpline, among those referred; (3) treatment reach—proportion who accepted counseling services, among those who had an outpatient or inpatient encounter at UCD; and (4) Helpline treatment reach—proportion who accepted counseling services, among those who accepted any services. We used clinical encounters with tobacco use rather than the UCD smoking population as our denominator for treatment reaches for comparability with other eReferral studies.[26]

For effectiveness, eReferral recipients who reported current tobacco use at the time of order were assessed for follow-up visit tobacco status. Multiple (second, third, or fourth) orders (n = 117) were not included in the primary analysis, for a total of 1,032 first eReferrals ([Fig. 1]); however quit outcomes for all orders are included in [Table 1]. Quit status was defined as a time-stamped, point prevalence status of not smoking at a follow-up clinic visit during one of four time periods (0–1, 1–3, 3–6, and 6–12 months). Using Chi-square analyses, the proportion of patients who had quit among patients with a follow-up clinic visit was described for a “complete case” analysis. The proportion of patients who had quit among all patients with a first eReferral was also evaluated for a “missing = smoking” analysis (primary outcome) which assumed any smoker who did not have a follow-up visit was still smoking.[27] We also compared the proportion of patients who quit between those who did or did not accept Helpline counseling services across the four-time periods.

Table 1 Quit rates of patients[b] with a first, second, third, and fourth eReferral order at clinic follow-up

	Quit, n (%)
	0–1 months	1–3 months	3–6 months	6–12 months
First eReferral n [a] = 1032	n [b] =223	n = 497	n = 510	n = 576
“Complete case”[c]	33 (14.8)	92 (18.5)	108 (21.2)	126 (21.9)
“Missing = smoking”[d]	33 (3.2)	92 (8.9)	108 (10.5)	126 (12.2)
Second eReferral n = 103	n = 27	n = 50	n = 59	n = 80
“Complete case”	4 (14.8)	10 (20)	11 (18.6)	10 (12.5)
“Missing = smoking”	4 (3.8)	10 (9.7)	11 (10.7)	10 (9.7)
Third eReferral n = 12	n = 3	n = 9	n = 9	n = 9
“Complete case”	1 (33.3)	2 (22.2)	2 (22.2)	0 (0)
“Missing = smoking”	1 (8.3)	2 (16.6)	2 (16.6)	0 (0)
Fourth eReferral N = 2	n = 0	n = 2	n = 2	n = 2
“Complete case”	0 (0)	0 (0)	0 (0)	0 (0)
“Missing = smoking”	0 (0)	0 (0)	0 (0)	0 (0)

^a Total number of first, second, third, or fourth eReferrals ordered.

^b Patients who returned for a follow-up clinic visit during the indicated time period with a documented smoking status.

^c “Complete case”: proportion of patients who quit among those with a documented smoking status at follow-up visit.

^d “Missing = smoking”: proportion of patients who quit among those with a first eReferral order (n = 1032), with the assumption that missing = smoking.

For adoption, the proportions of primary care, specialty, and inpatient eReferrals were calculated out of total eReferrals. The proportion of UCD providers who placed an eReferral was calculated. For Implementation, EHR modifications to enhance the utilization of the eReferral are described. For Maintenance, average monthly eReferrals are tracked over time.

Results

Implementation

The Centers for Disease Control and Prevention provided approximately $50,000 in funding to UCD in 2012 to build the first California bidirectional eReferral to the Helpline.[28] Funding went toward a computer programmer, interoperability manager, and physician champion. Current standard costs for the Helpline to connect with health systems via HL7 are $7,000 then $150 per month for maintenance.[29] The eReferral order was implemented as a standalone project to address the “refer” component of ask–advise–refer.[1] [2]

An EHR in-basket announcement was sent to all ordering providers on March 2013, with a web link to additional screenshots and information. The eReferral order was available as the same single order in all inpatient (discharge) and outpatient settings. No other eReferral promotional activity took place until 2015. Informal teaching took place within the resident teaching clinics. No provider alerts to order the eReferral were implemented primarily due to concern for provider alert fatigue. The eReferral order was integrated into a tobacco outpatient “SmartSet” in Fall 2015. This “SmartSet” was not accessible within the usual order entry box but was instead accessible at a side panel in an outpatient clinical encounter with other “SmartSets.” Several medication options and a referral to the UCD in-person group class were included in the “SmartSet.” Subsequently, the “SmartSet” with the referral orders was added to an inpatient discharge order set in November 2015. The “SmartSet” is automatically included if a hospital patient has a tobacco status of “current smoker.” The eReferral was modified in 2015 to include referrals for household smokers. After the study time period, the eReferral order was incorporated into a 2016 pediatric resident quality improvement project.

There were other statewide initiatives promoting the Helpline during this time which may have raised provider or patient awareness of the Helpline. The statewide Medi-Cal Incentives To Quit Smoking project's statewide outreach through health channels was 2012 to 2015.[30] The UC quits project was a collaboration across all five University of California health systems in implementing bidirectional eReferrals during the study time period.[31] [32]

Reach

During the study time period, 1,149 eReferrals were placed, reflecting 7.1% of 16,083 clinical encounters with a tobacco user. Of the orders received by the Helpline (n = 1,137), 52% of the referrals were successfully contacted. The rate of accepting services was 39% (430 of 1,149 eReferrals placed). The rate of accepting services was lower in the hospital 23% (50 of 215 orders) than the clinics 43% (399 of 934 orders). The Helpline treatment reach was 61.2% (263 callers out of 430 who accepted services).

The treatment reach for the health system was 1.6% (263 patients out of 16,083 clinical encounters with a tobacco user). UCD also offers group classes for tobacco cessation which have been an established service for years prior to the concurrent Helpline eReferral. Comparatively, the UCD group class treatment reach for the same time period was 2.3% (331 patients out of 14,109 outpatient clinical encounters). There were no fax referrals before or during the study. Assuming these patients were selected from either Helpline or group class services, the additive treatment reach would be 3.7%.

Effectiveness

[Table 1] shows the quit rates for 1,032 UCD patients who had a time-stamped tobacco status at a follow-up encounter after their first eReferral. Only one-fifth of patients had an encounter within 1 month but about half of patients had a follow-up encounter between 1 to 12 months. At 6 to 12 months of follow-up, which had the highest follow-up rate, the “complete case” quit rate was 21.9%, and the “missing = smoking” quit rate was 12.2%. The prior time periods of 1 to 3 months and 3 to 6 months differed by only 3 to 4% points lower in the “complete case” or “missing = smoking” quit rate categories. [Table 1] shows there were 103 second referrals, 12 third referrals, and two fourth referrals. At 6 to 12 months, second referrals had a 9.7% quit rate. Third or fourth eReferral groups had very few follow-ups.

Patients with first eReferrals who accepted Helpline services had a higher quit rate at 1 to 3 months (13.6 vs. 7.6%, p = 0.007) than those who did not accept services. However, comparisons at other time intervals were not statistically significant (data not shown).

Adoption

Diverse providers and clinics ordered the eReferral over time, even though group cessation classes were also available. Two hundred and seventy-one providers used the eReferral order at least once, which represents 17% of UCD providers. About 55% (n = 641) of referrals originated from primary care providers. The other referrals were 25% (n = 292) from specialty clinic providers, 19% (n = 215) from inpatient providers, and <1% (n = 5) from pediatric providers. Among the various UCD clinics, 86% (12 of 14) of primary care clinics and 20% (30 of 150) of specialty clinics had placed the eReferral order. The internal medicine residency clinic had the highest number of orders (over 200), followed by the family medicine residency clinic (over 100); other faculty-based primary care clinics averaged 26 orders. Other clinics with a higher volume of eReferral orders (>50 total orders over 3 years) included the UCD pain clinic and thoracic surgery cancer clinic. Almost two-thirds (64%) of all providers who used the eReferral once used it at least a second time.

Maintenance

The monthly eReferrals initially averaged 32 eReferrals per month, as seen in [Fig. 2]. [Fig. 2] shows that monthly referrals doubled in December 2015 after the eReferral order was incorporated into inpatient discharge orders for current tobacco users.

Fig. 2 Monthly eReferral order count (March 2013–February 2016).

Discussion

This study demonstrates that a quitline eReferral order can be successfully implemented and sustained in a health system with minimal promotion, without provider alerts and in conjunction with group classes, while maintaining reach and effectiveness similar to other studies. The UCD bidirectional eReferral with the Helpline is the first in California. This early adoption allowed this study to examine long-term implementation. Primary care, specialty care, and hospital care adopted the eReferral, demonstrating the utility is not specific to primary care. Adoption, implementation, and maintenance were likely enhanced with automation enhancements that were minimally intrusive to the provider as follows: (1) the bidirectional feedback to the ordering provider may explain why almost two-thirds of ordering providers used it at least twice, and (2) automation into the hospital discharge order set doubled monthly orders. This study focuses on a generalized provider-driven workflow model but workflow models that activate other health care team members could have greater reach than the 7.1% of UCD clinical encounters for patients with current tobacco use.

For reach, our contact rate of 52% is consistent with rates found in other studies. Vidrine et al[26] reported a contact rate of 41% in their randomized trial providing patients with electronic quitline referrals or the quitline number. Kruse et al[11] reported a slightly higher contact rate of 56% when connecting patients to an internal tobacco treatment coordinator which may represent increased willingness to engage with in-system resources.[11] Poor contact rates found throughout eReferral studies may reflect a wide variety of issues, such as patient readiness, contact information accuracy, scheduling, or acceptance of telephone services. Incorporating complementary technology, such as texting or mobile apps, may enhance smoker engagement in cessation, such as texting for appointment reminders.[33] [34] [35] The difference in rate of accepting services between clinic and hospital patients is not surprising, since hospital patients may be recovering from an acute illness. The Helpline treatment reach of 61% is comparable or higher than others reported in the clinical trials by Vidrine et al (68.7%)[26] and Fiore et al (30% connected but not necessarily treated).[12] The similar treatment reach between the Helpline and group class suggest that these offerings are not mutually exclusive and can be promoted concurrently. Our treatment reach (1.6%) is higher than the ask–advise–refer group (0.5%) but lower than an ask–advise–connect group (14%) in a study by Vidrine et al.[26] This may reflect our inclusion of inpatient eReferrals which had lower rates of accepting services.

For effectiveness, the “missing = smoking” quit rate of 12.2%, a conservative estimate, is comparable to 6 month quit rates of 11 to 12% seen in other studies of eReferrals to tobacco quitlines.[4] [36] Even the “complete case” quit rate of 21.9% may reflect an underestimate of the higher range, as only half of the patients who had a eReferral had a follow-up clinic visit within the year.[12] [26] We did not measure utilization of medications, which may improve quit rates, as these may not have been prescribed and documented in the EHR but used over the counter. There was also a concurrent Helpline offer for free nicotine patches mailed directly to patients in the Medi-Cal Incentives to Quit Smoking project (2012–2015)[30] which would also not have been documented in the EHR. We found that second eReferrals had a lower quit rate of 9.7% at 6 to 12 months. It may take smokers multiple attempts to quit which may be reflected in the ongoing success of the second eReferrals. The low number of third and fourth eReferrals limited interpretation.

For adoption, primary care, hospital care, and specialty care providers utilized the eReferral. Although only five eReferrals came from pediatrics, the option for referral for household smoker came late in the study time period, and the pediatrics department included the eReferral in an inpatient discharge order set after the study time period. Pediatric eReferrals were expected to increase after these inclusions, other studies by Jenssen et al have found that clinical decision support tools for parental tobacco treatment increased quitline enrollment.[37] [38] [39] Our finding that the hospital-based resident clinics had greater use of the eReferral order than faculty-based clinics may reflect a difference in patient or provider characteristics, resident–physician engagement, or promotional strategies. The hospital-based clinic patient population may represent a different population as patients may reflect the urban, underserved neighborhood, or follow-up hospital discharges for patients that live remotely. UCD group classes at the surrounding faculty-based clinics were concurrently promoted directly to these patients with permission by their providers. Group classes that were on-site and in-person may hold greater appeal than the Helpline to some patients. Direct-to-patient promotion by health systems may be an adjunctive method to engage smokers with quitlines. Targeted mailings to identified smokers, particularly with incentive messaging, including cessation medications, can stimulate calls to a quitline.[40]

For implementation and maintenance, a strength of our study is that, it demonstrates the impact in both hospital and outpatient settings which allows for more than one perspective.[7] The eReferral was launched as an outpatient order and later incorporated as a hospital discharge order. Patients referred in the outpatient setting had almost double the rate of accepting services from the Helpline than those in a hospital setting. Possible explanations include that outpatients may have had more trust with their primary care or specialty provider, or that discharged patients were too sick to engage or not available (e.g., transferred to a skilled nursing facility). From the maintenance perspective, eReferral orders doubled when the hospital discharge order set automatically included the tobacco “SmartSet” when linked to a current tobacco user. There was not an equivalent integrated and automated workflow in the outpatient setting, where the concern for an outpatient provider “best practice alert” linked to orders was that it may contribute to the provider's cognitive burden with additional clicks and “alert fatigue.” For either setting, it is possible that the bidirectional feedback to order providers may encourage providers to continue using the eReferral order, as almost two-thirds of providers had more than one order. Monthly orders did not increase over time suggesting providers did not start to use the eReferral more frequently after feedback. Further research may explore whether the bidirectional feedback is even necessary for providers, the messaging or timing of the bidirectional feedback, or the additional benefit of more provider training or integration into the clinical workflow. Use of an audio computer-assisted self-interview system to assess smoking status, or a noninterruptive clinical decision support tool to promote referral, may be other feasible resources.[39] [41]

Implementation and maintenance of the eReferral requires workflow considerations such as available support staff or culture and the institutional capacity for population health. Training licensed Vocational Nurses to record smoking status and place referrals in the Vidrine et al.[9] eReferral study resulted in a greater treatment reach (8%) than that of the UCD eReferral (1.8%) which relied on physician entry. Automated methods of connecting patients to tobacco cessation, such as interactive voice response calls after hospital discharge,[42] [43] are promising for increasing reach and follow-up. Future efforts may consider tobacco registries to track tobacco status and outcomes after interventions.[44] [45] The bidirectional eReferral feedback from a quitline can potentially contribute to such tobacco registries by tracking tobacco status at follow-up calls and describing interventions received. Interoperability between tobacco cessation EHR resources may be a model when developing clinical informatics tools, such as an open EHR.[46]

Limitations

Our study has several limitations. This was an observational study at a single-academic institution, not a randomized controlled trial. We identified smokers who had quit by whether their smoking status was time-stamped at a follow-up clinic visit which may have difficulties with validity.[45] Directly contacting all patients for follow-up smoking status, medications used, or biochemical validation would provide more accurate data but would have been less feasible and not reflect “real-world” practices. The effectiveness measure is limited by absence of a control population. The patients who had follow-up encounters may represent a unique patient population. We did not include an assessment of medication use as their utilization may not have been documented in the EHR. We did not include an assessment of the use of the “SmartSet” due to a lack of available data. The proportion of patients who completed counseling out of those who accepted services is lower than that of other state quitlines, the etiology of which was not within the scope of this study, limits the generalizability.

Conclusion

This “real world” implementation study demonstrates that an eReferral system can be successfully integrated into an academic health system, resulting in reach and quit rates comparable to those seen in other studies. As tobacco users may average 8 to 12 attempts to permanently quit,[2] the accessibility and integration of the quitline eReferral throughout the health system's EHR can help address tobacco at every potential clinical encounter. Advantages of this outpatient order included the versatility to be used in specialty clinic and hospital settings. Future efforts may also consider how population health strategies combined with clinical informatics may enhance tobacco treatment efforts outside of the clinical encounter setting.[46]

Clinical Relevance Statement

As health systems consider adding eReferrals to their electronic health records, stakeholders question how to effectively implement the eReferral in their system. Decisions, such as who will place the order, whether to make a bidirectional order, whether to require a BPA, or order set, will depend on the unique environment of an individual health system and will benefit from sharing prior implementation experiences. This paper not only describes one method of implementation across diverse clinical settings but additionally contains effectiveness data over a 3-year time frame.

Multiple Choice Questions

1. The incorporation of the UCD eReferral into the hospital discharge order set led to

   • A decrease in hospital eReferral orders.

   • No change in hospital eReferral orders.

   • An increase in hospital eReferral orders.

   • The removal of the eReferral order from the outpatient order sets.

   Correct Answer: The correct answer is option c. After the implementation of a discharge order set, we observed an increase in hospital eReferral orders.

2. The highest proportion of eReferrals came from

   • Resident clinics.

   • Faculty clinics.

   • Specialty clinics.

   • Hospital units.

   Correct Answer: The correct answer is option a. The highest proportion of eReferrals came from resident clinics at this academic site.

Conflict of Interest

None declared.

Acknowledgments

The authors wish to thank Carrie Kirby, MA and Sharon E. Cummins. PhD for their assistance with Helpline data reports on service outcomes and service-related issues.

Protection of Human and Animal Subjects

The Institutional Review Board of UCD determined this study was not human patients' research.

• References

• 1 Centers for Disease Control and Prevention (CDC). Quitting smoking among adults--United States, 2001-2010. Morb Mortal Wkly Rep 2011; 60 (44) 1513-1519
• 2 Fiore MC, Jaen CR, Baker TB. , et al; 2008 PHS Guideline Update Panel, Liaisons, and Staff. Treating tobacco use and dependence: 2008 update U.S. Public Health Service Clinical Practice Guideline executive summary. Respir Care 2008; 53 (09) 1217-1222
• 3 Verbiest M, Brakema E, van der Kleij R. , et al. National guidelines for smoking cessation in primary care: a literature review and evidence analysis. NPJ Prim Care Respir Med 2017; 27 (01) 2
• 4 Zhu SH, Anderson CM, Tedeschi GJ. , et al. Evidence of real-world effectiveness of a telephone quitline for smokers. N Engl J Med 2002; 347 (14) 1087-1093
• 5 Stead LF, Hartmann-Boyce J, Perera R, Lancaster T. Telephone counselling for smoking cessation. Cochrane Database Syst Rev 2013; 8 (08) CD002850
• 6 Sherman SE, Krebs P, York LS. , et al. Telephone care co-ordination for tobacco cessation: randomised trials testing proactive versus reactive models. Tob Control 2018; 27 (01) 78-82
• 7 Boyle R, Solberg L, Fiore M. Use of electronic health records to support smoking cessation. Cochrane Database Syst Rev 2014; 12 (12) CD008743
• 8 Sharifi M, Adams WG, Winickoff JP, Guo J, Reid M, Boynton-Jarrett R. Enhancing the electronic health record to increase counseling and quit-line referral for parents who smoke. Acad Pediatr 2014; 14 (05) 478-484
• 9 Vidrine JI, Shete S, Cao Y. , et al. Ask-advise-connect: a new approach to smoking treatment delivery in health care settings. JAMA Intern Med 2013; 173 (06) 458-464
• 10 Tindle HA, Daigh R, Reddy VK. , et al; Pennsylvania eReferral Workgroup. eReferral between hospitals and quitlines: an emerging tobacco control strategy. Am J Prev Med 2016; 51 (04) 522-526
• 11 Kruse GR, Kelley JH, Linder JA, Park ER, Rigotti NA. Implementation of an electronic health record-based care management system to improve tobacco treatment. J Gen Intern Med 2012; 27 (12) 1690-1696
• 12 Fiore M, Adsit R, Zehner M. , et al. An electronic health record-based interoperable eReferral system to enhance smoking quitline treatment in primary care. J Am Med Inform Assoc 2019; ocz044
• 13 Bernstein SL, Rosner J, DeWitt M. , et al. Design and implementation of decision support for tobacco dependence treatment in an inpatient electronic medical record: a randomized trial. Transl Behav Med 2017; 7 (02) 185-195
• 14 Leuthard JL, Beebe LA, Halstead L, Olson KD, Roysdon JW. Increased evidence-based tobacco treatment through Oklahoma hospital system changes. Am J Prev Med 2015; 48 (01) (Suppl. 01) S65-S70
• 15 Boudreaux ED, Abar B, Haskins B, Bauman B, Grissom G. Health evaluation and referral assistant: a randomized controlled trial to improve smoking cessation among emergency department patients. Addict Sci Clin Pract 2015; 10: 24
• 16 Adsit RT, Fox BM, Tsiolis T. , et al. Using the electronic health record to connect primary care patients to evidence-based telephonic tobacco quitline services: a closed-loop demonstration project. Transl Behav Med 2014; 4 (03) 324-332
• 17 Croyle RT, Morgan GD, Fiore MC. Addressing a core gap in cancer care - the nci moonshot program to help oncology patients stop smoking. N Engl J Med 2019; 380 (06) 512-515
• 18 Dexter PR, Perkins S, Overhage JM, Maharry K, Kohler RB, McDonald CJ. A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med 2001; 345 (13) 965-970
• 19 Ancker JS, Edwards A, Nosal S, Hauser D, Mauer E, Kaushal R. ; with the HITEC Investigators. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak 2017; 17 (01) 36
• 20 Collier R. Electronic health records contributing to physician burnout. CMAJ 2017; 189 (45) E1405-E1406
• 21 Shanafelt TD, Dyrbye LN, Sinsky C. , et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc 2016; 91 (07) 836-848
• 22 Peters DH, Adam T, Alonge O, Agyepong IA, Tran N. Implementation research: what it is and how to do it. BMJ 2013; 347: f6753
• 23 Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: a systematic review of use over time. Am J Public Health 2013; 103 (06) e38-e46
• 24 Glasgow RE, Klesges LM, Dzewaltowski DA, Estabrooks PA, Vogt TM. Evaluating the impact of health promotion programs: using the RE-AIM framework to form summary measures for decision making involving complex issues. Health Educ Res 2006; 21 (05) 688-694
• 25 Measuring Reach of Quitline Programs. Available at: https://cdn.ymaws.com/www.naquitline.org/resource/resmgr/docs/naqc_issuepaper_measuringrea.pdf . Accessed August 23, 2019
• 26 Vidrine JI, Shete S, Li Y. , et al. The Ask-Advise-Connect approach for smokers in a safety net healthcare system: a group-randomized trial. Am J Prev Med 2013; 45 (06) 737-741
• 27 Hedeker D, Mermelstein RJ, Demirtas H. Analysis of binary outcomes with missing data: missing = smoking, last observation carried forward, and a little multiple imputation. Addiction 2007; 102 (10) 1564-1573
• 28 Epic Clinical Programs. Smoking Cessation Intervention Using a Tobacco Quitline. Epic Earth. 2018 . Accessed November 2018
• 29 Helpline Referral Options. Available at: https://www.nobutts.org/helpline-referral-options . Accessed July 15, 2019
• 30 Tong EK, Stewart SL, Schillinger D. , et al. The Medi-Cal Incentives to Quit Smoking Project: Impact of Statewide Outreach Through Health Channels. Am J Prev Med 2018; 55 (6S2): S159-S169
• 31 Smoking Cessation Leadership Center. UC Quits. Available at: https://smokingcessationleadership.ucsf.edu/uc-quits . Accessed July 20, 2019
• 32 Kaslow AA, Romano PS, Schwarz E, Shaikh U, Tong EK. Building and scaling-up california quits: supporting health systems change for tobacco treatment. Am J Prev Med 2018; 55 (6S2): S214-S221
• 33 Abroms LC, Boal AL, Simmens SJ, Mendel JA, Windsor RA. A randomized trial of Text2Quit: a text messaging program for smoking cessation. Am J Prev Med 2014; 47 (03) 242-250
• 34 An LC, Betzner A, Schillo B. , et al. The comparative effectiveness of clinic, work-site, phone, and Web-based tobacco treatment programs. Nicotine Tob Res 2010; 12 (10) 989-996
• 35 Iacoviello BM, Steinerman JR, Klein DB. , et al. Clickotine, a personalized smartphone app for smoking cessation: initial evaluation. JMIR Mhealth Uhealth 2017; 5 (04) e56
• 36 Ray MN, Funkhouser E, Williams JH. , et al; National Dental PBRN Collaborative Group. Smoking-cessation e-referrals: a national dental practice-based research network randomized controlled trial. Am J Prev Med 2014; 46 (02) 158-165
• 37 Jenssen BP, Bryant-Stephens T, Leone FT, Grundmeier RW, Fiks AG. Clinical Decision support tool for parental tobacco treatment in primary care. Pediatrics 2016; 137 (05) e20154185
• 38 Jenssen BP, Muthu N, Kelly MK. , et al. Parent e-referral to tobacco quitline: a pragmatic randomized trial in pediatric primary care. Am J Prev Med 2019; 57 (01) 32-40
• 39 Jenssen BP, Shelov ED, Bonafide CP, Bernstein SL, Fiks AG, Bryant-Stephens T. Clinical decision support tool for parental tobacco treatment in hospitalized children. Appl Clin Inform 2016; 7 (02) 399-411
• 40 Hood-Medland EA, Dove MS, Stewart SL. , et al. Direct-to-member household or targeted mailings: incentivizing medicaid calls for quitline services. Am J Prev Med 2018; 55 (6S2): S178-S185
• 41 Trick WE, Deamant C, Smith J, Garcia D, Angulo F. Implementation of an audio computer-assisted self-interview (ACASI) system in a general medicine clinic: patient response burden. Appl Clin Inform 2015; 6 (01) 148-162
• 42 Rigotti NA, Chang Y, Rosenfeld LC. , et al. Interactive voice response calls to promote smoking cessation after hospital discharge: pooled analysis of two randomized clinical trials. J Gen Intern Med 2017; 32 (09) 1005-1013
• 43 Haas JS, Linder JA, Park ER. , et al. Proactive tobacco cessation outreach to smokers of low socioeconomic status: a randomized clinical trial. JAMA Intern Med 2015; 175 (02) 218-226
• 44 Regan S, Reid ZZ, Kelley JH. , et al. Smoking status confirmation by proxy: validation in a smoking cessation trial. Nicotine Tob Res 2016; 18 (01) 34-40
• 45 Parsons A, McCullough C, Wang J, Shih S. Validity of electronic health record-derived quality measurement for performance monitoring. J Am Med Inform Assoc 2012; 19 (04) 604-609
• 46 Wei PC, Atalag K, Day K. An open EHR approach to detailed clinical model development: tobacco smoking summary archetype as a case study. Appl Clin Inform 2019; 10 (02) 219-228