Appl Clin Inform 2019; 10(02): 180-188
DOI: 10.1055/s-0039-1679926
Research Article
Georg Thieme Verlag KG Stuttgart · New York

Effect of Sociodemographic Factors on Uptake of a Patient-Facing Information Technology Family Health History Risk Assessment Platform

R. Ryanne Wu
1   Center for Applied Genomics and Precision Medicine, Department of Medicine, Duke University School of Medicine, Durham, North Carolina, United States
2   Durham VA Cooperative Studies Program Epidemiology Center, Durham, North Carolina, United States
,
Rachel A. Myers
1   Center for Applied Genomics and Precision Medicine, Department of Medicine, Duke University School of Medicine, Durham, North Carolina, United States
,
Adam H. Buchanan
3   Genomic Medicine Institute, Geisinger, Danville, Pennsylvania, United States
,
David Dimmock
4   Rady Children's Institute for Genomic Medicine, San Diego, California, United States
,
Kimberly G. Fulda
5   The North Texas Primary Care Practice-Based Research Network and Family Medicine, University of North Texas Health Science Center, Fort Worth, Texas, United States
,
Irina V. Haller
6   Essentia Institute of Rural Health, Essentia, Duluth, Minnesota, United States
,
Susanne B. Haga
1   Center for Applied Genomics and Precision Medicine, Department of Medicine, Duke University School of Medicine, Durham, North Carolina, United States
,
Melissa L. Harry
6   Essentia Institute of Rural Health, Essentia, Duluth, Minnesota, United States
,
Catherine McCarty
7   University of Minnesota Medical School, Duluth Campus, Duluth, Minnesota, United States
,
Joan Neuner
8   Department of Medicine, Medical College of Wisconsin, Milwaukee, Wisconsin, United States
9   Center for Patient Care and Outcomes Research, Medical College of Wisconsin, Milwaukee, Wisconsin, United States
,
Teji Rakhra-Burris
1   Center for Applied Genomics and Precision Medicine, Department of Medicine, Duke University School of Medicine, Durham, North Carolina, United States
,
Nina Sperber
1   Center for Applied Genomics and Precision Medicine, Department of Medicine, Duke University School of Medicine, Durham, North Carolina, United States
10   Department of Population Health Sciences, Duke University School of Medicine, Durham, North Carolina, United States
11   Durham VA Health Services & Development Service, Durham, North Carolina, United States
,
Corrine I. Voils
12   William S. Middleton Memorial Veterans Hospital, Madison, Wisconsin, United States
13   Department of Surgery, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin, United States
,
Geoffrey S. Ginsburg
1   Center for Applied Genomics and Precision Medicine, Department of Medicine, Duke University School of Medicine, Durham, North Carolina, United States
,
Lori A. Orlando
1   Center for Applied Genomics and Precision Medicine, Department of Medicine, Duke University School of Medicine, Durham, North Carolina, United States
› Author Affiliations
Funding This study was funded by NIH (NCT01956773). The funder had no involvement in the design, conduct, data collection, analysis, or manuscript preparation. C. Voils' effort on this study was supported by a Research Career Scientist award from the Department of Veterans Affairs (RCS 14–443). This study was approved by the IRBs of all four participating institutions and the funders.
Further Information

Address for correspondence

R. Ryanne Wu, MD, MHS
Department of Medicine, Duke University School of Medicine
411 West Chapel Hill Street
Suite 600, Durham, NC 27701
United States   

Publication History

07 November 2018

18 January 2019

Publication Date:
13 March 2019 (online)

 

Abstract

Objective Investigate sociodemographic differences in the use of a patient-facing family health history (FHH)-based risk assessment platform.

Methods In this large multisite trial with a diverse patient population, we evaluated the relationship between sociodemographic factors and FHH health risk assessment uptake using an information technology (IT) platform. The entire study was administered online, including consent, baseline survey, and risk assessment completion. We used multivariate logistic regression to model effect of sociodemographic factors on study progression. Quality of FHH data entered as defined as relatives: (1) with age of onset reported on relevant conditions; (2) if deceased, with cause of death and (3) age of death reported; and (4) percentage of relatives with medical history marked as unknown was analyzed using grouped logistic fixed effect regression.

Results A total of 2,514 participants consented with a mean age of 57 and 10.4% minority. Multivariate modeling showed that progression through study stages was more likely for younger (p-value = 0.005), more educated (p-value = 0.004), non-Asian (p-value = 0.009), and female (p-value = 0.005) participants. Those with lower health literacy or information-seeking confidence were also less likely to complete the study. Most significant drop-out occurred during the risk assessment completion phase. Overall, quality of FHH data entered was high with condition's age of onset reported 87.85%, relative's cause of death 85.55% and age of death 93.76%, and relative's medical history marked as unknown 19.75% of the time.

Conclusion A demographically diverse population was able to complete an IT-based risk assessment but there were differences in attrition by sociodemographic factors. More attention should be given to ensure end-user functionality of health IT and leverage electronic medical records to lessen patient burden.


#

Background and Significance

Racial and socioeconomic disparities in health care have been well documented.[1] While there are numerous factors contributing to these disparities, one potential contributor is associated with health care providers' subconscious biases in risk assessment and referral patterns.[2] [3] Systematic risk assessment using information technology (IT) platforms has the potential to help mitigate racial and socioeconomic differences in health care access and outcomes. Through the use of IT platforms, populations can be assessed systematically for disease risk so that care is evaluated and distributed based on underlying risk without the influence of subconscious biases.

The potential value of an IT-based risk assessment directing health services to those at greatest need while reducing use of services among those least likely to benefit has been increasingly recognized. The purpose of risk assessment will vary across settings, ranging from decreasing hospital readmission rates to selecting appropriate medications for patients.[4] [5] [6] Within the precision medicine context, risk assessment has great potential benefit.[7] [8] [9] Although genetic testing for disease risk and medication adverse events is becoming more available, the health system still needs systematic ways to identify patients most likely to benefit from such interventions. Family health history (FHH)-based risk assessment is an important way to identify those at greatest risk for inherited conditions and most in need of increased screening and genetic testing services.[10] FHH as currently collected in routine care is frequently inadequate to perform risk assessment.[11] There are numerous barriers to adequate collection and use of this information including lack of patient education and awareness, lack of provider time and resources, and inadequate electronic health record systems for FHH collection and assessment.[12] [13] [14] [15] Numerous IT risk assessment tools (e.g., Family Healthware, Health Heritage) have been developed and have been shown to: improve the quality of FHH collected as compared with usual care,[7] [16] [17] increase the identification of at-risk individuals,[7] [18] [19] and impact health care practices of clinicians and patients.[20] [21] One of these tools, MeTree, is a web-based, patient-facing FHH risk assessment platform with guideline-informed clinical decision support that was developed by our group.[22] Little work has been done to evaluate the impact of patient demographics on the ability to complete FHH risk assessment tools. One small study of 70 individuals comparing an animated virtual counselor (VICKY) to the Surgeon General's MyFamilyHealthPortrait (MFHP) among an underserved population showed that VICKY was easier to use and identified more medical conditions among relatives than MFHP.[23] This was the first study to identify a concern about usability among underserved populations and suggested that a deeper evaluation of the impact of sociodemographics on feasibility was warranted.

To date, MeTree has been assessed in limited practice settings. Given patients' range of experiences and comfort with IT-based health applications, knowledge of and sharing of health information between family members, health information-seeking, cultural practices, and other patient demographics, we desired to ascertain differences in utilization of the platform in more diverse patient populations. We recently evaluated utilization of the platform in four diverse health care systems in different regions of the United States.[24]


#

Objective

In this paper, we report our findings regarding the usability of a FHH platform as measured by (1) the degree to which uptake of the intervention varied by patient demographics and (2) the impact of sociodemographics on the quality of the data collected within the platform.


#

Methods

Overview and Study Design

This was a pragmatic type III hybrid implementation-effectiveness trial across four diverse health care systems in the United States.[24] [25] The study design and outcomes were organized based on the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework.[26] Full details of the study protocol have been published and are summarized below.[24] This is a secondary analysis of impact of sociodemographics on study participation.


#

Setting

The four participating health care systems had diverse operational profiles, missions, and patient populations: Duke University, a suburban academic medical system with a moderately racially diverse patient population in central North Carolina; Essentia Health (Essentia), a rural integrated health system serving a predominantly white population across the upper Midwest; Medical College of Wisconsin (MCW), an urban academic medical system with a large black population in Milwaukee, Wisconsin; and University of North Texas Health Science Center (UNTHSC), an academic urban medical system with a large Hispanic, underserved population in Texas.


#

Study Process

All adult patients (age ≥ 18) of participating clinicians with an upcoming primary care visit were eligible to participate. Those with email addresses on file were invited to participate electronically by email. Patients without an email address on file and at sites not approved for email recruitment were sent paper invitations by post mail. Invitations were sent 3 weeks prior to an upcoming nonurgent appointment. Invitees were told that their medical provider had enrolled in the study, would receive their risk report, and would discuss their results with them at their next clinic visit. A follow-up invitation was sent to those who did not respond within 5 days. Interested patients were instructed to contact the central coordinator by email or phone to enroll. Once the coordinator was contacted by an eligible patient, the entire enrollment, consent, and study completion process was done electronically.

At each stage, the participant was emailed a link to complete the next step. Completion of one step automatically led to receipt of a link to complete the next step ([Fig. 1]). Once consented, participants were emailed a link to complete a baseline survey regarding health behaviors, health literacy, and health information seeking; then a link to the risk assessment application, which collected demographic information as well as personal and FHH. Risk reports generated by MeTree were immediately displayed to the participant within the program and made available to be downloaded; the provider report was uploaded to the electronic medical record (EMR) for the clinician for review, discussion, and action as clinically appropriate.

Zoom Image
Fig. 1 Study flow diagram.

#

Measures

Measures reported in this paper were collected from (1) MeTree (demographics, personal health history, and FHH), (2) the baseline survey (health literacy and information-seeking confidence), and (3) automated tracking of participant progression through the study. Participant demographic categories included age, sex, race (Asian, black, mixed race, American Indian/Alaskan Native, not reported, white), ethnicity (Hispanic, non-Hispanic), insurance type (Medicare, Medicaid, employer/private, other), and education (high school [HS] or less, community college, 4-year college, graduate school). Health literacy and information seeking data were only available for those who completed the baseline survey, and race and ethnicity data, which were collected by the platform, were available for those who started it. The baseline survey included a validated screener for health literacy (“How confident are you in filling out forms by yourself?”),[27] and assessment of confidence in information seeking (“Overall, how confident are you that you could get health-related advice or information if you needed it?”).[28]


#

Outcomes

We evaluated how participants' demographics were associated with two outcomes: (1) progression through each stage of the study (i.e., enrollment, consent, baseline survey completion, intervention completion) and (2) quality of FHH entered. Participants with race “not reported” (n = 327, 13.9%) and “multiracial” (n = 48, 2.0%) categories were kept in the analysis as separate groups to assess the impact of their education and insurance status on outcomes; however, impacts of race for these two groups are not reported as these results were considered uninformative. “Multiracial” was considered uninformative because it included subjects from a wide range of racial combinations, each of which was too sparse to represent its own group.

A high-quality FHH as defined by Bennett and commonly accepted should include: (1) three generations of relatives; (2) relatives' lineage (i.e., paternal or maternal); (3) relatives' gender; (4) an up-to-date FHH; (5) pertinent negatives in FHH noted; (6) age of disease onset in affected relatives; and for deceased relatives, the (7) age and (8) cause of death.[29] Criteria 1 to 5 are automatically collected based on how the risk assessment platform is structured. We assessed the remaining criteria of the number and percentage of relatives: (1) with age of onset reported on relevant conditions and (2) if deceased, with cause of death and (3) age of death reported. We also evaluated the percentage of relatives for which medical history was marked as unknown.


#

Statistical Analysis

Participant demographics were summarized using counts and percentages for ethnicity, race, education level, and insurance type and using mean and standard deviation for age in years. Bivariate analysis of differences in participants' demographics by study progression was assessed using Pearson's chi-squared tests or ANOVA F-tests. Intervention completion (i.e., of patients who started and completed it) was modeled as a function of education level, insurance, race, ethnicity, gender, age, health literacy, and information-seeking confidence, while controlling for study site. Backward stepwise logistic mixed effect regression was used to identify and eliminate nonsignificant fixed-effect model terms (p > 0.05) until all remaining terms were significant. Significance of each model term was assessed using likelihood ratio tests of nested models, and pairwise contrasts between levels of categorical model terms with three or more levels (e.g., education, insurance) were assessed using generalized linear hypothesis testing with Benjamini–Hochberg multiple testing corrections.

FHH quality measures of age of disease onset, cause of death, age at death, and unknown health history were aggregated to the family level, summarizing the number of instances with and without the feature reported in each family. For example, the number of deceased relatives with an age at death reported and not reported was determined for each participant. Proportions were aggregated at the family level, and then assessed for differences by the participant's demographics using grouped logistic fixed effect regression, with weights such that each family had equal weight, regardless of family size. Significance was assessed using likelihood ratio tests of nested models.


#
#

Results

All primary care patients from 19 primary care clinics across four health systems with upcoming nonurgent primary care appointments (N = 55,738) were invited to participate in the study. We enrolled 2,514 (4.51%) patients with a mean age of 57 ([Table 1]). Data on how enrolled participants differed from the underlying clinic populations have been previously reported.[30]

Table 1

Participant characteristics and study progression by demographics

Demographic group

Consented

N (%)

Completed baseline survey

N (%)

Started intervention

N (%)

Completed intervention

N (%)

Age (mean [SD])

56.76 (14.13)

57.00 (14.07)

57.03 (14.00)

57.02 (14.06)

Sex

 Female

1,722 (72.6)

1,680 (97.6)

1,611 (95.9)

1,309 (81.3) [a]

 Male

791 (72.3)

772 (97.6)

736 (95.3)

578 (78.5) [a]

Insurance

 Employer/private

1,629 (100)

1,603 (98.4) [b]

1,541 (96.1) [b]

1,280 (83.1) [b]

 Medicaid

67 (100)

63 (94) [b]

54 (85.7) [b]

36 (66.7) [b]

 Medicare

741 (100)

733 (98.9) [b]

703 (95.9) [b]

536 (76.2) [b]

 Other

22 (100)

19 (86.4) [b]

18 (94.7) [b]

14 (77.8) [b]

Education

 HS or less

235 (100)

228 (97)

213 (93.4)

150 (70.4) [a] [b]

 Community college

391 (100)

380 (97.2)

362 (95.3)

276 (76.2) [b]

 4-year college

709 (100)

700 (98.7)

682 (97.4)

563 (82.6) [a] [b]

 Graduate school

1,119 (100)

1,106 (98.8)

1,056 (95.5)

875 (82.9) [a] [b]

Race[c]

 Asian

29 (100)

21 (72.4) [a] [b]

 Black

165 (100)

143 (86.7) [b]

 Native American/Alaskan

3 (100)

3 (100) [b]

 White

1,776 (100)

1,606 (90.4) [a] [b]

Health literacy[d]

 Never

9 (100)

8 (88.9) [b]

5 (62.5) [b]

 Occasionally

8 (100)

7 (87.5) [b]

3 (42.9) [b]

 Sometimes

48 (100)

44 (91.7) [b]

31 (70.5) [b]

 Often

251 (100)

234 (93.2) [b]

171 (73.1) [b]

 Always

1,807 (100)

1,740 (96.3) [b]

1,424 (81.8) [b]

Information-seeking confidence[e]

 Not confident at all

7 (100)

6 (85.7)

3 (50) [b]

 A little confident

36 (100)

33 (91.7)

21 (63.6) [b]

 Somewhat confident

311 (100)

297 (95.5)

234 (78.8) [b]

 Very confident

923 (100)

885 (95.9)

710 (80.2) [b]

 Completely confident

850 (100)

817 (96.1)

666 (81.5) [b]

Note: All percentages in parentheses use the number from the preceding column as the denominator for the calculation. Statistically significant values are presented in bold.


a Significant differences (p-value < 0.05) seen on pairwise comparisons in multivariate model of intervention completion.


b Significant differences (p-value < 0.05) across the category in bivariate analysis of study stage progression.


c Race recorded within MeTree, so only available from that study stage. Participants in “no race reported” or “mixed race” category not included in race analysis.


d “How confident are you in filling out forms by yourself?”[27]


e “Overall, how confident are you that you could get health-related advice or information if you needed it?”[28]


Study Progression

Participants' progression through the study varied significantly by age, race, insurance type, education level, health literacy, and information-seeking confidence in bivariate analyses ([Table 1]). Age did not correlate with progression through initial stages of the study, but those of older age were less likely to finish the final step of completing the risk assessment once it was started (for every 10-year increase in participant age, odds of not completing increased by a factor of 1.22, p-value < 0.001). Intervention completion rates varied by race (p-value < 0.001). Progression by insurance type and education varied at every study stage (insurance: p-value < 0.001 at all stages; education: completed baseline survey, p-value = 0.04; started MeTree, p-value = 0.04; completed MeTree, p-value < 0.001). Low health literacy, as measured in the baseline survey, correlated with lack of progression at all subsequent steps (started MeTree, p-value = 0.05; completed MeTree, p-value < 0.001). Low information-seeking confidence also correlated with lack of intervention completion (p-value = 0.03). Progression did not vary by sex or ethnicity.

In multivariate logistic mixed effect modeling of the transition from intervention started to intervention completed, lack of completion was more commonly seen among those of older age (for every 10-year increase in age, odds of not completing increased by a factor of 1.22, p-value = 0.005), less education (p-value = 0.004), male sex (odds ratio [OR]female vs. male = 1.75, 95% confidence interval [CI]: 1.19–2.56, p-value = 0.005), and was variable by race (p-value = 0.009). Insurance type, health literacy, and information seeking were no longer significant covariates and eliminated from the model via backward stepwise regression. Pairwise comparisons of racial groups showed that Asians had the lowest odds of completing while whites had the highest odds. After multiple testing corrections, whites were significantly different from Asians (ORwhite vs. Asian = 5.51, 95% CI: 2.14–14.17, p = 0.006); black versus Asians was trending toward significance (ORblack vs. Asian = 4.66, 95% CI: 1.43–15.3, p = 0.08). Pairwise comparisons of education levels show that differences in intervention completion were driven by lower completion rates in those with HS education versus all other levels of education: community college (ORComm. coll. vs. HS = 2.24, 95% CI: 1.12–4.48, p-value = 0.04), 4-year college (OR4-year. coll. vs. HS = 2.91, 95% CI: 1.57–5.41, p-value = 0.002), or graduate level degrees (ORGrad. level vs. HS = 2.95, 95% CI: 1.67–5.23, p-value < 0.001).


#

Family Health History Quality

FHH quality metrics ([Table 2]) were used to evaluate any disparities in FHH reporting.[29] Some sociodemographic factors had significant correlation with quality metrics. Health literacy and information-seeking confidence had no correlation with any of the FHH quality metrics assessed.

Table 2

Frequency of reporting relative: age of onset, unknown medical history, cause and age of death

Participant demographics

Conditions with age of onset reported[a] % (CI)

Relatives with unknown medical history % (CI)

Deceased relatives with cause of death reported % (CI)

Deceased relatives with age of death reported % (CI)

Sex

 Female

86.25 (84.3–88.0) [b]

18.36 (16.3–20.5) [b]

86.25 (84.3–88.1)

93.27 (91.8–94.5)

 Male

91.46 (89.0–93.6) [b]

22.87 (19.6–26.4) [b]

83.97 (80.8–86.8)

94.85 (92.8–96.5)

Insurance

 Employer/private

87.31 (85.4–89.1)

16.86 (14.9–19.0) [b]

86.96 (85.0–88.7)

94.10 (92.7–95.3)

 Medicaid

84.05 (69.5–93.7)

25.92 (13.6–41.7) [b]

86.10 (72.5–94.8)

87.11 (73.8–95.4)

 Medicare

89.42 (86.6–91.8)

26.20 (22.6–30.0) [b]

82.05 (78.6–85.2)

93.48 (91.2–95.4)

 Other

89.53 (67.2–98.8)

17.37 (3.8–41.8) [b]

89.29 (66.9–98.7)

92.26 (71.2–99.5)

Education

 HS or less

87.75 (81.8–92.4)

26.54 (19.9–34.0)

81.78 (75.0–87.4)

90.32 (84.9–94.4)

 Community college

86.81 (82.5–90.5)

22.17 (17.5–27.3)

85.43 (80.9–89.3)

92.49 (89.0–95.2)

 4-year college

86.93 (84.0–90.0)

18.85 (15.8–22.2)

86.59 (83.6–89.3)

93.99 (91.8–95.8)

 Graduate school

88.80 (86.6–90.8)

18.36 (15.9–21.0)

85.54 (83.1–87.7)

94.59 (93.0–96.0)

Race

 Asian

90.92 (73.5–98.6)

26.38 (11.0–47.2)

80.82 (60.4–93.8)

85.50 (66.2–96.3) [b]

 Black

81.21 (74.2–87.1)

24.40 (17.9–31.9)

78.68 (71.5–84.8)

83.33 (76.7–88.8) [b]

 Native American/Alaskan

100 (NA)

36.30 (3.0–85.7)

68.89 (17.5–98.2)

93.33 (40.2–100) [b]

 White

88.44 (86.8–89.9)

19.26 (17.4–21.2)

86.38 (84.6–88.0)

94.84 (93.7–95.9) [b]

Abbreviations: CI, confidence interval; HS, high school.


Note: Statistically significant values are presented in bold.


a Includes only conditions entered where age of onset is requested.


b Significant differences (p-value < 0.05) between groups within the socio-demographic factor.


Age of Onset

Participants reported age of onset for the majority of diseases (87.85%, CI: 86.32–89.27%) entered for relatives. Women and those who were younger reported significantly fewer ages of onset for relatives' conditions (p-value = 0.001 for sex and for age, for every 10-year increase in age, odds of providing age of onset increased by a factor of 1.15, p-value = 0.005).


#

Cause of Death

Participants reported cause of death for the majority (85.55%, CI: 83.91–87.10%) of relatives marked as deceased. Older participants were less likely to report cause of death (for every 10-year increase in age, odds of not reporting cause of death increased by a factor of 1.25, p-value < 0.001). No other demographic factors were associated with significant differences in reporting.


#

Age of Death

Participants reported age of death for the majority (93.76%, CI: 92.61–94.80%) of relatives marked as deceased. Participants' race impacted age of death reporting, with Asians (85.50%) and blacks (83.33%) being less likely to report age of death (p-value < 0.001).


#

Unknown History

The proportion of relatives with an “unknown medical history” reported (19.75%, CI: 18.00–21.59%) was higher among male participants (p-value = 0.025), those with Medicaid and Medicare insurance (p-value < 0.001), and older participants (for every 10-year increase in participant age, odds of reporting unknown history increase by a factor of 1.32, p-value < 0.001) on bivariate analyses. Race and education had no effect. Multivariate fixed effect modeling showed that difference was driven entirely by participant age (for every 10-year increase in participant age, odds of reporting unknown history increased by 1.32, p-value < 0.001).


#
#
#

Discussion

While the systematic use of IT-based health applications such as MeTree may improve use of FHH for risk assessment and targeted prevention strategies, IT-based health applications could also exacerbate health disparities.[31] This study shows that using a web-based technology to collect family health histories directly from patients results in a high overall completion rate (75%) with no significant disparities among blacks or Hispanics as compared with white participants. This is a significant finding given ongoing concerns about health care disparities in general, and widening gaps in morbidity and longevity in minority patients.[32] [33]

However, several factors significantly correlated with lack of completion of the risk assessment platform, including less education, older age, male sex, and Asian race. There are several reasons that may have contributed to these findings, including lack of comfort with IT, lack of knowledge of one's FHH, and lack of time due to competing demands.[30] Older patients who have registered to access a patient portal are less likely to utilize it.[34] In addition, poor digital literacy, health literacy, and lack of internet access are all interrelated and well-documented barriers among the elderly and lower socioeconomic groups.[35] [36] [37] [38] [39] Collection of and knowledge about FHH among men have been shown to be significantly lower which may help explain their lower rates of completion.[40] [41] Better understanding of these barriers and their relationships will allow investment in interventions that can solve these challenges. Low numbers of Asian participants makes it difficult to draw strong conclusions about this group although Asian-Americans may have some unique cultural barriers that should be considered as highlighted by Chen and colleagues' study reporting a high perceived value of FHH among Asian-Americans, yet a significant lack of knowledge due to barriers such as distance from relatives and the perception of “healthy families.”[42] [43] Lastly, the higher engagement in clinical trials by Caucasians and those with higher education is a well-documented phenomenon, which is a difficult challenge to overcome.[44] [45]

For those who did complete the intervention, the quality of the data provided was high overall, with negligible differences by sociodemographic characteristics. Important to note, sociodemographic (e.g., race, education, insurance) differences did not correlate with lack of knowledge about relatives' medical histories. Age was the only significant correlate. This is likely due to (1) less clear medical diagnoses in previous generations, (2) the diaspora induced by World Wars I and II and the massive impact these wars had on mortality in young adults, and (3) higher number of deceased relatives among other factors.

The limitations of this analysis include the inability to capture the reason why individuals did not progress past a stage in the study. Further work needs to be done to understand these barriers. In addition, not all sociodemographic factors were available from the time of enrollment (e.g., race is captured in the platform). Thus fully understanding the impact of each factor on progression between each stage of the study was not possible. We also are limited in applying these results more broadly by the small number of minority participants. Additional studies are in development to more directly target non-White populations. Furthermore, the collinearity and imbalance in the features hypothesized to influence completion prohibited us from obtaining stable estimates for the complete multivariable mixed effect model of study completion. To overcome this limitation, we used backward stepwise regression to eliminate nonsignificant features. However, this approach results in p-values that may be biased due to the repeated analyses in stepwise regressions and parameter estimates that are biased/may not replicate.

Despite these limitations, the stepwise regression complements the bivariate analyses by estimating the effects in the context of other participant features (e.g., effect of education while holding race constant) and identifying which features remain associated with completion.

Given the potential benefits of technology to overcome barriers of access and implicit bias, future efforts to address these and other technology-related barriers that hinder enrollment and engagement with minorities and lower literacy groups are essential for improving the health of the U.S. population. Risk assessment tools like MeTree frequently already include tips to gather information from relatives, and education on the importance of specific types of information such as age of onset, glossaries and pop-up reminders to help clarify the type of information, or remind patients about missing data, respectively. These could be further enhanced. In addition, some potential solutions to address the technology barriers and expand utilization include development of additional patient resources, such as short step-by-step videos demonstrating how to complete an online survey or use a tablet (if applicable) and how to complete a FHH-based risk assessment. A small community pilot with 20 low-literacy American-Indians showed that they were highly engaged in collecting and documenting their FHH. However, when using a technology, like MeTree, their limited awareness of the body and diseases was a barrier for identifying where to enter disease information. Going forward, understanding these challenges in greater detail will allow developers to build technology that overcomes these difficulties, e.g., using augmented reality to select the location of the disease or speech to text so that the program enters the data for them. In addition, integration with the EMR through SMART-on-FHIR-enabled applications that allow seamless integration for patients and providers may help overcome some of the existing challenges.[46] [47] EMR integration has great potential as it permits access to data within the EMR, reducing the burden of data entry and improving the quality of risk stratification by using verified laboratory and diagnosis data imported from the patient's medical record.


#

Conclusion

There is potential to make a significant impact on population health by systematic identification of presymptomatic disease risk. Although there was little evidence of racial disparity in risk assessment completion in this study, there is room to improve in addressing barriers for older and less-educated individuals. Systematic, unbiased risk assessment has great potential to mitigate disparities currently seen in our health care systems at least at the stage of assessment (although it is only one piece of the puzzle and work must continue to address disparities in referral and care decision making). Yet even as we address racial disparities, we must be mindful of the risk of introducing new disparities for the elderly and less educated based on IT literacy in the process.


#

Clinical Relevance Statement

This study shows that a patient-facing risk assessment IT platform is of interest and accessible by patients from a wide range of racial and educational backgrounds. There are potential barriers to FHH-based health IT use among those who are older, with low education levels, and who are male. The quality of the FHH information that patients can provide is minimally impacted by their sociodemographics.


#

Multiple Choice Questions

  1. Family health history based risk assessment platforms have been shown to:

    • Identify disease risk at equal rates as routine care.

    • Change practice patterns.

    • Diagnose conditions in participants.

    • All of the above.

    Correct Answer: The correct answer is option b, change practice patterns. Use of these platforms has been shown to change screening and referral patterns within clinics so that they are more in line with current practice guidelines. FHH-based risk assessment platforms have been shown to increase identification of at-risk individuals as compared with current identification rates in routine care. The purpose of risk assessment is not to diagnose medical conditions in those who complete them but to identify what conditions they are at most risk for based on their family health history.

  2. In this study, it was shown that information technology based risk assessment platforms are less likely to be completed by:

    • Blacks.

    • Medicaid recipients.

    • Elderly.

    • Women.

    Correct Answer: The correct answer is option b. In multivariate modeling of intervention completion, those who were of older age, male sex, less education, and Asian race were less likely to complete the risk assessment. Insurance type did not have an effect.

  3. Which of the following is not a measure of family health history data quality?

    • Three generations of relatives.

    • Relative lineage.

    • Disease age of onset.

    • Relatives' current age.

    Correct Answer: The correct answer is option d. A high-quality FHH as defined by Bennett should include (1) three generations of relatives; (2) relatives' lineage (i.e., paternal or maternal); (3) relatives' gender; (4) an up-to-date FHH; (5) pertinent negatives in FHH noted; (6) age of disease onset in affected relatives; and for deceased relatives, the (7) age and (8) cause of death. Relatives' current age is not a key quality indicator.

  4. Which of the following groups were shown to be less likely to know their relatives' medical history (i.e., report “unknown history”)?

    • Women.

    • Elderly.

    • Asians.

    • Those with lower health literacy.

    Correct Answer: The correct answer is option b. On multivariate modeling, being older was the only significant factor associated with reporting “unknown history” for a relative. Race and health literacy had no impact on this outcome. Men were less likely to know relatives' medical history on bivariate analyses but this association was no longer present on multivariate modeling.


#
#

Conflict of Interest

Drs. Wu, Orlando, and Ginsburg and Mrs. Rakhra-Burris have a potential conflict of interest. They are cofounders of a company, MeTree&You that will provide MeTree as a clinical service. MeTree&You has not and will not receive any direct financial gain from this study or publication. The remaining authors declare that they have no conflicts of interest in the research.

Note

The views are those of the authors and do not reflect the Department of Veterans Affairs or United States Government.


Protection of Human and Animal Subjects

This study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was reviewed by the institutional review boards of all four participating health care systems (Duke, Essentia, MCW, and UNTHSC).


  • References

  • 1 Baicker K, Chandra A, Skinner JS. Geographic variation in health care and the problem of measuring racial disparities. Perspect Biol Med 2005; 48 (1, Suppl): S42-S53
  • 2 Williams DR, Wyatt R. Racial bias in health care and health: challenges and opportunities. JAMA 2015; 314 (06) 555-556
  • 3 Paradies Y, Truong M, Priest N. A systematic review of the extent and measurement of healthcare provider racism. J Gen Intern Med 2014; 29 (02) 364-387
  • 4 Kansagara D, Englander H, Salanitro A. , et al. Risk prediction models for hospital readmission: a systematic review. JAMA 2011; 306 (15) 1688-1698
  • 5 Pisters R, Lane DA, Nieuwlaat R, de Vos CB, Crijns HJGM, Lip GYH. A novel user-friendly score (HAS-BLED) to assess 1-year risk of major bleeding in patients with atrial fibrillation: the Euro Heart Survey. Chest 2010; 138 (05) 1093-1100
  • 6 Subramanian S, Bobashev G, Morris RJ, Hoover S. Personalized medicine for prevention: can risk stratified screening decrease colorectal cancer mortality at an acceptable cost?. Cancer Causes Control 2017; 28 (04) 299-308
  • 7 Cohn WF, Ropka ME, Pelletier SL. , et al. Health Heritage© a web-based tool for the collection and assessment of family health history: initial user experience and analytic validity. Public Health Genomics 2010; 13 (7–8): 477-491
  • 8 Elliott LS, Henderson JC, Neradilek MB, Moyer NA, Ashcraft KC, Thirumaran RK. Clinical impact of pharmacogenetic profiling with a clinical decision support tool in polypharmacy home health patients: a prospective pilot randomized controlled trial. PLoS One 2017; 12 (02) e0170905
  • 9 Larson EA, Wilke RA. Integration of genomics in primary care. Am J Med 2015; 128 (11) 1251.e1-1251.e5
  • 10 Welch BM, Wiley K, Pflieger L. , et al. Review and comparison of electronic patient-facing family health history tools. J Genet Couns 2018; 27 (02) 381-391
  • 11 Qureshi N, Wilson B, Santaguida P. , et al. Collection and use of cancer family history in primary care. Evid Rep Technol Assess (Full Rep) 2007; (159) 1-84
  • 12 Mikat-Stevens NA, Larson IA, Tarini BA. Primary-care providers' perceived barriers to integration of genetics services: a systematic review of the literature. Genet Med 2015; 17 (03) 169-176
  • 13 Berg AO, Baird MA, Botkin JR. , et al. National Institutes of Health state-of-the-science conference statement: family history and improving health. Ann Intern Med 2009; 151 (12) 872-877
  • 14 Gramling R, Nash J, Siren K, Eaton C, Culpepper L. Family physician self-efficacy with screening for inherited cancer risk. Ann Fam Med 2004; 2 (02) 130-132
  • 15 de Hoog CL, Portegijs PJ, Stoffers HE. Family history tools for primary care are not ready yet to be implemented. a systematic review. Eur J Gen Pract 2014; 20 (02) 125-133
  • 16 Hulse NC, Ranade-Kharkar P, Post H, Wood GM, Williams MS, Haug PJ. Development and early usage patterns of a consumer-facing family health history tool. AMIA Annu Symp Proc 2011; 2011: 578-587
  • 17 Wu RR, Himmel TL, Buchanan AH. , et al. Quality of family history collection with use of a patient facing family history assessment tool. BMC Fam Pract 2014; 15: 31
  • 18 Rubinstein WS, Acheson LS, O'Neill SM. , et al; Family Healthware Impact Trial (FHITr) Group. Clinical utility of family history for cancer screening and referral in primary care: a report from the Family Healthware Impact Trial. Genet Med 2011; 13 (11) 956-965
  • 19 Orlando LA, Wu RR, Beadles C. , et al. Implementing family health history risk stratification in primary care: impact of guideline criteria on populations and resource demand. Am J Med Genet C Semin Med Genet 2014; 166C (01) 24-33
  • 20 Ruffin IV MT, Nease Jr DE, Sen A. , et al; Family History Impact Trial (FHITr) Group. Effect of preventive messages tailored to family history on health behaviors: the Family Healthware Impact Trial. Ann Fam Med 2011; 9 (01) 3-11
  • 21 Orlando LA, Wu RR, Myers RA. , et al. Clinical utility of a Web-enabled risk-assessment and clinical decision support program. Genet Med 2016; 18 (10) 1020-1028
  • 22 Orlando LA, Buchanan AH, Hahn SE. , et al. Development and validation of a primary care-based family health history and decision support program (MeTree). N C Med J 2013; 74 (04) 287-296
  • 23 Wang C, Bickmore T, Bowen DJ. , et al. Acceptability and feasibility of a virtual counselor (VICKY) to collect family health histories. Genet Med 2015; 17 (10) 822-830
  • 24 Wu RR, Myers RA, McCarty CA. , et al; Family Health History Network. Protocol for the “Implementation, adoption, and utility of family history in diverse care settings” study. Implement Sci 2015; 10: 163
  • 25 Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care 2012; 50 (03) 217-226
  • 26 Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999; 89 (09) 1322-1327
  • 27 Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Fam Med 2004; 36 (08) 588-594
  • 28 NCI. Health Information National Trends Survey. Available at: https://hints.cancer.gov/view-questions-topics/question-details.aspx?red=1&qid=413 . Accessed August 29, 2018
  • 29 Bennett RL. The family medical history. Prim Care 2004; 31 (03) 479-495
  • 30 Wu RR, Myers RA, Sperber N. , et al. Implementation, adoption, and utility of family health history risk assessment in diverse care settings: evaluating implementation processes and impact with an implementation framework. Genet Med 2018; DOI: 10.1038/s41436-018-0049-x.
  • 31 Lyles C, Schillinger D, Sarkar U. Connecting the dots: health information technology expansion and health disparities. PLoS Med 2015; 12 (07) e1001852
  • 32 Cunningham TJ, Croft JB, Liu Y, Lu H, Eke PI, Giles WH. Vital signs: racial disparities in age-specific mortality among Blacks or African Americans - United States, 1999-2015. Morb Mortal Wkly Rep 2017; 66 (17) 444-456
  • 33 O'Connell J, Yi R, Wilson C, Manson SM, Acton KJ. Racial disparities in health status: a comparison of the morbidity among American Indian and U.S. adults with diabetes. Diabetes Care 2010; 33 (07) 1463-1470
  • 34 Gordon NP, Hornbrook MC. Differences in access to and preferences for using patient portals and other eHealth technologies based on race, ethnicity, and age: a database and survey study of seniors in a large health plan. J Med Internet Res 2016; 18 (03) e50
  • 35 Choi NG, Dinitto DM. The digital divide among low-income homebound older adults: Internet use patterns, eHealth literacy, and attitudes toward computer/Internet use. J Med Internet Res 2013; 15 (05) e93
  • 36 Estacio EV, Whittle R, Protheroe J. The digital divide: examining socio-demographic factors associated with health literacy, access and use of internet to seek health information. J Health Psychol 2017; DOI: 10.1177/1359105317695429.
  • 37 Murray E, Lo B, Pollack L. , et al. The impact of health information on the internet on the physician-patient relationship: patient perceptions. Arch Intern Med 2003; 163 (14) 1727-1734
  • 38 Morris A, Goodman J, Brading H. Internet use and non-use: views of older users. Univers Access Inf Soc 2007; 6 (01) 43-57
  • 39 Neves BB, Amaro F, Fonseca J. Coming of (old) age in the digital age: ICT usage and non-usage among olders adults. Sociol Res Online 2013; 18 (02) 1-14
  • 40 Hughes Halbert C, Welch B, Lynch C. , et al. Social determinants of family health history collection. J Community Genet 2016; 7 (01) 57-64
  • 41 Koehly LM, Peters JA, Kenen R. , et al. Characteristics of health information gatherers, disseminators, and blockers within families at risk of hereditary cancer: implications for family health communication interventions. Am J Public Health 2009; 99 (12) 2203-2209
  • 42 Chen L-S, Li M, Talwar D, Xu L, Zhao M. Chinese Americans' views and use of family health history: a qualitative study. PLoS One 2016; 11 (09) e0162706
  • 43 Kam S, Bylstra Y, Forrest L, Macciocca I, Foo R. Experience of Asian males communicating cardiac genetic risk within the family. J Community Genet 2018; 9 (03) 293-303
  • 44 Quay TA, Frimer L, Janssen PA, Lamers Y. Barriers and facilitators to recruitment of South Asians to health research: a scoping review. BMJ Open 2017; 7 (05) e014889
  • 45 Ford JG, Howerton MW, Lai GY. , et al. Barriers to recruiting underrepresented populations to cancer clinical trials: a systematic review. Cancer 2008; 112 (02) 228-242
  • 46 Alterovitz G, Warner J, Zhang P. , et al. SMART on FHIR Genomics: facilitating standardized clinico-genomic apps. J Am Med Inform Assoc 2015; 22 (06) 1173-1178
  • 47 Warner JL, Rioth MJ, Mandl KD. , et al. SMART precision cancer medicine: a FHIR-based app to provide genomic information at the point of care. J Am Med Inform Assoc 2016; 23 (04) 701-710

Address for correspondence

R. Ryanne Wu, MD, MHS
Department of Medicine, Duke University School of Medicine
411 West Chapel Hill Street
Suite 600, Durham, NC 27701
United States   

  • References

  • 1 Baicker K, Chandra A, Skinner JS. Geographic variation in health care and the problem of measuring racial disparities. Perspect Biol Med 2005; 48 (1, Suppl): S42-S53
  • 2 Williams DR, Wyatt R. Racial bias in health care and health: challenges and opportunities. JAMA 2015; 314 (06) 555-556
  • 3 Paradies Y, Truong M, Priest N. A systematic review of the extent and measurement of healthcare provider racism. J Gen Intern Med 2014; 29 (02) 364-387
  • 4 Kansagara D, Englander H, Salanitro A. , et al. Risk prediction models for hospital readmission: a systematic review. JAMA 2011; 306 (15) 1688-1698
  • 5 Pisters R, Lane DA, Nieuwlaat R, de Vos CB, Crijns HJGM, Lip GYH. A novel user-friendly score (HAS-BLED) to assess 1-year risk of major bleeding in patients with atrial fibrillation: the Euro Heart Survey. Chest 2010; 138 (05) 1093-1100
  • 6 Subramanian S, Bobashev G, Morris RJ, Hoover S. Personalized medicine for prevention: can risk stratified screening decrease colorectal cancer mortality at an acceptable cost?. Cancer Causes Control 2017; 28 (04) 299-308
  • 7 Cohn WF, Ropka ME, Pelletier SL. , et al. Health Heritage© a web-based tool for the collection and assessment of family health history: initial user experience and analytic validity. Public Health Genomics 2010; 13 (7–8): 477-491
  • 8 Elliott LS, Henderson JC, Neradilek MB, Moyer NA, Ashcraft KC, Thirumaran RK. Clinical impact of pharmacogenetic profiling with a clinical decision support tool in polypharmacy home health patients: a prospective pilot randomized controlled trial. PLoS One 2017; 12 (02) e0170905
  • 9 Larson EA, Wilke RA. Integration of genomics in primary care. Am J Med 2015; 128 (11) 1251.e1-1251.e5
  • 10 Welch BM, Wiley K, Pflieger L. , et al. Review and comparison of electronic patient-facing family health history tools. J Genet Couns 2018; 27 (02) 381-391
  • 11 Qureshi N, Wilson B, Santaguida P. , et al. Collection and use of cancer family history in primary care. Evid Rep Technol Assess (Full Rep) 2007; (159) 1-84
  • 12 Mikat-Stevens NA, Larson IA, Tarini BA. Primary-care providers' perceived barriers to integration of genetics services: a systematic review of the literature. Genet Med 2015; 17 (03) 169-176
  • 13 Berg AO, Baird MA, Botkin JR. , et al. National Institutes of Health state-of-the-science conference statement: family history and improving health. Ann Intern Med 2009; 151 (12) 872-877
  • 14 Gramling R, Nash J, Siren K, Eaton C, Culpepper L. Family physician self-efficacy with screening for inherited cancer risk. Ann Fam Med 2004; 2 (02) 130-132
  • 15 de Hoog CL, Portegijs PJ, Stoffers HE. Family history tools for primary care are not ready yet to be implemented. a systematic review. Eur J Gen Pract 2014; 20 (02) 125-133
  • 16 Hulse NC, Ranade-Kharkar P, Post H, Wood GM, Williams MS, Haug PJ. Development and early usage patterns of a consumer-facing family health history tool. AMIA Annu Symp Proc 2011; 2011: 578-587
  • 17 Wu RR, Himmel TL, Buchanan AH. , et al. Quality of family history collection with use of a patient facing family history assessment tool. BMC Fam Pract 2014; 15: 31
  • 18 Rubinstein WS, Acheson LS, O'Neill SM. , et al; Family Healthware Impact Trial (FHITr) Group. Clinical utility of family history for cancer screening and referral in primary care: a report from the Family Healthware Impact Trial. Genet Med 2011; 13 (11) 956-965
  • 19 Orlando LA, Wu RR, Beadles C. , et al. Implementing family health history risk stratification in primary care: impact of guideline criteria on populations and resource demand. Am J Med Genet C Semin Med Genet 2014; 166C (01) 24-33
  • 20 Ruffin IV MT, Nease Jr DE, Sen A. , et al; Family History Impact Trial (FHITr) Group. Effect of preventive messages tailored to family history on health behaviors: the Family Healthware Impact Trial. Ann Fam Med 2011; 9 (01) 3-11
  • 21 Orlando LA, Wu RR, Myers RA. , et al. Clinical utility of a Web-enabled risk-assessment and clinical decision support program. Genet Med 2016; 18 (10) 1020-1028
  • 22 Orlando LA, Buchanan AH, Hahn SE. , et al. Development and validation of a primary care-based family health history and decision support program (MeTree). N C Med J 2013; 74 (04) 287-296
  • 23 Wang C, Bickmore T, Bowen DJ. , et al. Acceptability and feasibility of a virtual counselor (VICKY) to collect family health histories. Genet Med 2015; 17 (10) 822-830
  • 24 Wu RR, Myers RA, McCarty CA. , et al; Family Health History Network. Protocol for the “Implementation, adoption, and utility of family history in diverse care settings” study. Implement Sci 2015; 10: 163
  • 25 Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care 2012; 50 (03) 217-226
  • 26 Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999; 89 (09) 1322-1327
  • 27 Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Fam Med 2004; 36 (08) 588-594
  • 28 NCI. Health Information National Trends Survey. Available at: https://hints.cancer.gov/view-questions-topics/question-details.aspx?red=1&qid=413 . Accessed August 29, 2018
  • 29 Bennett RL. The family medical history. Prim Care 2004; 31 (03) 479-495
  • 30 Wu RR, Myers RA, Sperber N. , et al. Implementation, adoption, and utility of family health history risk assessment in diverse care settings: evaluating implementation processes and impact with an implementation framework. Genet Med 2018; DOI: 10.1038/s41436-018-0049-x.
  • 31 Lyles C, Schillinger D, Sarkar U. Connecting the dots: health information technology expansion and health disparities. PLoS Med 2015; 12 (07) e1001852
  • 32 Cunningham TJ, Croft JB, Liu Y, Lu H, Eke PI, Giles WH. Vital signs: racial disparities in age-specific mortality among Blacks or African Americans - United States, 1999-2015. Morb Mortal Wkly Rep 2017; 66 (17) 444-456
  • 33 O'Connell J, Yi R, Wilson C, Manson SM, Acton KJ. Racial disparities in health status: a comparison of the morbidity among American Indian and U.S. adults with diabetes. Diabetes Care 2010; 33 (07) 1463-1470
  • 34 Gordon NP, Hornbrook MC. Differences in access to and preferences for using patient portals and other eHealth technologies based on race, ethnicity, and age: a database and survey study of seniors in a large health plan. J Med Internet Res 2016; 18 (03) e50
  • 35 Choi NG, Dinitto DM. The digital divide among low-income homebound older adults: Internet use patterns, eHealth literacy, and attitudes toward computer/Internet use. J Med Internet Res 2013; 15 (05) e93
  • 36 Estacio EV, Whittle R, Protheroe J. The digital divide: examining socio-demographic factors associated with health literacy, access and use of internet to seek health information. J Health Psychol 2017; DOI: 10.1177/1359105317695429.
  • 37 Murray E, Lo B, Pollack L. , et al. The impact of health information on the internet on the physician-patient relationship: patient perceptions. Arch Intern Med 2003; 163 (14) 1727-1734
  • 38 Morris A, Goodman J, Brading H. Internet use and non-use: views of older users. Univers Access Inf Soc 2007; 6 (01) 43-57
  • 39 Neves BB, Amaro F, Fonseca J. Coming of (old) age in the digital age: ICT usage and non-usage among olders adults. Sociol Res Online 2013; 18 (02) 1-14
  • 40 Hughes Halbert C, Welch B, Lynch C. , et al. Social determinants of family health history collection. J Community Genet 2016; 7 (01) 57-64
  • 41 Koehly LM, Peters JA, Kenen R. , et al. Characteristics of health information gatherers, disseminators, and blockers within families at risk of hereditary cancer: implications for family health communication interventions. Am J Public Health 2009; 99 (12) 2203-2209
  • 42 Chen L-S, Li M, Talwar D, Xu L, Zhao M. Chinese Americans' views and use of family health history: a qualitative study. PLoS One 2016; 11 (09) e0162706
  • 43 Kam S, Bylstra Y, Forrest L, Macciocca I, Foo R. Experience of Asian males communicating cardiac genetic risk within the family. J Community Genet 2018; 9 (03) 293-303
  • 44 Quay TA, Frimer L, Janssen PA, Lamers Y. Barriers and facilitators to recruitment of South Asians to health research: a scoping review. BMJ Open 2017; 7 (05) e014889
  • 45 Ford JG, Howerton MW, Lai GY. , et al. Barriers to recruiting underrepresented populations to cancer clinical trials: a systematic review. Cancer 2008; 112 (02) 228-242
  • 46 Alterovitz G, Warner J, Zhang P. , et al. SMART on FHIR Genomics: facilitating standardized clinico-genomic apps. J Am Med Inform Assoc 2015; 22 (06) 1173-1178
  • 47 Warner JL, Rioth MJ, Mandl KD. , et al. SMART precision cancer medicine: a FHIR-based app to provide genomic information at the point of care. J Am Med Inform Assoc 2016; 23 (04) 701-710

Zoom Image
Fig. 1 Study flow diagram.