CC BY-NC-ND 4.0 · ACI open 2022; 06(02): e123-e128
DOI: 10.1055/s-0042-1749191
Case Report

Design and Development of Halyos: A Patient-Facing Visual EHR Interface for Longitudinal Risk Awareness

Samson Mataraso*
1   Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States
2   Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, California, United States
3   Department of Bioengineering, University of California, Berkeley, Berkeley, California, United States
,
Vimig Socrates*
1   Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States
4   Department of Electrical Engineering and Computer Science, Case Western Reserve University, Cleveland, Ohio, United States
5   Department of Population and Quantitative Health Sciences, Case Western Reserve University, Cleveland, Ohio, United States
,
Fritz Lekschas
6   Harvard John A. Paulson School of Engineering and Applied Sciences, Cambridge, Massachusetts, United States
,
Nils Gehlenborg
1   Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States
› Author Affiliations
Funding This work was enabled by NIH grants U54HG007963 and R00HG007583.
 

Abstract

Objectives We have developed Halyos, a visual electronic health record (EHR) web application that complements existing patient portals. Halyos is designed to integrate with existing EHR systems to help patients interpret their health data.

Methods The Halyos application utilizes the Substitutable Medical Applications and Reusable Technologies on Fast Healthcare Interoperability Resources (SMART on FHIR) platform to create an interoperable interface that provides interactive visualizations of clinically validated risk scores and longitudinal data derived from a patient's clinical measurements.

Results These visualizations allow patients to investigate the relationships between clinical measurements and risk over time.

Discussion By enabling patients to set hypothetical future values for these clinical measurements, patients can see how changes in their health will impact their risks.

Conclusion Using Halyos, patients are provided with the opportunity to actively improve their health based on increased understanding of longitudinal information available in EHRs and to begin a dialogue with their providers.


#

Introduction

Patients are gaining greater access to data stored in electronic health records (EHRs) through patient-facing personal health records (PHRs). However, many patients have difficulties understanding their own health information.[1] Properly aggregating, representing, and explaining these data has the potential to empower patients to make informed decisions about their health, encourage preventive care, and promote collaborative decision-making between clinicians and patients by providing patients with an alternate source of personalized, interpretable health information. We introduce Halyos (http://halyos.gehlenborglab.org/) ([Fig. 1]) as a web-based application that allows patients to explore their health data through clinically validated risk scores for disease, visualize their longitudinal measurements, and understand how changes in their health (e.g., increased blood pressure) impact their future disease risk scores.

Zoom Image
Fig. 1 (a) Home screen of Halyos. Risk scores are calculated from patient measurements and displayed as percentages (1). Individual measurement values are displayed under the Measurements section (2) with a detailed view shown in [Fig. 2]. The dashboard also includes suggestive preventative care and environmental factors that affect health risks. (b) Expanded risk tile, which displays pictorial unit charts representing a patient's past, present, and future risk (3) with a written description of the risk score (4).

Although nearly 90% of hospitals provide the ability for patients to view, download, and transmit health records,[2] they cannot be used effectively as patients may struggle to understand the information presented to them,[1] [3] partially due to comprehension barriers.[4] Many users, especially those with low health literacy,[5] experience difficulty performing tasks related to health interpretation when using PHRs.[6] [7] Our interface expands upon existing work that uses data visualization to increase patients' abilities to understand their personal health data through risk-focused data aggregation.[8] [9] We have synthesized patients' clinical data into single quantifiable values using clinically validated risk scores. For example, to increase the interpretability of risk by patients without a medical degree, the Reynolds risk score[10] [11] was previously visualized (https://apps.smarthealthit.org/app/cardiac-risk) by showing patients their blood test results along with a scale of desirable to high risk of cardiovascular disease. Other solutions have allowed clinicians to explore risk scores to augment their understanding of the role of certain clinical variables in the ultimate risk outcome.[12] We complement and expand upon such existing solutions by designing a patient-facing application that not only presents a patient with their risk scores but also provides details about clinical measurements and external factors (e.g., pollen count) that allow a patient to actively engage with their own data.

As a proof of principle, our application introduces visualizations intended to enable patients to interpret clinical measurements and improve their health understanding. Further, we have created visualizations of patients' longitudinal data, which gives them the ability to compare their current health risks with their risk in the past and their hypothetical risk in the future. This comparison can provide patients insights into the way their health has changed over time. By creating a resource that allows patients to engage with and explore their health data, we aim to increase patients' understanding of their health, stimulate the use of preventive health services, encourage patients to initiate informed conversations with their clinicians, and improve patient outcomes. Previous work has shown that PHRs have helped patients become aware of relevant preventative health services and have more effective conversations with their clinical care team.[13]


#

Methods

Risk Scores

Twenty-eight risk scores were collected from a review of the literature and five were selected ([Table 1]) to be used for this proof of concept, based on their relative applicability to a variety of patients, their clinical validity, and availability of necessary clinical variables available in the sample EHR. Synthetic values for clinical variables (e.g., age, cholesterol, blood pressure) are extracted from a sample Fast Healthcare Interoperability Resources (FHIR) (https://www.hl7.org/fhir/index.html) data store ( http://hapi.fhir.org/ ) and used to compute the selected risk scores. While these risk scores may not necessarily be relevant to every patient, they demonstrate the broad applicability of our application. Halyos is designed to allow the easy addition of more applicable risk scores, as discussed in Section “Customization”. We have included a section in the measurements panel for patients to enter patient-provided information not conventionally stored in the EHR that are required by some risk scores, such as the family history of a certain disease or smoking status. These data can be conveniently entered by the user by toggling a checkbox (see [Fig. 1a], Section 2) and are used in the relevant risk score calculations during a user session, after which the data are erased to protect patient privacy.

Table 1

Risk calculators implemented in Halyos

Disease referenced

Measurements used

Cardiovascular disease (Reynolds risk score)[10] [11]

Gender; smoker; blood pressure; cholesterol; high-density lipoprotein cholesterol; high-sensitivity C-reactive protein; family history

Nonalcoholic fatty liver disease[28]

Age; body mass index; presence of impaired fasting glucose or diabetes; aspartate aminotransferase/alanine aminotransferase ratio; platelet count; albumin

Kidney failure from chronic kidney disease[29]

Age; sex; glomerular filtration rate; albumin creatinine

Mortality risk from chronic obstructive pulmonary disorder[30]

Blood urea nitrogen; mental status; heart rate; age

Risk of developing diabetes[31]

Age; body mass index; waist circumference; history of antihypertensive drug treatment; high blood glucose; gender


#

Design

The user interface was designed around the presentation of risk scores. Communication of health risk using risk personalized scores has been shown to improve patient risk estimation.[14] [15] We used pictorial unit charts to help patients visualize their risk, which has been shown to be effective in communicating risk to patients.[16] [17] We used a line graph to visualize how clinical measurements changed over time, which included an indication of the normal range (see [Fig. 2], Section 2). Contextualized temporal visualizations have been noted as an important design feature for patient-facing visualizations of personal health data.[18] Patients are also able to change the values of various clinical measurements (e.g., reduce their systolic blood pressure) to understand the impact the change would have on their risk of various diseases. Using these features as the core functionality of the patient portal, we designed and implemented a clean and responsive user interface with several visualizations to enable patient engagement.[19] The source code for the interface is available at https://github.com/hms-dbmi/halyos/.

Zoom Image
Fig. 2 Graph of a patient’s specific measurement values over time (1). The horizontal blue line represents the patient selected past date. The blue dot indicates which value is closest to that past date and therefore which value is being used to calculate the past risk score (2). The slider on the right allows a patient to manipulate their “future“ value, which is used to calculate the future risk score (3).

#

SMART on FHIR

The Substitutable Medical Applications and Reusable Technologies (SMART) on FHIR is a platform that allows third-party applications to send requests securely to EHR systems to retrieve patient data.[20] We utilized SMART on FHIR during development to enable the integration of personalized patient data. Other apps have been successfully implemented into an EHR with the SMART on FHIR platform,[21] and it is broadly recommended as a tool for integrating third-party apps into EHRs.[22]


#
#

Results

Risk Scores

When a patient first uses Halyos, the most prominent features are the risk scores located at the top ([Fig. 1], Section 1). Since all measurements required for a risk score may not have been taken simultaneously, we calculate current risk scores based on the most recent values of each of the measurements required. Upon clicking on a risk score, a section is displayed with in-depth information ([Fig. 1], Section 2). Here, we provide a risk visualization that displays a patient's past, present, and hypothetical future risk score as a pictorial unit chart of 100 dots ([Fig. 1], Section 3). Each black dot corresponds to one percentage point of risk, e.g., five dots indicate 5% risk. The dots change in size to indicate increases or decreases from a patient's current risk. By selecting a past date to compare with their present health, patients can explore how their health has changed. Pictorial unit charts have been shown to be effective at communicating risks in health care settings, particularly to those with low numeracy.[23] To aid with interpretability, only the measurements that contribute to selected risk scores are shown. This filtering draws a clear link between particular clinical measurements and their associated risks.


#

Clinical Measurements

The bottom left of the dashboard lists a patient's clinical measurements ([Fig. 1], Section 2), including a section for patients to enter relevant data not typically stored in the EHR. An arrow highlights how clinical measurements have changed since a user-specified past date. Clicking on a measurement reveals a visualization of longitudinal data. It shows the patient's data of the selected clinical measurement over time and includes typical reference ranges, as reported in the EHR ([Fig. 2], Section 1). Patients can horizontally pan, zoom, and brush the graph to explore a time period of interest. The future value is adjustable to help the patient understand the impact of a potential future measurement on relevant risk scores. Along with this visualization, text is displayed explaining the measurement. For the purpose of this article, we did not curate audience-appropriate definitions, but this can be done in a clinical setting.


#

Clinical Measurement Visualization

The patient's ability to select a past date on the graph, which is used to find the nearest measurements and calculate “past” risk scores, enables exploration of their health data over time. The measurement used for risk score calculations is displayed as a light blue dot ([Fig. 2], Section 2). These interactions build intuition for key drivers of various disease risks. For example, observing a decrease in the Reynolds risk score as blood pressure decreases enforces that blood pressure is important for a healthy cardiovascular system.


#

Customization

We designed Halyos to be as modular as possible, with a single configuration file containing all risk scores. To change or add a new risk score, a clinician can work with a health system's information technology (IT) team to identify the Logical Observation Identifiers Names and Codes (LOINC) codes of the measurements that are necessary and implement a function to calculate the new score. While Halyos implements five generally applicable risk scores as a proof of concept, the extensible design of Halyos enables the simple addition of more risk scores as long as the relevant measurements are available in the EHR.


#

External Data Integration

We used the U.S. Preventive Services Task Force application programming interface (API) to integrate suggested preventive care measures customized to a patient's gender and age. These suggestions have been shown to increase patient engagement with PHRs as well as utilization of these preventive services.[24] Several other non-EHR data sources are integrated to provide patients with a comprehensive view of their health such as local pollen levels, air quality, and flu cases. Pollen levels are taken from the AccuWeather API (https://developer.accuweather.com/), air quality from IQAir (https://airvisual.com/api), and flu cases from Flu Near You by HealthMap.[25] These specific data were chosen to be included based on insights from one human-centered design workshop conducted during the initial design of Halyos, which followed the methodology described in the citation.[26]


#
#

Discussion

One of the key contributions of Halyos is the way it presents health data in context, which can potentially lead a patient to a greater understanding of their own health. The temporal aspect of the clinical measurement visualization further contextualizes clinical data, as it allows patients to see changes in health over time. Viewing their past and future disease risks, and the contributing measurements, allow patients to gain intuition for how certain measurements may impact their health. We display recorded reference ranges for individual measurements where available, allowing patients to begin interpreting their risk. However, reference ranges are often not available for overall risk scores. Therefore, we emphasize the importance of clinician oversight to reduce the risk of misinterpretation. Halyos has been developed as a communication tool to facilitate conversation between patients and clinicians and not for a patient's independent use. In additional to measurement values, our application allows manually entered data to be used in risk scores, making it powerfully customizable to unique patient populations. Finally, we begin to integrate auxiliary information beyond medical measurements, such as environmental data and preventive care suggestions.

A major limitation of the work is that it was not prospectively evaluated. Here, Halyos is introduced as a proof of concept; a prospective evaluation would be needed prior to widespread adoption. Any implementation would also need to include training for clinicians, so they understand how to use the tool effectively with their patients. As another limitation, important data points are not captured within an EHR that can determine the trajectory of a person's health, such as exercise habits, eating habits, and drug use. While we have included functionality for user input of relevant non-EHR data, the level of patient engagement remains to be explored. Additionally, since EHR data are not collected at standard intervals, data categorized together as “present” measurements may actually have been measured months apart; to mitigate this, we plan to add a filter that limits measurements to within a provider-specified date range (and if no relevant data are available for a required measurement, the score will not be omitted). Patients should also have more granular information regarding the date/time that a particular measurement was recorded. Future work will also include an implementation of visual cues that alert the user to the recency of each measurement in the detailed page for the associated risk scores.

Using Halyos as a proof of principle, we demonstrate the feasibility of employing existing SMART on FHIR technology to calculate personalized risk scores and promote health data interpretability. By leveraging modern interoperability standards, we enable the addition of Halyos to the SMART on FHIR app gallery, allowing any FHIR-compliant EHR to securely provide our application with a patient's full data. The eventual goal is to integrate Halyos into a live EHR system; it is important to note that Halyos would only be able to integrate into an EHR that is FHIR compliant. Halyos addresses concerns raised by clinicians and patients in various studies, such as difficulty understanding medical terminology and lack of appropriate visualization.[6] [27] Future work will involve a user study to comprehensively assess the effectiveness of Halyos in improving personal health understanding, as well as its ability to help patients meet their health goals. Additionally, Halyos may be able to support shared decision-making, as the visualizations can better help a patient understand how a particular medical decision may impact their health. We also plan to provide deeper links between data sources, such as customizing preventive care suggestions within the context of the environment (e.g., suggest indoor exercises if outdoor air quality is poor). In further development, we intend to add functionality for patients to correct information in the EHR that may be incorrect, such as their zip code for determining environmental data (though these changes would not be propagated to the EHR itself). Finally, to promote broader adoption by clinicians unfamiliar with code, we plan to develop an administrative user interface into which hospital administrative staff could enter equations and LOINC codes. Halyos would automatically compute and integrate these new risk scores and allow clinicians to customize them to a particular patient. To ensure clinical accuracy, these new risk scores can then be validated by clinicians and accessed by all providers at the institution. Keeping in mind the increasing burden of EHRs, the design and implementation of this administrative interface would be tested for usability and integration into clinical workflows. The eventual goal is to integrate Halyos into a live EHR system. Using the SMART on FHIR specification, tools such as ours could be downloaded and securely integrated into a patient's standard health portal at their own discretion. This would allow widespread testing of its efficacy in increasing patient health.


#

Conclusion

Our work contributes representations of clinical data that aim to make a patient's health more interpretable through both data synthesis via risk scores and the ability to temporally explore personalized health data. As tools like SMART on FHIR improve patients' accessibility to their personal health data, developing meaningful ways for patients to explore and interact with this data becomes particularly important. To this end, we developed Halyos to aggregate and visualize patient data in a way that encourages exploration. This exploration was designed to allow patients to gain intuition about their health and stimulate conversation with the clinical care team, resulting in patients making more informed decisions about their care. This work is particularly timely, as interoperability increasingly allows for personalized medicine solutions to easily adapt to a patient's data.


#

Clinical Relevance Statement

Communicating basic measurements without clinical context can often overwhelm patients trying to evaluate their overall health, particularly those with limited health literacy. A unified, interactive design, as presented here, may allow patients to better understand the impact of individual clinical measurements on their future disease risk. Such awareness may help a patient take more agency over their health, communicate more effectively with their care providers, and make more informed decisions.


#
#

Conflict of Interest

None declared.

Acknowledgments

The authors acknowledge members of the Gehlenborg Laboratory and Susanne Churchill for mentorship and support. We would also like to acknowledge the participants of our user-centered design workshop.

Protection of Human and Animal Subjects

The human-centered design workshop is exempt under 45 CFR 46.101(b) Categories of Exempt Human Subjects Research Exemption 2 with no identifying information recorded.


* These authors contributed equally to this work.


  • References

  • 1 Sarkar U, Karter AJ, Liu JY. et al. The literacy divide: health literacy and the use of an internet-based patient portal in an integrated health system-results from the diabetes study of northern California (DISTANCE). J Health Commun 2010; 15 (02, Suppl 02): 183-196
  • 2 Office of the National Coordinator for Health Information Technology. Hospital progress to meaningful use by size, type, and urban/rural location. Published online August 2017. Available at: https://dashboard.healthit.gov/quickstats/pages/FIG-Hospital-Progress-to-Meaningful-Use-by-size-practice-setting-area-type.php
  • 3 Choi NG, Dinitto DM. The digital divide among low-income homebound older adults: Internet use patterns, eHealth literacy, and attitudes toward computer/Internet use. J Med Internet Res 2013; 15 (05) e93
  • 4 Goldzweig CL, Orshansky G, Paige NM. et al. Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review. Ann Intern Med 2013; 159 (10) 677-687
  • 5 Baker DW. The meaning and the measure of health literacy. J Gen Intern Med 2006; 21 (08) 878-883
  • 6 Tieu L, Schillinger D, Sarkar U. et al. Online patient websites for electronic health record access among vulnerable populations: portals to nowhere?. J Am Med Inform Assoc 2017; 24 (e1): e47-e54
  • 7 Tieu L, Sarkar U, Schillinger D. et al. Barriers and facilitators to online portal use among patients and caregivers in a safety net health care system: a qualitative study. J Med Internet Res 2015; 17 (12) e275
  • 8 Sox CM, Gribbons WM, Loring BA, Mandl KD, Batista R, Porter SC. Patient-centered design of an information management module for a personally controlled health record. J Med Internet Res 2010; 12 (03) e36
  • 9 Britto MT, Jimison HB, Munafo JK, Wissman J, Rogers ML, Hersh W. Usability testing finds problems for novice users of pediatric portals. J Am Med Inform Assoc 2009; 16 (05) 660-669
  • 10 Ridker PM, Buring JE, Rifai N, Cook NR. Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA 2007; 297 (06) 611-619
  • 11 Ridker PM, Paynter NP, Rifai N, Gaziano JM, Cook NR. C-reactive protein and parental history improve global cardiovascular risk prediction: the Reynolds Risk Score for men. Circulation 2008; 118 (22) 2243-2251
  • 12 Krause J, Perer A, Ng K. Interacting with predictions: visual inspection of black-box machine learning models. Paper presented at: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems; May 2016: 5686-5697
  • 13 Ant Ozok A, Wu H, Garrido M, Pronovost PJ, Gurses AP. Usability and perceived usefulness of Personal Health Records for preventive health care: a case study focusing on patients' and primary care providers' perspectives. Appl Ergon 2014; 45 (03) 613-628
  • 14 Welschen LMC, Bot SDM, Kostense PJ. et al. Effects of cardiovascular disease risk communication for patients with type 2 diabetes on risk perception in a randomized controlled trial: the @RISK study. Diabetes Care 2012; 35 (12) 2485-2492
  • 15 Harle CA, Downs JS, Padman R. Effectiveness of personalized and interactive health risk calculators: a randomized trial. Med Decis Making 2012; 32 (04) 594-605
  • 16 Ancker JS, Senathirajah Y, Kukafka R, Starren JB. Design features of graphs in health risk communication: a systematic review. J Am Med Inform Assoc 2006; 13 (06) 608-618
  • 17 Zikmund-Fisher BJ. Risk reduction to motivate specific action using an icon array display. Visualizing Health. Published January 27, 2014. Available at: http://www.vizhealth.org/
  • 18 Turchioe MR, Myers A, Isaac S. et al. A systematic review of patient-facing visualizations of personal health data. Appl Clin Inform 2019; 10 (04) 751-770
  • 19 Irizarry T, DeVito Dabbs A, Curran CR. Patient portals and patient engagement: a state of the science review. J Med Internet Res 2015; 17 (06) e148
  • 20 Mandel JC, Kreda DA, Mandl KD, Kohane IS, Ramoni RB. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc 2016; 23 (05) 899-908
  • 21 Scalia P, Ahmad F, Schubbe D. et al. Integrating option grid patient decision aids in the epic electronic health record: case study at 5 health systems. J Med Internet Res 2021; 23 (05) e22766
  • 22 Karhade AV, Schwab JH, Del Fiol G, Kawamoto K. SMART on FHIR in spine: integrating clinical prediction models into electronic health records for precision medicine at the point of care. Spine J 2021; 21 (10) 1649-1651
  • 23 Galesic M, Garcia-Retamero R, Gigerenzer G. Using icon arrays to communicate medical risks: overcoming low numeracy. Health Psychol 2009; 28 (02) 210-216
  • 24 Krist AH, Woolf SH, Bello GA. et al. Engaging primary care patients to use a patient-centered personal health record. Ann Fam Med 2014; 12 (05) 418-426
  • 25 Chunara R, Aman S, Smolinski M, Brownstein JS. Flu Near You: an online self-reported influenza surveillance system in the USA. Online J Public Health Inform 2013; 5 (01) DOI: 10.5210/ojphi.v5i1.4456.
  • 26 Kerzner E, Goodwin S, Dykes J, Jones S, Meyer M. A framework for creative visualization-opportunities workshops. IEEE Trans Vis Comput Graph 2018; 25 (01) 748-758
  • 27 Haggstrom DA, Saleem JJ, Russ AL, Jones J, Russell SA, Chumbler NR. Lessons learned from usability testing of the VA's personal health record. J Am Med Inform Assoc 2011; 18 (01, Suppl 01): i13-i17
  • 28 Angulo P, Hui JM, Marchesini G. et al. The NAFLD fibrosis score: a noninvasive system that identifies liver fibrosis in patients with NAFLD. Hepatology 2007; 45 (04) 846-854
  • 29 Tangri N, Stevens LA, Griffith J. et al. A predictive model for progression of chronic kidney disease to kidney failure. JAMA 2011; 305 (15) 1553-1559
  • 30 Shorr AF, Sun X, Johannes RS, Yaitanes A, Tabak YP. Validation of a novel risk score for severity of illness in acute exacerbations of COPD. Chest 2011; 140 (05) 1177-1183
  • 31 Lindström J, Tuomilehto J. The diabetes risk score: a practical tool to predict type 2 diabetes risk. Diabetes Care 2003; 26 (03) 725-731

Address for correspondence

Nils Gehlenborg, PhD
Department of Biomedical Informatics, Harvard Medical School
10 Shattuck Street, Boston, MA 02115
United States   

Publication History

Received: 26 January 2021

Accepted: 03 February 2022

Article published online:
22 December 2022

© 2022. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

  • References

  • 1 Sarkar U, Karter AJ, Liu JY. et al. The literacy divide: health literacy and the use of an internet-based patient portal in an integrated health system-results from the diabetes study of northern California (DISTANCE). J Health Commun 2010; 15 (02, Suppl 02): 183-196
  • 2 Office of the National Coordinator for Health Information Technology. Hospital progress to meaningful use by size, type, and urban/rural location. Published online August 2017. Available at: https://dashboard.healthit.gov/quickstats/pages/FIG-Hospital-Progress-to-Meaningful-Use-by-size-practice-setting-area-type.php
  • 3 Choi NG, Dinitto DM. The digital divide among low-income homebound older adults: Internet use patterns, eHealth literacy, and attitudes toward computer/Internet use. J Med Internet Res 2013; 15 (05) e93
  • 4 Goldzweig CL, Orshansky G, Paige NM. et al. Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review. Ann Intern Med 2013; 159 (10) 677-687
  • 5 Baker DW. The meaning and the measure of health literacy. J Gen Intern Med 2006; 21 (08) 878-883
  • 6 Tieu L, Schillinger D, Sarkar U. et al. Online patient websites for electronic health record access among vulnerable populations: portals to nowhere?. J Am Med Inform Assoc 2017; 24 (e1): e47-e54
  • 7 Tieu L, Sarkar U, Schillinger D. et al. Barriers and facilitators to online portal use among patients and caregivers in a safety net health care system: a qualitative study. J Med Internet Res 2015; 17 (12) e275
  • 8 Sox CM, Gribbons WM, Loring BA, Mandl KD, Batista R, Porter SC. Patient-centered design of an information management module for a personally controlled health record. J Med Internet Res 2010; 12 (03) e36
  • 9 Britto MT, Jimison HB, Munafo JK, Wissman J, Rogers ML, Hersh W. Usability testing finds problems for novice users of pediatric portals. J Am Med Inform Assoc 2009; 16 (05) 660-669
  • 10 Ridker PM, Buring JE, Rifai N, Cook NR. Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA 2007; 297 (06) 611-619
  • 11 Ridker PM, Paynter NP, Rifai N, Gaziano JM, Cook NR. C-reactive protein and parental history improve global cardiovascular risk prediction: the Reynolds Risk Score for men. Circulation 2008; 118 (22) 2243-2251
  • 12 Krause J, Perer A, Ng K. Interacting with predictions: visual inspection of black-box machine learning models. Paper presented at: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems; May 2016: 5686-5697
  • 13 Ant Ozok A, Wu H, Garrido M, Pronovost PJ, Gurses AP. Usability and perceived usefulness of Personal Health Records for preventive health care: a case study focusing on patients' and primary care providers' perspectives. Appl Ergon 2014; 45 (03) 613-628
  • 14 Welschen LMC, Bot SDM, Kostense PJ. et al. Effects of cardiovascular disease risk communication for patients with type 2 diabetes on risk perception in a randomized controlled trial: the @RISK study. Diabetes Care 2012; 35 (12) 2485-2492
  • 15 Harle CA, Downs JS, Padman R. Effectiveness of personalized and interactive health risk calculators: a randomized trial. Med Decis Making 2012; 32 (04) 594-605
  • 16 Ancker JS, Senathirajah Y, Kukafka R, Starren JB. Design features of graphs in health risk communication: a systematic review. J Am Med Inform Assoc 2006; 13 (06) 608-618
  • 17 Zikmund-Fisher BJ. Risk reduction to motivate specific action using an icon array display. Visualizing Health. Published January 27, 2014. Available at: http://www.vizhealth.org/
  • 18 Turchioe MR, Myers A, Isaac S. et al. A systematic review of patient-facing visualizations of personal health data. Appl Clin Inform 2019; 10 (04) 751-770
  • 19 Irizarry T, DeVito Dabbs A, Curran CR. Patient portals and patient engagement: a state of the science review. J Med Internet Res 2015; 17 (06) e148
  • 20 Mandel JC, Kreda DA, Mandl KD, Kohane IS, Ramoni RB. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc 2016; 23 (05) 899-908
  • 21 Scalia P, Ahmad F, Schubbe D. et al. Integrating option grid patient decision aids in the epic electronic health record: case study at 5 health systems. J Med Internet Res 2021; 23 (05) e22766
  • 22 Karhade AV, Schwab JH, Del Fiol G, Kawamoto K. SMART on FHIR in spine: integrating clinical prediction models into electronic health records for precision medicine at the point of care. Spine J 2021; 21 (10) 1649-1651
  • 23 Galesic M, Garcia-Retamero R, Gigerenzer G. Using icon arrays to communicate medical risks: overcoming low numeracy. Health Psychol 2009; 28 (02) 210-216
  • 24 Krist AH, Woolf SH, Bello GA. et al. Engaging primary care patients to use a patient-centered personal health record. Ann Fam Med 2014; 12 (05) 418-426
  • 25 Chunara R, Aman S, Smolinski M, Brownstein JS. Flu Near You: an online self-reported influenza surveillance system in the USA. Online J Public Health Inform 2013; 5 (01) DOI: 10.5210/ojphi.v5i1.4456.
  • 26 Kerzner E, Goodwin S, Dykes J, Jones S, Meyer M. A framework for creative visualization-opportunities workshops. IEEE Trans Vis Comput Graph 2018; 25 (01) 748-758
  • 27 Haggstrom DA, Saleem JJ, Russ AL, Jones J, Russell SA, Chumbler NR. Lessons learned from usability testing of the VA's personal health record. J Am Med Inform Assoc 2011; 18 (01, Suppl 01): i13-i17
  • 28 Angulo P, Hui JM, Marchesini G. et al. The NAFLD fibrosis score: a noninvasive system that identifies liver fibrosis in patients with NAFLD. Hepatology 2007; 45 (04) 846-854
  • 29 Tangri N, Stevens LA, Griffith J. et al. A predictive model for progression of chronic kidney disease to kidney failure. JAMA 2011; 305 (15) 1553-1559
  • 30 Shorr AF, Sun X, Johannes RS, Yaitanes A, Tabak YP. Validation of a novel risk score for severity of illness in acute exacerbations of COPD. Chest 2011; 140 (05) 1177-1183
  • 31 Lindström J, Tuomilehto J. The diabetes risk score: a practical tool to predict type 2 diabetes risk. Diabetes Care 2003; 26 (03) 725-731

Zoom Image
Fig. 1 (a) Home screen of Halyos. Risk scores are calculated from patient measurements and displayed as percentages (1). Individual measurement values are displayed under the Measurements section (2) with a detailed view shown in [Fig. 2]. The dashboard also includes suggestive preventative care and environmental factors that affect health risks. (b) Expanded risk tile, which displays pictorial unit charts representing a patient's past, present, and future risk (3) with a written description of the risk score (4).
Zoom Image
Fig. 2 Graph of a patient’s specific measurement values over time (1). The horizontal blue line represents the patient selected past date. The blue dot indicates which value is closest to that past date and therefore which value is being used to calculate the past risk score (2). The slider on the right allows a patient to manipulate their “future“ value, which is used to calculate the future risk score (3).