Integration of Risk Scores and Integration Capability in Electronic Patient Records

Abstract

Background Digital availability of patient data is continuously improving with the increasing implementation of electronic patient records in physician practices. The emergence of digital health data defines new fields of application for data analytics applications, which in turn offer extensive options of using data. Common areas of data analytics applications include decision support, administration, and fraud detection. Risk scores play an important role in compiling algorithms that underlay tools for decision support.

Objectives This study aims to identify the current state of risk score integration and integration capability in electronic patient records for cardiovascular disease and diabetes in German primary care practices.

Methods We developed an evaluation framework to determine the current state of risk score integration and future integration options for four cardiovascular disease risk scores (arriba, Pooled Cohort Equations, QRISK3, and Systematic Coronary Risk Evaluation) and two diabetes risk scores (Finnish Diabetes Risk Score and German Diabetes Risk Score). We then used this framework to evaluate the integration of risk scores in common practice software solutions by examining the software and inquiring the respective software contact person.

Results Our evaluation showed that the most widely integrated risk score is arriba, as recommended by German medical guidelines. Every software version in our sample provided either an interface to arriba or the option to implement one. Our assessment of integration capability revealed a more nuanced picture. Results on data availability were mixed. Each score contains at least one variable, which requires laboratory diagnostics. Our analysis of data standardization showed that only one score documented all variables in a standardized way.

Conclusion Our assessment revealed that the current state of risk score integration in physician practice software is rather low. Integration capability currently faces some obstacles. Future research should develop a comprehensive framework that considers the reasonable integration of risk scores into practice workflows, disease prevention programs, and the awareness of physicians and patients.

Keywords

Risk factors - medical records - patient records - clinical decision support - software

Background and Significance

Digital availability of patient data is continuously improving with the increasing implementation of electronic patient records in physician practices. A survey reveals that, by 2019, nearly 80% of the physician practices in Germany had digitized at least most of their documentation, thus continuing the development that had reached 73% in 2018.[1] [2] In the United States, the use of electronic health records (EHRs) by office-based physicians in 2019 was even close to 90%.[3] Findings from a recent review show that physicians in inpatient and outpatient care spend nearly 40% of their working time with EHRs, while physicians in outpatient care already exceed this percentage.[4]

The increase of available electronic patient records provides data analytics with a new field of application and with opportunities to expand the uses of patient data. Common areas of data analytics applications include decision support, administration, and fraud detection.[5] Decision support has received a lot of attention from users, as well as in academic research. Recent systematic reviews revealed that decision support has achieved improvements in care operations,[6] practitioners' performance, and medical outcomes.[7] Areas of application include prevention of hospital readmissions,[8] suicide risk prediction,[9] or pharmacist clinical monitoring.[10]

In clinical contexts, decision support systems are defined as

software that [is] designed to be a direct aid to clinical decision-making, in which the characteristics of an individual patient are matched to a computerized clinical knowledge base and patient-specific assessments[,] or recommendations are then presented to the clinician or the patient for a decision.[11]

Standard components of decision support tools are patient data, medical knowledge, and inference mechanisms including prediction rules.[12]

Quantitative absolute risk assessments in the form of risk scores are a prediction rule that has proven to be of value for physicians as well as for patients.[13] Risk scores rely on prognostic or predictive models and usually serve medical staff to assess the outcome of certain clinical procedures or identify patients at risk.[14] Quantitative risk assessment can be both part of a larger decision support system and a stand-alone application. Several success factors that have been identified for clinical decision support tools provide valuable design criteria for stand-alone risk scores, too. Studies that surveyed tools that integrated decision support into professional and patient strategies and tools that provided decision support directly to the patient showed that automated advice was more successful than on-demand options.[15] However, risk scores that display automated advice are premised on automated score calculation. The latter aspect is likely to be critical for the success of risk score implementation, according to studies that have recognized interoperability as an unresolved problem in the development of clinical decision support systems.[16]

Risk score integration and integration capability have not yet been systematically examined in primary care software. Hence, the functioning of risk score calculation in currently available tools is unknown, too. Analyses of risk scores in the present health care software contribute to develop this area in three ways. First, they raise awareness; second, they help specify software development requirements and, lastly, they identify where further research is needed.

Objective

This study aims to identify the current state of risk score integration and integration capability in electronic patient records for cardiovascular disease and diabetes mellitus in German primary care practices. Both medical conditions are of high relevance to society and largely preventable.

Methods

Existing frameworks for assessing risk scores in software products focus on the problem of automated risk score calculation.[17] [18] For example, Aakre et al examined the programmability of clinical score calculation by tracing how structured data emerged in EHRs of the Mayo Clinic.[17] We extended existing approaches with our objective to consider the extent to which manual input is required to gather missing data. To achieve this goal, we developed a framework to evaluate the current state of risk score integration and future integration options. We then used this framework to evaluate the integration of risk scores in practice software by examining the software and making inquiries with the software provider.

Framework Development

The framework of evaluation for this research was designed with a dual focus on, first, risk score integration and, second, options of risk score integration that we conceptualized as “data accessibility.”

Risk score integration comprised existing options of risk score calculation within systems (internal) as well as interfaces to external score systems. Our examination of internal risk score calculation considered, in addition, whether the score calculator automatically used the information that was available in the system or prompted manual input.

Our second focus was on data accessibility, the concept we established to understand risk score calculation in light of further options to either automate internal risk scores or integrate additional scores. We designed a matrix ([Fig. 1]) with the following dimensions: data standardization, in order to determine the degree of data retrievability, and data availability, to rate the effort of data collection.

Fig. 1 Data accessibility evaluation matrix.

Data standardization can assume three levels, indicated by the letters A to C. Category A comprises standardized data that are provided in a separate textbox. This format is usually applicable for data on gender, age, etc. Data in category B can be characterized as notes that describe, for example, the treatment of a specific disease. This information is heterogeneous, yet sufficiently standardized to generate queries. Category C contains information that is manually entered in text fields that have no standardized data format. This is typically individually specific information about a family disease history, for example.

Data availability encompasses three categories, labeled 1 to 3. Category 1 contains information that can only be obtained from external sources such as laboratories, including cholesterol levels. Category 2 refers to information that is not necessarily stored in the system but can be obtained by the physician during examinations or patient interviews. Examples of category B data are blood pressure or body mass index. Category 3 information is always available as it is basic claims data such as patient's date of birth. Overall, data accessibility is best in categories A and 3, and worst in categories C and 1.

Relevant risk scores for our study encompass scores that predict cardiovascular disease and diabetes mellitus. Our selection of scores is based on the guideline recommendations of German, British, European, and American physician societies. The American College of Cardiology/American Heart Association recommends the application of pooled cohort equations (PCEs) for cardiovascular disease.[19] The European Society of Cardiology's joint guidelines prefer the Systematic Coronary Risk Evaluation (SCORE).[20] The German College of General Practitioners and Family Physicians recommends the arriba score,[21] and the National Institute for Health and Care Excellence suggests QRISK3.[22] The German guidelines for diabetes risk assessment, currently due for updates, refer to the German Diabetes Risk Score (GDRS) and FINDRISK, an adapted version of the Finnish Diabetes Risk Score (FINDRISC).[23] American and British guidelines provide a selection of available scores for physicians to choose from.[24] [25] Our performance assessment, therefore, focuses on the two recommended scores in greater detail. [Table 1] provides an overview of our sample and counts of the variables used in the score calculations. For a list of the variables, see [Supplementary Appendix 1] (available in the online version).

Table 1 Risk scores included in this study

Cardiovascular risk scores	Diabetes risk scores
- Pooled cohort equations: 9 variables - Systematic Coronary Risk Evaluation: 5 variables - arriba: 9 variables - QRISK3: 21 variables	- German Diabetes Risk Score: 10 variables - Finnish Diabetes Risk Score (adapted version): 8 variables

Software Assessment

We collected information about options of risk score integration from our software assessment. Our selection of relevant software was based on installation statistics for medical specializations published by the Kassenärztliche Bundesvereinigung.[26] Primary care statistics are most suitable for the purpose of this research. Here, observations about the top 10 systems for primary care account for 77% of the primary care practices and 40% across all medical specialties.

We obtained trial versions of the software under examination, or, if unavailable, asked for an online tutorial to explore their risk scores and data availability. This way, we were able to explore nine of the top ten software solutions. One solution was designated for phase-out by the software provider. Therefore, neither a trial version nor an online tutorial was available. In addition to trials and tutorials, we examined the integration of risk scores by sending queries to the providers' sales representative. [Supplementary Appendix 2] (available in the online version) shows the software in our sample, [Supplementary Appendix 3] (available in the online version) is a list of the questions we sent to providers. [Fig. 2] is an outline of the research approach.

Fig. 2 Research approach.

Results

Risk Score Integration

Our evaluation showed that the most widely integrated risk score for cardiovascular disease is arriba, as recommended by German medical guidelines. Arriba is based on the Framingham formula and predicts a 10-year risk of cardiovascular events.[27] It comprises nine risk factors.[28] Primary care physicians have free access to the arriba module for cardiovascular prevention.[29] Every software version in our sample provided an interface to arriba or offered to implement one. However, the programs seemed to vary the data they supplied for arriba. A more detailed analysis of data transfers was impossible with trials that lack an active interface to arriba. However, arriba provides a technical summary of the interface. Two out of the nine variables (gender and date of birth) are available as standard data. The remaining seven variables (family history, antihypertensive treatment, systolic blood pressure, total cholesterol, high-density lipoprotein cholesterol, diabetes, and smoking status) refer to preassigned selection/input fields.

Only one provider integrated additional cardiovascular risk scores, including PROCAM/Framingham. No software in our sample offered risk scores for diabetes.

Data Accessibility

The availability of data is a universal feature that can be judged independently from the software. We assigned values of accessibility to score variable data, ranging from 1: difficult (additional effort required to obtain data, usually by consulting external sources) to 3: easy (basic claims data). Our evaluation of data standardization, by contrast, considered the distinctive presentation of variables in each software system in categories between A (separate textbox using a standardized format) and C (free text), as described above. We found similar levels of data standardization in the different software systems. Therefore, the presentation of our results is not organized by software systems but a median summary. [Figs. 3] [4] [5] [6] show the results for cardiovascular risk scores. The size of the circles varies according to the number of variables per category. The figure's caption provides detailed information.

Fig. 3 Data accessibility matrix PCE (A1: two variables, A2: two variables, A3: three variables, B3: one variable, C2: one variable). PCE, pooled cohort equations.

Fig. 4 Data accessibility matrix SCORE (A1: one variable, A2: two variables, A3: two variables). SCORE, Systematic Coronary Risk Evaluation.

Fig. 5 Data accessibility matrix arriba (A1: two variables, A2: two variables, A3: three variables, B3: one variable, C2: one variable).

Fig. 6 Data accessibility matrix QRISK3 (A1: two variables, A2: four variables, A3: nine variables, B3: four variables, C2: two variables).

Results on data availability were mixed for all the scores evaluated. At least one variable in each score concerns cholesterol levels that necessitate laboratory testing. Nonetheless, most data are available in the system or obtainable by patient inquiry. A more differentiated picture emerged of data standardization. All SCORE variables, for example, are documented in a standardized way, while almost 30% of the QRISK3 variables show medium-to-low standardization levels. Overall, SCORE is the model with the smallest number of variables and the best data accessibility, followed by PCE and arriba, which use similar variables.

[Figs. 7] and [8] display the results for the diabetes risk scores in our sample.

Fig. 7 Data accessibility matrix GDRS (A2: four variables, A3: one variable, C2: five variables). GDRS, German Diabetes Risk Score.

Fig. 8 Data accessibility matrix FINDRISK (A1: one variable, A2: two variables, A3: two variables, B3: one variable, C2: three variables). FINDRISK, Finnish Diabetes Risk Score.

A direct comparison of the two scores reveals that GDRS data are better available since no variable presupposes laboratory tests. Half of the data are usually in the system, while the other half can be obtained from patients. FINDRISK requires less accessible data on blood glucose levels to determine cardiovascular risks. However, FINDRISK variables use more standardized data than GDRS, which considers nutrition in a more detailed way than FINDRISK (three questions vs. one question).

Discussion

This study focused on risk score integration and integration capability in electronic patient records for primary care practices in Germany. We concentrated on cardiovascular and diabetes diseases, both of which have a strong impact on society as a whole although their risk factors are well known and in part preventable by lifestyle changes.

Our results for risk score integration range overall from 0 to several risk scores. However, all systems provided an interface to the arriba risk calculator, which determines the risk score recommended by German guidelines. There is limited evidence of risk score integration in primary care in other countries. Yet, we have reason to assume that integration is rather low in most countries, as hospital-based studies from the United States or Spain suggest.[17] [30] We found evidence of existing integration options in the United Kingdom, however, not for all primary care practices.[31]

Our integration capability assessment revealed a nuanced picture. The data availability for cardiovascular risk calculation was strongly diminished by cholesterol levels, which can usually be accessed by means of laboratory blood assessments. Therefore, risk score calculation is only possible if recent test results are available. Data standardization failed for the variables “race/ethnicity” and “family history of cardiovascular disease,” and sometimes for “smoking status.” Consequently, risk score calculations cannot be completely automated. An analysis by Aakre et al, who tested the current capacity of automatic clinical score calculations extensively with the electronic medical records of the Mayo Clinic, confirmed that complete automation cannot be realized at present. The atherosclerotic cardiovascular disease risk estimator (PCE) achieved programmability of 89%. QRISK2, the predecessor of QRISK3, had a programmability of 85%, and a modified version of the Framingham coronary heart disease risk score was automated to 86%.[17] Aakre et al's analysis did not encompass SCORE and arriba.

Automated risk score calculations made progress in two other areas. Within the past years, several standards have been developed for data standardization and interoperability.[32] A higher level of interoperability will increase the availability of data for risk score calculation. Another key factor is the development of more advanced information retrieval technologies, which can extract data from less standardized formats, such as free text.

Nevertheless, more issues need to be solved to achieve valid results. The data quality is primarily a technical problem. The prediction accuracy of risk scores does not only depend on the validity of the score model but also on the quality of input data.[33] Today, professionals are chiefly involved in providing the data that physician software stores. However, this is likely to change when data generated by patients will be integrated, as well. For example, CompuGroup Medical and Medatixx, two major physician practice software companies, already provide solutions that allow patients to share health data with their physicians.[34] [35] An ethical issue is the question of how risk scores should figure in decisions about further diagnostics or treatments. If these decisions depend on the risk scores alone, failures of risk prediction models would become intolerable,[36] especially because of the performance differences that scores inherit from different populations.[37] [38] [39]

Our analysis has several limitations. First, our focus on German software systems means that our results are not necessarily valid for other countries. However, studies from the United States or the Netherlands reveal similar issues of data availability and data standardization.[40] [41] Therefore, our findings and suggestions for improvement might apply internationally. In addition, our analytical framework does not consider any country-specific characteristics and thus can be used for similar analyses in other countries. Second, our analysis is based on software trials. These trials might not always be the latest available version and lack some functions. Our inquiries with the software contact persons admittedly only reduced these shortcomings but did not fully eliminate the risk of missing risk score-related functions. Test versions also might not reveal issues related to practice setup or customized features. To a certain degree, every software is customizable to fit a physician practice's individual needs. Hence, our results might not be valid for every practice. Finally, we neglected implementation issues beyond calculating the score itself, such as workflow integration or alert fatigue. However, these aspects must be taken into consideration to succeed with the implementation of risk scores.[42] [43]

Conclusion

None of the scores for diabetes risk assessment played a role in the most used practice software. The most prominent cardiovascular risk score, displayed in every software, was not directly integrated but accessible through an interface. Overall, our assessment revealed that the current state of risk score integration in physician practice software is rather low and unlikely to advance quickly due to obstacles that persist also in the integration capability, including data standardization. This issue will be settled when more advanced information retrieval technologies become available. However, our results reveal the current gap between what could be possible and what is technologically feasible, and between the progress in research and the quotidian practices of care.

Finally, technological solutions are only one aspect of implementing risk scores in the real world. Future research should develop a comprehensive framework to promote the reasonable integration of risk scores into practice workflow, disease prevention programs, and the awareness of physicians and patients.

Clinical Relevance Statement

Risk scores can be a valuable tool for physicians as well as for patients. However, in order to become an inherent part of preventive care, it is essential to facilitate their implementation by keeping additional effort for physicians as low as possible. The best way to achieve this objective is to integrate risk scores into practice software and provide automated score calculation.

Multiple Choice Questions

1. What is a common pitfall when it comes to clinical decision support system implementation?

   • Wrong algorithms

   • Wrong hardware

   • Poor interoperability

   • Poor planning

   Correct Answer: The correct answer is option c. Decision support systems often exist as stand-alone systems or systems, which communicate with each other ineffectively. In the future, users are very likely to benefit from the ongoing development of and agreements on interoperability standards.

2. Which of the following statements on arriba score is correct?

   • arriba a diabetes risk score

   • arriba comprises five variables

   • arriba is based on the Framingham formula

   • arriba is mandatory due to the German medical guidelines

   Correct Answer: The correct answer is option c. arriba is a 9-variable risk score for cardiovascular disease, based on the Framingham formula. It is recommended by the German Medical guidelines, however not mandatory.

Conflict of Interest

None declared.

Note

The present work was performed in fulfillment of the requirements for obtaining the degree “Dr. rer. biol. hum.” by Ann-Kathrin Heider from the Friedrich-Alexander-Universität Erlangen-Nürnberg.

Protection of Human and Animal Subjects

Human and/or animal subjects were not included in the project.

• References

• 1 KBV. PraxisBarometer Digitalisierung. Accessed March 8, 2021 at: https://www.kbv.de/media/sp/PraxisBarometer-Digitalisierung_2018.pdf
• 2 KBV. PraxisBarometer Digitalisierung 2019. Accessed March 8, 2021 at: https://www.kbv.de/media/sp/KBV_Praxisbarometer_Digitalisierung_2019.pdf
• 3 National Center for Health Staistics. . 2019 National Electronic Health Records Survey Public Use File National Weighted Estimates, Accessed May 15, 2022 at: https://www.cdc.gov/nchs/data/nehrs/2019NEHRS-PUF-weighted-estimates-508.pdf
• 4 Pinevich Y, Clark KJ, Harrison AM, Pickering BW, Herasevich V. Interaction time with electronic health records: a systematic review. Appl Clin Inform 2021; 12 (04) 788-799
• 5 Islam MS, Hasan MM, Wang X, Germack HD, Noor-E-Alam M. A systematic review on healthcare analytics: application and theoretical perspective of data mining. Healthcare (Basel) 2018; 6 (02) 54
• 6 Kwan JL, Lo L, Ferguson J. et al. Computerised clinical decision support systems and absolute improvements in care: meta-analysis of controlled clinical trials. BMJ 2020; 370: m3216
• 7 Kruse CS, Ehrbar N. Effects of computerized decision support systems on practitioner performance and patient outcomes: systematic review. JMIR Med Inform 2020; 8 (08) e17283
• 8 Wu CX, Suresh E, Phng FWL. et al. Effect of a real-time risk score on 30-day readmission reduction in Singapore. Appl Clin Inform 2021; 12 (02) 372-382
• 9 Walker RL, Shortreed SM, Ziebell RA. et al. Evaluation of electronic health record-based suicide risk prediction models on contemporary data. Appl Clin Inform 2021; 12 (04) 778-787
• 10 Schreier DJ, Lovely JK. Optimizing clinical monitoring tools to enhance patient review by pharmacists. Appl Clin Inform 2021; 12 (03) 621-628
• 11 Sim I, Gorman P, Greenes RA. et al. Clinical decision support systems for the practice of evidence-based medicine. J Am Med Inform Assoc 2001; 8 (06) 527-534
• 12 Pereira AM, Jácome C, Amaral R, Jacinto T, Fonseca JA. Real-time clinical decision support at the point of care. In Agache I, Hellings P, eds. Implementing Precision Medicine in Best Practices of Chronic Airway Diseases. MA: Academic Press; 2019: 125-133
• 13 Black JA, Campbell JA, Parker S. et al. Absolute risk assessment for guiding cardiovascular risk management in a chest pain clinic. Med J Aust 2021; 214 (06) 266-271
• 14 Vogenberg FR. Predictive and prognostic models: implications for healthcare decision-making in a modern recession. Am Health Drug Benefits 2009; 2 (06) 218-222
• 15 Van de Velde S, Heselmans A, Delvaux N. et al. A systematic review of trials evaluating success factors of interventions with computerised clinical decision support. Implement Sci 2018; 13 (01) 114
• 16 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3: 17
• 17 Aakre C, Dziadzko M, Keegan MT, Herasevich V. Automating clinical score calculation within the electronic health record: a feasibility assessment. Appl Clin Inform 2017; 8 (02) 369-380
• 18 Perry WM, Hossain R, Taylor RA. Assessment of the Feasibility of automated, real-time clinical decision support in the emergency department using electronic health record data. BMC Emerg Med 2018; 18 (01) 19
• 19 Goff Jr DC, Lloyd-Jones DM, Bennett G. et al; American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 2014; 129 (25, Suppl 02) S49-S73
• 20 Piepoli MF, Hoes AW, Agewall S. et al; ESC Scientific Document Group. 2016 European guidelines on cardiovascular disease prevention in clinical practice: The Sixth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice (constituted by representatives of 10 societies and by invited experts) Developed with the special contribution of the European Association for Cardiovascular Prevention and Rehabilitation (EACPR). Eur Heart J 2016; 37 (29) 2315-2381
• 21 Ludt S, Angelow A, Baum E. et al. S3-Leitlinien Hausärztliche Risikoberatung zur kardiovaskulären Prävention, AWMF-Register-Nr. 053 -024 DEGAM-Leitlinie Nr. 19. Hrsg Deutsche Gesellschaft für Allgemeinmedizin und Familienmedizin e.V. 2017. Accessed May 15, 2022 at:. www.awmf.org/leitlinien/detail/ll/053-024.html
• 22 National Institute of Clinical Excellence (NICE). Surveillance report 2018–Cardiovascular disease: risk assessment and reduction, including lipid modification. NICE guideline CG181. 2014. Accessed May 15, 2022 at: https://www.nice.org.uk/guidance/cg181/resources/surveillance-report-2018-cardiovascular-disease-risk-assessment-and-reduction-including-lipid-modification-2014-nice-guideline-cg181-4724759773/chapter/Surveillance-decision?tab=evidence
• 23 American Diabetes Association. Classification and diagnosis of diabetes: standards of medical care in diabetes-2021. Diabetes Care 2021; 44 (Supplement1): 15-33
• 24 NICE. Type 2 diabetes: prevention in people at high risk. Accessed May 15, 2022 at: https://www.nice.org.uk/guidance/ph38
• 25 Bundesärztekammer (BÄK). Kassenärztliche Bundesvereinigung (KBV), Arbeitsgemeinschaft der Wis-senschaftlichen Medizinischen Fachgesellschaften (AWMF). Nationale VersorgungsLeitlinie Therapie des Typ-2-Diabetes – Langfassung, 1. Auflage. Version 4. 2013. Accessed May 15, 2022 at: https://www.leitlinien.de/themen/diabetes/archiv/pdf/therapie-des-typ-2-diabetes/dm-therapie-1aufl-vers4-lang.pdf
• 26 Kassenärztliche Bundesvereinigung. Installationsstatistiken von Softwaresystemen. Accessed February 15, 2021 at: https://www.kbv.de/html/7023.php
• 27 Donner-Banzhoff N, Keller H, Sadowski E-M. et al. arriba Werkstattbericht. Accessed February 15, 2021 at: https://arriba-hausarzt.de/uploads/files/arriba_werkstattbericht.pdf
• 28 Grammer TB, Dressel A, Gergei I. et al. Cardiovascular risk algorithms in primary care: results from the DETECT study. Sci Rep 2019; 9 (01) 1101
• 29 GPZK gGmbH.. Arriba fuer Hausaerzte. Accessed February 12, 2021 at: https://arriba-hausarzt.de/zugang-arriba/arriba-f%C3%BCr-haus%C3%A4rzte
• 30 Martinez-Millana A, Argente-Pla M, Valdivieso Martinez B, Traver Salcedo V, Merino-Torres JF. Driving type 2 diabetes risk scores into clinical practice: performance analysis in hospital settings. J Clin Med 2019; 8 (01) 107
• 31 Public Health England. NHS Health Check Best practice guidance. Accessed May 15, 2022 at: https://www.healthcheck.nhs.uk/seecmsfile/?id=1480
• 32 HIMSS. Interoperability in Healthcare. Accessed May 15, 2022 at: https://www.himss.org/resources/interoperability-healthcare
• 33 Miller DD. The medical AI insurgency: what physicians must know about data to practice with intelligent machines. NPJ Digit Med 2019; 2: 62
• 34 CGM Mobile Services GmbH. CGM LIFE. . Accessed March 14, 2021 at: https://cgmlife.de/
• 35 medatixx GmbH & Co. KG. x.patient. Accessed March 14, 2021 at: https://medatixx.de/praxissoftware/zusatzloesungen/xpatient
• 36 Amarasingham R, Patzer RE, Huesch M, Nguyen NQ, Xie B. Implementing electronic health care predictive analytics: considerations and challenges. Health Aff (Millwood) 2014; 33 (07) 1148-1154
• 37 Ruwanpathirana T, Owen A, Reid CM. Review on cardiovascular risk prediction. Cardiovasc Ther 2015; 33 (02) 62-70
• 38 Glümer C, Vistisen D, Borch-Johnsen K, Colagiuri S. DETECT-2 Collaboration. Risk scores for type 2 diabetes can be applied in some populations but not all. Diabetes Care 2006; 29 (02) 410-414
• 39 Buijsse B, Simmons RK, Griffin SJ, Schulze MB. Risk assessment tools for identifying individuals at risk of developing type 2 diabetes. Epidemiol Rev 2011; 33 (01) 46-62
• 40 Shelov E, Muthu N, Wolfe H. et al. Design and implementation of a pediatric ICU acuity scoring tool as clinical decision support. Appl Clin Inform 2018; 9 (03) 576-587
• 41 Wasylewicz A, van de Burgt B, Weterings A. et al. Identifying adverse drug reactions from free-text electronic hospital health record notes. Br J Clin Pharmacol 2022; 88 (03) 1235-1245
• 42 Lee TC, Shah NU, Haack A, Baxter SL. Clinical implementation of predictive models embedded within electronic health record systems: a systematic review. Informatics (MDPI) 2020; 7 (03) 25
• 43 Noble D, Mathur R, Dent T, Meads C, Greenhalgh T. Risk models and scores for type 2 diabetes: systematic review. BMJ 2011; 343: d7163

Publication History

Received: 22 February 2022

Accepted: 13 July 2022

Article published online:
07 September 2022