Power of Heuristics to Improve Health Information Technology System Design

Abstract

Background Clinical decision-making can be prone to error if health system design does not match expert clinicians' higher cognitive skills. There is a gap in understanding the need for the importance of heuristics in clinical decision-making. The heuristic approach can provide cognitive support in designing intuitive health information systems for complex cases.

Objective We explored complex decision-making by infectious diseases (ID) clinicians focusing on fast and frugal heuristics. We hypothesized that ID clinicians use simple heuristics to understand complex cases using their experience.

Methods The study utilized cognitive task analysis and heuristics-based decision modeling. We conducted cognitive interviews and provided clinicians with a fast-and-frugal tree algorithm to convert complex information into simple decision algorithms. We conducted a critical decision method–based analysis to generate if–then logic sentences from the transcript. We conducted a thematic analysis of heuristics and calculated the average time to complete and the number of crucial information in the decision nodes.

Results A total of 27 if–then logic heuristics sentences were generated from analyzing the data. The average time to construct the fast-and-frugal trees was 1.65 ± 0.37 minutes, and the average number of crucial pieces of information clinicians focused on was 5.4 ± 3.1.

Conclusion Clinicians use shortcut mental models to reduce complex cases into simple mental model algorithms. The innovative use of artificial intelligence could allow clinical decision support systems to focus on creative and intuitive interface design matching the higher cognitive skills of expert clinicians.

Keywords

clinical decision support - heuristics - data processing - expert system - algorithm

Introduction

We focus on essential information cues and make decisions based on our bounded rationality in our daily decision-making.[1] [2] Cognitive limitations of working memory result in the use of heuristics. Heuristics provide a take-the-best cue approach by minimizing information processing and selecting the best optimization. They have been commonly thought to create bias in decision-making.[1] Research has demonstrated in many fields that the take-the-best approach or less-is-more and computational model perform faster and more accurately than the traditional approach of information overload.[3] [4] [5] [6] [7] A common heuristic is “less is more.”[3] Unnecessary information creates information overload and makes the clinical decision flawed. The “less is more” heuristic has the potential for more accurate inferences and better predictions in an information space that creates unnecessary information overload.[3] [8] Unnecessary information increases the chances of error.[1] This is especially relevant in clinical medicine.

Expert clinicians use many variations of a “rule of thumb”—or shortcut mental models—for complex cases to balance timeliness and accuracy during decision-making.[9] [10] Much of the resulting work has incorporated the idea of optimization by considering the role of organizing information in sound reasoning. However, the medical community was quick to guard against heuristics owing to bias and faulty decision-making.[11] [12] This bias is based on clinicians ignoring a vital piece of information. Despite this, the last three decades of heuristics research has revealed that clinical experts make better decisions by ignoring some information and focusing on the most relevant indicators for optimal clinical outcome.[13] An example of this approach involves fast-and-frugal tree (FFT) heuristics. The FFT is a type of shortcut mental model that involves an algorithm approach. The “fast-and-frugal” approach to decision-making has achieved widespread popularity in various areas, including decision analysis inpatient care.[4] [13] [14] The overall goal of the heuristic is to develop robust and straightforward decisions using minimal but crucial information. This results in more effective information processing and better decision-making in complex domains such as infectious diseases (ID) medicine. In clinical medicine, ID represents a challenging and complex domain.[15] [16] [17] The complexity of emerging diseases such as coronavirus disease 2019 (COVID-19), environmentally persistent organisms, and increasing antibiotic resistance contribute to the ID domain's complex information environment.[18] For example, understanding and dealing with pathogens with limited information can make clinicians apprehensive. However, the less-is-more approach supports limited information processing for better decisions. Therefore, clinical decision support systems (CDSSs) are crucial to support ID clinicians. CDSS interventions in the ID domain include microbiology understanding, using information visualizations, and optimizing treatment and therapy outcomes.[16] [19] [20]

Previous research in the design of health information systems focused on cognitively supporting clinicians by creating user-friendly CDSS embedded within the electronic health records (EHRs).[21] [22] Most current EHRs are not designed to help clinicians' cognition.[20] [22] Clinician's cognition refers to the higher cognitive skills to ponder complex clinical problems. Previous research has shown that the complex approach of showing all information leading to cognitive overload may not be suitable for heuristics-based decision-making by experts.[23] [24] [25] Such systems with unnecessary information cues on the screen will create information overload and may increase errors.[26] Complaints from clinicians center around the inability of information technology (IT) to support the high level of reasoning required to execute complex clinical tasks.[27] As we are designing and implementing our health IT system and CDSSs, exploring and understanding the value and power of heuristics in the system design are essential. Thus, there is a gap in understanding how expert clinicians process complex cases using a shortcut mental model.

In the present study, we explored complex decision-making by ID clinicians and focused on fast and frugal heuristics to simplify complex decisions into simple shortcut mental models. We hypothesized that ID clinicians use their vast experience to apply simple heuristics to reduce the practical complexity of complex cases.

Methods

Study Design

In this study, we used cognitive task analysis (CTA) and heuristics-based decision modeling. We conducted CTA, a systematic method for understanding and describing experts' complex reasoning in performing complex tasks.[27] We used “combinatorics” of CTA, combining the critical decision method (CDM) with essential incident interviews.[24] Combinatorics is a method to compare, converge, merge, and adapt methods from CTA. CTA data can be analyzed in a myriad of ways. CTA can better understand the overall data when merged. We have used the RATS (relevance of study question, appropriateness of qualitative methodology, transparency of procedure, and soundness of interpretive approach) protocol to analyze data from transcriptions of the interviews qualitatively.[28] The RATS protocol ensures CTA data curation and analysis are performed in an organized format for data reproducibility. The RATS protocol provides standardized guidelines for qualitative research methods, including CTA, cognitive work analysis, and other human factors methodology. We used the FFT algorithm to reduce complex decisions into more straightforward decision steps for decision modeling. Our main goal was to understand the value of the FFT algorithm in reducing complex cases. We used the SQUIRE (Standards for Quality Improvement Reporting Excellence) reporting guidelines for the results.[29]

Settings

We conducted the study at the University of Utah and the Salt Lake City Veterans Administrations Hospital. The study was approved by the respective institutional review boards (IRBs). All participants provided oral consent, and the University of Utah IRB approved verbal consent.

Participants

We conducted the cognitive interviews and decision-modeling tasks with 10 ID expert clinicians, defined as board-certified ID physicians with more than 5 years of full-time work experience in the ID field.

Procedure

Interviews

The first author conducted semistructured interviews based on the CDM, a type of CTA.[24] Each ID expert was asked to describe a recent complex case challenging diagnostics or treatment. The first author explained to the ID experts the purpose and objective of the study. Then, ID experts were told to think about a very complex case that took considerable effort and time.

A semistructured interview script was piloted with two ID clinicians and was refined. At the end of the interview, participants were asked to provide basic demographic information. Each interview lasted around 50 minutes. All interviews were recorded and transcribed. Audio transcripts were transcribed using a third-party company called Datagain. The company uses software and manual checks to ensure the integrity of transcription. Datagain is a Health Insurance Portability and Accountability Act (HIPAA)-compliant transcription company that uses human transcription for health care data. All patient identifiers were removed.

Decision Model

To understand fast-and-frugal decisions, we provided an FFT ([Fig. 1]). There is at least one exit leaf in the FFT algorithm at every decision node (rectangular box in [Fig. 1]). For every checked cue, at least one outcome can lead to a decision. Thus, an FFT algorithm leads to the well-established simple heuristics strategy for comparison called “take the best.”[30] Each rectangular box represents a decision node and, based on binary (yes/no) decisions, another node starts. Therefore, FFT heuristics depend on a search-stop-decision rule. After our interviews, we asked each participant to construct the complex case verbally. The research team member explained to each clinician completing the FFT in detail. Each clinician then filled out an FFT to practice based on a sample complex case. After the sample case FFT was filled, the research team member explained the correct procedure and meanings of different nodes in detail. Once the ID experts verbally clarified their understanding of filling out the FFT, the research team member provided them with an empty FFT to complete for the specific complex case. The participants also verbalized in detail the rationale of the FFT algorithm. We noted the time to complete the algorithm and calculated the number of information cues for each decision made at each step/node of the algorithm. We hypothesize that ID experts will take more time than usual to deliberate on a complex case. Therefore, collecting time data on completing the FFTs can give us a better understanding of the overall decision-making process. The time data will confirm if experts are using heuristics to solve a complex case. Generally, complex cases should take longer than simple ones that follow evidence-based guidelines.

Fig. 1 Fast-and frugal-tree decision algorithm used in the present study. Each node branches out based on binary decisions.

Data Analysis

Interviews

The research team conducted a qualitative analysis of the interview transcripts. The research team developed a codebook for conducting the iterative thematic analysis. First, all research team members met, identified, and parsed data sections for creating the codebook. After renaming each concept, the team members discussed, merged, and finally developed a group of terms to be included in the codebook. Once the codebook was created, the team members conducted an iterative analysis. The iterative inductive analysis independently identified the following three parts of transcripts: (1) sections related to FFT mental quirks focusing on sensemaking, (2) sections related to cognitive strategies, and (3) sections about coping with the complexity of the case. Three researchers independently conducted data analyses and then met as a team to complete the full transcripts. Each researcher had more than 10 years of experience in qualitative coding. The research team first created a sentence structure extraction from the CTA. The analysis revealed the texts and sentences related to heuristics. Once the specific paragraphs and sentences were identified, the research team conducted the content analysis. The content analyses were conducted in four phases: (1) initial review, (2) data coding, (3) synthesis, and (4) grouping codes into meaningful sentences.[31] [32] The researchers coded the heuristics independently and later met to discuss the codes. The researchers then examined the codes, merged similar codes, and finally reached a consensus on a meaningful sentence representing if–then logic. One of the simplest ways our brain understands heuristics is by creating a cause–effect or if–then relationship from the FFT trees. All FFT algorithms have if–then decision trees. Thus, the if–then clause can support the most effective way to store information for future purposes. For example, if a transcript focused on “looking at the big picture instead of wallowing in given data,” then an if–then logic was formulated such as “if big picture conflicts with minutia, go with big-picture.” The multidisciplinary research team sought a group consensus at the end of each iteration, and the resulting codes were used for subsequent iterations. Finally, the team consensually merged all codes into if–then logic–based meaningful sentences. To reduce any biases in coding, the interrater reliability of Cohen's kappa was also calculated. We used the qualitative software program Atlas ti 7.0 to conduct data analyses. Atlas ti version 19.1 was used for data analysis.

Decision Model Results

Two researchers (doctors) analyzed the FFT algorithms from ID physician's transcripts. Each FFT decision node was analyzed for a specific number of information cues. Decision nodes were examined with the cues that were vital for decision-making. For example, if “patient with other signs of systemic infections, increased WBC, positive culture from blood,” three information cues were recorded. The research team calculated the number of information cues for each complex case and the time to complete each fast-and-frugal decision algorithm.

Results

Interviews

The ID experts (n = 10) had an average of 19 years of experience; two were female, and eight were male. The results matched the study goal of understanding if expert ID clinicians can reduce complex information into FFT algorithms. The results demonstrate that ID clinicians were very efficient in developing the FFT for all the 10 complex cases. The content analysis produced 27 meaningful if–then logic sentences, which the ID experts used to deal with complex cases. For example, antibiotic resistance is a global issue. Our results showed several if–then sentences related to topics such as “if repeated antibiotics then are more conservative about antibiotic use” and “if no reason to change, do the usual.” The complete list is provided in [Table 1]. The final interrater reliability of Cohen's kappa was 0.86.

Table 1 If–then heuristics from a qualitative data analysis of interviews conducted in the present study

If big-picture conflicts with minutia, go with big-picture

If applicable clinical pathway exists, follow the pathway

If no reason to change, do the usual

If abnormal context, do not expect normal results

If cause makes sense, consider it. Otherwise, look for causes that make sense

If repeated antibiotics, then be more conservative about antibiotics use

If no cause, then continue to look for cause and watch

If context + history is not compelling, then be conservative and watch

If no reason to deviate from the conservative path, continue on that path

If stuck, hope for luck

If risk is high, go with safest bet

If patient likes it, then try that first

If context (family or personal History) takes us into dangerous space, then consider other options

If in over your head, punt

If don't have definitive answer, even if doesn't affect care, then have low threshold for requesting more input

If sunk cost, leave it on ocean floor

if no diagnosis, GET ONE

If actions do not meet standard of care AND harm comes, THEN you are at fault

If can eliminate diagnosis, then can more quickly reach decision

If can categorize, then make assumptions

If find a trail, then follow it closely

If matches pattern, then follow typical course for pattern

If have data but not answer, then take best guess based on data

If safe to get more information, then wait and get more information

If history available, then get it.

If unexpected, then react accordingly and in proportion to risk

If context conflicts with guidelines, then go with context

Example Cases

Two of the problematic cases are described in the following:

“Patient is a 62-year-old, white male Veteran who presented to the VA with a history of back pain. It turns out that for the past, maybe even a couple of months, he's been falling and hurting his back and has been going to the clinic. They first treated him for the usual injuries, brain, et cetera, but the pain continued after about another month. They went ahead and got an MRI. T10, T11 spine, and the disc space between them were destroyed, and osteomyelitis was diagnosed. The care team saw the MRI with the radiologist, which turned out to be an infection. The risk factor may have been that the patient had done dental work about four to six weeks ago, so he may have been bacteremia at that time, which transmitted into the disk space. We're going to ask interventional radiology to put a needle in that space and see if we can get some tissue for microbiology and pathology. The care team started Vancomycin and third-generation cephalosporin, but the infection is unclear.”

“ The patient, a 57-year-old male, came to the VA 45 days ago for community-acquired pneumonia. However, after being treated for ten days, he developed a Vancomycin-resistant enterococcus (VRE) infection. The care team started the patient on Daptomycin based on clinical guidelines. However, after being treated for 15 days, the VRE persisted. There are no guidelines for patients who did not respond to Daptomycin. The options were to give patients another off-label antibiotic or keep the patient on Daptomycin. It was unclear when to stop the medication and if starting other medications at this point can significantly improve or worsen the overall situation for the patient.”

Decision Model

The 10 ID experts completed the FFT algorithm for each complex case in 1.65 ± 0.37 minutes. All ID experts average reduced their complex cases to simple heuristics in less than 2 minutes.

For the decision model, the ID experts analyzed 5.4 ± 3.1 information cues. One ID expert used five decision nodes, while another used four nodes. The remaining ID experts used only three decision nodes to complete the FFT algorithm. Two examples of FFTs are provided in [Fig. 2]. Each of the 10 FFT algorithms generated is an independent case and is unrelated to each other.

Fig. 2 Two examples of fast-and-frugal algorithms (A, B). The infectious diseases experts interviewed in the present study were able to reduce the complex cases within each decision node.

Our results show that even very complex cases described by ID experts were solved using a very short time. However, this also may be due to diverse complex cases unique to each ID expert.

Discussion

Our findings are similar to those of previous studies on the efficacy and effectiveness of heuristics.[30] [33] [34] [35] Previous studies also found that the fast and frugal algorithm supports efficient and effective strategies for experts to deal with complex problems. Studies found that improving judgment clarifies the importance of understanding the degree to which different heuristics can be employed in the clinical domain and their actual usage. We agree that other domains of medicine may end with different types of heuristics.

One performance benefit of using heuristics could be redundant processing information, as repetitive tasks become quicker and easier to execute and understand and faster to communicate.[36] The results from this study may also be explained within fuzzy trace theory (FTT).[37] FTT is focused on the fact that judgment and decision-making rely preferentially on the gist representations of information instead of verbatim presentations. The preference to operate on the crudest gist increases with experience or expertise, as seen in our ID clinicians' study. For example, in one of the most complex cases described in the present study ([Fig. 2A, B]), the clinician focused on vital information cues from patient history such as urinary tract infections, sepsis/unstable, and blood pressure/mental status that correlated those with previous experience in treating similar patients. This gist-based thinking is not merely the retrieval of instances experienced in the past with other identical ID patients. Still, it is instead the distillation of the meaning of past experiences into an intuitive, bottom-line interpretation.

The clinical training is exhaustive and ensures that physicians adhere to clinical guidelines. Therefore, it can be mentioned that FFTs can be a simple projection of the clinical guidelines or rules. However, in this study using CDM, we ensured that clinicians focus only on a complex case. A case is complex when clinical guidelines are exhausted and no evidence-based information is available. The study focused on understanding ID experts' mechanisms to manage complex cases. The study findings still point toward the simple FFTs that experts use, similar to the if–then logic of clinical guidelines. These findings have significant implications for the health IT system design.

Implications for Health IT System Design

The current design of the Health IT system does not reflect the high-level reasoning of clinical experts.[38] As a result, the disjointed EHRs and their lack of usability in available CDSS have introduced new medical errors into the health care system.[39] [40] [41] [42] [43] [44] [45] EHR usability and CDSS design remain critical challenge in health care, with implications for medical errors, patient safety, and clinician burnout.[46] [47] Current system designers assume that more information supports clinical reasoning and will thus lead to better decisions.[46] [48] As a result, the current system design includes cluttered interfaces with extra information that does not support novice or expert clinicians. Even simple pharmacogenomics decision support systems are cluttered with laboratory results without pertinence.[43] [49] [50] [51] However, results from this study suggest that heuristics may require fewer data while simultaneously producing better results using the concept of less is more. A concern, particularly from a liability perspective, is whether ignored information could cause biases. Our current CDSSs do not have the full capabilities to understand the higher cognition of expert clinicians. We provide future designers with some opportunities to create innovative and intuitive design features that may support these if–then statements.

The results from this study can potentially improve future designs of health IT. For example, “If matches pattern, then goes with the pattern.” This can be translated to provide clinicians with pattern-matching analytics. If patients' laboratory values are consistent and within normal limits with the start of the antibiotic, then the interface can show the clinician a time-series graph of the usual pattern. Such visualization can help the clinician understand the patient's overall situation. Another example is on the heuristics “If repeated antibiotics then are more conservative about antibiotics use.” The current systems do not differentiate between different types of medications in terms of providing alerts. Advanced system designers may provide specific overprescribing of antibiotics-related alerts to ID clinicians. Also, the study informs designers to be aware of different cognitive biases that arise from shortcut thinking, which may be detrimental to patient safety. With the advent of machine learning and artificial intelligence, it is possible to have automated systems check for such biases related to heuristics.[52] [53] For example, system designers may assume that using FFT heuristics could cause ID experts to ignore information such as critical laboratory values. The system can flag any unusual data for the clinician using advanced machine learning that warrants immediate attention. Heuristics may offer an opportunity to mimic the higher cognitive thinking of clinicians. However, we must acknowledge the bias and errors that may come with heuristics. Therefore, future machine learning algorithms may check for biases and fix them automatically.[54] [55] For example, if clinicians use heuristics and overlook some test results, the machine learning algorithm may prompt an intelligent alert to clinicians to double-check parameters. Our results' if–then statements may prove that system designers may design interfaces with pivotal information. At the same time, background machine learning checks should focus on reducing errors and biases resulting from simple heuristics thinking. Previous research has shown that not considering specific cues and not learning from reading extensively different aspects of patients' history may result in misdiagnosis. Our study also emphasizes the rationale for considering more information in deliberation. However, the current cluttered EHR design may hinder clinicians from looking into all aspects of the patient. Future intuitive interfaces taking consideration of heuristics and artificial intelligence methods may fill in the gap.[56] [57] [58]

Future studies may evaluate the coverage of the 27 if–then logic heuristics for their usefulness in system design. Developing a metric to quantify the outcome of these heuristics in system design may be necessary. Displaying critical information that matches the high-level reasoning techniques used by clinical experts could ultimately improve the design of health IT systems and save lives by reducing errors. Future studies should validate and possibly extend the heuristics highlighted in this study. For example, knowing what specific information ID experts ignore when creating advanced CDSS tools will be essential. Future studies may also look into other specialized fields such as cardiology or rheumatology to identify similar FFT heuristics. The results may validate this study's results in terms of matching the higher cognition of expert clinicians.

Limitations

Even though experts may focus on limited crucial information, we acknowledge that similar outcomes may not be the same for novice clinicians who might require more information cues to feel confident. More research into the area of novice clinicians is needed as well. Also, the smaller sample size of 10 participants may curb the findings' generalizability. There could be variabilities in coding heuristics statements. However, the researchers piloted and created a coding list to reduce biases. Also, in this study, we did not evaluate whether the FFTs generated were the efficient rule. We assumed that expert ID clinicians would make the optimum decision to improve their patients. Future studies may test these FFT algorithms for evaluations. We acknowledge that the culture of antibiotic stewardship affects how ID experts make treatment decisions. Antibiotic resistance is a global issue leading to resistant bacteria due to overprescribing antibiotics. Many of the overprescribing allude to the culture of prescribing to ensure clinicians feel safe. Such training on overprescribing may influence the study results. However, we only focused this study on ID clinicians with expertise and awareness of antibiotic resistance issues. Also, many if–then logic from the results may not be feasible to be translated into improving health IT system design. Any CTA study comes with recall and recency bias. We acknowledge that recall bias can be a problem. However, we asked each clinician to tell us a case that is freshly remembered with enough details in their minds. Some of the decision heuristics were nonspecific and may not be applied to the actual design of an intuitive interface. Not all of these heuristics can be used to improve health system design. Also, these results may not apply to novice clinicians. The if–then statements are based on the mental shortcut that expert clinicians use. The sentences may not represent complete and total explanations of the specific situation. The data analysis team included ID clinicians. Complexity is a phenomenon that is unique to a particular individual. One complex case to an ID clinician may not be complex for another ID clinician. We acknowledge the personal bias in the cases presented. Our research cannot claim that the resulting if–then statements can be used as building blocks for health IT system design. However, this research provides opportunities for future studies to incorporate heuristics into the health interface systems design for intuitive design.

Conclusion

In this study, we have demonstrated that ID experts can reduce very complex cases using FFT heuristics using if–then logic. Our results identified 27 if–then logic–based heuristics that ID clinicians use while dealing with highly complex cases. All 10 ID expert clinicians completed the FFT heuristics in less than 2 minutes. Relying on the concept of less is more as a tool for medical decision-making may help clinicians make accurate, transparent, and quick decisions. Expert clinicians may deal with complex cases better with the information presented in a more methodological and at-the-point of care style. Future health care system designers should consider adopting the idea of “less is more” in the health IT system design to reduce errors by matching system design to the higher-level cognitive abilities of the expert clinicians who will use the systems.

Clinical Relevance Statement

Our results focus on the expert clinicians' decision-making within the context of less-is-more heuristics. Future health IT system designers should focus on intuitive design by creating interfaces that match the cognition of expert clinicians found in our results. The current frustrations with information overload in the graphical interface can be solved by understanding heuristics and using less vital information cues for safer clinical practice.

Conflict of Interest

None declared.

Acknowledgment

We express our sincere gratitude to the Infectious Diseases clinicians who participated in our study and provided valuable insights.

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was reviewed by the University of Utah Institutional Review Board.

Author Contributions

All the authors contributed to the conception of the work, surveying participants, data curation, and drafting the paper.

• References

• 1 Gigerenzer G, Hertwig R, Pachur T. Heuristics: The Foundations of Adaptive Behavior. New York, NY: Oxford University Press; 2011
• 2 Herbert W. Heuristics revealed [internet]. APS Obs 2010 23(08). Accessed at: https://www.psychologicalscience.org/observer/heuristics-revealed
• 4 Peters E, Dieckmann N, Dixon A, Hibbard JH, Mertz CK. Less is more in presenting quality information to consumers. Med Care Res Rev 2007; 64 (02) 169-190
• 4 Banks AP, Gamblin DM, Hutchinson H. Training fast and frugal heuristics in military decision making. Appl Cogn Psychol 2020; 34 (03) 699-709
• 5 Gigerenzer G, Kurzenhäuser S. Fast and frugal heuristics in medical decision making. In: Bibace R, Laird JD, Noller KL, Valsiner J. Science and Medicine in Dialogue: Thinking through Particulars and Universals. Westport, CT: Praeger Publishers/Greenwood Publishing Group; 2005: 3-15
• 6 Islam R, Weir C, Fiol GD. Heuristics in managing complex clinical decision tasks in experts' decision making. IEEE Int Conf Healthc Inform 2014; 2014: 186-193
• 7 Islam R, Mayer J, Clutter J. Supporting novice clinicians cognitive strategies: system design perspective. IEEE EMBS Int Conf Biomed Health Inform 2016; 2016: 509-512
• 8 Narins CR, Dozier AM, Ling FS, Zareba W. The influence of public reporting of outcome data on medical decision making by physicians. Arch Intern Med 2005; 165 (01) 83-87
• 9 Anderson N, Fuller R, Dudley N. ‘Rules of thumb’ or reflective practice? Understanding senior physicians' decision-making about anti-thrombotic usage in atrial fibrillation. QJM 2007; 100 (05) 263-269
• 10 André M, Borgquist L, Foldevi M, Mölstad S. Asking for ‘rules of thumb’: a way to discover tacit knowledge in general practice. Fam Pract 2002; 19 (06) 617-622
• 11 Whelehan DF, Conlon KC, Ridgway PF. Medicine and heuristics: cognitive biases and medical decision-making. Ir J Med Sci 2020; 189 (04) 1477-1484
• 12 Eva KW, Norman GR. Heuristics and biases–a biased perspective on clinical reasoning. Med Educ 2005; 39 (09) 870-872
• 13 Martignon L, Vitouch O, Takezawa M, Forster M. Naive and yet enlightened: from natural frequencies to fast and frugal decision trees. In: Hardman D, Macchi L. eds. Thinking: Psychological Perspectives on Reasoning, Judgment and Decision Making. Wiley; 2003. 30. ;15(06): 189-211
• 14 Gigerenzer G, Kurzenhäuser S. Fast and frugal heuristics in medical decision making. Science and medicine in dialogue: Thinking through particulars and universals. 2005. 30. 3-15
• 15 Fauci AS, Morens DM. The perpetual challenge of infectious diseases. N Engl J Med 2012; 366 (05) 454-461
• 16 Carroll LN, Au AP, Detwiler LT, Fu TC, Painter IS, Abernethy NF. Visualization and analytics tools for infectious disease epidemiology: a systematic review. J Biomed Inform 2014; 51: 287-298
• 17 Fong IW. Challenges in the control and eradication of malaria. In: Fong IW, ed. Challenges in Infectious Diseases. New York, NY: Springer; 2013: 203-231
• 18 Roosan D, Weir C, Samore M. et al. Identifying complexity in infectious diseases inpatient settings: an observation study. J Biomed Inform 2017; 71S: S13-S21
• 19 Fong IW. Challenges in Infectious Diseases. New York, NY: Springer; 2013
• 20 Islam R, Weir CR, Jones M, Del Fiol G, Samore MH. Understanding complex clinical reasoning in infectious diseases for improving clinical decision support design. BMC Med Inform Decis Mak 2015; 15 (01) 101
• 21 Roosan D, Del Fiol G, Butler J. et al. Feasibility of population health analytics and data visualization for decision support in the infectious diseases domain: a pilot study. Appl Clin Inform 2016; 7 (02) 604-623
• 22 Roosan D, Samore M, Jones M, Livnat Y, Clutter J. Big-data based decision-support systems to improve clinicians' cognition. IEEE Int Conf Healthc Inform 2016; 2016: 285-288
• 23 Roosan D, Law A, Karim M, Roosan M. Improving team-based decision making using data analytics and informatics: protocol for a collaborative decision support design. JMIR Res Protoc 2019; 8 (11) e16047
• 24 Hoffman RR, Crandall B, Shadbolt N. Use of the critical decision method to elicit expert knowledge: a case study in the methodology of cognitive task analysis. Hum Factors 1998; 40 (02) 254-276
• 25 Harteis C, Billett S. Intuitive expertise: theories and empirical evidence. Educ Res Rev 2013; 9: 145-157
• 26 Jones SS, Rudin RS, Perry T, Shekelle PG. Health information technology: an updated systematic review with a focus on meaningful use. Ann Intern Med 2014; 160 (01) 48-54
• 27 Islam R, Weir C, Del Fiol G. Clinical complexity in medicine: a measurement model of task and patient complexity. Methods Inf Med 2016; 55 (01) 14-22
• 28 Booth A, Hannes K, Harden A, Noyes J, Harris J, Tong A. COREQ (Consolidated Criteria for Reporting Qualitative Studies). In: Moher D, Altman DG, Schulz KF, Simera I, Wager E. eds. Guidelines for Reporting Health Research: A User's Manual. John Wiley & Sons; 2014: 214-226
• 29 Ogrinc G, Mooney SE, Estrada C. et al. The SQUIRE (Standards for QUality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Qual Saf Health Care 2008; 17 (Suppl 1): i13-i32
• 30 Newell BR, Weston NJ, Shanks DR. Empirical tests of a fast-and-frugal heuristic: Not everyone “takes-the-best.”. Organ Behav Hum Decis Process 2003; 91 (01) 82-96
• 31 Neuendorf KA. The Content Analysis Guidebook. Thousand Oaks, CA: Sage Publications; 2017
• 32 Stemler S. Pract Assess, Res Eval 2002; •••: 3 Retrieved March. 2001
• 33 Katsikopoulos KV, Pachur T, Machery E, Wallin A. From Meehl to fast and frugal heuristics (and back): new insights into how to bridge the clinical—actuarial divide. Theory Psychol 2008; 18 (04) 443-464
• 34 Gibbons LJ, Stoddart K. ‘Fast and frugal heuristics’: clinical decision making in the emergency department. Int Emerg Nurs 2018; 41: 7-12
• 35 Bobadilla-Suarez S, Love BC. Fast or frugal, but not both: decision heuristics under time pressure. J Exp Psychol Learn Mem Cogn 2018; 44 (01) 24-33
• 36 Jenny MA, Pachur T, Lloyd Williams S, Becker E, Margraf J. Simple rules for detecting depression. J Appl Res Mem Cogn 2013; 2 (03) 149-157
• 37 Brainerd CJ, Reyna VF. Fuzzy-trace theory and memory development. Dev Rev 2004; 24 (04) 396-439
• 38 Patel VL, Yoskowitz NA, Arocha JF, Shortliffe EH. Cognitive and learning sciences in biomedical and health instructional design: A review with lessons for biomedical informatics education. J Biomed Inform 2009; 42 (01) 176-197
• 39 Graber ML, Siegal D, Riah H, Johnston D, Kenyon K. Electronic health record-related events in medical malpractice claims. J Patient Saf 2019; 15 (02) 77-85
• 40 Kim MO, Coiera E, Magrabi F. Problems with health information technology and their effects on care delivery and patient outcomes: a systematic review. J Am Med Inform Assoc 2017; 24 (02) 246-250
• 41 Palojoki S, Mäkelä M, Lehtonen L, Saranto K. An analysis of electronic health record-related patient safety incidents. Health Informatics J 2017; 23 (02) 134-145
• 42 Roosan D. The promise of digital health in healthcare equity and medication adherence in the disadvantaged dementia population. Pharmacogenomics 2022; 23 (09) 505-508
• 43 Roosan D, Hwang A, Roosan MR. Pharmacogenomics cascade testing (PhaCT): a novel approach for preemptive pharmacogenomics testing to optimize medication therapy. Pharmacogenomics J 2021; 21 (01) 1-7
• 44 Roosan D, Karim M, Chok J, Roosan M. Operationalizing Healthcare Big Data in the Electronic Health Records Using a Heatmap Visualization Technique. In: Proceedings of the 13th International Joint Conference on Biomedical Engineering Systems and Technologies 2020 5. 361-368 DOI: 10.5220/0008912503610368
• 45 Roosan D, Hwang A, Law AV, Chok J, Roosan MR. The inclusion of health data standards in the implementation of pharmacogenomics systems: a scoping review. Pharmacogenomics 2020; 21 (16) 1191-1202
• 46 Kroth PJ, Morioka-Douglas N, Veres S. et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open 2019; 2 (08) e199609-e199609
• 47 Bosworth HB, Zullig LL, Mendys P. et al. Health information technology: meaningful use and next steps to improving electronic facilitation of medication adherence. JMIR Med Inform 2016; 4 (01) e9
• 48 Kim E, Baskys A, Law AV, Roosan MR, Li Y, Roosan D. Scoping review: the empowerment of Alzheimer's disease caregivers with mHealth applications. NPJ Digit Med 2021; 4 (01) 131
• 49 Roosan D, Wu Y, Tran M, Huang Y, Baskys A, Roosan MR. Opportunities to integrate nutrigenomics into clinical practice and patient counseling. Eur J Clin Nutr 2022; DOI: 10.1038/s41430-022-01146-x.
• 50 Sayer M, Duche A, Nguyen TJT. et al. Clinical implications of combinatorial pharmacogenomic tests based on cytochrome P450 variant selection. Front Genet 2021; 12: 719671
• 51 Li Y, Duche A, Sayer MR. et al. SARS-CoV-2 early infection signature identified potential key infection mechanisms and drug targets. BMC Genomics 2021; 22 (01) 125
• 52 Beam AL, Kohane IS. Big data and machine learning in health care. JAMA 2018; 319 (13) 1317-1318
• 53 Pellegrini E, Ballerini L, Hernandez MDCV. et al. Machine learning of neuroimaging for assisted diagnosis of cognitive impairment and dementia: a systematic review. Alzheimers Dement (Amst) 2018; 10: 519-535
• 54 Roosan D, Law AV, Roosan MR, Li Y. Artificial intelligent context-aware machine-learning tool to detect adverse drug events from social media platforms. J Med Toxicol 2022; 18 (04) 311-320
• 55 Roosan D, Wu Y, Tatla V. et al. Framework to enable pharmacist access to health care data using Blockchain technology and artificial intelligence. J Am Pharm Assoc (Wash DC) 2022; 62 (04) 1124-1132
• 56 Roosan D, Chok J, Baskys A, Roosan MR. PGxKnow: a pharmacogenomics educational HoloLens application of augmented reality and artificial intelligence. Pharmacogenomics 2022; 23 (04) 235-245
• 57 Roosan D, Chok J, Karim M. et al. Artificial intelligence-powered smartphone app to facilitate medication adherence: protocol for a human factors design study. JMIR Res Protoc 2020; 9 (11) e21659
• 58 Roosan D, Li Y, Law A. et al. Improving medication information presentation through interactive visualization in mobile apps: human factors design. JMIR Mhealth Uhealth 2019; 7 (11) e15940

Publication History

Received: 22 September 2021

Accepted: 20 September 2022

Article published online:
09 December 2022