Physician Perceptions of the Electronic Problem List in Pediatric Trauma Care

Keywords

problem-oriented model - patient safety - human factors engineering - pediatric trauma - care coordination

Background and Significance

Context of Pediatric Trauma

Pediatric trauma is the leading cause of death among children aged 1 to 18 years.[1] The emergent and complex pediatric trauma care process can be hazardous to children.[2] In particular, communication and coordination among care team members can be affected by incomplete information, uncertainty, and time pressure.[3] Traumatically injured children generally experience many transitions of care and are treated by multiple clinicians including physicians from different services.[4] [5] We previously identified 53 roles involved in the pediatric trauma care process at a children's hospital.[5] [6] These roles include different groups of physicians that may not be involved in every transition but still require a clear assessment of the patient's hospital course, including a list of the patient's injuries, treatments, current status, and relevant past medical history. One consequence of suboptimal information flow, for example, known injuries and suspected problems, are missed injuries, which, according to one study, occur in 16% of pediatric trauma patients.[7]

Emergence of the Electronic Problem List

A possible solution for documenting and communicating information about the patient and his/her injuries is use of the electronic problem list (PL), a standard part of the electronic health record (EHR). In 1968, Dr. Larry Weed introduced the concept of the PL, which has become the focus of the problem-oriented medical record in a computerized system. Weed[8] advocated that physicians take a systematic approach to the medical record by organizing data around each problem to avoid missing details and improving continuity of care. The increased use of the EHR offers an opportunity for physician notes to be organized around problems on the PL and associated patient data (e.g., imaging and medications).[9] For example, the problem-oriented model allows EHR users to click on the PL and view a dynamic display of relevant labs, imaging, procedure data, and consultant notes.[10] Another approach is problem-based documentation, where clinicians document assessments and plans for each problem on the PL.[11] [12] The increasing use of the EHR may help to implement Weed's vision and better support physician work so that recorded problems are connected to the relevant patient information in other parts of the EHR.[10] [13]

Research on the Electronic Problem List: Outpatient Settings

Most studies on the electronic PL are performed in outpatient care settings.[14] [15] [16] [17] [18] [19] [20] [21] Makam et al[18] reported that 70% of primary care providers (PCPs) who responded to a survey on the use of and satisfaction with the EHR thought that the PL was helpful. Research in the outpatient setting has also examined physician use of the PL. Analysis of a random selection of 100,000 medical records demonstrated that PCPs were more likely to add problems to the PL than specialists.[22] Accuracy and completeness of the PL remain major issues.[15] [17] For example, PL completeness for diabetes patients was measured in a retrospective analysis of EHR data from 10 healthcare facilities (e.g., academic medical center, community hospital, and regional health system). Results showed that PL completeness ranged from 60 to 99%, as measured by the ratio of outpatients having diabetes coded on their PL to the number of patients with HbA1c levels greater than 7%.[21] A survey with mostly PCPs incorporated clinical scenarios to understand their actions toward the PL,[23] showing differing opinions about PL content, particularly which problems to include (e.g., family history and surgeries), and whether the PL should be structured with discrete or free-text fields. To address issues of PL accuracy and completeness, informatics methods such as natural language processing have been used to populate the electronic PL.[16] [19] [24] [25] [26] [27] Electronic alerts in 28 primary care clinics used medication and laboratory-problem associations to identify undocumented problems for 17 conditions;[27] subsequent alerts for physicians to add these problems increased documentation during a 6-month period from 3,739 problems to 10,016 problems. PL accuracy and completeness can also be improved by better understanding how physicians perceive and would like to use the PL.

Extending Research on the Electronic Problem List from Outpatient to Inpatient Settings

Differences between outpatient and inpatient settings may shed light on variations in physician PL usage and help understand why PL content may be incomplete or inaccurate. Wright et al[28] performed 264 hours of observations and interviewed 63 clinicians across multiple specialties to understand their PL use. They identified several themes that described PL utilization behaviors, especially noting (1) ownership and responsibility for maintaining the PL, (2) presentation and organization to support automatically sorting or grouping problems in the EHR, and (3) accuracy of the PL as a reliable source of information. Zhou et al[29] studied how the PL supported information sharing between different clinicians, for example, specialist, general medicine hospitalists, and PCPs. They performed more than 750 hours of field observations, reviewed patient medical records, and interviewed physicians and nurse practitioners. They found that PCPs were more likely to update the PL because it saved time during the patient's next visit and helped to maintain continuity of care. Some specialists thought that because they spent significant time writing clinical notes, spending extra time adding problems to the PL was unnecessary. Zhou et al[29] argued that mixed perceptions about PL content results in not all clinicians using it for the same purpose. For example, one group of physicians used the PL as a place to put comments about patients they wanted another physician to know about. Understanding physician views and perceptions of the PL is necessary to improve its design and better support physicians' information needs.

The PL could be designed to avoid fragmented care planning and support care coordination by integrating priorities from various clinical disciplines. Collins et al[30] argued that the collaborative nature of a shared PL among different disciplines has design implications to support clinicians with overlapping information needs. Based on a literature review, they proposed five sociotechnical requirements to support the patient-centered PL. One of the sociotechnical requirements is to categorize problems based on priority ranking for discipline-specific needs. The researchers suggested that PL users view their priority rankings of problems alongside others' to understand different perspectives and improve PL management. Improving PL use for different groups of users could result in a more complete and accurate PL and, therefore, enhanced care coordination.

Only one study evaluated PL use in an inpatient pediatric setting; PL use was measured by at least one problem documented on the PL at the time of discharge.[11] A series of interventions, for example, resident and fellow training on the PL, handouts, and teaching about PL use during rounds, was implemented to improve PL documentation. Hospitalists and residents on the documentation committee provided feedback on interventions by sending biweekly e-mails with daily PL usage graphs; PL use improved from 27 to 97%.[11] Research on the PL in pediatric settings is limited, especially studies describing how physicians think it could be used in a pediatric inpatient setting.

Objectives

Our study aims to describe perceptions of physicians from different services involved in acute pediatric trauma (i.e., physicians in pediatric emergency department [ED], pediatric surgery, pediatric anesthesiology, and pediatric intensive care unit [PICU]) about the potential PL functionality. We identify how physicians define the PL and its goals, characteristics, and information elements and compare PL perceptions of physicians from different services involved in the acute pediatric trauma care process.

Methods

Setting and Participants

This study is part of a larger project aimed at developing health IT (information technology) design requirements to support care transitions for pediatric trauma.[31] The participating hospital is an American College of Surgeons certified level 1 pediatric and adult trauma center[32] [33] with an 87-bed children's hospital, a 21-bed PICU, and 8 pediatric operating rooms. The pediatric trauma center receives between 250 and 300 level 1 and 2 pediatric trauma cases per year (∼50 level 1 patients and 250 level 2 patients per year) and between 400 and 450 unleveled traumas per year. There were 1,487 pediatric traumas between 2013 and 2017.[5] The participating hospital implemented a system-wide EHR (Epic Systems Corporation, Verona, Wisconsin, United States) in 2008. This preliminary, exploratory study was granted exemption from the University of Wisconsin-Madison Institutional Review Board.

Interview data collection occurred between July and November 2016. We used purposive sampling to interview three various physicians (resident, fellow, attending) in each of the four services: ED, surgery, anesthesia, and PICU.[34] We asked members of our research team, that is, attending physicians from pediatric emergency medicine (EM), surgery, anesthesia, critical care, and hospitalist services, and the hospital's pediatric trauma nursing program manager, for suggestions on how to identify potential participants. We e-mailed potential participants with a description of the project. Participation was voluntary, and all participants provided verbal consent. We interviewed seven males and five females.

Data Collection Methods

Based on discussion with experts at the Johns Hopkins University and the University of Wisconsin-Madison and on the literature, we developed an initial version of the PL interview guide. Using the initial version of the interview guide, we interviewed three physicians on our research team and, based on their feedback, we made some minor revisions to questions in the guide ([Appendix A]). We used the updated version of the interview guide for the rest of the interviews.

Pairs of human factors researchers conducted the 12 semistructured interviews. The average interview duration was 54 minutes (range: 41 to 88 minutes), for a total of 10 hours and 44 minutes. Interviews were audio-recorded and transcribed by a professional transcription service. The interview guide contained questions on the ideal PL, not the current PL functionality, including definition, use and content that should be included. See [Appendix A] for the full interview guide.

Data Analysis Methods

Interview transcripts were uploaded to Dedoose, a qualitative data analysis software, and coded by the human factors researchers. Excerpts from the interview transcripts related to the PL were coded in three categories: (1) goals, (2) characteristics, and (3) patient-related information. Multiple goals, characteristics, and patient-related information elements emerged from the inductive, multistep coding process; these lists were created and used in Dedoose. Two researchers first individually coded a single interview in Dedoose and met to review their coding. The researchers discussed disagreement in coding and reached consensus about the coding structure after discussion. The process was repeated in Dedoose for a second interview. This allowed refinement of the three categories of (1) goals, (2) characteristics, and (3) patient-related information. Two researchers separately coded the rest of the interviews in Dedoose and continually discussed their coding to assure consistency. Four researchers reviewed the coding iteratively, and refined or combined codes and revised definitions as the data analysis proceeded until saturation was achieved.[35]

We presented our preliminary findings to all research team members, a form of member checking, which is a strategy to ensure rigor in qualitative data analysis.[34] Members of the research team reviewed the definitions and suggested combining certain codes (e.g. pain with medications); their feedback was incorporated in the final coding. We computed frequencies for each goal and characteristic (see the Total column in [Tables 1] and [2]), and patient-related information (see the Include/exclude column in [Table 3]) across all the four services. We computed totals for the four different services across all goals, as well as all characteristics to compare which service mentioned goals or characteristics more often.

Table 1 PL Goals by Service and Policy sorted by frequency

PL goals	Definitions	Services
		ED, N = 3	Surgery, N = 3	Anesthesia, N = 3	PICU, N = 3	Total, N = 12
1. To communicate with others	The PL helps physicians communicate with other clinicians involved or getting involved in the patient's care (i.e., what has been done and needs to be done). These clinicians may be distributed across services, environments, time, etc.	✓✓	✓✓	✓✓	✓✓✓	9
2. To make sense of the patient's problems[a]	The PL provides an overall assessment of what's going on with the patient; this is particularly helpful when meeting or caring for the patient the first time.	✓✓	✓	✓✓	✓✓✓	8
3. To document the patient's problems[a]	Patient's injuries and problems are recorded as they are identified; this helps to avoid missed injuries. All problems, big and small, should be captured.	✓✓	✓	✓	✓✓✓	7
4. To make decisions about the care plan[a]	The PL helps with what to do next to care for the patient, e.g., to define, review, and revise the care plan. It is helpful to anticipate and plan how to proceed with caring for the patient.	✓	✓	✓	✓✓	5
5. To know who is involved in the patient's care	The PL helps to identify who is caring for the patient.		✓✓		✓✓	4
Total		7	7	6	13

Abbreviations: ED, emergency department; PICU, pediatric intensive care unit; PL, problem list.

Note: A checkmark indicates that one interviewee mentioned that goal.

^a Indicates that the goal was mentioned in the hospital policy.

Table 2 PL Characteristics by Service and Policy sorted by frequency

PL characteristics	Definitions	Services
		ED, N = 3	Surgery, N = 3	Anesthesia, N = 3	PICU, N = 3	Total
1. Completeness[a]	Reflecting changes over time, since trauma occurred	✓✓✓	✓✓✓	✓✓	✓✓✓	11
2. Efficiency	Fast to use; not too much information	✓✓	✓✓	✓✓✓	✓✓✓	10
3. Accessibility	Available when and where needed	✓✓	✓✓✓	✓✓	✓✓	9
4. Multiple users[a]	Shared and supporting multiple roles and their perspectives	✓✓	✓✓	✓	✓✓	7
5. Organized	Order of the problems structured by organ system or injury priority	✓✓	✓✓✓	✓	✓	7
6. Created before arrival	To prepare before patient arrival	✓✓	✓	✓	✓✓	6
7. Representing uncertainty[a]	Represents uncertainty of patient's problem	✓✓		✓	✓	4
Total		15	14	11	14

Abbreviations: ED, emergency department; PICU, pediatric intensive care unit; PL, problem list.

Note: A checkmark indicates that one interviewee mentioned the characteristic.

^a Indicates that the characteristic was mentioned in the hospital policy.

Table 3 Patient-related Information by Service and Policy (sorted by frequency)

Patient-related Information	Definition	Services
		ED, N = 3	Surgery, N = 3	Anesthesia, N = 3	PICU, N = 3	Include/exclude
1. Medications	Patient's medications	✓✓✓	✓✓✓XX	✓✓	✓✓✓	11 include, 2 exclude
2. Injuries[a]	List of the patient's injuries	✓✓✓	✓✓✓	✓	✓✓✓	10 include
3. Past medical history	Patient's past medical history	✓✓	✓✓XX	✓✓✓	✓✓	9 include, 2 exclude
4. Plan of care	Ongoing management of patient's problem	✓✓✓	✓✓X	✓X	✓✓✓	9 Include, 2 exclude
5. Allergies	Patient's allergies	✓✓	✓✓X	✓✓	✓✓	8 include, 1 exclude
6. Care completed	What has been done, specifically the care the patient has received	✓✓✓X	✓✓✓	✓	✓X	8 include, 2 exclude
7. Vitals	Patient's vitals	✓✓	✓✓✓XXX	✓	✓✓XX	8 include, 5 exclude
8. Events	What happened to the trauma patient (i.e., the events)	✓✓✓		✓	✓✓✓	7 include
9. Labs and imaging	Tests performed to detect, diagnose, or monitor any injury	✓✓	✓✓✓X	✓	✓X	7 include, 2 exclude
10. Teams involved	List of teams caring for the patient	✓✓	✓✓✓	✓		6 include
11. Non-trauma issues[a]	Active, ongoing issues are unrelated to the trauma (e.g., diabetes or asthma)	✓	✓XX	✓✓X	✓✓X	6 include, 4 exclude
12. Difficult airways	Patient's breathing status	✓✓		✓✓	✓	5 include
13. Date of birth	Patient's date of birth	✓	✓	✓	✓	4 include
14. Patient name	Patient's name	✓		✓		2 include
15. Patient weight	Patient's weight			✓✓		2 include
16. Immunizations	Immunizations the patient has	✓			✓	2 include
17. Last meal	The last meal the patient had	✓		✓		2 include
18. Prior anesthetics	Prior anesthetics that have been given			✓✓		2 include
19. IV access	IV access the patient has		X	✓	✓	2 include, 1 exclude
20. Gender	Patient's gender	✓				1 include
21. All inputs and outputs	The problem list should include urine output and intake			✓		1 include
22. Social support	Social support of the patient, who is with the patient, etc.			✓		1 include

Abbreviations: ED, emergency department; IV, intravenous; PICU, pediatric intensive care unit.

Note: A checkmark (✓) indicates that one interviewee mentioned the information. The number in front of “include” indicates a row total for how many interviewees mentioned to include that information on the PL. An X indicates that one participant said that piece of information should not be included on the PL. The number in front of “exclude” indicates a row total for how many interviewees mentioned to not include that information on the PL. Occasionally, an interviewee mentioned to include and exclude a piece of information in the same interview.

^a Indicates the patient-related information was mentioned in hospital policy to be included on the PL.

We obtained a copy of the hospital's PL policy (formally called “PL etiquette”), which was also coded for goals, characteristics, and patient-related information. We compared our interview results to the coded hospital's PL policy.

Results

Goals of the Problem List

[Table 1] shows the five PL goals, their definitions, and the frequencies with which each PL goal was mentioned across the four services (EM, surgery, anesthesia, PICU), as well as whether the specific PL goal was included in the hospital's PL policy. Four of the five PL goals were mentioned by at least one participant in each service: documenting the patient's problems, making sense of the patient's problems, making decisions about the care plan, and communicating with others. PICU physicians mentioned goals of the PL more frequently than physicians in other services (see the Total column in [Table 1]).

Nine of the 12 participants mentioned the PL goal of communicating with others. A PICU fellow mentioned that the PL should be used to communicate with physicians involved in the child's care, “…closing the loop … getting the word out to everybody … what's important to a surgeon … [and] what's important to me when I'm trying to take care of the patient overnight.”

Eight of the 12 participants mentioned the PL goal of making sense of the patient's problems. An EM attending said the PL allows him to know of other medical problems, “So a problem list for me is … a quick way to assess, does this patient have any other medical problems?” A PICU attending talked about how the PL can inform him of treatment, “So a good problem list really informs all of the treatment and who is needed to be part of the care team as well.”

Of the 12 participants, seven mentioned the PL goal of documenting the patient's problems; these participants were mostly from EM, surgery, and PICU. A surgery resident mentioned that the PL helps to distinguish who has what injury, “… especially trauma patients, all the patients tend to kind of blend together so it helps us distinguish who has what injury, what side is the injury on, who's managing that injury, what do we need to do for that injury?” A PICU resident said that the PL is a list of the patient's medical problems, “… A list of what the patient's medical problem [are] and conditions are … that are being addressed in their current hospitalization.”

Five of the 12 participants, mostly from surgery and PICU, mentioned the PL goal of making decisions about the care plan. An EM resident said that the PL helps in decisions regarding next steps, “Any pertinent labs … [are] vital information [that] guides your next step … if they don't know that information, they don't know what they're doing.” A surgery resident talked about the PL including the care plan, “…Our problem list is a list of injuries that the patient has. … if it's well updated … the plan for those injuries, who's consulting,.... [and] what follow-up or further things that they need.”

Four of the 12 participants, all from surgery and the PICU, mentioned the PL goal of knowing who is involved in care. One surgery attending described the usefulness of knowing the other surgical services involved in the child's care, “It would be helpful to … know where everybody is at … this bone fracture … was managed by orthopedics …”

Characteristics of the Problem List

[Table 2] shows the seven PL characteristics, their definitions, and frequencies across all four services (EM, surgery, anesthesia, PICU), as well as the coding of the hospital's PL policy. Six of the seven PL characteristics were mentioned by at least one participant in each service: completeness, efficiency, accessibility, multiple users, organized, PL created before arrival, and PL representing uncertainty. Overall, physicians in all four services mentioned most of the characteristics (see the Total row in [Table 2]).

Of the 12 participants, 11 mentioned the PL characteristic of completeness. A PICU resident talked about how it was helpful to have a PL with all of the problems there, “Patients who are medically complex … have a very, very thorough problem list ... And it's very helpful … that all the problems are in there.” An EM resident mentioned the need for every problem to be addressed, “Because a lot of times, what happens is you get these little injuries. They go to the next phase, and it gets forgotten about … I don't think any problem is too small. I think it all needs to be addressed, especially in kids.”

Of the 12 participants, 10 mentioned the PL characteristic of efficiency. An EM attending talked about the need to communicate information efficiently, “How do we continue to communicate that information efficiently in a way that's value-added and helpful to handoffs in those transitions of care?” An anesthesia attending mentioned the challenge of getting to key information quickly, “I have to sift through telephone encounters … nurses calling the family … It's hard to get the pertinent points quickly.”

Of the 12 participants, nine mentioned the PL characteristic of accessibility. An anesthesia resident indicated that the PL should be easy to find, “Something that would be streamlined … [and] accessible … when you log into [EHR], it comes up, or … you can find easily that's clear and concise.” A PICU fellow talked about the fact that information needs to be easily accessible, for example, in a paper format, “If it's an easy-to-access thing, before the patient arrives … we could print it off … I could have that sheet with me in the room and … then I'm not tied to a computer.”

Of the 12 participants, seven mentioned the PL characteristic of multiple users; these interviewees were mostly from EM, surgery, and PICU. A PICU attending said, “We're always building that problem list. It's always … a construct in all of our minds. The question is, how do we take all of our minds and put it into a ‘group think’, a group-[generated] list that we can all sign off and say … I agree, that adequately describes this patient.”

Seven of the 12 participants, mostly from EM and surgery, mentioned the PL characteristic of organized. The PL could, for instance, be organized by organ system, as indicated by a surgery attending, “I would rather have it by organ systems, because it makes you think … am I missing something somewhere?” A PICU attending talked about organizing the PL in the context of other important information, “So a problem existing just by itself is not enough. There needs to be subcategories under that in terms of … what is the degree of that injury and what has [been] done about it, and then, ideally, what is planned to be done about it.”

Of the 12 participants, six mentioned the need for the PL to be created before arrival. A surgery resident said, “I think we should gather all the information that we can from that time when the trauma happened and input it into our system.”

Four of the 12 participants from EM, anesthesia, and PICU mentioned the PL characteristic of representing uncertainty. An EM resident talked about the need to indicate that certain problems may not be fully identified and have a degree of uncertainty, “We don't know at that time, what all the problems are. We could say trauma or spleen injury, but we don't know exactly [what] that spleen injury is going to be, [maybe] they're going to remove the spleen.”

Patient-Related Information Associated with the Problem List

Physicians described a total of 22 information elements. [Table 3] lists the patient-related information, their definitions, and frequencies, denoted by “include” and “exclude,” across all four services (EM, surgery, anesthesia, PICU), as well as the coding of the hospital's PL policy. The 22 PL patient-related information elements ranged from situation/background, such as patient name and gender, to objective data such as medications and vitals, as well as assessment information such as injuries and plan of care. Of the 22 PL patient-related information elements, 10 were mentioned by physicians in the four services: medications, injuries, plan of care, past medical history, allergies, care completed, vitals, labs and imaging, non-trauma issue, and date of birth. A few participants mentioned information elements that should be excluded from the PL, and other physicians could mention the same information elements as elements that should be included in the PL. For example, a surgery resident mentioned that vitals should be excluded and later mentioned that vitals should be included, “I can't see an injury that a vital sign is going to necessarily line up … in those situations … even vitals in the field versus vitals when they arrive, that's important.”

There is general agreement about medications, injuries, and past medical history to be included on or connected to the PL, as these elements were mentioned by 11, 10, and 9 participants, respectively. There is less agreement about including in the PL or connecting the PL to gender, all inputs and outputs, and social support, as these information elements were each mentioned by one participant.

Of 12 participants, 10 mentioned injuries as one of the PL patient-related information elements. An EM attending suggested that blood and pain control for injuries should be on the PL, “… If I have a critical patient … with shock, head injury, belly trauma … we gave blood … We gave pain control … not everything falls in perfectly in a problem list.” An anesthesia resident mentioned having “a story” of the injuries on the PL, including information about what happened, airway access, blood given, and allergies, “… Having … more of a story of the injuries, what happened … they have an airway … two IVs. They're getting a unit of blood... They have multiple long bone fractures. They're known to have asthma and are allergic to sulfa.”

Coding of the Hospital's Problem List Policy

Three of the five PL goals identified by the interviewees are addressed in the PL policy: documenting the patient's problems, making sense of the patient's problems, and making decisions about the care plan ([Table 1]). Three of the seven characteristics are also mentioned in the PL policy: completeness, multiple users, and uncertainty ([Table 2]). The PL policy includes 2 of the 22 patient-related information: injuries and non-trauma issues ([Table 3]).

Discussion

When interviewed about the ideal electronic PL, 12 physicians across four services involved in pediatric trauma care mentioned five goals for the PL, seven characteristics of the PL, and 22 patient-related information elements. Many of the characteristics and patient-related information elements were not found in the hospital's PL policy, suggesting a gap in the formal PL functionality (as described in the policy) to meet physician information needs.

Previous studies, primarily conducted in outpatient care settings, have described a lack of consensus about what should be included on the PL.[23] [28] [29] Our results also demonstrate challenges in achieving consensus about information elements that should be included in or connected to the PL. Nine of the 22 patient-related information were identified by some physicians as related to the PL or as elements that should be excluded from the PL ([Table 3]): for example, medications, past medical history, plan of care, and allergies. In all instances, there were more mentions of inclusion in the PL than exclusion from the PL. Physicians from different services involved in pediatric trauma care somewhat agree that many information elements in the EHR are related to the PL. More than 75% of the 22 patient-related information elements were mentioned by at least two participants from different services ([Table 3]). These findings have design implications for the electronic PL to be organized around related patient data, for example, imaging and medications. Organizing the PL around other relevant patient data besides diagnoses and injuries could help support physician clinical thinking and coordination of care.[8] [9] [10] [13]

Our results confirm findings of previous PL research[14] [15] [16] [17] [18] [19] [21] [28] that completeness and accuracy are two important characteristics of the PL. The PL should be designed to support overlapping information needs between physicians involved in the complex pediatric trauma care process as most participants mentioned the PL goal of communicating with others and PL characteristic of completeness ([Tables 1] and [2]). Physicians described using the PL to communicate with other physicians distributed across services and environments. The amount of information and organization of the PL may depend on the child's stage of care, for example, ED versus PICU. Therefore, PL goals and characteristics must not be considered independently as they interact. In light of the systems approach recommended by human factors and systems engineering,[36] [37] the PL should be designed to support interactions between goals and characteristics to have a positive impact on patient safety by avoiding missed injuries while supporting effective and efficient communication and coordination during in-hospital transitions, for example, from the ED to the operating room to PICU.

There are some limitations to this study. First, our findings are limited by the small sample size of 12 interviews with physicians involved in pediatric trauma. The data may not capture the entire range of physicians' perceptions of the PL, for example, hospitalists who may care for children with trauma injuries after a PICU stay. Second, we collected data at a single, academic institution. Third, we interviewed only physicians who are the primary users and generators of the PL; understanding the perspective of other healthcare professions, for example, nurses, who interact with the child and use the PL in the EHR would be useful. Fourth, we focused on in-hospital transitions. Therefore, it is difficult to determine whether our results are generalizable to other children's hospitals of varying trauma level verification or nonteaching hospitals. More work is needed to evaluate and improve the design of the PL so that it accomplishes the goals and characteristics identified in this study.

Conclusion

Physicians described the PL as more than a tool to document and share a list of problems or injuries suffered by pediatric trauma patients. Physicians from four services involved in pediatric trauma mentioned many other information elements connected to problems on the PL, for example, medications and past medical history. A PL should support physician cognitive work and the collaborative nature of the pediatric trauma care process. Future studies could build on our results and examine the importance of the goals, characteristics, and information elements of the PL as perceived by physicians involved in pediatric trauma in other children's hospitals.

Clinical Relevance Statement

Our results have design implications for the electronic PL to be organized around relevant patient data that could help support physician clinical thinking and coordination of care. The electronic PL should be designed to balance its goals and characteristics and have a positive impact on patient safety, for example, avoid missed injuries, and support communication and coordination during transitions.

Multiple Choice Questions

1. Physicians think of the PL as follows:

   • The PL is just a list of problems, injuries, and diagnoses of the patient, which are not related to any other parts of the EHR.

   • The PL is a list of problems, injuries, and diagnoses of the patient, which should be connected to other parts of the EHR, such as imaging and medications.

   Correct Answer: The correct answer is option b. The results of our study clearly show that physicians talk about the PL in relation to many other patient-related information elements.

2. The literature shows that the following characteristics of PL remain an issue.

   • Completeness.

   • Customizable.

   • Not shareable.

   • Duplicative.

   Correct Answer: The correct answer is option a. PL completeness is often mentioned in the literature, as well as by our study participants.

3. The hospital policy on PL describes the following information elements:

   • List of injuries and non-trauma issues.

   • List of non-trauma issues and medications.

   • List of medications and vitals.

   • List of vitals and allergies.

   Correct Answer: The correct answer is option a. The results of our study show that hospital policy mentioned to include injuries and non-trauma issues on the PL.

• References

• 1 CDC. 10 Leading Causes of Death Age Group, United States – 2016. Available at: https://www.cdc.gov/injury/images/lc-charts/leading_causes_of_death_age_group_2016_1056w814h.gif . Accessed April 18, 2018
• 2 Centers for Disease Control Prevention. Injury Prevention and Control. Available at: https://www.cdc.gov/injury/wisqars/index.html . Accessed April 18, 2018
• 3 Hoonakker PLT, Wooldridge AR, Hose B-Z. , et al. Things Falling through the Cracks: Information Loss during Pediatric Trauma Care Transitions. Paper presented at the Congress of the International Ergonomics Association, Florence, Italy, 2018
• 4 Durojaiye AB, McGeorge NM, Puett LL. , et al. Mapping the flow of pediatric trauma patients using process mining. Appl Clin Inform 2018; 9 (03) 654-666
• 5 Wooldridge AR, Carayon P, Hoonakker P. , et al. Complexity of the pediatric trauma care process: implications for multi-level awareness. Cogn Tech Work 2018 Doi: 10.1007/s10111-018-0520-0
• 6 Wooldridge AR, Carayon P, Hoonakker P. , et al. Understanding Team Complexity in Pediatric Trauma Care. Paper presented at the Human Factors and Ergonomics in Healthcare Symposium New Orleans, Louisiana, 2017
• 7 Soundappan SVS, Holland AJA, Cass DT. Role of an extended tertiary survey in detecting missed injuries in children. J Trauma 2004; 57 (01) 114-118
• 8 Weed LL. Medical records that guide and teach. N Engl J Med 1968; 278 (11) 593-600
• 9 Hodge CM, Narus SP. Electronic problem lists: a thematic analysis of a systematic literature review to identify aspects critical to success. J Am Med Inform Assoc 2018; 25 (05) 603-613
• 10 Buchanan J. Accelerating the benefits of the problem oriented medical record. Appl Clin Inform 2017; 8 (01) 180-190
• 11 Bakel LA, Wilson K, Tyler A. , et al. A quality improvement study to improve inpatient problem list use. Hosp Pediatr 2014; 4 (04) 205-210
• 12 Li RC, Garg T, Cun T. , et al. Impact of problem-based charting on the utilization and accuracy of the electronic problem list. J Am Med Inform Assoc 2018; 25 (05) 548-554
• 13 Schiff GD, Bates DW. Can electronic clinical documentation help prevent diagnostic errors?. N Engl J Med 2010; 362 (12) 1066-1069
• 14 Ancker JS, Kern LM, Edwards A. , et al; HITEC Investigators. How is the electronic health record being used? Use of EHR data to assess physician-level variability in technology use. J Am Med Inform Assoc 2014; 21 (06) 1001-1008
• 15 Banerjee ES, Gambler A, Fogleman C. Adding obesity to the problem list increases the rate of providers addressing obesity. Fam Med 2013; 45 (09) 629-633
• 16 Bui AAT, Taira RK, El-Saden S, Dordoni A, Aberle DR. Automated medical problem list generation: towards a patient timeline. Stud Health Technol Inform 2004; 107 (Pt 1): 587-591
• 17 Daskivich TJ, Abedi G, Kaplan SH. , et al. Electronic health record problem lists: accurate enough for risk adjustment?. Am J Manag Care 2018; 24 (01) e24-e29
• 18 Makam AN, Lanham HJ, Batchelor K. , et al. Use and satisfaction with key functions of a common commercial electronic health record: a survey of primary care providers. BMC Med Inform Decis Mak 2013; 13 (01) 86
• 19 McEvoy D, Gandhi TK, Turchin A, Wright A. Enhancing problem list documentation in electronic health records using two methods: the example of prior splenectomy. BMJ Qual Saf 2018; 27 (01) 40-47
• 20 Wright A, Pang J, Feblowitz JC. , et al. A method and knowledge base for automated inference of patient problems from structured data in an electronic medical record. J Am Med Inform Assoc 2011; 18 (06) 859-867
• 21 Wright A, McCoy AB, Hickman TT. , et al. Problem list completeness in electronic health records: a multi-site study and assessment of success factors. Int J Med Inform 2015; 84 (10) 784-790
• 22 Wright A, Feblowitz J, Maloney FL, Henkin S, Bates DW. Use of an electronic problem list by primary care providers and specialists. J Gen Intern Med 2012; 27 (08) 968-973
• 23 Holmes C, Brown M, Hilaire DS, Wright A. Healthcare provider attitudes towards the problem list in an electronic health record: a mixed-methods qualitative study. BMC Med Inform Decis Mak 2012; 12: 127
• 24 Cillessen FHJM, de Vries Robbé PF, Biermans MCJ. A hospital-wide transition from paper to digital problem-oriented clinical notes. A descriptive history and cross-sectional survey of use, usability, and satisfaction. Appl Clin Inform 2017; 8 (02) 502-514
• 25 Cillessen FH, de Vries Robbé PF. Modeling problem-oriented clinical notes. Methods Inf Med 2012; 51 (06) 507-515
• 26 Meystre SM, Haug PJ. Randomized controlled trial of an automated problem list with improved sensitivity. Int J Med Inform 2008; 77 (09) 602-612
• 27 Wright A, Pang J, Feblowitz JC. , et al. Improving completeness of electronic problem lists through clinical decision support: a randomized, controlled trial. J Am Med Inform Assoc 2012; 19 (04) 555-561
• 28 Wright A, Maloney FL, Feblowitz JC. Clinician attitudes toward and use of electronic problem lists: a thematic analysis. BMC Med Inform Decis Mak 2011; 11 (01) 36
• 29 Zhou X, Zheng K, Ackerman M, Hanauer D. Cooperative Documentation: The Patient Problem List as a Nexus in Electronic Health Records. Paper presented at the Proceedings of the ACM Conference on Computer Supported Cooperative Work, Seattle, WA, February 11–15, 2012
• 30 Collins S, Tsivkin K, Hongsermeier T, Dubois D, Nandigam HK, Rocha RA. A continuum of sociotechnical requirements for patient-centered problem lists. Stud Health Technol Inform 2013; 192: 889-893
• 31 Center for Quality and Productivity Improvement. Teamwork and Care Transitions in pediatric Trauma Understanding Clinical Information Needs and Healthcare Decision Making Processes in the Context of Health Information. Available at: http://cqpi.wisc.edu/teamwork-and-care-transitions-in-pediatric-trauma/ . Accessed August 3, 2018
• 32 Acosta CD, Kit Delgado M, Gisondi MA. , et al. Characteristics of pediatric trauma transfers to a level i trauma center: implications for developing a regionalized pediatric trauma system in California. Acad Emerg Med 2010; 17 (12) 1364-1373
• 33 Notrica DM, Brown D, Garcia-Filion P. Development of a pediatric level 1 trauma center at a freestanding children's hospital: staff attitudes and perceptions before and after trauma designation. J Pediatr Surg 2011; 46 (09) 1764-1770
• 34 Devers KJ. How will we know “good” qualitative research when we see it? Beginning the dialogue in health services research. Health Serv Res 1999; 34 (5 Pt 2): 1153-1188
• 35 Creswell JW. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. 4th ed. Thousand Oaks, CA: Sage; 2014
• 36 Carayon P, Schoofs Hundt A, Karsh BT. , et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care 2006; 15 (Suppl. 01) i50-i58
• 37 Carayon P, Wetterneck TB, Rivera-Rodriguez AJ. , et al. Human factors systems approach to healthcare quality and patient safety. Appl Ergon 2014; 45 (01) 14-25