Face and Content Validity of a Probe Tube Placement Training Simulator

Abstract

Background:

Probe tube placement is an important skill audiologists must learn to make real-ear measurements in an audiology clinic. With current evidence-based guidelines recommending insertion of the probe tube within 5 mm of the tympanic membrane (TM) for proper acoustical measurements, students must be well trained to ensure they are capable to perform this placement in clinical practice. This is not always the case as it has been found that real-ear measurements are not performed in a clinic as often as required. To address this, a simulator consisting of a 3D-printed ear model and an optical tracking system was developed to provide a training system for students to practice probe tube placement and to provide a method to evaluate competency before starting clinical practicum placements. Two simulators were developed, an adult model and a pediatric model.

Purpose:

To assess the face and content validity of the two probe tube placement simulators (adult and pediatric) and define barriers and facilitators to implementing this system into an educational setting.

Research Design:

Participants followed the setup and operating instructions designed to guide them through each functionality of the simulator. A questionnaire was used to assess face and content validity, applicability to an educational setting, and to determine perceived barriers and facilitators to using the probe tube simulators for training purposes. Five additional probe tube placements with each simulator were performed in which distance-to-TM was recorded.

Study Sample:

Twelve participants with significant probe tube placement experience.

Data Collection and Analysis:

Participants rated each question in the questionnaire from 0% to 100% depending on their level of agreement. Averages and standard deviations (SDs) were compiled and presented for each section (face validity, content validity, and applicability to an educational setting). Final facilitators and barriers for the simulator were compiled and the top answers of each are presented. The five quantitative probe tube placement measurements for each participant were averaged, SDs were calculated, and contacts with the TM while placing the probe tube were recorded.

Results:

The average face validity score over all questions for the adult model was 65% (SD = 18.2) whereas the pediatric model received a score of 64% (16.4). The overall content validity average score was 78.7% (17) and applicability to an educational setting had an average score of 80% (5.33). The average distance-to-TM across all trials and participants was 3.74 mm (1.82) for the adult model and 2.77 mm (0.94) for the pediatric model with only one participant exceeding the recommended maximum of 5 mm. Listed shortcomings of the current simulator included realism of the 3D-printed ear, ease of insertion of an otoscope tip into the ear, ability to visualize the ear canal “landmarks” and the TM, and foam tip insertion experience.

Conclusions:

Results were generally very positive for the simulator, and future iterations will look to improve the flexibility and texture of the ear, as well as the otoscopic view of the ear canal and TM.

Key Words

content validity - education - face validity - hearing aids - probe tube placement - real-ear - simulation - training

INTRODUCTION

The insertion of a small diameter soft probe tube into a patient’s ear canal to perform real-ear measurements is a common clinical skill in audiology that must be mastered to properly verify hearing aid fittings. Practice standards and clinical practice guidelines recommend that real-ear probe microphone measurements be performed on every patient who is being provided with hearing aids ([ISO, 2001]; [Valente, 2006]; [BSA, 2007]; [CASLPO, 2016]). According to these guidelines, as well as recent literature, the probe tube must be placed within 5 mm of the tympanic membrane (TM) to obtain the most accurate measurements ([Dirks and Kincaid, 1987]; [Bagatto et al, 2006]; [Moodie, Pietrobon, et al, 2016]; [Vaisberg et al, 2016]). Contact with the TM must be avoided as it will cause physical discomfort, can cause a patient to lose confidence in the technical skills of their clinician, and, in pediatric cases, can lead to an early appointment conclusion because of lack of cooperation ([Moodie, Rall, et al, 2016]).

Probe microphone real-ear measurements have had several levels of resistance to overcome since their introduction in 1979, and there continues to be a lack of widespread use in clinical practice ([Mueller, 2005]; [2014]; [Mueller and Picou, 2010]; [Ross, 2014]; [Sonic, 2014]; [Moodie, Rall, et al, 2016]). Although probe microphone measurements have been largely simplified over time and are continuing to be taught in educational settings around the world, there are still a large number of clinics that do not perform these measurements. Varying reasons for this lack of measurements include the complexity of modern hearing aids and the fitting software included with them, cost of the equipment, and poor training programs ([Mueller, 2005]; [Mueller and Picou, 2010]; [Mueller, 2014]). Although training programs have improved in recent years and there is importance placed on the fitting and verification of hearing aids using evidence-based practice, there continues to be the need for additional hands-on experience within the educational setting to foster proficiency and confidence in the skill of probe tube placement.

[Moodie, Rall, et al (2016)] found that approximately 12% of pediatric audiologists who own real-ear measurement equipment do not perform real-ear measurements because of lack of confidence. This may be attributed to a gap between the real-ear verification procedures that are being taught and a proper outlet to practice these learned techniques. Presently, the method of practicing probe tube placement during clinical training is with classroom volunteers/classmates. One disadvantage of this approach is that students and instructors lack an objective method to quantify the distance from the probe tip to the TM to determine if the probe tube is placed appropriately for the most comfortable and accurate real-ear measurements. This means the student may receive informal subjective feedback from nonexpert peers during practice or instructor feedback during scheduled laboratory sessions. However, instructor feedback is not available during practice sessions outside of scheduled instructional time. If students are practicing independently, they are unable to receive any expert feedback on whether they are placing the probe within the distance required for accurate measurements. In addition, students lack preclinical exposure to various anatomies (i.e., pediatric-sized ears) not found in a classroom with their peers.

A common method to provide additional preclinical training and to evaluate if a student is prepared for clinical practice is with the use of training simulators. Simulation has been shown to provide an opportunity to all students to obtain additional nonpatient experience before entering clinical practice ([Issenberg et al, 1999]; [Kunkler, 2006]; [Brown, 2017]). Providing students with a probe tube placement simulator could reduce the amount of teaching time required by the instructor and improve student competency in probe tube placement before entering their clinical practicum. Simulation could also be used for an evaluation method to ensure that probe tube placement is completed properly and students are achieving the expected competency standard before working with patients.

A training simulator was developed at Western University to help train students with probe tube placement, provide objective feedback to help them improve their skills with the placement, and improve their confidence in accurately making the measurement before starting their clinical practicum experience. This simulator allows instructors to obtain quantitative feedback on student performance to help determine if students are ready to perform real-ear measurements in the clinical practice setting. Validation of this simulator is required before using it in a clinical or educational setting. Face and content validity is typically an initial step in validation ([Gallagher et al, 2003]). Face validity refers to an assessment of the realism of the simulation compared with the real situation ([Schout et al, 2010]; [Huang et al, 2015]) whereas content validity refers to evaluating whether the simulator is useful in training ([Gallagher et al, 2003]; [Huang et al, 2015]). These two forms of validation provide valuable feedback on the simulator and highlight specific areas that may need to be improved before more rigorous validation.

The objective of this study is to evaluate the face and content validity of the probe tube placement simulator, as well as the barriers/facilitators to implementation of the simulator in educational settings.

MATERIALS AND METHODS

Simulator

The training simulator consists of a 3D-printed ear combined with an optical system for tracking the probe microphone and for measuring its location relative to the TM; both the printed ear model and the tracking system are integrated into a styrofoam head model as shown in [Figure 1]. The ear model was developed directly from X-ray computed tomography (CT) scans to mimic realistic anatomy. The outer ear was printed using a Stratasys Objet 500 Connex3 3D printer with TangoPlus FLX 930 material (Stratasys Ltd., Eden Prairie, MN) at a shore value (hardness) of 27A. To allow the optical tracking system (see below) to view the probe clearly, the ear canal portion of the ear was printed in a transparent material (VeroClear-RGD810) and coated in a latex paint to further increase transparency.

Figure 1 Simulator hardware. (A) Adult simulator for training in probe tube placement, (B) optical tracking system of the ear canal inside the adult simulator, and (C) pediatric simulator.

The optical tracking system of the simulator consists of a Microsoft LifeCam HD-3000 camera mounted inside the styrofoam head, with the styrofoam ear being replaced with the 3D-printed ear. The camera was interfaced with a custom MATLAB (Mathworks, Natick, MA) program, providing a graphical user interface and using the image processing toolbox to analyze the position of the probe microphone inside the canal. The program splits the usage of the simulator into two modes of operation: practice mode and test mode. The practice mode provides a live camera feed of the ear canal and displays real-time probe-to-TM distance, allowing the user to see exactly how the probe’s position inside the canal is changing at any time during the insertion. The test mode does not provide real-time feedback, and instead, requires the user to press “start” before the probe is inserted and press “finish” once the probe is in its final position. The users’ probe-to-TM distance, time to completion, and a final image of the probe inside the canal are displayed after the user has finished. In test mode, live tracking of the probe’s position is still occurring but with the results hidden. If at any point the TM is contacted with the probe, a warning sound is outputted to alert the trainee. The program was exported into a stand-alone program, which the participant can launch without the need for a MATLAB installation.

Two ear models were created, resulting in two different simulators; one to represent an adult ear and another for a pediatric ear. The canal length of the adult model was 32 mm whereas the pediatric ear canal length was 15 mm, as defined by the CT scans. These two anatomies were used to provide a range of variation that may exist between individuals in a clinic. Both simulators are comprised of the same internal components and used the same user interface.

Participants

This study was approved by the Western University Health Research Ethics Board (REB 109083). Participants were recruited through the National Centre for Audiology at Western University and comprised of 12 clinicians and researchers, with probe tube placement experience from 3 to 37 years. Ten of 12 of these participants are/were employed as course instructors or clinical supervisors in an audiology training program and were at some point responsible for the teaching of novice audiologists in courses, whereas the remainder have had experience teaching novice clinicians in roles such as teaching assistants and/or external clinical practicum supervisors. In addition, all participants have clinical experience performing real-ear measures for the purposes of hearing aid fitting.

Protocol

The structure of this study consisted of three sections: (1) operation and evaluation of the adult simulator; (2) operation and evaluation of the pediatric simulator; and (3) content validity, applicability to an educational setting, and barriers and facilitators to use. For all evaluations, a “think-aloud” approach was used ([Boren and Ramey, 2000]; [Johnson et al, 2005]; [Brooke et al, 2012]) in which participants were audio- and videotaped to capture their physical use and thought processes while using the simulators. The study also included participants completing a questionnaire aimed to provide quantitative evaluations of both simulators (see the following paragraphs). Sections (1) and (2) were identical, with the exception of the use of the different models: adult and pediatric.

During sections (1) and (2), the participant was given setup and operating instructions for the specific simulator they were using. The participant would follow the instructions, which guided them through each feature and aspect of the simulator, including assessing the realism of the 3D-printed ear, otoscopic usage with the simulator, probe insertion in both practice and test modes, interpretation of results after/during insertion in both practice and test modes, and foam tip insertion. Once the participant was comfortable with the simulator and had completed the setup and operating instructions, they were presented with the questionnaire to fill out based on their experience with the simulator they had just used. They were first asked to complete the face validity section of the questionnaire to assess the realism of specific aspects of the simulator. Once the face validity questions were completed, the participant was required to perform five consecutive probe tube placements in which the final probe-to-TM distance was recorded. Any contacts with the TM were recorded. Finally, once it was performed on both simulators, the remainder of the questionnaire was completed to evaluate the total content validity, applicability to an educational setting of both simulators, and to find specific facilitators and barriers to their implementation into a clinical education setting.

Questionnaire

A questionnaire was developed for this study that aimed to assess the following relative to the two simulator models: face validity, content validity, applicability to an educational setting, and barriers and facilitators to implementation in clinical education settings. Questions for face validity are summarized as items in [Table 1] whereas questions for content validity, applicability, and facilitators/barriers are summarized in [Table 2]. Questionnaire data were collected during a visit session via SurveyMonkey™.

Table 1 Summary of Face Validity Questions Used to Assess the Realism of the Adult and Pediatric Models

No.	Sections 1 and 2: Face Validity
1	Appearance of the ear
2	Shape of the ear
3	Texture of the ear
4	Stiffness of the ear
5	Otoscopic view of the ear
6	Length of simulator ear canal
7	Presence of relevant anatomical features
8	Proportionality of the ear to the head
9	Sturdiness of the head
10	Adjustability of the head
11	Ability to set up probe microphone equipment on simulator
12	Ability to properly position the probe microphone lanyard
13	Foam tip insertion experience
14	Time required to perform insertion
15	Total probe placement experience

Table 2 Summary of Questions on Content Validity (3A), Recommendations on the Applicability to an Educational Setting (3B), and Facilitators and Barriers to Implementation

No.	Section 3A: Content Validity
1	Educate student on anatomical landmarks
2	Educate student on otoscopic usage
3	Provides high-quality opportunity to practice probe placement
4	Provides high-quality probe placement evaluation method
5	Assists students in identifying their skill level
6	Assists instructor in identifying a student’s skill level
7	Simulator is not too time consuming in an educational setting
8	Simulator is not too difficult to use in an educational setting
9	Simulator can be used to train students on all aspects of probe placement
	Section 3B: Applicability to an Educational Setting
10	This simulator provides a more valid approach for teaching probe placement
11	The simulator should be implemented within clinical education programs
12	The simulator should be implemented in professional development programs
13	There will be widespread acceptance of this simulator in clinical education programs
	Section 3C: Facilitators and Barriers
14	List the top three facilitators to implementing this simulator in an educational setting
15	List the top three barriers to implementing this simulator in an educational setting

The questionnaire used a scale from 0% to 100% in intervals of 10%, with 0% indicating a strong disagreement, 50% indicating neither an agreement nor disagreement, and 100% showing a strong agreement. Participants were also provided an opportunity to provide written feedback for each item. The questionnaire was developed in conjunction with several audiologists from the National Centre for Audiology to ensure all aspects of the simulator and the procedure were properly assessed.

Statistical Analysis

The average and standard deviation (SD) of each questionnaire section was found, as well as the average probe-to-TM distance and SD resulting from the repeated placements for each simulator. The content of the open-ended responses was examined to see how they could be used to refine, revise the models, and to develop a better understanding of the barriers/facilitators to using the simulators in educational settings. The facilitators and barriers provided by the participants were ranked in terms of the most mentioned topics and the top seven facilitators and barriers are presented.

RESULTS

Face Validity (Sections 1 and 2)

Average participant rating for the realism of the adult model was 65% (SD = 18.2) whereas the average rating of the child model was 64% (SD = 16.4). Ratings per question are shown in [Figure 2]. Four questions out of 12 were given a negative rating (<50%) for both simulators. These questions pertained to the evaluation of the texture of the ear, stiffness of the ear, otoscopic view of ear “landmarks” and/or the TM, and the foam tip insertion experience.

Figure 2 Box plot of face validity results for the adult and pediatric simulator, corresponding to the questionnaire represented by [Table 1]. White represents results from the adult model whereas gray represents results from the pediatric simulator.

Distance-to-TM Results (Sections 1 and 2)

The average distance-to-TM for all participants combined was 3.74 mm (SD = 1.82) for the adult model with TM contact 12% of the time across all trials. With the child model, participants achieved an average distance-to-TM of 2.77 mm (SD = 0.94) with TM contact 5% of the time.

Content Validity and Applicability to Educational Settings (Section 3)

The content validity (section 3A, i.e., questions 1–9 of [Table 2]) was intended for evaluating the teaching value of this simulator and had an average score of 78.7% (SD = 17.0) with only one question producing a negative response. The applicability to an educational setting (section 3B, i.e., questions 10–13 of [Table 2]) had an average score of 80.0% (SD = 5.33) with no negative responses being reported. All the above data are seen in [Figure 3], and all recorded facilitators and barriers can be seen in [Table 3].

Figure 3 Box plot of content validity results for both simulators, corresponding to the questions asked in [Table 2].

Table 3 Facilitators and Barriers in Implementing This Simulator in an Educational Setting

Rank	Facilitators	Barriers		Solution
1	Ability to learn and practice in a controlled, low-stress, safe environment	Stiffness and texture	→	Future ears made of silicon will increase flexibility and increase skin texture realism
2	Visual and auditory feedback received after insertion of probe	Lack of anatomical landmarks and difficulty visualizing canal	→	Allow usage of otoscope light and optimize otoscopic image by discriminating the common landmarks
3	Ability to obtain accurate results of probe-to-eardrum distance measurement	Potential high cost	→	Materials used have low cost and simulator will be an inexpensive teaching tool
4	Simulator features and ease of use	Some aspects not realistic (movement, shoulders)	→	Future iterations will look at adding partial shoulders to the model and possible actuation of the base of the simulator
5	Realistic aspects of simulator	Not acoustically accurate	→	Future iterations will test real-ear measures at each stage of development to confirm proper acoustical measurements
6	Provides ear anatomies which would not have previously been possible to practice on	Suboptimal user interface	→	Major usability issues have been found and a new user interface will look to fix these issues
7	Alternate uses	Lack of multiple anatomies	→	More simulator options will be later available with more potential patient anatomies (2+ adult anatomies and 2+ child anatomies)

Note: Rank 1 showing the number one facilitator and the number one barrier in implementing this simulator into an educational setting, as suggested by participants.

DISCUSSION

Simulator systems are typically evaluated for face and content validity at an early product development stage, to glean systematic feedback before developing final versions for use in educational programs ([Gallagher et al, 2003]; [Wolfe, 2013]). This study completed a first-level face and content validity evaluation of a prototype-simulated patient designed to assist in teaching probe tube placement before real-ear measurement, by allowing experienced clinical instructors of audiology coursework to use the simulator and provide structured feedback. For the first level of evaluation for this simulator, results from this study were generally positive, with most of the questions receiving >50% rating. By using experts in probe tube placement, we obtained feedback from audiologists who have had extensive experience performing clinical probe tube placements and in most cases, have taught students and other professionals how to properly place a probe tube within both adult and child-sized ears. These participants are aware of the barriers that exist in learning probe tube placement techniques and are representative of professionals who may make use of a product such as this in professional situations such as course or laboratory instruction, or in clinical practice.

The face validity average of 65% (SD = 18.2) for the adult simulator and 64% (SD = 16.4) were reasonable considering the questions that lowered this score. The lowest scores reported for both models were attributed to the texture and stiffness of the 3D-printed ear, foam tip insertion experience, and the otoscopic view of the ear canal and TM. The lack of realism in the texture and stiffness of the ear causing an unsatisfactory foam tip insertion may be due to the method of creating the ear. By using a multimaterial printer to print the ear, the shore value (hardness) was customized according to material properties of human auricular cartilage found in the literature. With Young’s modulus (i.e., intrinsic stiffness) values ranging from 0.8 to 8 MPa ([Nimeskern et al, 2015]; [Griffin et al, 2016]; [Bos et al, 2017]), a lower value of 1 MPa was chosen for this model and converted to an approximate shore value of 27A, using Gent’s relationships ([Gent, 1958]). This value of 27A is presently the softest material available to be 3D printed, meaning any future improvements of the stiffness and texture of the ear will require a different method, such as silicon molds, to create the ear. Additional material properties will also need to be considered, as flexibility was a large issue, with clinicians unsuccessfully attempting to open the canal by pulling the posterior part of the pinna up and back. Using materials such as silicon will likely increase the flexibility of the ear and result in more positive ratings by clinicians. The unrealistic otoscopic view may be due to how the simulator was optimized to improve the accuracy of the probe-to-TM distance measurement. For the camera to best locate the probe inside the canal, the ear canal was made as transparent as possible and an internal light was situated behind the camera. As both of these design decisions seemed to degrade the otoscopic image of the ear canal and TM, further optimization will be needed to improve the otoscopic image while not negatively impacting the camera’s view of the canal. In addition, the TM will be further discriminated from the ear canal to ensure the user can visualize the ear canal using the TM as a consistent landmark.

Quantitative results showed that the average probe tube-to-TM distance was within the 5-mm guideline with only one participant having an average of their five placements greater than the recommended 5 mm. It was observed that participants primarily in research and participants who had the least clinical experience achieved a closer placement of the probe tip to the TM whereas those who primarily work in a clinic achieved farther distances from the TM (perhaps to avoid any accidental contact with the TM). As all participants were relative experts who routinely perform probe tube placements, it is reasonable to expect that all results are within proper distances from the TM. On a person-to-person basis, most participants were consistent with their own five placements, showing that their technique for placement is repeatable and well-practiced, as may not be the case with students.

Individual question scores for content validity and applicability to an educational setting (sections 3A and 3B, respectively) were both generally high whereas receiving low scores for the same topics that received low scores for face validity (texture, stiffness, and otoscopic view of the ear). In addition, two neutral responses were given regarding (a) no presence of anatomical landmarks and (b) otoscope usage. These two aspects of probe placement are difficult with the simulator in its current form as the landmarks are not easily distinguishable, and the lack of flexibility in the pinna and ear canal makes clinical otoscopy difficult. The improvements mentioned previously (producing silicon ears with improved shore values and more distinct landmarks) will help address these issues.

The facilitators and barriers (as seen in [Table 3]) provided by the participants outline the strengths and weaknesses of the current iteration of the simulator. The top three major barriers to successfully implementing this in an educational setting include the current texture and stiffness of the ear, lack of landmarks, and the potential cost for educational institutions to implement this into their program. To address the third barrier, as seen in [Figure 1], the materials used in this simulator are relatively inexpensive, with the 3D-printed ear being the most expensive part. As future iterations of the ear may include silicon molds instead of using 3D printing, this cost will decrease further, making this an extremely affordable simulator, capable of being purchased by most institutions. Other barriers listed include the lack of realism, inability to record accurate acoustical measurements of the ear, suboptimal user interface, and only having two anatomies to practice. These issues will be addressed with future iterations of the design.

The facilitators listed ([Table 3]) demonstrate the uses of this simulator and the benefits it may have in an educational setting. Although improvements are needed, the simulator allows students to practice probe tube placement in a controlled, low-stress environment while receiving visual, auditory, and quantitative feedback to help them progress their skill level before advancing to clinic. In addition, a well-built, relatively inexpensive head simulator with a variety of realistic ears developed from CT scans may also be used for other applications. Opportunities to use the simulator to assess a clinician’s skill, to know the exact distance from the TM for research purposes, and to test real-ear measurement systems using this probe-to-TM distance are a few examples. Within an educational setting, and with specific additional features, a simulator such as this could be used for practicing the fitting an adult ear with a receiver-in-the-ear or slim tubes of the correct size, cutting earmold tubing to size for correct positioning of a behind-the-ear aid, setting up for complex real-ear measurement setups such as contralateral routing of signal (CROS and BICROS) or open fittings and, using monitoring headphones. Once improved, this model could be used to improve skills such as otoscope usage and basic understanding of the anatomy and variation that may exist between individuals.

CONCLUSION

The probe tube placement simulator is a novel tool for instructors and students to gain experience in probe tube placement before entering clinical practice. The results of the face and content validity are encouraging for this simulator and show a clear set of characteristics of the simulator which must be improved before any widespread use. With the participants’ final opinion that this would be recommended for use in clinical education programs, these pressing issues will be explored, and future iterations will be tested with experts and students to ensure its success in an educational setting.

Abbreviations

CT: computed tomography

SD: standard deviation

TM: tympanic membrane

No conflict of interest has been declared by the author(s).

Funding for this project was provided through an Ontario Graduate Scholarship (2016/2017) and through the Natural Sciences and Engineering Research Council of Canada.

• REFERENCES

• Bagatto MP, Seewald RC, Scollie SD, Tharpe AM. 2006; Evaluation of a probe-tube insertion technique for measuring the real-ear-to-coupler difference (RECD) in young infants. J Am Acad Audiol 17 (08) 573-581
• Boren MT, Ramey J. 2000; Thinking aloud: reconciling theory and practice. IEEE Trans Prof Commun 43 (03) 261-278
• Bos EJ, van der Laan K, Helder MN, Mullender MG, Iannuzzi D, van Zuijlen PP. 2017; Noninvasive measurement of ear cartilage elasticity biomechanical information for tissue engineering. Plast Reconstr Surg Glob Open 5: 1-6
• British Society of Audiology (BSA). (2007) Guidance on the use of real ear measurement to verify the fitting of digital signal processing hearing aids. Br Soc Audiol 1–20.
• Brooke RE, Isherwood S, Herbert NC, Raynor DK, Knapp P. 2012; Hearing aid instruction booklets: employing usability testing to determine effectiveness. Am J Audiol 21 (02) 206-214
• Brown DK. 2017; Simulation before clinical practice: the educational advantages. Audiol Today 29 (05) 16-24
• College of Audiologists and Speech-Language Pathologists of Ontario (CASLPO). (2016) Practice standards for the provision of hearing aid services by audiologists. Toronto, ON: CASLPO. http://www.caslpo.com/sites/default/uploads/files/PS_EN_Practice_Standards_for_the_Provision_of_Hearing_Aid_Services_By_Audiologists.pdf
• Dirks DD, Kincaid GE. 1987; Basic acoustic considerations of ear canal probe measurements. Ear Hear 8 (05) (Suppl) 60S-67S
• Gallagher AG, Ritter EM, Satava RM. 2003; Fundamental principles of validation, and reliability: rigorous science for the assessment of surgical education and training. Surg Endosc 17 (10) 1525-1529
• Gent AN. 1958; On the relation between indentation hardness and Young’s modulus. Trans Inst Rubber Ind 34: 46-57
• Griffin MF, Premakumar Y, Seifalian AM, Szarko M, Butler PEM. 2016; Biomechanical characterisation of the human auricular cartilages; implications for tissue engineering. Ann Biomed Eng 44 (12) 3460-3467
• Huang C, Cheng H, Bureau Y, Agrawal SK, Ladak HM. 2015; Face and content validity of a virtual-reality simulator for myringotomy with tube placement. J Otolaryngol Head Neck Surg 44 (01) 1-8
• International Standards Organization (ISO). (2001) Acoustics–procedures for the measurement of real-ear acoustical characteristics of hearing aids (ISO 12124:2001).
• Issenberg SB, McGaghie WC, Hart IR, Mayer JW, Felner JM, Petrusa ER, Waugh RA, Brown DD, Safford RR, Gessner IH, Gordon DL, Ewy GA. 1999; Simulation technology for health care professional skills training and assessment. JAMA 282 (09) 861-866
• Johnson CM, Johnson TR, Zhang J. 2005; A user-centered framework for redesigning health care interfaces. J Biomed Inform 38 (01) 75-87
• Kunkler K. 2006; The role of medical simulation: an overview. Int J Med Robot 2 (03) 203-210
• Moodie S, Pietrobon J, Rall E, Lindley G, Eiten L, Gordey D, Davidson L, Moodie KS, Bagatto M, Haluschak MM, Folkeard P, Scollie S. 2016; Using the real-ear-to-coupler difference within the American Academy of Audiology Pediatric Amplification Guideline: protocols for applying and predicting earmold RECDs. J Am Acad Audiol 27 (03) 264-275
• Moodie S, Rall E, Eiten L, Lindley G, Gordey D, Davidson L, Bagatto M, Scollie S. 2016; Pediatric audiology in North America: current clinical practice and how it relates to the American Academy of Audiology Pediatric Amplification Guideline. J Am Acad Audiol 27 (03) 166-187
• Mueller HG. 2005; Probe-mic measures: hearing aid fitting’s most neglected element. Hear J 58 (10) 21-22 24, 26, 28, 30
• Mueller HG. 2014 20Q: Real-Ear Probe-Microphone Measures—30 Years of Progress?. AudiologyOnline. http://www.audiologyonline.com/articles/20q-probe-mic-measures-12410 . Accessed November 17, 2016
• Mueller HG, Picou E. 2010; Survey examines popularity of real-ear probe-microphone measures. Hear J 63 (05) 27-32
• Nimeskern L, Pleumeekers MM, Pawson DJ, Koevoet WL, Lehtoviita I, Soyka MB, Röösli C, Holzmann D, van Osch GJ, Müller R, Stok KS. 2015; Mechanical and biochemical mapping of human auricular cartilage for reliable assessment of tissue-engineered constructs. J Biomech 48 (10) 1721-1729
• Ross T. (2014) Fear and Loathing of Verification. MedRx website. http://medrx-usa.com/Website/pages/medrx-FearLoathing-of-Verification.html . Accessed November 17, 2016.
• Schout BMA, Hendrikx AJM, Scheele F, Bemelmans BLH, Scherpbier AJJA. 2010; Validation and implementation of surgical simulators: a critical review of present, past, and future. Surg Endosc 24 (03) 536-546
• Sonic. (2014) Real Ear Fit: Combining Real Ear Verification with Best Fit Fast. http://www.sonici.com/~/media/Files/Guides/Spotlights%20and%20Tech%20Papers/UK/Spotlight_Real%20Ear%20Fit_UK_6_14.ashx . Accessed April 10, 2017.
• Vaisberg JM, Macpherson EA, Scollie SD. 2016; Extended bandwidth real-ear measurement accuracy and repeatability to 10 kHz. Int J Audiol 55 (10) 580-586
• Valente, M. (2006) Guideline for audiologic management of the adult patient. Audiology Online. https://www.audiologyonline.com/articles/guideline-for-audiologic-management-adult-966 . Accessed November 10, 2016.
• Wolfe BE. 2013; The value of pilot studies in clinical research: a clinical translation of the research article titled “In search of an adult attachment stress provocation to measure effect on the oxytocin system”. J Am Psychiatr Nurses Assoc 19 (04) 192-194