Modeling End-users’ Acceptance of a Knowledge Authoring Tool

Summary

Objectives: Knowledge bases comprise a vital component in the classic medical expert system model, yet the knowledge acquisition process by which they are created has been characterized as highly iterative and labor-intensive. The difficulty of this process underscores the importance of knowledge authoring tools that satisfy the demands of end-users. The authors hypothesize that the acceptability of a knowledge authoring tool for the creation of medical knowledge base content can be predicted by an accepted model in the information technology (IT) field, specifically the Technology Acceptance Model (TAM).

Methods: An online survey was conducted amongst knowledge base authors who had previously established experience with the authoring tool software. The Likert-based questions in the survey were patterned directly after accepted TAM constructs with minor modifications to particularize them to the software being used. The results were analyzed using structural equation modeling.

Results: The TAM performed well in predicting end-users’ behavioral intentions to use the knowledge authoring tool. Five out of seven goodness-of-fit statistics indicate that the model represents the behavioral intentions of the authors well. All but one of the hypothesized relationships specified by the TAM were significant with p values less than 0.05.

Conclusions: The TAM provides an adequate means by which development teams can anticipate and better understand what aspects of a knowledge authoring tool are most important to their target audience. Further research involving other behavioral models and an expanded user base will be necessary to better understand the scope of issues that factor into acceptability.

Keywords

Knowledge acquisition - knowledge bases - software acceptance - clinical information systems - Technology Acceptance Model

• References

• 1 Shortliffe EH. et al An artificial intelligence program to advise physicians regarding antimicrobial therapy. Comput Biomed Res 1973; 6 (06) 544-60.
• 2 Barnett GO. et al DXplain. An evolving diagnostic decision-support system. Jama 1987; 258 (01) 67-74.
• 3 Miller R, Masarie FE, Myers JD. Quick medical reference (QMR) for diagnostic assistance. MD Comput 1986; 3 (05) 34-48.
• 4 Warner Jr HR. Iliad: moving medical decisionmaking into new frontiers. Methods Inf Med 1989; 28 (04) 370-2.
• 5 Berner ES. et al Performance of Four Computer- Based Diagnostic Systems. N Engl J Med 1994; 330 (025) 1792-6.
• 6 Berner E, Jackson J, Algina J. Relationships among performance scores of four diagnostic decision support systems. J Am Med Inform Assoc 1996; 3 (03) 208-15.
• 7 Perry CA. Knowledge bases in medicine: A review. Bulletin of the Medical Library Association 1990; 78 (03) 271-82.
• 8 Smirnov A. et al Knowledge logistics as an intelligent service for healthcare. Methods Inf Med 2005; 44 (02) 262-4.
• 9 Degoulet P, Fieschi M, Chatellier G. Decision support systems from the standpoint of knowledge representation. Methods Inf Med 1995; 34 1-2. 202-8.
• 10 Barreiro A. et al A modular knowledge base for the follow-up of clinical protocols. Methods Inf Med 1993; 32 (05) 373-81.
• 11 Giuse DA, Giuse NB, Miller RA. Evaluation of long-term maintenance of a large medical knowledge base. J Am Med Inform Assoc 1995; 2 (05) 297-306.
• 12 Compton P. et al Maintaining an expert system. In Quinlan JR. (ed.) Applications of Expert Systems. Turing Institute Press in association with Addison-Wesley 1989; pp 366-84.
• 13 Warner HR, Sorenson DK, Bouhaddou O. Knowledge engineering in health informatics. NewYork: Springer; 1997
• 14 Lincoln M. Medical Education Applications. In Berner E. (ed.) Clinical Decision Support Systems.. New York: Springer-Verlag; 1999. p 110
• 15 Yu VL. Conceptual obstacles in computerized medical diagnosis. J Med Philos 1983; 8 (01) 67-75.
• 16 Miller R, Geissbuhler A.. Clinical Diagnostic Decision Support Systems - An Overview. In Berner E.. Clinical Decision Support Systems.. New York: Springer-Verlag; 1999. pp 19-21.
• 17 Balas EA, Boran SA. Managing clinical knowledge for healthcare improvement. In: Yearbook of Medical Informatics.. Bethesda, MD: NLM; 2000. pp 65-70.
• 18 Johansson B. et al Database and knowledge base integration - a data mapping method for Arden Syntax knowledge modules. Methods Inf Med 1996; 35 4-5. 302-8.
• 19 Spreckelsen C, Spitzer K. Formalising and acquiring model-based hypertext in medicine: an integrative approach. Methods Inf Med 1998; 37 (03) 239-46.
• 20 Alberdi E, Taylor P, Lee R. Elicitation and representation of expert knowledge for computer aided diagnosis in mammography. Methods Inf Med 2004; 43 (03) 239-46.
• 21 Grutter R, Fierz W. An electronic study form to support collaborating agents in the management of clinical knowledge. Methods Inf Med 1999; 38 (03) 154-7.
• 22 Giuse DA, Giuse NB, Miller RA. Consistency enforcement in medical knowledge base construction. Artif Intell Med 1993; 5 (03) 245-52.
• 23 Giuse DA, Giuse NB, Miller RA. A tool for the computer-assisted creation of QMR medical knowledge base disease profiles. Proc Annu Symp Comput Appl Med Care 1991; pp 978-9.
• 24 Walton JD. et al Graphical access to medical expert systems: III. Design of a knowledge acquisition environment. Methods Inf Med 1987; 26 (03) 78-88.
• 25 Jenders RA, Dasgupta B.. Assessment of a knowledge- acquisition tool for writing Medical Logic Modules in the Arden Syntax. Proc AMIA Annu Fall Symp 1996; pp 567-71.
• 26 Shiffman RN. et al Bridging the guideline implementation gap: a systematic, document-centered approach to guideline implementation. J Am Med Inform Assoc 2004; 11 (05) 418-26.
• 27 Musen MF, RW Grosso WE, Noy NF, Grubezy M, Gennari JH. Component-based support for building knowledge acquisition systems. In: Conference on Intelligent Information Processing of the International Federation for Processing World Computer Congress. Beijing 2000
• 28 The Protégé Ontology Editor and Knowledge Acquisition System. Available at http://protege. stanford.edu/index.html AccessedAug 28 2004
• 29 Hulse NC. et al. Application of an XML-based Document Framework to Knowledge Content Authoring and Clinical Information System Development. Proc AMIA Symp 2003; p 870.
• 30 Intermountain Healthcare. Available at http://www.intermountainhealthcare.org Accessed May 1 2006
• 31 Intermountain Healthcare Clinical Programs Available at http://www.intermountainhealthcare.org/xp/ihc/physician/clinicalprograms Accessed May 1 2006
• 32 Hougaard J. Developing evidence-based interdisciplinary care standards and implications for improving patient safety. Int J Med Inf 2004; 73 7-8. 615-24.
• 33 Hulse NC. et al The Knowledge Authoring Tool: A Flexible XML-based Knowledge Authoring Environment. J Am Med Inform Assoc 2005; 12 (04) 418-30.
• 34 Davis FD. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly, 1989; 13 (03) 319-40.
• 35 Bagozzi RP, Davis FD, Warshaw PR. Development and Test of a Theory of Technological Learning and Usage. Human Relations 1992; 45 (07) 659-86.
• 36 Ajzen I, Fishbein M. Understanding attitudes and predicting social behaviour. Eaglewood Cliffs, NJ: Prentice-Hall 1980
• 37 Hu PJ. et al Examining the Technology Acceptance Model Using Physician Acceptance of Telemedicine Technology. Journal of Management of Information Systems 1999; 16 (02) 91-112.
• 38 Chismar WG, Wiley-Patton S.. Test of the technology acceptance model for the internet in pediatrics. Proc AMIA Symp 2002; pp 155-9.
• 39 Wilson EV, Lankton NK. Modeling patients’ acceptance of provider-delivered e-health. J Am Med Inform Assoc 2004; 11 (04) 241-8.
• 40 Hooff B v d. et al Knowledge Sharing in Knowledge Communities in Communities and Technologies. M. Huysman (ed.). The Netherlands: Kluwer Academic Publishers 2003; pp 119-41.
• 41 Adams DA, Nelson RR, Todd PA. Perceived Usefulness, Ease of Use, and Usage of Information Technology - a Replication. MIS Quarterly 1992; 16 (02) 227-47.
• 42 Heilman G, White D.. On General Application of the Technology Acceptance Model. Available at http://mcb.unco.edu/pdfs/WPS/TAM-2.doc Accessed Sep. 15 2004
• 43 Schumacker RE, Lomax RG. A Beginner’s Guide to Structural Equation Modeling. 2nd ed. Mahwah, NJ: Lawrence Erlbaum Associates Inc; 2004. p 498
• 44 SPSS Amos 5.0 Available at http://www.spss.com/amos/ Accessed Nov 15 2004
• 45 Nunally JC. Pyschometric Theory. 2nd ed. New York: McGraw-Hill; 1978
• 46 Hoyl RH. (ed.) Structural Equation Modeling: Concepts, Issues and Applications. Thousand Oaks, CA: Sage; 1995
• 47 Bollen KA. Overall Fit in Covariance Structural Models: Two Types of Sample Size Effects. Psychological Bulletin 1990; 107 (02) 256-9.
• 48 Hoelter J. The analysis of covariance structures: goodness-of-fit indices. Soc Methods & Research 1983; 11: 325-44.
• 49 Venkatesh V, Davis FD. A theoretical extension of the Technology Acceptance Model: Four longitudinal field studies. Management Science 2000; 46 (02) 186-204.