Graphical Access to Medical Expert Systems: V. Integration with Continuous-Speech Recognition

Abstract:

This paper describes three prototypes of computer-based clinical record-keeping tools that use a combination of window-based graphics and continuous speech in their user interfaces. Although many of today’s commercial speech-recognition products achieve high rates of accuracy for large grammars (vocabularies of words or collections of sentences and phrases), they can only “listen for” (and therefore recognize) a limited number of words or phrases at a time. When a speech application requires a grammar whose size exceeds a speech-recognition product’s limits, the application designer must partition the large grammar into several smaller ones and develop control mechanisms that permit users to select the grammar that contains the words or phrases they wish to utter. Furthermore, the user interfaces they design must provide feedback mechanisms that show users the scope of the selected grammars. The three prototypes described were designed to explore the use of window-based graphics as control and feedback mechanisms for continuous-speech recognition in medical applications. Our experiments indicate that window-based graphics can be effectively used to provide control and feedback for certain classes of speech applications, but they suggest that the techniques we describe will not suffice for applications whose grammars are very complex.

Key-Words

Continuous-Speech Recognition - Multimodal Human-Computer Interfaces - Sublanguages - Oncology

• REFERENCES

• 1 Tsuji S, Shortliffe EH. Graphical access to a medical expert system: I. Design of a knowledge engineer’s interface. Meth Inf Med 1986; 25: 62-70.
• 2 Lane CD, Walton JD, Shortliffe EH. Graphical access to medical expert systems: II. Design of an interface for physicians. Meth Inf Med 1986; 25: 143-50.
• 3 Walton JD, Musen MA, Combs D. et al. Graphical access to medical expert systems: III. Design of a knowledge-acquisition environment. Meth Inf Med 1987; 26: 78-88.
• 4 Isaacs E, Wulfman C, Rohn J. et al. Graphical access to medical expert systems: IV. Experiments for determining the role of spoken input. Meth Inf Med 1993; 32: 18-32.
• 5 Shortliffe EH. Computer programs to support clinical decision making. JAMA 1987; 258: 61-6.
• 6 Bergeron B, Locke S. Speech recognition as a user interface. M. D. Computing 1990; 07: 329-34.
• 7 Levinson S, Roe D. A perspective on speech recognition. IEEE Communications Magazine January, 1990; 28-34.
• 8 Peacocke RD, Graf DH. An introduction to speech and speaker recognition. IEEE Computer 1990; 23: 26-33.
• 9 Robbins AH, Horowitz DM, Srinivasan M. Speech controlled radiology report generation. Radiology 1987; 164: 569-73.
• 10 Robbins AH, Vincent ME, Shaffer K. et al. Radiology reports: assessment of a 5,000-word speech recognizer. Radiology 1988; 167: 853-5.
• 11 Swett HA, Fisher P, Pradeep M. et al. tThe IMAGE/ICON system: voice activated intelligent image display for radiologic diagnosis. In: Proceedings of the Thirteenth Annual Symposium on Computer Applications in Medical Care. Kingsland, ed. Washington DC: IEEE Computer Society Press; 1989: 977-8.
• 12 Shortliffe EH, Scott AC, Bischoff MB. et al. ONCOCIN: an expert system for oncology protocol management. In: Proceedings of the Seventh International Joint Conference on Artificial Intelligence . Vancouver BC: 1981: 876-81.
• 13 Bischoff MB, Shortliffe EH, Scott AC. et al. Integration of a computer-based consultant into the clinical setting. In: Proceedings of the Seventh Annual Symposium on Computer Applications in Medical Care . Baltimore, MD: 1983: 149-52.
• 14 Tu SW, Kahn MG, Musen MA. et al. Episodic monitoring of time-oriented data for heuristic skeletal-plan refinement. Commun ACM 1989; 32: 1439-55.
• 15 Meisel WS, Anikst MT. Efficient representation of speech for recognition. Speech Technology February/March, 1991; 96-100.
• 16 Lane CD. PDISERVE (Internal Memorandum). Stanford: Section on Medical Informatics, Stanford University; 1988
• 17 Lane CD. SCOMPILE (Internal Memorandum). Stanford: Section on Medical Informatics, Stanford University; 1988
• 18 Lane C. The Ozone O₃ Reference Manual (Technical Report). Stanford: Knowledge Systems Laboratory, Stanford University; 1986
• 19 Burton RR, Brown JS. Toward a natural-language capability for computer-assisted instruction. In: O’Neil H. ed. Procedure for Instructional Systems Development . New York: Academic Press; 1979: 273-313.
• 20 Sager N. Sublanguage: Linguistic phenomenon, computational tool. In: Grishman R, Kittredge R. eds. Analyzing Language in Restricted Domains: Sublanguage Description and Processing . London: Lawrence Erlbaum Associates; 1986: 1-17.
• 21 Bates B. A Guide to Physical Examination . Philadelphia: Lippincott; 1987
• 22 Grosz BJ, Jones KS, Webber BL. Semantic interpretation. In: Grosz BJ, Jones KS, Webber BL. eds. Readings in Natural Language Processing . Los Altos: Morgan Kaufmann; 1986: 161-6.
• 23 Shiftman S, Wu AW, Poon AD. et al. Building a speech interface to a medical diagnostic program. IEEE Expert. February, 1991: 41-50.
• 24 Leitner G. The Voice Navigator™ User’s Manual . Cambridge: Henry Nigro; 1989
• 25 Schmandt C, Ackerman MS, Hindus D. Augmenting a window system with speech input. IEEE Computer 1990; 23: 50-6.
• 26 Shiftman S, Lane C, Johnson K. et al. The integration of a continuous speech-recognition system with the QMR Diagnostic Program. In: Frisse M. ed. Proceedings of the Sixteenth Annual Symposium on Computer Applications in Medical Care. New York: McGraw Hill; 1992: 767-71.
• 27 Johnson K, Poon A, Shiftman S. et al. A history-taking system that uses continuous speech recognition. In: Frisse M. ed. Proceedings of the Sixteenth Annual Symposium on Computer Applications in Medical Care . New York: McGraw Hill; 1992: 757-61.
• 28 Poon A, Johnson K, Fagan L. Augmented transition networks as a representation for knowledge-based history-taking systems. In: Frisse M. ed. Proceedings of the Sixteenth Annual Symposium on Computer Applications in Medical Care . New York: McGraw Hill; 1992: 762-6.
• 29 Andreshak JC, Lumelsky S, Chang IF. et al. Medication charting via computer gesture recognition (Research Report). T. J. Watson Research Division, Yorktown Heights, NY, IBM Research Division 1990
• 30 Brocklehurst ER. The NPL electronic paper project. Int J Man – Machine Studies 1991; 34: 69-95.