A Linguistic Approach to Relevance Judgment

A lexically-oriented approach to the judgment of relevance is proposed, and data derived from medical literature is presented to illustrate the technique. This approach to relevance judgments assumes that the user is guided in his judgments primarily by a similarity of words in the question and in the document. A sample of questions submitted by medical researchers and the output obtained in response to them were examined.

Für die Relevanzbeurteilung wird ein lexikalisch orientiertes Näherungsverfahren vorgeschlagen. Die Technik des Verfahrens wird an Material aus dem medizinischen Schrifttum erläutert. Das Verfahren geht von der Annahme aus, daß der Benutzer in seinem Urteil von der Gleichartigkeit der Wörter in seiner Anfrage und in dem ermittelten Dokument geleitet wird. Eine von medizinischen Wissenschaftlern eingesandte Anfragen-Stichprobe und die dazu erhaltenen Maschinenausdrucke werden näher untersucht.

^* Work performed under PHS Grant FR-00118.

• References

• 1 Chomskt N.. Aspects of the Theory of Syntax. The M.I.T. Press; Boston: 1965
• 2 Fairthorne R. A.. Similarity and Stability of Textual Interests. In: Information Retrieval in Action. Cleveland: Press of Western Reserve University; 1963. pp. 183-197.
• 3 Francis W. N.. Structure of American English. New York: Ronald Press; Inc.: 1958
• 4 Goffman W., and Newill V. A.. Methodology for Test and Evaluation of Information Retrieval Systems. Inform. Stor. Retr 3: 19-25 1966; [Also appears as Comparative Systems Laboratory Technical Report No. 2 (CSL:TR-2), Cleveland: Western Reserve University, July 1964, 19 p.].
• 5 Goldwyn A. J.. Purpose and Objectives of the Comparative Systems Laboratory. [Paper presented at the Congresso Internazionale Documentazione Informazione Scientifico Tecnica,; Rome, Italy: February 7, 1964: 21 p. Also appears as Comparative Systems Laboratory Technical Report No. 1 (CSL:TR-1), Cleveland: Western Reserve University, August 1964, 21 p.].
• 6 Harris Z.. The Elementary Transformations. University of Pennsylvania; TDAP No. 54 (pre-publication copy), 1964
• 7 Melton J. S.. Automatic Processing of Metallurgical Abstracts for the Purpose of Information Retrieval. (Interim Report NSF-3, Center for Documentation and Communication Research, School of Library Science, Western Reserve University; Cleveland, Ohio: July 1965. 2 Vols.).
• 8 Rees A. M.. Information Needs and Patterns of Usage. In: Information Retrieval in Action. Cleveland: Press of Western Reserve University; 1963. pp. 17-23.
• 9 Rees A. M., and Sahacevic T.. The Measurability of Relevance. [Progress in Information Science and Technology, Vol. 3, Proceedings of the Annual Meeting (1966) of the American Documentation Institute, October 3-7, Santa Monica, Calif: Adrianne Press; 1966. pp. 225-234 [Also appears as Comparative Systems Laboratory Technical Report No. 7 (CSL:TR-7), Cleveland: Western Reserve University, August 1966, 19 p.].
• 10 Resnick A., and Savage T. R.. The Consistency of Human Judgments of Relevance. Amer. Doc 25: 93-95 1964;
• 11 Vickery B. C.. On Retrieval Systems Theory. London: Butterworths; 1961
• 12 Zuix C. G., and Baumanis G. J.. Textual Correlates of Relevance: Preliminary Investigations. [Comparative Systems Laboratory Technical Report No. 9 (CSL:TR-9), Cleveland: Western Reserve University; June 1967: 47 p.].