Citation Rates of Medical German-Language Journals in English-Language papers - Do They Correlate With the Impact Factor, and Who Cites?

Table of Contents

Background and objective: Several publications are warning that the German language is no longer needed for transmission of scientific data. One of the causes may be the Impact Factor (IF), which appears to be derived predominantly from Anglo-American journals. The aim of this study was to check actual international attention paid to German- language journals, i. e. their citation frequencies in English-language papers. Are these citing rates in English-language articles correlated to the IF, and from where do citing articles originate?

Methods: Of 25 arbitrarily selected > 85 % German-language medical journals, IF as well as language distributions of citing articles were determined by searching publication years 1995-2000 in Science Citation Index (SCI). MEDLINE and EMBASE were used as supplementary retrieval systems.

Results: (i) The sample journals displayed an average IF = 0.357. A 99 % correlation (Pearson factor r = 0.987; n = 25) was observed between our ” constructed ” IF 2000 and IF published in Journal Citation Report 2000. This proves Stegmann’s IF determination method (31) to be valid. On the average, 53 % German-language and 45 % English-language articles between 1995-2000 cited the 1995-1999’ contributions of the studied journals. No correlation was observed between IF vs. rates of citing articles in English (r < 0.1). 64 % of citing English-language articles showed corporate sources in Germany/ Austria/ Switzerland, and 13.5 % authors’ institutions in USA.

Conclusions: (i) An IF 1 is, obviously, very hard to attain by German-language journals. ISI’s differentiation between Citing vs. Cited-only Journals (the latter often serving as MEDLINE/ EMBASE sources) during derivation of IF appears unjustified. (ii) English now serves as the predominant communication language in sciences in German-speaking countries, but has not supplanted the German language. Our study reveals remarkable international attention rates remaining.

Studies by Navarro ([21] - [23]) showed a decline in the German language since 1945 as the language of medicine in German-speaking countries. In Germany, Austria and German-speaking Switzerland, English became the dominant language of science in the early 1970 s. A parallel development was observed in France regarding the substitution of English for French ([24]). As early as the 1950 s, English had gained much terrain in the Netherlands ([26]), French-speaking Switzerland ([22]), Scandinavia ([5]) and Spain ([25]). Boettiger ([5]) found that in 1983, 74 % of the authors in the Deutsche Medizinische Wochenschrift made reference to English-language works. In representative Anglo-American journals, practically all the cited references were articles in English. However, this Swedish author found in a MEDLINE literature search, as a control, a verifiable share of only 54 % of English-language articles regarding the corresponding article subjects.

Both scientists and editors of medical journals are equally concerned with this development. Dietrich ([7]) found that authors focusing on the reputation and international dissemination of journals lean toward English as a language of publication. This was also true of publishers who enter international markets or who lack any support by national organizations. The readers are for the most part interested in readily accessible documents also in terms of language, whereas language barriers regarding research activities are non-existent.

Beller ([3]) came to the conclusion that German is now unnecessary for the communication of scientific data. German-language journals should therefore concentrate on continuing education and provide opportunities for the original works of graduate students.

Finzen (9) warned, however, against the separation of research activity and national languages. A developed territory with 100 million people should be in a position to cultivate its own scientific language. [ M. Middeke of the Dtsch Med Wochenschr in a recent renowned newspaper - Frankfurter Allgemeine Sonntags-Zeitung of 3^rd Febr. 2002 - in this context spoke of a needful ”conservation of the variety of species”, as in environmental biology]. Haller in Zurich repeatedly asked ([15], [16]): ’’Is the Impact Factor killing the German language?’’

According to all the authors cited ([3], [7], [9], [15]), the predominance of English in medicine and science in the age of information technology can be attributed to tools developed in the United States:

• Index Medicus with the online database version MEDLINE ([8], [20])

• Science Citation Index (SCI)

[One may add, in the area of chemistry/ biochemistry, the Chemical Abstracts Service (CAS).]

A secondary product of SCI is the Journal Impact Factor ([13]). It has been used worldwide since the mid-1990s to evaluate the quality of scientific journals and even individual researchers and professor’s chairs applicants ([4], [10], [14], [18] ).

The impact factors ([12]) of life science journals are published annually in the Journal Citation Reports/Science Editions (JCR) of the Institute for Scientific Information-Thomson Scientific. It measures the citation frequency as represented by the cited references in the articles contained in the 5,600 SCI source journals selected by ISI. The formula for the year 2000 is presented in Fig. [1]:

Fig. 1 Journal Impact Factor ([11]): formula for the year 2000.
*) as documented in the bibliographic databases of ISI.

Citable works include original works, short communications from studies as well as review articles. In some very exceptional cases German-language journals received an IF above one. The procedure for determining the IF has raised there is a series of critical questions ([10], [15], [19], [27] - [31], [34]). Some in the United States, too, are wondering about this procedure ([2], [11]).

As an indicator for the international visibility of German-language journals, the level of worldwide citations can be useful; in that light, the IF is one measure. Furthermore, Boettiger and Navarro in their examination of articles from a series of volumes of representative German ([5], [21]), Austrian ([23]) and Swiss ([22]) journals determined in what language the cited references had been written.

As another visibility criterion, we consider the percentage of citations of German-language medical journals in the English-language literature as pertinent. In particular, the following questions were asked:

1. What has been the contribution of authors writing in English to the IF of German-language journals since the mid-1990s?

2. Is there a correlation between the scale of citations in English-language literature and the IF?

3. What percentage of English-language citing articles comes from institutions in German-speaking countries?

4. How often, for example, are German medical journals cited in articles that are (co-)authored in the United States?

Relative to question 2, J. Stegmann’s IF construction method ([31]) was applied to those periodicals that did not obtain an IF for the year 2000. So far no validation of this method has been described in the literature. Hence, before applying it, for example, to the 2000 data, the statistical correlation between the ISI IF and the constructed IF should be determined.

Methodology

Twenty-five German, Austrian and Swiss medical journals with predominantly German-language articles (over 85 %) for the publishing period 1995-July 2001 were selected. The basic set was composed using German-titled journals listed in Journal Citation Reports 1998 for Austria, Germany and Switzerland as well as German-titled periodicals from an Internet list of non-JCR journals (33) with a calculable (constructed) IF. Next, forty-five journals were randomly selected from this set: 30 JCR titles and 15 Internet titles. Using a combined search routine, SCISEARCH+MEDLINE+EMBASE, at DIMDI (8), journals with up to 15 % content devoted to English-language articles (i. e. >85 % German-language) for the period 1995-July 2001were found. According to our specifications for the subsequent selection, Austria and Switzerland should each have two periodicals represented. The other 21 were randomly selected.

The constructed impact factors for the year 2000 were calculated using the Stegmann method ([31], [32]) with SCI searches - for journals without an ISI IF with the help of MEDLINE+EMBASE databases at DIMDI ([8]). The latter served to retrieve citable 1998 and 1999 works not documented in the SCI, in order to obtain a denominator value higher than zero in the IF formula (Fig. [1]). Included among the citable documents are original articles, short communications and review articles - the so-called meaty articles. Citation frequencies of non-SCI periodicals were determined by searching Referenced Journal/Referenced Year data fields in DIMDI’s SCI version.

As a gauge of the relationship between the ISI IF and the constructed IF (Stegmann IF), Pearson’s correlation coefficient (r) was used. Where r = 1.00 , there is a 100 % positive linear relationship; where r = -1.00 , there is 100 % negative linear relationship. In the SCI, citations during 1995 - 2000 of the 1995 - 1999 meaty articles of the selected journals were searched, and then the language distribution of the citing articles was determined. Statistical relationships between the language shares of the citing articles and the IF were again determined using the correlation coefficient.

For each journal it was asked which country during 1995 - 2000 was the source of the English-language citing articles for the 1995 - 1999 publications of the 25 sample journals. The bibliographic data in SCISEARCH ([8]) with detailed and, in some cases, multiple entries in the field corporate source (institutional addresses) revealed how many were authored or co-authored in a German-speaking country (Germany, Austria and Switzerland) or in the United States. Such an approach could lead to the multiple counting of articles as in the case of a collaborative project by a German and a United States institution.

Results

Impact Factors of German-Language Journals

Table [1] provides an overview of the percentages of English-language contents as well as three different IF parameters for the journals examined.

Table 1 Comparison of Impact Factors (IF) taken from Journal Citation Reports (JCR) 1998 and 2000, and of ”constructed” IF 2000 ([31]) regarding a sample of 25 mainly German-language medical journals.

Journal 1	% English-lang. works published between 1995 - July 20012	IF 1998 (ISI)3	IF 2000 (ISI) 3	IF 2000 (constructed)4
1.Akt Ernährungsmed	5,3	ND 3)	ND	0,200
2.Akt Rheumatol	0,6	0,321	0,260	0,370
3.Anästhesist	1,6	0,977	0,829	0,838
4.Anästh Intensivmed.	2,9	0,575	0,646	0,605
5.Arbeitsmed Sozmed Umw	1,2	ND	ND	0,106
6.Chirurg	1,8	0,932	0,721	0,724
7.Dtsch Med Wochenschr	1,1	0,652	0,788	0,783
8.Hautarzt	0,9	0,479	0,535	0,550
9.HNO	1,7	0,675	0,722	0,700
10.Internist	2,3	0,329	0,277	0,268
11.Med Klin	1,5	0,380	0,390	0,380
12.Med Mon Pharmazeuten	0	ND	ND	0,033
13.Med Welt	1,0	0,075	0,061	0,061
14.Monatsschr Kinderheilk	1,2	0,191	0,140	0,168
15.Münch Med Wschr	0	ND	ND	0,039
16.Nervenarzt	3,6	0,696	0,641	0,594
17.Notfall-Med	0	ND	ND	0,005
18.ROFO -Fortschr Röntgen	3,4	0,901	1,005	0,980
19.Schmerz	0,3	0,443	0,500	0,524
20.Schweiz Med Wochenschr 5	11,8	0,296	0,258	0,294
21.Schweiz Rundschau Med Prax	0,2	ND	ND	0,034
22.Wien Med Wschr (WMW)	5,0	ND	ND	0,121
23.Wien Tierärztl MonSchr	10,7	0,236	0,287	0,262
24.Z ärztl Fortb Qualitätssich	0,5	ND	ND	0,144
25.Z Kardiol	15,3	0,760	0,874	0,859
Average	3,0	0,357	0,357	0,385
1=alphabetical order. 2= Values searched in Science Citation Index (SCI) or, if not represented in SCI, in a MEDLINE + EMBASE combination. All document types in the period 1995- July 2001 were included. 3= Values taken from JCRs 1998 and 2000 edited by the Institute for Scientific Information (ISI). IF 1998 relates to 1996+1997 meaty articles cited in 1998, and IF 2000 to contributions in 1998+1999 cited in 2000 - as documented by the ISI systems. ND = no data present in ISI’s JCR. 4=Determination from search results in SCI, or - in the case of ND - in the combined SCI+MEDLINE+EMBASE according to J. Stegmann (31, 32). In the SCI data fields Referenced Journal vs. Referenced Year at DIMDI, citations in year 2000 of meaty articles of Publication Years 1998+1999 were evaluated. 5=Determination from search results in SCI, or - in the case of ND - in the combined SCI+MEDLINE+EMBASE according to J. Stegmann (31, 32). In the SCI data fields Referenced Journal vs. Referenced Year at DIMDI, citations in year 2000 of meaty articles of Publication Years 1998+1999 were evaluated.

Determination from search results in SCI, or - in the case of ND - in the combined SCI+MEDLINE+EMBASE according to J. Stegmann ([31], [32]). In the SCI data fields Referenced Journal vs. Referenced Year at DIMDI, citations in year 2000 of meaty articles of Publication Years 1998 + 1999 were evaluated.

since Jan. 2001: Swiss Medical Weekly (100 % English-language)

The 25 journals with more than 85 % of their articles in German for 1995-July 2001 were arranged alphabetically in Table [1]. The proportion of articles published in English had a negligible relationship to the IF of the journals, as indicated by the correlation coefficient of 0.234 between the series on English share and the ISI 2000 impact factors.

Along with our constructed 2000 IF, the ISI IF for 2000 and 1998 were used for comparison. In general, the table reveals large similarities among the three IF parameters.

The journal Aktuelle Ernährungsmedizin, not represented in the JCR, had a constructed 2000 IF of 0.2. The constructed IF of the other non-JCR periodicals (no. 5, 12, 15, 17, 21, 22, and 24) were below 0.2. The value 0.2 was suggested by Froemter (10) on behalf of the Arbeitsgemeinschaft der Wissenschaftlich-Medizinischen Fachgesellschaften (AMWF) as a ”lump sum” to assign to German-language journals without an ISI IF. Comparison of the 1998 and the 2000 impact factors indicated that 10 of the journals (no. 4, 7, 8, 9, 11, 18, 19, 23, 25) in the last two years increased their impact factors - DMW, RöFo and Z Kardiol by more than 0.1. In fact, RöFo broke the knowledge sound barrier for German periodicals (IF = 1.0).

The comparison of 1998 and 2000 impact factors indicated a statistical correlation of almost 98 % (r = 0.975; n = 17 ). So for these two years, the impact factors of German-language journals seemed rather static.

The constructed impact factors are valid measures. The correlation coefficient between the 25 ISI impact factors and the constructed IF for 2000 was 0.987 (a 99 % correlation). If the impact factors only of the 17 SCI source journals are used in the comparison, a direct linear relationship is revealed: r = 0.9925 ! Hence, the Stegmann method was considered reasonable.

Language of Articles Citing the Selected Journals

According to results listed in Table [2] , the impact factor of the German-language journals examined was related with an average of 44.8 % to the references of authors writing in English for the specified time period.

Table [2] Shares and Origins of English-language Articles Citing during the Period 1995-2000 the ‚meaty’ Works published between 1995 - 1999 in a sample of 25 mainly German-language Medical Journals.

Cited Journal1	Articles published between 1995-2000 who cite the meaty works that appeared between 1995 - 1999 in the listed 25 journals
	% shares in German	% in English	% in English from institutions in D, A, CH 2, 3 (i. e., English share = 100 %)	% of articles (co-)authored in the USA 3 (English share = 100 % )
1.Akt Ernährungsmed	66,7	33,3	83,3	13,3
2.Akt Rheumatol	75,7	23,1	79,5	7,7
3.Anästhesist	49,6	48,8	54,5	16,7
4.Anästh Intensivmed.	63,7	35,6	69,8	12,2
5.Arbeitsmed Sozmed Umw	34,0	66,0	89,1	4,7
6.Chirurg	54,0	44,2	59,8	11,1
7.Dtsch Med Wochenschr	57,0	41,4	60,0	13,7
8.Hautarzt	33,4	63,8	55,0	15,3
9.HNO	60,0	39,1	61,4	14,0
10.Internist	76,5	23,1	87,2	3,4
11.Med Klin	49,9	47,9	56,9	16,3
12.Med Mon Pharmazeuten	60,0	40,0	66,7	33,3
13.Med Welt	60,4	39,6	74,5	3,6
14.Monatsschr Kinderheilk	45,6	53,9	82,0	8,5
15.Münch Med Wschr	51,4	45,9	45,4	25,0
16.Nervenarzt	49,5	47,7	62,7	16,8
17.Notfall-Med	71,9	25,0	75,0	0
18.ROFO -Fortschr Röntgen	50,1	48,1	66,0	12,7
19.Schmerz	72,5	27,5	76,3	8,8
20.Schweiz Med Wochenschr4	32,3	62,2	45,0	19,4
21.Schweiz Rundschau Med Prax	37,1	55,6	35,7	15,5
22.Wien Med Wschr (WMW)	28,2	66,0	54,6	17,9
23.Wien Tierärztl MonSchr	48,2	48,5	37,8	12,6
24.Z ärztl Fortb Qualitätssich	61,4	37,0	58,8	17,6
25.Z Kardiol	42,4	55,9	60,5	16,9
Mittelwert	53,3	44,8	63,9	13,5
Standardabweichung	13,9	12,8	14,6	7,1
1=alphabetical order. 2=D = Germany; A = Austria; CH = Switzerland 3=Multiple counting possible if multiple entries in SCI data field Corporate Source (Author’s address) exist, as with collaborative projects of US and German institutions 4= since Jan. 2001: Swiss Medical Weekly (100% Engl)

During the period 1995 - 2000, eight of the 25 journals (nos. 5, 8, 14, 20, 21, 22, 23, 25) were cited more often in English-language than in German-language articles published in 1995 - 1999. The total of citations in other languages, in French for example, played a minor role with an average of 1.5 %.

Relationship Between Languge of Citing Articles and Impact Factor

There was no relationship found between the various impact factor measures in Table [1] and the percentage of citations in English-language articles (Table [2] ): all the relevant correlation coefficients were less than 0.1

Even the relationship between publication share in English (Table [1] ) and level of citation in English-language articles (Table [2]) was extremely weak: r = 0.355 .

These two findings were surprising and correlate possibly with the overwhelming contribution of institutions in German-language countries as providers of the English-language citing works (Table [2]).

Publication Countries of English-Language Citing Articles

Table [2] shows that, on average, 64 % of the English-language 1995 - 2000 articles which cited articles from the German-language medical journals published during 1995 - 1999 had institutional addresses in Germany, Austria or Switzerland. The percentage of United States institutions in English-language citing articles averaged 13.5 %. In other words, approximately 20 - 26 % of the English-language citing articles were distributed over countries not mentioned here. Samples indicate high percentages for EU countries. Since possible double counting in publications with multiple institutional addresses cannot be eliminated, only an estimate can be given here.

Discussion

Impact Factor of German-Language Journals

Table [1] lends itself to some speculation regarding the IF. If German-language journals, such as nos. 1, 5, 12, 15, 17, 21, 22 and 24, were listed in the SCI as well as in EMBASE or MEDLINE, they would have a JCR IF, since there would be a value in the ISI literature documentation system in order to form the denominator of the IF formula (Fig. [1]). Furthermore, the cited references in the individual publications would be stored at ISI and could be added to the numerators in the IF calculations processed there.

The appropriate Referenced Journal/Referenced Year entries of the SCI records (RJ, Table [3] - showing one complete document in SCI) form the numerator of the IF formula. Unlike the SCI, the databases of MEDLINE, EMBASE, Biological Abstracts, etc., do not include cited references (RF). The SCI is thus the only available citation analysis instrument in medicine. If the article depicted in Table [3] had appeared hypothetically in the journal Zeitschr ärztl Fortbild Qualsich (ZaeFQ), the references under RF/RJ would not have shown up in the derivation of the ISI IF. A value for the denominator of the formula is not available at ISI since the SCI does not list ZaeFQ as a source (datafield SO).

Table 3 One example of a complete unit record (i. e. publication referral to (7)) in the Science Citation Index (SCI).

1/1 of 1 DIMDI-SCISEARCH COPYRIGHT ISI 2001

ND: 351VR005430001

AU: Dietrich GV; Hempelmann G

TI: How well-positioned is a German-worded publication?

SO: ANASTHESIOLOGIE INTENSIVMEDIZIN NOTFALLMEDIZIN SCHMERZTHERAPIE

VOL.35 [N9],PG:543 - 544,2000 SEP

LA: GERMAN

CS: UNIV GIESSEN, ANASTHESIOL & OPERAT INTENS MED ABT, RUDOLF BUCHHEIM STR 7;D-35385 GIESSEN; GERMANY

DT: EDITORIAL MATERIAL

JSC: BA ANESTHESIOLOGY; ...

RF: BOLDT J,1999,V34,P131 RJ: ANASTH INTENSIV NOTF

BOLDT J,1999,V34,P542 RJ: ANASTH INTENSIV NOTF

HALLER U,1999,V7,P39 RJ: CHIRURG S

HAUTEVILLE D,1995,V146,P29 RJ: ANN MED INTERNE

LEHRL S,1999,V175,P141 RJ: STRAHLENTHER ONKOL

MALECK WH,2000,V35,P559 RJ: ANASTHESIOL INTENSIV

SCISEARCH version at DIMDI (8):

ND = Number of Document

AU = Authors

TI = Title

SO = Source

CS = Corporate Source

LA = Language

DT = Document Type

JSC = Journal Section Code

RF = References

RJ = Referenced Journal

However, those non-SCI journals have entries throughout in the Referenced Journal field in the SCI, especially if they are cited by SCI source journals. With respect to the non-SCI non-JCR journals with recognizable citations in the SCI field RF, Stegmann (31, 32) used the search results from MEDLINE, EMBASE and other databases to the determine the denominator of the IF formula. Here he strictly resorts to the ISI methodology, however, on an enlarged and so more representative basis. The Stegmann IF that we calculated in this work indicated a 99 % statistical correlation to the corresponding ISI IF. The method proves to be valid here. Slight differences in the individual impact factors can be explained by the different search results attributable to the (sometimes very) multiple codes for one and the same journal in the SCI under RJ.

Periodicals such as ZaeFQ are considered by ISI as ’’cited-only-journals’’ in constrast to ’’source journals’’ (’’citing journals’’). In order to assess the quality of a science article, its content, of course, must be assessed, i. e., in a journal of a given quality. The omission of an article cannot automatically be based on the fact that the journal publishing it does not show up in a particular database (namely here, in an ISI product). Especially if the journal is indexed by the other recognized information systems MEDLINE and EMBASE and if its articles have been peer reviewed. Hence, the journal classification practice of ’’citing’’ and ’’cited only’’ is fundamentally without merit in the derivation of such a widely used quality assessment instrument as the IF. This was already demonstrated for the area of emergency medicine (11, 19).

For the non-SCI journals from Germany, Austria and Switzerland, general experience leads one to assume that the individual articles cite a considerable amount of German-language publications as references, notably those from the home or neighbour hospitals/ universities. If German-language non-SCI journals indexed by MEDLINE or EMBASE also showed up in the SCI there would be higher values in the IF numerator presumably for all journals from German-speaking countries.

In fact, Table [1] suggests the following. If the eight non-SCI journals were also listed by ISI, then they would, at the moment, rarely have IF values greater than 0.2. The AWMF suggestion mentioned earlier (10) seems at this point realistic.

German As a Scientific Language

A moderate citation frequency at the international level was found for the selected journals, which were mostly German-language medical publications (over 85 %)s, including tradition and distinguished periodicals. It is remarkable that, on average, over 44 % of the citations of the journals examined appeared in the English-language scientific literature. This rate is independent of the IF in our selection.

Table [2] shows that in the mid- and end-1990 s two-thirds of the English-language citing articles come from institutions in German-speaking countries. This also partially explains the weak correlation to the IF. So, English has been the prevailing scientific language of communication for quite some time in German-speaking areas. However, the results are not so bad as they seem in light of the ubiquitousness of English in this era of information technology. According to Dietrich (17), the average IF of German-language journals is 0.367 higher that the IF for languages other than English. We found an average ISI IF of 0.357 for our sample for the years 1998 and 2000.

Our research also indicates that more than 13 % of the English-language citing articles were authored or coauthored in the United States. This is quite interesting, since Boettiger (5) already established in 1983 that Americans quite rarely cite works from abroad. This observation was confirmed by Ojasoo (27) for international journals on urology with a description of a ’’transatlantic rift’’: United States publications tended to cite one another, while, on the other hand, European journals cited one another somewhat more frequently. Bookstein (6), proposing an augmented calculation for ”own-language citation bias”, found that in the area of sociology two German-language journals were top in this kind of bias.

Keul (17) reported on a dual phenomenon with respect to citation behavior in German-language psychology. Data from the Social Science Citation Index indicated that there were two equally large categories of authors. One group wrote mainly in English and cited internationally. The second group published in German, but tended to ignore research by the first group. Our Table [2] shows German, Austrian or Swiss institutional addresses for 64 % of the English-language citing works. Without a doubt, this contributes to the international visibility of the journals selected. In that respect, physicians publishing in English in the three countries regularly and eagerly refer to their colleagues who write in German.

Every era has had its dominant scientific language(s). The lingua franca of today is indisputably English. German and French, equally important national languages since the 19th century, have suffered a tremendous decline since World War II - or least since the 1970 s. Ammon (1) noted: ’’The German language has lost rank during the course of the century. English is making headway everywhere, so in science...French colleagues are sensing the same thing. For them, this turn of events is even more troublesome because of the active language policy of the French Academy...’’

Scientific journals such as those reviewed here can guarantee, however, that researchers and practitioners will not turn their backs on the national language. Through presentation of what is ascertained and what is new, such journals are still eminently significant for professional training and continuing education. Our findings suggest that they still have an distinct international readership.

Translation in English by M. Smith and E. Garfield, Chairman Emeritus of ISI, for purposes of ASIS&T, SIG-Metrics.

Authorized by G. Winkmann ([e-mail]), with kind assistance of Georg Thieme Verlag Stuttgart ([e-mail] , [e-mail]).

References

• 1 Ammon U. Deutsch als Wissenschaftssprache.  Spektrum der Wissenschaft. 1992;  124 117-118; 124
• 2 Barnaby D P, Gallagher E J. Alternative to the Science Citation Index impact factor as an assessment of emergency medicine’s scientific contributions.  Ann Emerg Med. 1998;  31 78-82
• 3 Beller F K. Die Zukunft der deutschen Sprache in der Wissenschaft.  Gynakol-geburtshilfliche Rundsch. 2000;  40 (1) 50-54
• 4 Benitez-Bribiesca L. The impact factor of medical journals: its use and misuse.  Arch Med Research. 1999;  30 161-162
• 5 Boettiger L E. Reference lists in medical journals - language and length.  Acta med scand. 1983;  214 (1) 73-77
• 6 Bookstein A, Yitzhaki M. Own-language preference: a new measure of ”relative language self-citation”.  Scientometrics. 1999;  46 (2) 337-348
• 7 Dietrich G V, Hempelmann G. Welchen Stellenwert hat eine Publikation in deutscher Sprache?.  Anasthesiol Intensivmed. 2000;  35(9) 543-544
• 8 DIMDI. Deutsches Institut für Medizinische Dokumentation und Information (DIMDI), Gate Database Searching.  Internet URL http://www.dimdi.de/.
• 9 Finzen A, Hoffmann-Richter U, Dittmann V, Haug H -J. Deutsch lesen - Englisch schreiben. Fachzeitschriften zwischen Science Citation Index und Nulltarif.  Psychiat Prax. 1996;  23 (1) 1-3
• 10 Froemter E, Braehler E, Langenbeck U, Meenen N M, Usadel K H. Das AWMF-Modell zur Evaluierung publizierter Forschungsbeiträge in der Medizin.  Dtsch Med Wochenschr. 1999;  124 910-915
• 11 Gallagher E J, Barnaby D B. Evidence of methodological bias in the derivation of the Science Citation Index impact factor.  Ann Emerg Med. 1998;  31 83-86
• 12 Garfield E. The Impact Factor.  Internet URL www.isinet.com/isi/hot/essays/journalcitationreports/7.html (as of Febr. 2002).
• 13 Garfield E. How can impact factors be improved?.  Br Med J. 1997;  313 411-413
• 14 Gießler A. Zur Evaluierung der Forschungsleistung eines Universitätsklinikums.  Dtsch Med Wochenschr. 2000;  125 979-983
• 15 Haller U, Hepp H, Reinold E. Tötet der ”Impact Factor” die deutsche Sprache?.  Gynakol-geburtshilfliche Rundsch. 1997;  37 117-118
• 16 Haller U, Hepp H, Reinold E. Tötet der ”Impact Factor” die deutsche Sprache?.  Chirurg. 1999;  70 (2) Suppl 39-41
• 17 Keul A G, Gigerenzer G, Stroebe W. Wie international ist die Psychologie in Deutschland, sterreich und der Schweiz? Eine SSCI-Analyse.  Psychol Rundsch. 1993;  44 (4) 159-169
• 18 Lehrl S. Der Impact-Faktor als Bewertungskriterium wissenschaftlicher Leistungen - das Recht auf Chancengleichheit.  Strahlenther Onkol. 1999;  175 141-153
• 19 Meenen N M. Der Impact-Faktor - ein zuverlässiger scientometrischer Parameter?.  Rontgenpraxis. 1998;  51 266-271
• 20 National Library of Medicine . Creator of MEDLINE.  Internet URL http://www.nlm.nih.gov.
• 21 Navarro F A. Englisch oder Deutsch? Die Sprache der Medizin aufgrund der in der Deutschen Medizinischen Wochenschrift erschienenen Literaturangaben (1920 bis 1995).  Dtsch Med Wochenschr. 1996;  121 (50) 1561-1566
• 22 Navarro F A. Die Sprache der Medizin in der Schweiz von 1920 bis 1995.  Schweiz Med Wochenschr. 1997;  127 (38) 1565-1573
• 23 Navarro F A. Die Sprache der Medizin in sterreich (1920 - 1995).  Wiener Klin Wochenschr. 1996;  108 (12) 363-369
• 24 Navarro F A. L’importance de l’anglais et du francais sur la base de references bibliographiques de travaux originaux publies dans la Presse Medicale (1920 - 1995).  Presse Med. 1995;  33 1547-1551
• 25 Navarro F A. El idioma de la medicina a traves de las referencias bibliograficas de los articulos originales publicados en Medicina Clinica durante 50 anos (1945 - 1995).  Med Clinica. 1996;  107 (16) 608-613
• 26 Navarro F A. De taal in de geneeskunde afgeleid uit in literatuurreferenties van oorspronkelijke stukken in het Nederlands Tijdschrift voor Geneeskunde (1930 - 1995).  Nederl Tijdschr Geneeskd. 1996;  140 1263-1267
• 27 Ojasoo T, Dor J C. Citation bias in medical journals.  Scientometrics. 1999;  45 81-94
• 28 Schoonbaert D, Roelants G. Citation analysis for measuring the value of scientific publications: quality assessment tool or comedy of errors?.  Trop Med Int Health. 1996;  1 739-752
• 29 Seglen P O. Why the impact factor of journals should not be used for evaluation of research.  Br Med J. 1997;  314 498-502
• 30 Seglen P O. Citations and journal impact factors: questionable indicators of research quality.  Allergy. 1997;  52 1050-1056
• 31 Stegmann J. How to evaluate impact factors.  Nature. 1997;  390 550
• 32 Stegmann J. Building a list of journals with constructed impact factors.  J Documentation. 1999;  55 310-324
• 33 Stegmann J. Constructed Impact Factors (CIF) for journals not listed in the Journal Citation Report (JCR).  Internet URL http://www.medizin.fu-berlin.de/medbib/CIF/cif.html. 1999; 
• 34 Winkmann G, Schweim H G. Medizinisch-biowissenschaftliche Datenbanken und der Impact Faktor.  Dtsch Med Wochenschr. 2000;  38 1133-1141

correspondence