| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Special Report |
1 From Service de Radiologie (L.A., P.D., M.L., L.M.C., J.M.T.) and Unité de Santé Publique (P.Y.B.), Hôpital Saint-Antoine, 184 rue du Faubourg Saint-Antoine, 75012 Paris, France. Received May 30, 2003; revision requested August 12; revision received October 7; accepted November 12. Address correspondence to L.A. (e-mail: lionel.arrive@sat.ap-hop-paris.fr).
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMATERIALS AND METHODS RESULTS DISCUSSIONREFERENCES |
|---|
MATERIALS AND METHODS: The 1995 RSNA Scientific Program was jointly reviewed by two readers. A total of 1,897 abstracts were included in the study. A Medline search of articles published between 1996 and 2000 was then conducted to identify articles written by the first, second, and/or last authors of all abstracts published in the 1995 RSNA Scientific Program. The year of publication, journal, country origin of the abstract, subspecialty, and nature of the research (ie, human, animal, or technical) were recorded. Publication rates were compared by using multiple logistic regression analysis.
RESULTS: Six hundred thirty-five abstracts were expanded into manuscripts that were subsequently published in Medline-indexed journals; thus, the publication rate was 33%. A study was published less than 3 years after the 1995 scientific assembly in 595 (94%) cases. Genitourinary radiology and chest radiology studies had the highest publication rates (46% and 48%, respectively; P < .01), whereas physics studies had the lowest (24%, P < .001). Technical studies (24%, P < .001) were less likely than human studies (36%, P < .001) to be published. The publication rate also differed substantially according to the country origin of the abstract. The articles were published in a total of 109 journals and chiefly in Radiology (211 cases, 33% of published studies).
CONCLUSION: One-third of original studies presented orally at the 1995 scientific assembly were subsequently published in Medline-indexed journals. More articles were published in Radiology than in any other identified journal.
© RSNA, 2004
Index terms: Radiological Society of North America • Radiology and radiologists, research • Radiology and radiologists, socioeconomic issues
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTS DISCUSSIONREFERENCES |
|---|
Oral presentation at a national or international scientific conference has the advantage of enabling rapid communication. Subsequent publication in a scientific journal is the natural and final outcome of the research cycle because by the time of publication, the investigation will have been subjected to thorough manuscript preparation, peer review, and extensive analysis of the results. Oral presentation has the drawback of only reaching those who are present at the conference and/or those who have access to the conference proceedings, which are reported in very brief summaries. In addition, unlike the methodologic quality of fully published studies, the methodologic quality of conference abstracts is difficult to evaluate at the time that they are submitted (1).
It has been reported that the rate of publication of presentations at scientific conferences ranges from 11% to 78%; these data suggest that at least one-quarter of studies that are initially presented at conferences are never fully published in scientific journals (1–3). High publication rates are sometimes considered to be indicative of the quality of the scientific meeting. On the other hand, publication rates have been shown to depend on a variety of factors, some of which are independent of the quality of the research (4).
The aim of this study was to determine the rate at which original studies that were presented orally at the 1995 Radiological Society of North America (RSNA) scientific assembly were published in Medline-indexed journals and to identify factors predictive of publication.
|
MATERIALS AND METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTS DISCUSSIONREFERENCES |
|---|
The two readers jointly classified the remaining 1,897 abstracts according to (a) the radiologic subspecialty, as indicated in the program book; (b) the country from which the abstract was submitted, as indicated in the program book (three countries are cited in the program book); and (c) the study subjects and materials involved—specifically, patients and/or healthy volunteers, animals, or technical materials. Abstract subjects and material were classified as patients and/or healthy volunteers if the abstract contained both human and animal or technical data, as animals if it contained both animal and technical data but no human data, and as technical material if it contained neither human nor animal data.
Study Search and Data Collection
The two readers (L.A., P.D.) then jointly estimated the subsequent publication rate for the corresponding studies by scanning Medline on the PubMed server (www.ncbi.nlm.nih.gov/PubMed/) for the 5-year period following the RSNA 1995 Scientific Assembly—that is, January 1996 through December 2000. The results of several studies have shown that the majority of articles are published within 5 years after the material is presented at a conference (6–10).
The two readers conducted the Medline search by using the first author’s family name and the initial(s) of his or her first name(s). If more than 50 references were identified, the search was repeated by using both the first author’s and the second author’s last names and first-name initials. If at this second search they did not identify any publications corresponding to the abstract in question, the search was repeated by using both the first author’s and the last author’s last names and first-name initials. If this search was unfruitful, the search was repeated by using both the first author’s last name and first-name initials and the radiologic subspecialty. If the first search based on the first author’s last name and first-name initials yielded no publications, the search procedure was repeated, starting with the second author’s family name and first-name initials, as described above for the first author.
Only original articles corresponding to the abstract were selected; letters, reviews, and editorials were excluded. The concordance between the information contained in the summary of the published article and that cited in the abstract of the oral presentation was verified. We did not select articles that simply described the same information given in the abstract in question (eg, general considerations on the use of a contrast medium vs a precise application of the same contrast medium in a specific setting). In contrast, we selected articles that addressed more restricted subject matter compared with the information presented in the corresponding abstract, as this truncation might have occurred during the review process. We did not select articles in which the number of subjects, animals, or experiments described was more than three times that cited in the abstract because such abstracts probably corresponded to very preliminary work that had been followed up in a major study that bore little resemblance to the initial protocol. The publication rate was calculated by dividing the number of abstracts by the number of resulting published articles. The name of the journal, year, and language in which each article was published were noted.
Statistical Analyses
We then examined whether the following factors were associated with the publication rate: the radiologic subspecialty, the country from which the meeting abstract originated, and the type of research (ie, human, animal, or technical). This analysis was performed by using logistic regression of the dichotomous variable of publication versus nonpublication and the factors just mentioned (ie, radiologic subspecialty, country origin, and research type). A variable was considered influential on the basis of likelihood ratio test results at the .05 level. To ease interpretation, we adopted an approach that enabled us to compare a given factor with all other factors rather than with a reference-level factor. Therefore, each factor was coded as a binary variable and independently examined in the logistic regression. A univariate analysis of each of the three factors listed above was first performed; then, it was repeated with adjustments made according to the two other factors. All statistical analyses were performed by using computer software (R Software, version 1.7.1; The R Foundation for Statistical Computing, Vienna, Italy).
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTS DISCUSSIONREFERENCES |
|---|
|
|
Nearly all of the articles were published in English-language journals (614 publications, 97%), whereas 13 were published in German; six, in Italian; one, in Japanese; and one, in Spanish. With regard to the year of publication, 301 (47%) articles were published in 1996; 213 (34%) articles, in 1997; 81 (13%) articles, in 1998; 25 (4%) articles, in 1999; and 15 (2%) articles, in 2000 (Table 3). Thus, 94% (n = 595) of the articles were published during the first 3 years after the 1995 scientific assembly.
|
The country from which the abstracts originated also influenced the likelihood of subsequent publication. Studies from Japan and the Netherlands had the highest publication rates (47% and 59%, respectively), whereas studies from Korea had the lowest publication rate (22%). The odds ratio for presentations from the Netherlands—as opposed to presentations from all other countries—being published was 2.95 (P = .003); when any effects of subspecialty and research type were accounted for, the odds ratio was 2.8 (P = .005). On the other hand, the odds ratio for presentations from Korea—as opposed to presentations from all other countries—being published was 0.54 (P = .06); when any effects of subspecialty and research type were accounted for, the odds ratio was 0.5 (P = .036) (Table 2).
Finally, studies involving patients and healthy volunteers had the highest publication rate (503 [36%] of 1,389 studies), whereas technical studies had the lowest publication rate (96 [24%] of 395 studies). The odds ratio for human studies—as opposed to animal or technical studies—being published was 1.58 (P < .001); after any effects of subspecialty and country origin were accounted for, the odds ratio was 1.32 (P = .04). On the other hand, the odds ratio for technical studies—as opposed to human or animal studies—being published was 0.57 (P < .001); after any effects of subspecialty and country origin were accounted for, the odds ratio was 0.69 (P = .02).
When we restricted the analysis to English-language publications, the odds ratios for publication according to radiologic subspecialty and study type were unchanged. Likewise, with the exception of the odds ratios for Germany (0.6 with vs 0.83 without English-language restriction) and Italy (0.57 with vs 1.06 without English-language restriction), all odds ratios based on country origin were globally unchanged. Authors from these two countries were more likely than were authors from other countries to have their presentation-based articles published in their national language (13 [22%] of 59 articles for German authors, six [38%] of 16 articles for Italian authors).
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTS DISCUSSIONREFERENCES |
|---|
The method used to identify published articles that is described herein has been used in several previously performed studies and is based on the last name of the first author, followed, when necessary, by the last names of the second and last authors and the cross matching of the last name of the first author with the radiologic subspecialty (8,10,11).
We did not search for articles published before 1996 because our aim was to determine the rate of subsequent publication of orally presented abstracts. The period covered in this survey was the 5 years after the 1995 RSNA Scientific Assembly. Although the publication rate decreased markedly during the 4th and 5th years of our survey, it is likely that we missed some articles that were published in 2001 and 2002. Juzych et al (6) and Nguyen et al (7) found that the mean times to publication after oral presentations at ophthalmology and orthopedics conferences were 13 and 16 months, respectively. Yentis et al (8) reported that 90% of articles were published within 3 years after oral presentations at anesthesiology conferences. Callaham et al (9) reported that 88% of articles were published within 3 years after oral presentations at medical conferences.
Our results are consistent with these reports: We found that 94% of articles were published within 3 years after the 1995 RSNA Scientific Assembly. More recently, Bhandari et al (10) reported that the mean time to publication after oral presentation at the 63rd Annual Meeting of the American Academy of Orthopaedic Surgeons was 17.6 months ± 12 (SD) (median time, 4 months; range, 1–56 months).
We searched for published articles in a single database (Medline) by using a single search engine (PubMed). Although the results of some relatively older studies have suggested that document searches on electronic servers lead to the identification of only about 50% of target references (12,13), Berry et al (14) reported that their Medline search for articles on medical imaging was nearly exhaustive: 94% of target references were identified.
Although the search technique based on authors’ last names has been used in this type of study previously, to our knowledge its efficiency has not been assessed (8,10,11). Errors in the search may occur owing to the misspelling of the first author’s last name or to the fact that the first author of the abstract is not an author of the resulting full article. We minimized these risks by repeating unfruitful searches by using the last name of the second author and, if necessary, the last name of the last author. Thus, the 33% publication rate that we observed in this study is likely to be slightly lower than the actual publication rate. It is important to note that we included all 1,897 abstracts that were presented at the 1995 scientific assembly. Thus, the use of a questionnaire-based survey of the authors in question—an approach that has been used after smaller conferences—would have been extremely difficult.
The 33% publication rate for articles on studies presented orally at the 1995 RSNA scientific assembly is lower than the publication rates reported after other North American conferences, which range from 44% for the combination of studies presented at the meetings of four North American anesthesiologist groups (8) to 78% for studies presented at the American Society of Clinical Oncology Meeting (1). In contrast, the publication rate reported herein is identical to that for neuroradiologic studies presented at the 1993 RSNA scientific assembly (33%) (15). A meta-analysis revealed a publication rate of 51% for 2,391 abstracts (16). In a recent Cochrane Library review of publication rates following abstract presentations, 46 studies described in a total of 15,985 abstracts were identified (3); publication rates ranged from 11% to 78% (median, 45%) (1–3).
The majority of articles related to work presented at the 1995 RSNA scientific assembly were published in English-language journals, confirming that most authors consider English to be the best language in which to present their findings. It should also be noted that 63% (n = 1,202) of the abstracts that were presented at the 1995 RSNA scientific assembly originated in the United States.
We found that the publication rate varied according to the country origin of the orally presented abstract. In particular, the abstracts that originated in Japan and the Netherlands were published far more frequently than were abstracts that originated in other countries. However, these two countries generated only 78 (Japan) and 32 (the Netherlands) orally presented abstracts, compared with the 1,202 abstracts from the United States. Similarly, the frequency of publication of studies in some radiologic subspecialties—especially chest radiology and genitourinary radiology—was markedly higher than that of studies in other subspecialties, particularly physics, the most heavily represented subspecialty. Finally, oral presentations of technical studies were expanded and published less frequently than were oral presentations of human studies.
It was not possible to correlate the subsequent publication rate with specific characteristics of the oral presentations, such as methodologic quality, scientific originality, and the positive or negative nature of the results, because these data could not always be extracted from the 1995 RSNA Scientific Program (5).
Various factors may explain why some oral presentations are not subsequently published as manuscripts. The rejection of submitted manuscripts has only a minor role—although this varies according to the journal—according to Weber et al (17). This group found that only 20% of the orally presented original studies that had not been published had been submitted to a journal. In their questionnaire-based study, Weber et al surveyed 266 investigators who had made oral presentations to the Society for Academic Emergency Medicine and had not had their work published in full. Two hundred twenty-three (84%) of these investigators completed the questionnaire. One hundred seventy-nine of these investigators had not submitted a manuscript for publication because of the following stated reasons: There was not enough time (42%), the acceptance of the manuscript was thought to be unlikely (20%), the results of the study were not believed to be important enough (12%), there was too much trouble with the co-authors (9%), it was not worth the trouble to submit the work (7%), there were other articles describing similar findings (6%), the results of the statistical analysis were not positive (4%), and other reasons (22%) (17).
More recently, Sprague et al (18) consulted 71 authors who gave oral presentations at the annual conference of the American Academy of Orthopaedic Surgeons but did not subsequently submit the presented work for publication. These investigators offered the following explanations for not submitting manuscripts: There was no time to prepare a manuscript for possible publication (47%), the study would still be ongoing 5 years after the oral presentation (31%), a co-author was responsible for writing the manuscript (20%), there was difficulty with the co-authors (17%), and submitting a manuscript had low priority (13%) (18).
The low publication rate (33%) for the oral presentations given at the 1995 RSNA Scientific Assembly is probably due to the factors just described—namely, a lack of time, a belief that the study results were unimportant, pessimism regarding the likelihood of acceptance, and a conflict with the co-authors. In addition, it is possible that the rejection of submitted manuscripts was more important in this study than it was in the study of Weber et al (17), who reported that only 20% of the orally presented studies that had not been published had been submitted to a journal. Characteristics inherent to the field of radiology could lead to a higher rejection rate. Although most of the articles published after the 1995 scientific assembly were in a handful of journals, we identified articles in a total of 109 journals. Finally, the particularly rapid nature of technologic advances in radiology might explain why some studies that are accepted for oral presentation are not deemed worthy of publication some months or years later.
There were several limitations in the present study. We did not search for articles published before 1996 because our aim was to determine the rate of subsequent publication of oral presentations so that a comparison with the subsequent publication rate for studies in other subspecialties in medicine would be feasible. However, it is likely that some articles were published at the end of 1995. Similarly, it is likely that we missed some articles published more than 5 years after the RSNA scientific assembly. Our search was restricted to Medline-indexed journals, and it is possible that some articles were published in journals that are not indexed in Medline.
Finally, according to Radiology policy, all scientific abstracts that are accepted by RSNA should be submitted as a manuscript to Radiology first. One can understand how a rejection by Radiology could discourage authors from submitting manuscripts to other journals. Therefore, it is possible that the rejection of a submitted manuscript was more important in this study than it was in the study of Weber et al (17). However, we have no data on either the manuscripts that were submitted to Radiology first or those that were submitted to other journals without having been submitted to Radiology. These factors explain why the 33% publication rate reported herein probably is slightly lower than the actual publication rate.
This low rate of publication deprives the scientific community of potentially interesting results, and it also prevents these results from being included in meta-analyses and systematic reviews, especially for uncommon diseases (19). In addition, from an author’s viewpoint, in personal publications lists, which are important for grant applications and career advancement, oral presentations carry far less weight than do written publications. From a similar viewpoint, for younger investigators the educational value of scientific writing is lost when no attempt is made to publish a study that was presented in the form of an abstract. Finally, the nonpublication of original studies is detrimental for publishers specializing in radiology. Although Radiology was by far the leading target journal in this survey, only 11% (n = 211) of the 1,897 studies presented as abstracts at the 1995 RSNA Scientific Assembly were published in this journal.
The nonpublication of original studies has other consequences: Abstracts presented at medical and scientific conferences are sometimes referenced, especially since electronic publication now makes them more readily available (20). Bhandari et al (10) reviewed the latest editions of several major orthopaedic textbooks and found that in 53%–63% of the chapters, at least one abstract from an international meeting was referred to. Although many peer-reviewed journals prohibit the referencing of abstracts in published articles, abstracts are referenced in textbooks and routinely cited at lectures. However, contrary to the methodologic quality of published studies, the methodologic quality of abstracts presented at conferences is difficult to evaluate at the time of their submission (1,21–23).
Another concern is the confusion on the part of some authors as to whether a study that is described only in an abstract constitutes a randomized controlled trial. Scherer et al (16) surveyed 79 abstract authors who reported results from randomized controlled trials and found that nine of these authors, who had explicitly stated in the abstract that "treatment assignment was randomized" indicated the opposite in the survey questionnaire. Thus, methodologically flawed studies can enter the scientific information chain via abstracts presented at conferences.
Another problem, which is just as worrying, is the nonpublication of negative results, which leads to publication bias (24). Klassen et al (25), in analyzing the publication rate for randomized controlled trials presented as abstracts at meetings of the Society for Pediatric Research from 1992 to 1995, found that the results of almost 64% of the published studies, as compared with the results of only 43.5% of studies that were never published (P < .001), favored new therapies. In the field of radiology, such practices can lead to an overestimation of the precision of a radiologic technique due to—for example—the publication of only the most favorable studies. Publication bias is particularly problematic in pooled studies such as meta-analyses because it can lead to an overestimation of the degree of the effect (19). It is worth noting that meta-analyses are increasingly being used in radiologic research (26).
In conclusion, one-third of the original studies initially presented orally as abstracts at the 1995 RSNA scientific assembly were subsequently published in Medline-indexed journals. Articles were published less than 3 years after the meeting in 94% of cases. More articles were published in Radiology than in any other identified journal. Genitourinary radiology and chest radiology studies had the highest publication rates, whereas physics studies had the lowest. Human studies had the highest publication rate, whereas technical studies had the lowest. The publication rate also differed significantly according to the country origin of the abstract.
|
FOOTNOTES |
|---|
Author contributions: Guarantor of integrity of entire study, L.A.; study concepts, L.A., J.M.T.; study design, L.A., P.D.; literature research, L.A., P.D.; data acquisition, all authors; data analysis/interpretation, L.A., P.D., M.L., L.M.C.; statistical analysis, P.Y.B.; manuscript preparation and revision/review, all authors; manuscript definition of intellectual content, L.A.; manuscript editing, L.A., P.Y.B., M.L., L.M.C., J.M.T.; manuscript final version approval, L.A., L.M.C., P.D.
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTS DISCUSSIONREFERENCES |
|---|
This article has been cited by other articles:
|
|
 |
|
  L. Arrive, M. Lewin, P. Dono, L. Monnier-Cholley, C. Hoeffel, and J.-M. Tubiana Redundant Publication in the Journal Radiology Radiology, June 1, 2008; 247(3): 836 - 840. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. Miguel-Dasit, L. Marti-Bonmati, R. Aleixandre, P. Sanfeliu, and D. Bautista Publication of Material Presented at Radiologic Meetings: Authors' Country and International Collaboration Radiology, May 1, 2006; 239(2): 521 - 528. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| RADIOLOGY | RADIOGRAPHICS | RSNA JOURNALS ONLINE |
