Citations to Conference Papers Indicate They Are Declining in Importance across All Discipline Areas

Abstract

A Review of:
Lisée, Cynthia, Vincent Larivière and Eric Archambault. Conference Proceedings as a Source of Scientific Information: A Bibliometric Analysis.‛ Journal of the American Society for Information Science and Technology 59.11 (2008): 1776-84.

Objective – To compare the impact and ageing of conference proceedings with that of scientific literature in general, as reflected in citation characteristics.

Design – Citation analysis.

Setting – Thomson’s Science Citation Index, Social Science Citation Index, and Arts and Humanities Citation Index (CD-ROM version).

Subjects – Conference proceedings citations.

Methods – The Thomson citation indexes were searched to identify all citations to conference proceedings in natural sciences and engineering (NSE) and social sciences and humanities (SSH) from 1980 to 2005. Keywords in English, Spanish, Italian and German, truncated terms (such as ‘bienn’), single letters (such as P), and numbers were combined to retrieve all possible citations. Additional filters to exclude citations to publications other than proceedings were applied to the P search results, which had accounted for 75% of the total results. The references remaining in the P search set were validated using Google Scholar and WorldCat. Finally, two random samples of 1,000 references were checked manually to determine the extent of false positives and false negatives in the results.

Main Results – The study’s findings are presented for NSE and SSH separately, with 1.7% of NSE citations and 2.5% of SSH citations referring to conference proceedings. The total number of citations to proceedings has increased over the period 1980-2005, however, citations to proceedings in NSE and SSH as a proportion of all citations decreased during this time. A small increase in the average number of proceedings citations per paper was found for NSE and SSH. When this increase is compared to the overall increase in references per paper over this period, the share of proceedings citations per paper has decreased. Of all fields in NSE and SSH, only engineering has increased the proportion of proceedings citations, rising from 7% to 10% in the period studied. In 2005, the share of proceedings citations in NSE (excluding engineering) was below 3%, and for SSH it was below 1.5%.

The share of proceedings citations varies across different fields within NSE and SSH. Engineering fields and computer science range from around 5% (general engineering) to 19.6% (computers) in the share of proceedings citations, with only five of the 109 NSE fields having 10% or more as a share of proceedings citations. SSH has only one field (ergonomics, 7.6%) with a share of proceedings citations over 5%. Transport studies has a share of proceedings just under 5%, followed by the field information science & library science with proceedings citations at 3.3%.

In relation to the ageing characteristics of proceedings citations overall, the findings show a median age of 4.0 years compared with 6.1 years for citations to literature in general. The difference between the age of NSE cited proceedings and NSE cited literature in general had decreased during the period specified. In 1980, the median age of NSE cited proceedings was 6.3 years compared with 9.3 years for NSE citations to literature in general. In 2005, the median ages were 8.4 years and 10.1 years, respectively. The median age of SSH cited proceedings in 2005 was 10.3 years, compared with 14.2 years for all SSH cited literature. Cited literature in general is older for SSH (14.2 years) than NSE (10.1 years), but the age difference between proceedings cited in the two discipline areas is almost half this. A number of fields in NSE (such as physics, chemistry, and engineering) indicate a greater difference between the age of cited proceedings and literature in general, while for others (such as biology and biomedical research) the ageing characteristics are similar. In SSH, the difference between age of cited proceedings and literature in general is greater. Fine arts and psychology proceedings citations are 43% younger than citations to literature in general; literature cited proceedings are 42% younger, and social sciences 31% younger. Humanities are an exception, with cited proceedings only 11% younger than citations to literature in general.

Conclusion – Only 2% of all citations are to conference proceedings in NSE and SSH combined; a proportion that has declined over the 25-year period studied. While there was an increase in the average number of (all) citations per paper during this time, proceedings citations per paper have seen only a very slight increase. These findings are true of all fields studied, with the exception of engineering-related fields which have enjoyed an increase of over 2% in proceedings citations in the period studied. The results also indicate the importance of proceedings in the field of computers. The authors speculate that in these fields, proceedings are regarded as ‚more than just prototypes, but rather as the final products of scientific research.‛ Due to the higher proportion of proceedings citations in engineering and computer science fields, they should be considered for analysis in bibliometric studies.

Despite arriving at this conclusion, the authors suggest that computer scientists might consider publishing their papers through channels other than conference proceedings to ‚maximize their scientific impact [original italics].‛ They support this statement by noting that although proceedings citations in computer science represent 20% of total citations, a study of Australian computer science research output (Butler) found proceedings comprise over 60% of all publications in computer science. The authors suggest that the difference between the proportion of proceeding published and the proportion of proceedings cited indicate that their scientific impact does not seem to be all that important.‛

In all fields, proceedings are cited sooner after publication and they cease to be cited earlier than literature in general. These results indicate that proceedings deliver more current information and cutting edge research findings than literature in general. The differences between ageing of proceedings citations and of literature in general lead the authors to conclude that conference proceedings serve different functions and have different life cycles depending on the community they serve.‛