An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-Makers

By Andrew J. Oswald. Social Science Research Network, 2007

234 citations recorded by the source corpus.

In universities all over the world, hiring and promotion committees regularly hear the argument: “this is important work because it is about to appear in prestigious journal X”. Moreover, those who allocate levels of research funding, such as in the multi-billion pound Research Assessment Exercise in UK universities, often come under pressure to assess research quality in a mechanical way by using journal prestige ratings. This paper’s results suggest that such tendencies are dangerous. It uses total citations over a quarter of a century as the criterion. The paper finds that it is far better to publish the best article in an issue of a medium-quality journal like the Oxford Bulletin of Economics and Statistics than to publish the worst article (or often the worst 4 articles) in an issue of a top journal like the American Economic Review. Implications are discussed.

Subfields: Information Systems and Management, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/8b3e802652066e6ebbc33e1676a0440e4c8dcb8b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems and Management

In Statistics and Probability

Citations in the corpus

Journal Quality in Mathematics Education (2017, methodology): “Journal rankings are used by many governments as a measure of research output and quality (Auranen & Nieminen, 2010) and are factored into decisions about resource allocation (Oswald, 2007).”
Citations in Economics: Measurement, Uses and Impacts (2018, methodology): “Inspired by Oswald (2007), consider articles published in the Economic Journal and the Review of Economics and Statistics in 2007–08, the same years as the later sample of J5, and using the same selection criteria (ten+ pages, all in regular issues of journals, no comments, citations through…”
A review of theory and practice in scientometrics (2015, methodology): “If used for assessing individual researchers or papers the JIF is unrepresentative (Oswald, 2007).”
The Elite Brain Drain (2009, unknown): “As pointed out by Starbuck (2005), Oswald (2007a) and others, even prestigious journals publish large numbers of papers that make little impact.”
Tracking the evolution of waste recycling research using overlay maps of science. (2011, background): “Yet journal-based evaluation has been severely criticised as being inappropriate for this role (Seglen, 1997; Oswald, 2007).”