The Impact Factor Fallacy

By Frieder Michel Paulus, Nicole Cruz, S. Krach. 2017

81 citations recorded by the source corpus.

The use of the journal impact factor (JIF) as a measure for the quality of individual manuscripts and the merits of scientists has faced significant criticism in recent years. We add to the current criticism in arguing that such an application of the JIF in policy and decision making in academia is based on false beliefs and unwarranted inferences. To approach the problem, we use principles of deductive and inductive reasoning to illustrate the fallacies that are inherent to using journal based metrics for evaluating the work of scientists. In doing so, we elaborate that if we judge scientific quality based on the JIF or other journal based metrics we are either guided by invalid or weak arguments or in fact consider our uncertainty about the quality of the work and not the quality itself.

Subfields: Information Systems, Statistics and Probability, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

1. Tristan Thrush
2. Douwe Kiela
3. Yacine Jernite
4. Alexandra Sasha Luccioni
5. Margaret Mitchell

In Statistics and Probability

1. Douwe Kiela
2. Yacine Jernite
3. Margaret Mitchell

In Evaluation & Benchmarks

1. Tristan Thrush
2. Douwe Kiela
3. Angelina McMillan-Major
4. Yacine Jernite
5. Alexandra Sasha Luccioni
6. Margaret Mitchell
7. Nazneen Rajani
8. Mohit Bansal

Citations in the corpus

• Measuring Data (2022, background): “If measures are reported without the right contextualization, they can mislead and provide an undeserved veneer of objectivity to subjective or incomplete evaluations (Paulus et al., 2017).”