A statistical interpretation of term specificity and its application in retrieval

By Karen Spärck Jones. 2021

5,376 citations recorded by the source corpus.

The exhaustivity of document descriptions and the specificity of index terms are usually regarded as independent. It is suggested that specificity should be interpreted statistically, as a function of term use rather than of term meaning. The effects on retrieval of variations in term specificity are examined, experiments with three test collections showing in particular that frequently‐occurring terms are required for good overall performance. It is argued that terms should be weighted according to collection frequency, so that matches on less frequent, more specific, terms are of greater value than matches on frequent terms. Results for the test collections show that considerable improvements in performance are obtained with this very simple procedure.

Subfields: Artificial Intelligence, Computer Networks and Communications, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Percy Liang
  4. Luke S. Zettlemoyer
  5. Graham Neubig
  6. Yejin Choi
  7. Yoshua Bengio
  8. Noah A. Smith

In Computer Networks and Communications

  1. Grgoire Montavon
  2. Zachary Chase Lipton
  3. Jaime Teevan
  4. K. Müller
  5. Michael J. Cafarella
  6. Satish Narayanasamy
  7. Chun-Hung Hsiao
  8. Yuting Chen

In Information Retrieval & Search

  1. Sebastian Riedel
  2. Gautier Izacard
  3. Edouard Grave
  4. Wen-tau Yih
  5. Fabio Petroni
  6. Jamie Callan
  7. Patrick Lewis
  8. Zhiyuan Liu

Citations in the corpus