Learning Semantic Similarity for Very Short Texts
By Cedric De Boom, Steven Van Canneyt, Steven Bohez, Thomas Demeester, Bart Dhoedt. 2015
86 citations recorded by the source corpus.
Levering data on social media, such as Twitter and Facebook, requires information retrieval algorithms to become able to relate very short text fragments to each other. Traditional text similarity methods such as tf-idf cosine-similarity, based on word overlap, mostly fail to produce good results in this case, since word overlap is little or non-existent. Recently, distributed word representations, or word embeddings, have been shown to successfully allow words to match on the semantic level. In order to pair short text fragments -- as a concatenation of separate words -- an adequate distributed sentence representation is needed, in existing literature often obtained by naively combining the individual word representations. We therefore investigated several text representations as a combination of word embeddings in the context of semantic pair matching. This paper investigates the effectiveness of several such naive techniques, as well as traditional tf-idf similarity, for fragments of different lengths. Our main contribution is a first step towards a hybrid method that combines the strength of dense distributed representations -- as opposed to sparse term matching -- with the strength of tf-idf based methods to automatically reduce the impact of less informative terms. Our new approach outperforms the existing techniques in a toy experimental set-up, leading to the conclusion that the combination of word embeddings and tf-idf information might lead to a better model for semantic content within very short text fragments.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mónica Marrero
- Thomas Demeester
- Bart Dhoedt
- Nava Tintarev
- Noor Akhmad Setiawan
- Claudia Hauff
- Steven Van Canneyt
- Cedric De Boom
Citations in the corpus
- Representation learning for very short texts using weighted word embedding aggregation (2016, methodology): “In previous work we showed that using an Euclidean distance function leads to a much better separation between related and non-related pairs than the more often used cosine distance, so we will also use the Euclidean distance throughout our experiments here (De Boom et al. (2015a)).”
- Explaining Credibility in News Articles using Cross-Referencing (2019, methodology): “Future implementations should consider employing more sophisticated short-text similarity methods such as those based on word and sentence embeddings [6, 11].”
- Cosine similarity to determine similarity measure: Study case in online essay assessment (2016, methodology): “A number of methods such as Term Frequency-Inverse Document Frequency (TF-IDF) applied for finding information on social media content [1].”