Top2Vec: Distributed Representations of Topics

By D. Angelov. arXiv.org, 2020

515 citations recorded by the source corpus.

Topic modeling is used for discovering latent semantic structure, usually referred to as topics, in a large collection of documents. The most widely used methods are Latent Dirichlet Allocation and Probabilistic Latent Semantic Analysis. Despite their popularity they have several weaknesses. In order to achieve optimal results they often require the number of topics to be known, custom stop-word lists, stemming, and lemmatization. Additionally these methods rely on bag-of-words representation of documents which ignore the ordering and semantics of words. Distributed representations of documents and words have gained popularity due to their ability to capture semantics of words and documents. We present $\texttt{top2vec}$, which leverages joint document and word semantic embedding to find $\textit{topic vectors}$. This model does not require stop-word lists, stemming or lemmatization, and it automatically finds the number of topics. The resulting topic vectors are jointly embedded with the document and word vectors with distance between them representing semantic similarity. Our experiments demonstrate that $\texttt{top2vec}$ finds topics which are significantly more informative and representative of the corpus trained on than probabilistic generative models.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/fda2a8b03fb15a2d8b5c5aeb01d1c0b27f0b006b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

BERTopic: Neural topic modeling with a class-based TF-IDF procedure (2022, unknown): “Several approaches have started simplifying the topic building process by clustering word-and document embeddings (Sia et al., 2020; Angelov, 2020).”
Do travelers' reviews depend on the destination? An analysis in coastal and urban peer‐to‐peer lodgings (2021, methodology): “In particular, following BERTopic's approach (Grootendorst, 2020) based on Top2Vec (Angelov, 2020), our research firstly transforms our corpus into 768‐dimensional vectors leveraging a pre‐trained sentence transformer model optimized for semantic textual similarity (Reimers & Gurevych, 2019; cf. sen”
A Topic Modeling Comparison Between LDA, NMF, Top2Vec, and BERTopic to Demystify Twitter Posts (2022, methodology): “Model 3: Top2Vec Top2Vec (Angelov, 2020) is a comparatively new algorithm that uses word embeddings.”
MTEB: Massive Text Embedding Benchmark (2022, background): “Natural language embeddings power a variety of use cases from clustering and topic representation (Aggarwal and Zhai, 2012; Angelov, 2020) to search systems and text mining (Huang et al.”
PyThaiNLP: Thai Natural Language Processing in Python (2023, unknown): “They also combined our word vectors with Top2Vec ( Angelov, 2020) to perform topic modeling and improve customer experience.”
Granite Embedding R2 Models (2025, unknown): “…the relevance of a document to a given query can be determined by the similarity of their embeddings (Dunn et al., 2017; Xiong et al., 2020; Neelakantan et al., 2022; Zamani et al., 2018; Zhao et al., 2020), but also in document clustering (Angelov, 2020) and text classification (Sun et al., 2019).”
Granite Embedding Models (2025, unknown): “…the relevance of a document to a given query can be determined by the similarity of their embeddings (Dunn et al., 2017; Xiong et al., 2020; Neelakantan et al., 2022)(Zamani et al., 2018; Zhao et al., 2020), but also in document clustering (Angelov, 2020) and text classification (Sun et al., 2019).”
Length-Induced Embedding Collapse in PLM-based Models (2024, unknown): “…vectors that capture the semantic information of texts—are essential for many NLP applications, including text analysis (Aggarwal and Zhai, 2012; Angelov, 2020), question answering (Tan et al., 2023; Xu et al., 2024), web search (Zhao et al., 2024; Yates et al., 2021), and retrieval-augmented…”
Mapping Scientific Knowledge in Systematic Literature Reviews Through Embeddings and Bibliometric Techniques (2026, unknown): “More recent neural topic-modeling and density-based approaches - such as BERTopic [73], Top2Vec [74], and HDBSCAN-based clustering [75] combined with UMAP dimensionality reduction [76] - share our reliance on trans-former embeddings but differ in the clustering and topic-representation stages.”