Tangent-CFT: An Embedding Model for Mathematical Formulas

By Behrooz Mansouri, Shaurya Rohatgi, Douglas W. Oard, Jian Wu, C. Lee Giles, Richard Zanibbi. 2019

98 citations recorded by the source corpus.

When searching for mathematical content, accurate measures of formula similarity can help with tasks such as document ranking, query recommendation, and result set clustering. While there have been many attempts at embedding words and graphs, formula embedding is in its early stages. We introduce a new formula embedding model that we use with two hierarchical representations, (1) Symbol Layout Trees (SLTs) for appearance, and (2) Operator Trees (OPTs) for mathematical content. Following the approach of graph embeddings such as DeepWalk, we generate tuples representing paths between pairs of symbols depth-first, embed tuples using the fastText n-gram embedding model, and then represent an SLT or OPT by its average tuple embedding vector. We then combine SLT and OPT embeddings, leading to state-of-the-art results for the NTCIR-12 formula retrieval task. Our fine-grained holistic vector representations allow us to retrieve many more partially similar formulas than methods using structural matching in trees. Combining our embedding model with structural matching in the Approach0 formula search engine produces state-of-the-art results for both fully and partially relevant results on the NTCIR-12 benchmark. Source code for our system is publicly available.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Ji-rong Wen
2. Kun Zhou
3. Zhi Tang
4. Katharina Morik
5. Liangcai Gao
6. Ke Yuan
7. Shuai Peng
8. Shijin Wang

In Computational Theory and Mathematics

1. Kun Zhou
2. Ji-rong Wen
3. Katharina Morik
4. Shijin Wang
5. Xin Zhao
6. Zheng Gong
7. Jing Sha
8. Iddo Drori

Citations in the corpus

• Continual Pre-training of Language Models for Math Problem Understanding with Syntax-Aware Memory Network (2022, methodology): “, Operator Tree (Zanibbi and Blostein, 2012)), then network embedding method (Mansouri et al., 2019) and graph neural network (Song and Chen, 2021)”
• Semantic Search in Millions of Equations (2020, methodology): “[16] propose to embed equations using fastText, a method originally designed for computing word embeddings, while in our previous work [20] we compute embeddings with a similar embedding learning task and convolutional neural networks (see Section 4.”
• MathBERT: A Pre-Trained Model for Mathematical Formula Understanding (2021, methodology): “Several research tasks on mathematical formula, including Mathematical Information Retrieval(MIR) [Yuan et al. , 2016; Davila and Zanibbi, 2017; Mansouri et al. , 2019], Mathematical Formula Understanding (MFU) [Jiang et al. , 2018; Yuan et al. , 2020] and so forth, have continuously attracted…”