Set-Theoretic Alignment for Comparable Corpora

By Thierry Etchegoyhen, Andoni Azpeitia. 2016

28 citations recorded by the source corpus.

We describe and evaluate a simple method to extract parallel sentences from comparable corpora. The approach, termed STACC, is based on expanded lexical sets and the Jaccard similarity coefficient. We evaluate our system against state-of-theart methods on a large range of datasets in different domains, for ten language pairs, showing that it either matches or outperforms current methods across the board and gives significantly better results on the noisiest datasets. STACC is a portable method, requiring no particular adaptation for new domains or language pairs, thus enabling the efficient mining of parallel sentences in comparable corpora.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Graham Neubig
2. Luke S. Zettlemoyer
3. Holger Schwenk
4. Edouard Grave
5. Furu Wei
6. Michael Auli
7. Angela Fan
8. Vishrav Chaudhary

In Natural Language Processing

1. Holger Schwenk
2. Graham Neubig
3. Luke S. Zettlemoyer
4. Vishrav Chaudhary
5. Edouard Grave
6. Angela Fan
7. Mikel Artetxe
8. Michael Auli

Citations in the corpus

• Beyond English-Centric Multilingual Machine Translation (2020, methodology): “For example, methods aligned documents on metadata (Resnik, 1999) or sentences based on Jaccard distance or neural representations (Etchegoyhen and Azpeitia, 2016; Buck and Koehn, 2016).”
• Filtering and Mining Parallel Data in a Joint Multilingual Space (2018, methodology): “The best performing system “VIC” is based on the so-called STACC method which was shown to achieve state-of-the-art performance (Etchegoyhen and Azpeitia, 2016).”
• Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings (2018, methodology): “STACC uses seed lexical translations induced from IBM alignments, which are combined with set expansion operations to score translation candidates through the Jaccard similarity coefficient (Etchegoyhen and Azpeitia, 2016; Azpeitia et al., 2017, 2018).”
• WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia (2019, methodology): “The STACC method uses seed lexical translations induced from IBM alignments, which are combined with set expansion operations to score translation candidates through the Jaccard similarity coefficient (Etchegoyhen and Azpeitia, 2016; Azpeitia et al., 2017, 2018).”
• CCMatrix: Mining Billions of High-Quality Parallel Sentences on the Web (2019, methodology): “The STACC method uses seed lexical translations induced from IBM alignments, which are combined with set expansion operations to score translation candidates through the Jaccard similarity coeffi-cient (Etchegoyhen and Azpeitia, 2016; Azpeitia et al., 2017, 2018).”
• Overview of the Second BUCC Shared Task: Spotting Parallel Sentences in Comparable Corpora (2017, methodology): “Final sentence similarity is computed by their STACC method (Set-Theoretic Alignment for Comparable Corpora, (Etchegoyhen and Azpeitia, 2016)), which extends basic word overlap by taking into account non-matched words that share a long enough common prefix, as well as numbers and capitalized true-cas”
• Extracting Parallel Sentences from Comparable Corpora with STACC Variants (2018, methodology): “For our participation in the BUCC 2018 shared task on extracting parallel sentences from comparable corpora, we followed the STACC approach of (Etchegoyhen et al., 2016; Etchegoyhen and Azpeitia, 2016), which is based on Jac-card similarity (Jaccard, 1901) over lexical sets, with additional set…”