Byte Pair Encoding: a Text Compression Scheme That Accelerates Pattern Matching
By Takuya Kida, Shuichi Fukamachi, Masayuki Takeda, Ayumi Shinohara, Takeshi Shinohara, Setsuo Arikawa. 1999
Subfields: Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Thomas Scialom
- Paul-Alexis Dray
- Jacopo Staiano
- Sylvain Lamprier
- Benjamin Piwowarski
- Ananda Theertha Suresh
- Rajiv Mathews
- Michael D. Riley
Citations in the corpus
- MLSUM: The Multilingual Summarization Corpus (2020, methodology): “Neither model is pretrained, nor based on LSTM (Hochreiter and Schmidhuber, 1997), and they both use BPE tokenization (Shibata et al., 1999).”
- Federated Learning of N-Gram Language Models (2019, methodology): “To train neural models on morphologically rich languages, subword segments such as byte-pair encodings or word-pieces (Shibata et al., 1999; Schuster and Nakajima, 2012; Kudo, 2018) are typically used.”
- Effective Sentence Scoring Method Using BERT for Speech Recognition (2019, methodology): “The target sequence is processed in 5K case-insensitive sub-word units created via unigram byte-pair encoding Shibata et al. (1999).”
- Yi: Open Foundation Models by 01.AI (2024, methodology): “We use byte-pair encoding (BPE) [69] implemented in the SentencePiece framework [40], to tokenize the pretraining data.”
- Meaningless yet meaningful: Morphology grounded subword-level NMT (2018, methodology): “(Shibata et al., 1999) 3.1 BPE as subword unit BPE works as subword segmentation method for both NMT (Sennrich et al., 2016) and SMT (Kunchukuttan and Bhattacharyya, 2017).”
- Baichuan 2: Open Large-scale Language Models (2023, unknown): “We use byte-pair encoding (BPE) (Shibata et al., 1999) from SentencePiece (Kudo and Richardson, 2018) to tokenize the data.”
- ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction (2020, background): “Motivated by the intuition that rare and unknown words can often be decomposed into multiple known subwords, BPE finds the best word segmentation by iteratively and greedily merging frequent pairs of characters [32].”