Japanese and Korean voice search

By Mike Schuster, Kaisuke Nakajima. 2012

980 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer (2017, methodology): “Similar to GNMT, to effectively deal with rare words, we used sub-word units (also known as “wordpieces") (Schuster & Nakajima, 2012) for inputs and outputs in our system.”
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation (2016, neutral): “Using a human side-by-side evaluation on a set of isolated simple sentences, it reduces translation errors by an average of 60% compared to Google’s phrase-based production system.”
Memory-Efficient Adaptive Optimization for Large-Scale Learning (2019, methodology): “Both models were trained on a 4 ⇥ 4 Cloud TPU-v2 using the Lingvo [24] sequence modeling framework, with 32K word-pieces [21] for each language pair.”
Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates (2018, methodology): “3 Subword segmentations with language model 3.1 Byte-Pair-Encoding (BPE) Byte-Pair-Encoding (BPE) (Sennrich et al., 2016; Schuster and Nakajima, 2012) is a subword segmentation algorithm widely used in many NMT systems 1 .”
End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures (2019, neutral): “All AMs are token-based, outputting 10k word pieces [19], although this is just a choice for simplicity in this comparative study, not the result of a limitation.”
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations (2020, methodology): “Most recent work [51, 58, 17, 42] uses the better performing word pieces [50] for both models.”
BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition (2021, methodology): “Two tokenization schemes are used for the transcripts: graphemes or word-piece models (WPMs) [65], the details of”
Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition (2020, methodology): “A 1024-token word-piece-model (WPM) [52] constructed from the transcripts of the LibriSpeech training set is used to tokenize the transcripts.”
BPE-Dropout: Simple and Effective Subword Regularization (2019, methodology): “Other segmentation algorithms include Creutz and Lagus (2006), Schuster and Nakajima (2012), Chitnis and DeNero (2015), Kunchukuttan and Bhattacharyya (2016), Wu and Zhao (2018), Banerjee and Bhattacharyya (2018).”
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition (2019, methodology): “The WPM for Lib-riSpeech 960h is constructed using the training set transcripts.”
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation (2016, background): “hemodelwithallmultilingualdataconsistingof multiplelanguagepairsatonce,possiblyafterover-orundersamplingsomeofthedatatoadjustforthe relativeratioofthelanguagedataavailable. Weuseasharedwordpiecemodel [18,21]acrossallthesource andtargetdatausedfortraining,usuallywith32,000wordpieces. Alltrainingiscar”
Improved Noisy Student Training for Automatic Speech Recognition (2020, methodology): “We tokenize the transcripts using a WPM model [37] with vocabulary size 16k constructed from the clean 100h subset transcripts.”
LiT: Zero-Shot Transfer with Locked-image text Tuning (2021, methodology): “BERT uses a WordPiece (WP) tokenizer [49, 64], and all others use the SentencePiece (SP) tokenizer [34], a component which we also ablate with the ViT model.”
The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation (2018, methodology): “Each sentence was encoded into a sequence of sub-word units obtained by first tokenizing the sentence with the Moses tokenizer, then splitting tokens into subword units (also known as “wordpieces”) using the approach described in (Schuster and Nakajima, 2012).”
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling (2019, methodology): “Available tokenizers include VocabFileTokenizer which uses a look-up table provided as a ﬁle, BpeTokenizer for byte pair encoding [18], and WpmTokenizer for word-piece models [17].”
State-of-the-Art Speech Recognition with Sequence-to-Sequence Models (2017, methodology): “The word piece models [21] used in this paper are sub-word units, ranging from graphemes all the way up to entire words.”
Large scale distributed neural network training through online distillation (2018, methodology): “We used 256 dimensional input embeddings and a vocabulary of 24006 word pieces (Schuster & Nakajima, 2012), including sentence and paragraph start and end tokens, out of vocabulary (generally non-English characters), and end of document.”
w2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training (2021, methodology): “For transcript tokenization, we use a 1024token WordPiece model [36] that is constructed from the transcripts of the training set used by fine-tuning.”