The language-independent bottleneck features

By Karel Veselý, Martin Karafiát, Frantisek Grézl, M. Janda, E. Egorova. 2012

225 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

Unsupervised Pretraining Transfers Well Across Languages (2020, background): “Closer to our work, several works have shared the parameters of a neural network encoder, using feedforward networks [3, 1, 2] or LSTM [11].”
Libri-Light: A Benchmark for ASR with Limited or No Supervision (2019, neutral): “languages [5, 6] or pretraining using unsupervised objectives [7, 8].”
Unsupervised Speech Representation Learning Using WaveNet Autoencoders (2019, methodology): “e routinely used as input representations in other computer vision tasks [29]. Similarly, the speech community has used bottleneck features extracted from networks trained on phoneme prediction tasks [30], [31] as feature representations for speech recognition systems. Likewise, in natural language ”
Automatic speech recognition for under-resourced languages: A survey (2014, background): “[Thomas, 2012], [Thomas 2012b], and [Vesely, 2012] demonstrated how to use data from multiple languages to extract features for an under-resourced language and, hence, improve ASR performance.”
Sequence-Based Multi-Lingual Low Resource Speech Recognition (2018, methodology): “Unlike [5], we do not have any bottleneck layer, and the whole model is sequence trained based on CTC loss.”
Deep maxout networks for low-resource speech recognition (2013, methodology): “In tandem systems, we use DNNs to generate phone posteriors or bottleneck features (BNF), and build normal GMMHMM models with the discriminative front-end [3, 4, 5, 6].”
Language Independent End-to-End Architecture For Joint Language and Speech Recognition (methodology): “DNN-based approaches are also used to produce multilingual bottleneck features [26] [27].”
Semi-supervised ensemble DNN acoustic model training (2017, background): “confidence data as different resources, a multi-task training architecture inspired from multi-lingual modeling [10, 11, 12, 13, 14, 15] can be applied for semi-supervised training.”
Transfer Learning of Language-independent End-to-end ASR with Language Model Fusion (2018, background): “There are various approaches to leverage other languages: (a) to train a model multilingually (multi-task learning with other languages), and then further fine-tune to a particular language [6], and (b) to adapt a multilingual model to a new language using transfer learning [6–9] and additional feat”
Massively Multilingual Adversarial Speech Recognition (2019, methodology): “Popular approaches include the use of multilingual bottleneck features (Vesely et al., 2012) as well as multilingual model training before ﬁne-tuning to a target language (Scanzio et al., 2008; Vu et al., 2012).”
Multilingual deep neural network based acoustic modeling for rapid language adaptation (2014, methodology): “INTRODUCTION HMM/DNN hybrid systems that use deep neural networks (DNNs) to estimate the emission probabilities of the hidden Markov model (HMM) states [1–4] were successfully applied to large vocabulary continous speech recognition (LVCSR) and led to a significant improvement in various tasks with…”
Data augmentation for low resource languages (2014, background): “Another group relies on an alternative form of mapping, such as phone-to-phone [29] or hidden layer unit-to-target as in the MLP-based work of [30, 14].”
Improving low-resource CD-DNN-HMM using dropout and multilingual DNN training (2013, background): “There has been a number of works dedicated to training multilingual bottleneck features [11, 12, 13, 14] or probabilistic posterior features [5, 12] used in tandem systems.”
Investigation of transfer learning for ASR using LF-MMI trained neural networks (2017, methodology): “Supervised training using out-of-domain data is also a form of pretraining and it has been used to learn multilingual bottleneck features in [10, 11].”
Semi-Supervised and Cross-Lingual Knowledge Transfer Learnings for DNN Hybrid Acoustic Models Under Low-Resource Conditions (2016, background): “Shared hidden layers (SHL) are trained with multilingual data while softmax layers of the BNF extractor are language dependent [8, 12, 14].”
Linguistic Unit Discovery from Multi-Modal Inputs in Unwritten Languages: Summary of the “Speaking Rosetta” JSALT 2017 Workshop (2018, background): “Another strand of research focuses on building ASR systems using speech data from multiple languages, thus trying to create universal or crosslingual ASR systems [7, 8, 9, 10].”
An empirical evaluation of zero resource acoustic unit discovery (2017, unknown): “In our study, we explore a state-of-the-art multilingual time delay neural network (TDNN) technique to generate robust cross-lingual acoustic features in zero-resource setting, and the hope is that as one moves to new languages, this data driven feature extraction approach will work as-is, without…”
Multilingual bottle-neck feature learning from untranscribed speech (2017, methodology): “However, different from [22], our BNFs are learned without any manual transcription.”