Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi

By Michael McAuliffe, Michaela Socolof, Sarah Mihuc, M. Wagner, Morgan Sonderegger. 2017

1,549 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. James R. Glass
5. Wei-Ning Hsu
6. Hung-yi Lee
7. Zhou Zhao
8. Frank K. Soong

In Natural Language Processing

1. Tao Qin
2. Tie-Yan Liu
3. Xu Tan
4. Graham Neubig
5. Lior Wolf
6. Philipp Koehn
7. Yonatan Belinkov
8. Hung-yi Lee

In Speech & Audio

1. Xu Tan
2. Tao Qin
3. Wei-Ning Hsu
4. Tie-Yan Liu
5. Sheng Zhao
6. Hung-yi Lee
7. Yi Ren
8. Zhou Zhao

Citations in the corpus

• ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reordering (2024, methodology): “As shown in Figure 2, ELLA-V first constructs a hybrid sequence H : , 1 of acoustic and phoneme tokens, structured as: P 1 , P 2 , . . . , P n , BOS , P 1 , ⟨ C 1 ⟩ : , 1 , EOP , P 2 , ⟨ C 2 ⟩ : , 1 , EOP , . . . , P n , ⟨ C n ⟩ : , 1 , EOP , EOS It is worth noting that the MFA (Montreal Forced Alig”
• Audiobox: Unified Audio Generation with Natural Language Prompts (2023, background): “In addition, many prior studies (Le et al., 2023; Wang et al., 2023a) represent transcripts as phoneme sequences and using the off-the-shelf Montreal Forced Aligner (McAuliffe et al., 2017) for aligning the training data.”
• Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale (2023, methodology): “The Montreal Forced Aligner (MFA) [40] is used to phonemize and force align the transcript based on the MFA phone set.”
• Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance (2021, methodology): “To train the frame-wise phoneme classifier, we align transcript and speech using a forced alignment tool, the Montreal Forced Aligner (MFA) (McAuliffe et al., 2017), and extract the frame-level phoneme label ŷ.”
• A Survey on Neural Speech Synthesis (2021, methodology): “The works leveraging external alignment tools [381, 91, 226, 189] can be divided into several categories according to the used alignment tools: 1) Encoder-decoder attention: FastSpeech [284] obtains the duration label from the attention alignments of an autoregressive acoustic model.”
• AdaSpeech: Adaptive Text to Speech for Custom Voice (2021, methodology): “We use MFA (McAuliffe et al., 2017) to extract the alignment between the phoneme and mel-spectrogram sequence, which is used to prepare the input of the phoneme-level acoustic encoder.”
• FastSpeech 2: Fast and High-Quality End-to-End Text to Speech (2020, methodology): “The results are listed in Table 4b and it can be seen that more accurate duration information improves the voice quality of FastSpeech, which verifies the effectiveness of our improved duration from MFA.”
• Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders (2019, methodology): “We obtain force-aligned phoneme sequences with the Montreal Forced Aligner [24], where there are 72 possible phone classes.”
• DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism (2021, methodology): “…al. 2019), we 1) split each whole song into sentence pieces following DeepSinger (Ren et al. 2020) and train a Montreal Forced Aligner tool (MFA) (McAuliffe et al. 2017) model on those sentence-level pairs to obtain the phoneme-level alignments between song piece and its corresponding lyrics; 2)…”
• Adaspeech 2: Adaptive Text to Speech with Untranscribed Data (2021, methodology): “The ground-truth duration is obtained from a forced-alignment tool [15], and is used to train a duration predictor that built upon the phoneme encoder.”
• M2-CTTS: End-to-End Multi-Scale Multi-Modal Conversational Text-to-Speech Synthesis (2023, methodology): “…backbone and we adopt unsupervised duration modeling [17] rather than using supervised duration model with external aligner, such as Montreal Forced Aligner [18], because external aligners have risk of out-of-distribution problem and soft alignment has more flexibility for expressive performance.”
• M4Singer: A Multi-Style, Multi-Singer and Musical Score Provided Mandarin Singing Corpus (2022, methodology): “2) Montreal forced alignment (MFA) [28] is used to align original lyrics and audio.”
• EfficientTTS: An Efficient and High-Quality Text-to-Speech Architecture (2020, background): “FastSpeech 2 (Ren et al., 2021) improves the two-staged training of FastSpeech by introducing an external aligner named forced alignment (McAuliffe et al., 2017), but forced alignment itself requires an unsupervised training.”
• Scaling Effect of Self-Supervised Speech Models (2021, methodology): “Phoneme sequences are first aligned using the Montreal Forced Aligner [22].”
• Fcl-Taco2: Towards Fast, Controllable and Lightweight Text-to-Speech Synthesis (2021, methodology): “The is converted by MFA tool [22] into phoneme and pinyin sequences with tones, which are adopted as the inputs of TTS models for EN and CN experiments, respectively.”
• Surprisal-Triggered Conditional Computation with Neural Networks (2020, methodology): “Mini-Librispeech 3 is a 5-hour subset of the 960-hour LibriSpeech dataset [78] with transcripts but no phoneme labels; we used the Montreal Forced Aligner [79] to obtain label sequences from the transcripts.”
• Hierarchical Prosody Modeling for Non-Autoregressive Speech Synthesis (2020, methodology): “MFA is used to convert the transcripts into phoneme sequences and discover the alignments between the phoneme sequences and the utterances.”
• Incremental Text-to-Speech Synthesis with Prefix-to-Prefix Framework (2019, methodology): “We choose the ground-truth audios from test set as the inputs and extract the ending time of each chunk in those audios by the Montreal Forced Aligner (McAuliffe et al., 2017).”