Composition of Deep and Spiking Neural Networks for Very Low Bit Rate Speech Coding

By Milos Cernak, Alexandros Lazaridis, Afsaneh Asaei, Philip N. Garner. 2016

29 citations recorded by the source corpus.

Most current very low bit rate (VLBR) speech coding systems use hidden Markov model (HMM) based speech recognition and synthesis techniques. This allows transmission of information (such as phonemes) segment by segment; this decreases the bit rate. However, an encoder based on a phoneme speech recognition may create bursts of segmental errors; these would be further propagated to any suprasegmental (such as syllable) information coding. Together with the errors of voicing detection in pitch parametrization, HMM-based speech coding leads to speech discontinuities and unnatural speech sound artifacts. In this paper, we propose a novel VLBR speech coding framework based on neural networks (NNs) for end-to-end speech analysis and synthesis without HMMs. The speech coding framework relies on a phonological (subphonetic) representation of speech. It is designed as a composition of deep and spiking NNs: a bank of phonological analyzers at the transmitter, and a phonological synthesizer at the receiver. These are both realized as deep NNs, along with a spiking NN as an incremental and robust encoder of syllable boundaries for coding of continuous fundamental frequency (F0). A combination of phonological features defines much more sound patterns than phonetic features defined by HMM-based speech coders; this finer analysis/synthesis code contributes to smoother encoded speech. Listeners significantly prefer the NN-based approach due to fewer discontinuities and speech artifacts of the encoded speech. A single forward pass is required during the speech encoding and decoding. The proposed VLBR speech coding operates at a bit rate of approximately 360 bits/s.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

Audiodec: An Open-Source Streaming High-Fidelity Neural Audio Codec (2023, background): “NNbased encoder-decoder structures also have been proposed for different handcraft acoustic features such as spectrograms [14], phonological features, and pitches [15].”
Cascaded Cross-Module Residual Learning towards Lightweight End-to-End Speech Coding (2019, methodology): “Many DNN methods [11][12] take inputs in time-frequency (TF) domain from short time Fourier transform (STFT) or modified discrete cosine transform (MDCT), etc.”
Low Bit-rate Speech Coding with VQ-VAE and a WaveNet Decoder (2019, methodology): “In the speech domain, end-to-end coding operating at rates as low as 369 bps by use of extracted acoustic features [5] and an end-to-end optimized wideband codec performing on par with the adaptive multirate wideband (AMR-WB) codec at 9-24 kbps [6] have both recently been proposed.”
End-To-End Optimized Speech Coding with Deep Neural Networks (2017, background): “[8] proposed a nearly end-to-end design for a very-low-bitrate low-quality speech coder in 2016; however, their pipeline still required extraction of acoustic features and pitch (and was also quite complex, composing several different deep and spiking neural networks together).”