End-to-End Neural Speech Coding for Real-Time Communications

By Xue Jiang, Xiulian Peng, Chengyu Zheng, Huaying Xue, Yuan Zhang, Yan Lu. 2022

39 citations recorded by the source corpus.

Deep-learning based methods have shown their advantages in audio coding over traditional ones but limited attention has been paid on real-time communications (RTC). This paper proposes the TFNet, an end-to-end neural speech codec with low latency for RTC. It takes an encoder-temporal filtering-decoder paradigm that has seldom been investigated in audio coding. An interleaved structure is proposed for temporal filtering to capture both short-term and long-term temporal dependencies. Furthermore, with end-to-end optimization, the TFNet is jointly optimized with speech enhancement and packet loss concealment, yielding a one-for-all network for three tasks. Both subjective and objective results demonstrate the efficiency of the proposed TFNet.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

High Fidelity Neural Audio Compression (2022, methodology): “…(Kleijn et al., 2018; Valin & Skoglund, 2019b; Lim et al., 2020; Kleijn et al., 2021; Zeghidour et al., 2021; Omran et al., 2022; Lin et al., 2022; Jayashankar et al., 2022; Li et al.; Jiang et al., 2022), where most methods are based on quantizing the latent space before feeding it to the decoder.”
Cross-Scale Vector Quantization for Scalable Neural Speech Coding (2022, methodology): “On the other hand, some researchers propose end-to-end neural networks based on the vector-quantized variational autoencoder framework (VQ-VAE [9]) where the encoder, codebook and the decoder are learned in a joint fashion [10, 11, 12, 13, 14, 15].”
Disentangled Feature Learning for Real-Time Neural Speech Coding (2022, methodology): “Content encoder Following the similar design as TFNet [6], the content encoder E c includes several 2D causal convolutional layers followed by causal temporal ﬁltering modules.”