Vector Quantization

By Robert M. Gray. 2017

2,461 citations recorded by the source corpus.

Subfields: Control and Systems Engineering, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Efficiency & Compression

Citations in the corpus

Efficient Natural Language Response Suggestion for Smart Reply (2017, background): “Product quantization works by decomposing the high-dimensional vectors into low-dimensional subspaces and then quantizing them separately [4].”
SoundStream: An End-to-End Neural Audio Codec (2021, background): “In vector quantization [28], a point in a high-dimensional space is mapped onto a discrete set of code vectors.”
High Fidelity Neural Audio Compression (2022, background): “Results for EnCodec working at 6 kbps, EnCodec with Residual Vector Quantization (RVQ) at 6 kbps, and Opus at 6 kbps, and MP3 at 64 kbps are reported in Table 4.”
RepCodec: A Speech Representation Codec for Speech Tokenization (2023, methodology): “Both k-means (Lloyd, 1982) and VQ (Gray, 1984) are algorithms to discretize a high-dimensional vector into a discrete label.”
HiFi-Codec: Group-residual Vector quantization for High Fidelity Audio Codec (2023, neutral): “The most relevant related works to ours is the SoundStream [8] and Encodec [7] models, where these methods propose to use a fully convolutional encoder-decoder architecture with a Residual Vector Quantization (RVQ) [20, 21]”
Discovering Discrete Distributed Representations (1990, neutral): “It also turns out, much to my dismay, that ICL is virtually identical to a conventional technique in data compression known as multistep vector quantization (Gray, 1984).”
Part-dependent Label Noise: Towards Instance-dependent Label Noise (2020, result): “In contrast to principal components analysis (PCA) [2] and vector quantization (VQ) [14] that learn holistic but not parts-based representations, NMF allows additive but not subtractive combinations.”
Incremental Slow Feature Analysis: Adaptive Low-Complexity Slow Feature Updating from High-Dimensional Input Streams (2012, methodology): “many well-known unsupervised feature extractors (Abut, 1990; Jolliffe, 1986; Comon, 1994; Lee and Seung, 1999; Kohonen, 2001; Hinton, 2002), which ignore dynamics, and 2.”
Unsupervised Learning for Expressive Speech Synthesis (2017, methodology): “The clustering algorithm is k-means, concretely VQ, as by [12].”
Regeneration Learning: A Learning Paradigm for Data Generation (2023, background): “, PECO [22]), residual quantizers [30, 88, 16], or leverage diffusion models to learn the hidden representation [52].”
Compact 3D Gaussian Representation for Radiance Field (2023, background): “…is repeated up to the final stage, as a result, the selected indices and the codebook from each stage collectively represent the original vector. propose a codebook learned to represent representative geometric attributes, including scale and rotation, by employing vector quantization (VQ) [14].”
QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks (2024, methodology): “As presented in AQLM and here, fine-tuning is essentially a hybrid method between pure PTQ and full QAT that requires significantly less data and compute than full QAT.”
Prototype-based Neural Network Layers: Incorporating Vector Quantization (2018, background): “G RAY stated [14]: “A vector quantizer is a system for mapping a sequence of continuous or discrete vectors into a digital sequence suitable for communication over or storage in a digital channel.”
Autoregressive Image Generation using Residual Quantization (2022, methodology): “For the nearest neighbor search, product quantization (PQ) [15] approximates a vector as the sum of linearly independent vectors in the codebook.”
Finite Scalar Quantization: VQ-VAE Made Simple (2023, background): “Vector quantization (VQ), initially introduced by Gray (1984), has recently seen a renaissance in the context of learning discrete representations with neural networks.”
Feature Quantization Improves GAN Training (2020, methodology): “Vector quantization (VQ) (Gray, 1984) has been used in various settings, including clustering (Equitz, 1989), metric learning (Schneider et al.”
Self-Supervised Speech Representation Learning: A Review (2022, methodology): “niques include k-means clustering [11], mixture models [12], autoencoders [13], and non-negative matrix factorization [14].”
LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS (2023, background): “Previous works [11, 23, 42] have shown that learning a discrete and compact representation not only contributes to visual understanding but also improves the robustness of models.”
Multiscale Quantization for Fast Similarity Search (2017, background): “2 Related Works The original idea of product quantization traces back to early works of signal processing [14, 12].”