Accelerating Large-Scale Inference with Anisotropic Vector Quantization

By Ruiqi Guo, Philip Sun, Erik M. Lindgren, Quan Geng, David Simcha, Felix Chern, Sanjiv Kumar. 2019

574 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Efficiency & Compression

Citations in the corpus

Improving language models by retrieving from trillions of tokens (2021, methodology): “We use the SCaNN library (Guo et al., 2020) to achieve this.”
Adaptive Semiparametric Language Models (2021, background): “An early version of a neural language model that includes a retrieval component is presented in Guu et al. (2018).”
MURAL: Multimodal, Multitask Retrieval Across Languages (2021, background): “trieval via approximate nearest neighbors search (Guo et al., 2020) and can predict similarity within and across modalities (Parekh et al.”
Sparse, Dense, and Attentional Representations for Text Retrieval (2020, methodology): “Time and Space Analysis Figure 7 compares the running time/quality trade-off curves for DEBERT and ME-BERT on the MS MARCO passage task using the ScaNN (Guo et al., 2020) library on a 160”
Nearest Neighbor Machine Translation (2020, methodology): “eams, or querying smaller datastores, with relatively minor trade-offs in performance, as we will see in Section 5. Developing faster nearest neighbor search tools remains an active area of research (Guo et al., 2020). 4 EXPERIMENTS 4.1 SINGLE LANGUAGE-PAIR TRANSLATION To test whether kNN-MT can imp”
Hurdles to Progress in Long-form Question Answering (2021, positive): “During inference, we perform a maximum inner-product search (MIPS) with the ScaNN library (Guo et al., 2020) to efﬁciently ﬁnd the top K documents.”
Transformer Memory as a Differentiable Search Index (2022, methodology): “For the T5-based dual encoders, we train with contrastive learning on the NQ pairs until convergence ( ≈ 10K steps) and obtain top-k nearest neighbors with a system similar to ScaNN (Guo et al., 2020).”
MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text (2022, methodology): “Specifically, we apply the backbone encoder fθ to encode a query q, and use maximum inner product search (MIPS (Guo et al., 2020)) over all of the memory candidates m ∈M to find the Top-K nearest neighbors TopK(M|q) = [m1, · · · ,mk].”
COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List (2021, background): “As a collection of dense vectors, it is also possible to organize each inverted list as an approximate search index (Johnson et al., 2017; Guo et al., 2019) to further speed up search.”
Controllable Semantic Parsing via Retrieval Augmentation (2021, methodology): “In actual deployment, brute force retrieval might be too slow, and fast nearest neighbor methods (Johnson et al., 2021; Guo et al., 2020) could be used to speed up the retriever.”
Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages (2021, background): “, 2020; Artetxe and Schwenk, 2019), scalable approximate nearest neighbor search (Johnson et al., 2017a; Subramanya et al., 2019; Guo et al., 2020), and optical character recognition (OCR) of Indic scripts in rich text documents.”
Condenser: a Pre-training Architecture for Dense Retrieval (2021, methodology): “By pre-encoding the corpus into MIPS (Johnson et al., 2017; Guo et al., 2020) index, retrieval can run online with millisecond-level latency.”
Dense Text Retrieval Based on Pretrained Language Models: A Survey (2022, methodology): “ScaNN [268] is developed with quantizationbased approaches, which achieves excellent performance on ANN-Benchmarks.”
Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performance (2021, methodology): “• ScaNN [20]: It uses a new loss to train PQ and recently achieved state-of-the-art results on ANN benchmarks.”
Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval (2021, methodology): “By pre-encoding the corpus into MIPS index, retrieval can run on-line with millisecond-level latency (Johnson et al., 2017; Guo et al., 2020).”
Re-Imagen: Retrieval-Augmented Text-to-Image Generator (2022, methodology): “We experimented with both CLIP and BM25 similarity scores, and retrieval was implemented with ScaNN (Guo et al., 2020).”